What is the problem?
For certain values of the tunnels.anonymous field, I see the below exception. Note that this is somewhat intermittent. The exception will not always show:
/srv/service/node_modules/mariadb/lib/misc/errors.js:61 return new SqlError(msg, sql, fatal, info, sqlState, errno, additionalStack, addHeader); ^ SqlError: (conn=265, no: 1406, SQLState: 22001) Data too long for column 'anonymous' at row 1 sql: INSERT INTO tunnels (operator, type, anonymous) VALUES OêÔ'] at Object.module.exports.createError (/srv/service/node_modules/mariadb/lib/misc/errors.js:61:10) at PacketNodeEncoded.readError (/srv/service/node_modules/mariadb/lib/io/packet.js:511:19) at Query.readResponsePacket (/srv/service/node_modules/mariadb/lib/cmd/resultset.js:46:28) at PacketInputStream.receivePacketBasic (/srv/service/node_modules/mariadb/lib/io/packet-input-stream.js:104:9) at PacketInputStream.onData (/srv/service/node_modules/mariadb/lib/io/packet-input-stream.js:169:20) at Socket.emit (node:events:513:28) at addChunk (node:internal/streams/readable:315:12) at readableAddChunk (node:internal/streams/readable:289:9) at Socket.Readable.push (node:internal/streams/readable:228:10) at TCP.onStreamRead (node:internal/stream_base_commons:190:23) { text: "Data too long for column 'anonymous' at row 1", sql: 'INSERT INTO\n' + ' tunnels (operator, type, anonymous)\n' + ' VALUES\n' + " (?, ?, ?) - parameters:['¨',-26,'𰄛\x8DOêÔ']", fatal: false, errno: 1406, sqlState: '22001', code: 'ER_DATA_TOO_LONG' }
Steps to reproduce problem
- Save the JSON from "Reproduction data" below as a .gz file (e.g. reprod.json.gz) into the ipoid/tmp directory
- If necessary, start up docker in the ipoid directory (e.g. docker compose up -d)
- Initialise the database: docker compose exec web node init-db.js
- Run this command: docker compose exec -e FEED_PATH=./tmp/reprod.json.gz web node import-data.js
- If the exception does not appear first time, you may have to repeat step 4 multiple times until you do
Environment
ipoid commit 9bac83bb26f9b808edc3756284a8183a4e21c5c2, running on docker.
Reproduction data
{"as": {}, "risks": ["LOGIN_BRUTEFORCE", "GEO_MISMATCH", "TUNNEL", "CALLBACK_PROXY", "CALLBACK_PROXY", "TUNNEL", "CALLBACK_PROXY", "GEO_MISMATCH", "GEO_MISMATCH", "WEB_SCRAPING", "TUNNEL"], "organization": null, "location": {"city": "\u00d8", "country": "|", "state": null}, "infrastructure": -111, "client": {"behaviors": [false, "6", null]}, "tunnels": [{"operator": "\u00a8", "type": -26, "entries": [false, false, null, true, 16756, null, true, "\u001c", "\u00d2\u0005", null], "anonymous": "\ud880\udd1b\u008dO\u00ea\u00d4"}], "ip": "3205c46e-9770-46c2-939a-a793075d4672"}