Transform invalid JSON

Good morning,

i have a problem due to a file that consists of not 100% valid json.

The file in general is built like this:

{}
{}
{}
{}

This means, for a proper json file, it misses [] around and a comma at the end of each line.

Due to the fact, that I’m receiving an error while reading the file (which is correct of course) I’m not sure if I need to provide some scripting/custom nodes before the file can be processed from n8n or if I could use the function node and still grab the data so I thought asking you first before I’m starting to try my luck.

Thank you very much in advance and have a great day.

BR
Benji

Hola @Benjamin_Exner, so your file consists of one valid JSON Object per line and looks something like this?

{"message": "foo"}
{"message": "bar"}
{"message": "baz"}
{"message": "qux"}

Would you read this file through a binary node? If so, you could convert your binary data first and then split it up on each line break using a workflow like this (adjusting the path as necessary, of course):

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "filePath": "C:\\Users\\Tom\\Desktop\\example.txt"
      },
      "name": "Read Binary File",
      "type": "n8n-nodes-base.readBinaryFile",
      "typeVersion": 1,
      "position": [
        450,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "let results = [];\n\n// splitting our only input item into an array of strings on new lines and looping through them\nfor (item of items[0].json.data.split(/\\r?\\n/).filter(e => e)) {\n  // parsing the JSON and pushing it to our results\n  results.push({\n    json: JSON.parse(item)\n  });\n}\n\nreturn results;"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        850,
        300
      ]
    },
    {
      "parameters": {
        "setAllData": false,
        "options": {}
      },
      "name": "Move Binary Data",
      "type": "n8n-nodes-base.moveBinaryData",
      "typeVersion": 1,
      "position": [
        650,
        300
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Read Binary File",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Read Binary File": {
      "main": [
        [
          {
            "node": "Move Binary Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Move Binary Data": {
      "main": [
        [
          {
            "node": "Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

This would leave you with a proper n8n data structure after the Function node (with individual items):

3 Likes

Nice! Thank you very much, will try in a second.

1 Like

Works like a charm :-)! Thank you @MutedJam

2 Likes

Awesome, thanks so much for confirming!