Edit fields node getting wrong input

Questions
,
1

Describe the problem/error/question

I have an Edit fields node which picks an input from a kafka trigger but is interpreting this input in a quite wrong way. The input from Kafka trigger node has the following schema:

[
  {
    topic: string,
    message: string
  }
]

as in

[
  {
    "message": "{\"user_id\": 475, \"name\": \"Theresa Davidson\", \"phone\": \"502-890-7469\", \"address\": \"2701 Samuel Summit Suite 938\\nRyanbury, PR 78954\", \"ssn\": \"261-08-6677\", \"created_at\": \"2024-01-09 16:39:05\", \"updated_at\": \"2024-01-09 16:39:05\"}",
    "topic": "users"
  }
]

I want to convert the message field to a JSON object (further named payload) exactly like in the user-bets node (which is working ok) but for some reason is getting the input as the output. Initially I duplicated the user-bets to JSON node and then suspecting some parameter could be wrongly leaked I added a fresh new Edit fields node with no different result.

What is the error message (if any)?

ERROR: The ‘JSON Output’ in item 0 contains invalid JSON

{ “topic”: “users”, “payload”: {“user_id”:475,“name”:“Theresa Davidson”,“phone”:“502-890-7469”,“address”:“2701 Samuel Summit Suite 938\nRyanbury, PR 78954”,“ssn”:“261-08-6677”,“created_at”:“2024-01-09 “16”:“39”:05”,“updated_at”:“2024-01-09 “16”:“39”:05”} }; Original input: { topic: “users”, payload: {“user_id”:475,“name”:“Theresa Davidson”,“phone”:“502-890-7469”,“address”:“2701 Samuel Summit Suite 938\nRyanbury, PR 78954”,“ssn”:“261-08-6677”,“created_at”:“2024-01-09 16:39:05”,“updated_at”:“2024-01-09 16:39:05”} }

=====

Stack

NodeOperationError: The 'JSON Output' in item 0 contains invalid JSON
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Set/v2/raw.mode.js:44:15)
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Set/v2/SetV2.node.js:233:57)
    at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:696:42)
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:656:68

Please share your workflow

But separating the workflows it is happening anyway :point_down:

Share the output returned by the last node

Output returned by the last node (the Kafka trigger) is pinned in both trigger nodes. Expected output is:

{ 
  “topic”: “users”, 
  “payload”: {
    “user_id”:475,
    “name”:“Theresa Davidson”,
    “phone”:“502-890-7469”,
    “address”:“2701 Samuel Summit Suite 938\nRyanbury, PR 78954”,
    “ssn”:“261-08-6677”,
    “created_at”:“2024-01-09 “16”:“39”:05”,
    “updated_at”:“2024-01-09 “16”:“39”:05”
  }
}

Information on your n8n setup

  • n8n version: 1.20.0 (NODE_VERSION=18.18.2, YARN_VERSION=1.22.19, NODE_ENV=production, N8N_RELEASE_TYPE=stable)
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app): docker-compose
  • Operating system: Docker on MacOS 14
2

Hey @telekosmos,

I have just ran the test workflow and it looks fine to me, Can you tell me which node has the incorrect output and what you are expecting the output to be?

3

Hi, thanks for the quick response

The node who is giving the error is users to JSON, I’m attaching an screenshot here with the error and the input payload from the previous node (copied from the pinned data in the previous node) is:

[
  {
    "message": "{\"user_id\": 280, \"name\": \"Theresa Davidson\", \"phone\": \"502-890-7469\", \"ssn\": \"261-08-6677\", \"created_at\": \"2024-01-09 17:40:44\", \"updated_at\": \"2024-01-09 17:40:44\"}",
    "topic": "users"
  }
]

expected payload is:

{ 
  “topic”: “users”, 
  “payload”: {
    “user_id”:280,
    “name”:“Theresa Davidson”,
    “phone”:“502-890-7469”,
    “ssn”:“261-08-6677”,
    “created_at”:“2024-01-09 17:40:44”,
    “updated_at”:“2024-01-09 17:40:44”
  }
}

Screenshot 2024-01-09 at 19.05.48
Screenshot 2024-01-09 at 19.05.481920×1175 116 KB

Thanks again!

pd: you can ignore the message field, it is there just as control field to test the input

4

Hi @telekosmos. Try this in the JSON Output expression:

{{  JSON.stringify({
  topic: $input.item.json.topic,
  payload:  JSON.parse($input.item.json.message)
}) }}

Is that what you expected?

5

Ok, I’ve got it: the format of the created_at and updated_at fields in the message field in the input are crashing the transformation because of the colons (:). Actually it can be seen (with difficultly) in the error message: “updated_at”:“2024-01-09 “16”:“39”:05”, which is obviously malformed.

@david001’s solution works but I can’t understand why, as the JSON.stringify returns just an string, not an object… so I suspect something is working behind the scenes which makes the dates value with the colon to crash and the JSON.stringify works… bit counter-intuitive for me :roll_eyes:

Although the issue seems to be settled, I’m going to leave it open for a bit in the case someone wants to suggest or leave a comment clarifying if something is working behind the scenes as I suspect

Thanks all!

6

Hi @telekosmos. Agree. The error is a bit weird, specially with the formatting of the fields with a colon. I’ve checked the code for the Set node and found this piece of code which apparently is performing several replaces and might be the source of the parsing issue