Workflow crash for unknown reason

I have a workflow, where I backup a clients SaaS solution via REST API to Nextcloud.
Workflow runs through several hundred pages with no issues, but then it suddenly says it crashed for an unknown reason. Execution seems to stop just before a code piece, that has run for 212 times, with no issues and there is no new parameters for this run.

error given just says “Workflow execution process did crash for an unknown reason!”

{
  "meta": {
    "instanceId": "e15f21b7a1b70a913ee82847a038522916b3dd0d1a1cc7474a1d665386f81537"
  },
  "nodes": [
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1640,
        520
      ],
      "id": "66d04a21-5de2-4b0c-b02c-0c1c506caefd"
    },
    {
      "parameters": {
        "url": "=https://[URL]/api/v1/{{$node[\"Set apiCall\"].json[\"apiCall\"]}}",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "page",
              "value": "={{ $node[\"Set pages\"].json[\"newPage\"] }}"
            }
          ]
        }
      },
      "name": "GET apiCall",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 2,
      "position": [
        720,
        500
      ],
      "id": "f7d1f52b-533d-4438-beb3-6097ee4a0575"
    },
    {
      "parameters": {
        "path": "=/Backups/{{ $node[\"Set apiCall\"].json[\"apiCall\"] }}_page_{{$node[\"GET apiCall\"].json[\"meta\"][\"page\"]}}.json",
        "fileContent": "={{$json}}"
      },
      "name": "Save to Nextcloud",
      "type": "n8n-nodes-base.nextCloud",
      "typeVersion": 1,
      "position": [
        1260,
        340
      ],
      "id": "e81ece32-14c7-4b6b-9a92-a43da118f8a9"
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "number": [
            {
              "value1": "={{ $json[\"curPage\"] }}",
              "value2": "={{ $json[\"totalPage\"] }}"
            }
          ]
        }
      },
      "name": "Check if pagination?",
      "type": "n8n-nodes-base.if",
      "position": [
        1800,
        920
      ],
      "typeVersion": 1,
      "id": "83a8b3c3-f8f4-445b-b80a-0c87b37719c7"
    },
    {
      "parameters": {
        "conditions": {
          "string": [
            {
              "value1": "={{$node[\"Set apiCall\"].json[\"apiCall\"]}}",
              "value2": "settings"
            }
          ]
        }
      },
      "name": "Settings?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1020,
        660
      ],
      "id": "ff4e290f-93f1-4da5-a0e8-2b46c33c818e"
    },
    {
      "parameters": {
        "triggerTimes": {
          "item": [
            {
              "mode": "everyWeek",
              "hour": 12,
              "weekday": "6"
            }
          ]
        }
      },
      "name": "Cron - Sat 12:00",
      "type": "n8n-nodes-base.cron",
      "typeVersion": 1,
      "position": [
        -20,
        520
      ],
      "id": "34356d78-609c-45d0-bc83-919281b7aaf6"
    },
    {
      "parameters": {
        "jsCode": "// Code here will run once per input item.\n// More info and help: https://docs.n8n.io/nodes/n8n-nodes-base.functionItem\n// Tip: You can use luxon for dates and $jmespath for querying JSON structures\nlet apiCall = \"\";\nlet jsonData = \"\";\n\ntry {\n  apiCall = $node[\"Set\"].json[\"apiCall\"];\n  if (apiCall == \"products\") {\n        apiCall = \"customers\";\n        jsonData = '{{$node[\"GET apiCall\"].json[\"customers\"]}}'\n  }\n\n    else if (apiCall == \"customers\") {\n        apiCall = \"customer_assets\";\n        jsonData = '{{$node[\"GET apiCall\"].json[\"assets\"]}}'\n  }\n\n    else if (apiCall == \"customer_assets\") {\n        apiCall = \"tickets\";\n        jsonData = '{{$node[\"GET apiCall\"].json[\"tickets\"]}}'\n  }\n  \n    else if (apiCall == \"tickets\") {\n        apiCall = \"contacts\";\n        jsonData = '{{$node[\"GET apiCall\"].json[\"contacts\"]}}'\n  }\n\n    else if (apiCall == \"contacts\") {\n        apiCall = \"settings\";\n        jsonData = '{{$node[\"GET apiCall\"].json[\"settings\"]}}'\n  }\n  \n}\n\ncatch(err) {\n    apiCall = \"products\";\n    jsonData = '{{$node[\"GET apiCall\"].json[\"products\"]}}'\n}\n\nconst apiDATA = {\n  apiCall: apiCall,\n  jsonData: jsonData\n};\nreturn apiDATA;"
      },
      "id": "5b0c24ba-973f-4a80-a14f-26f1c45e195a",
      "name": "Set apiCall",
      "type": "n8n-nodes-base.code",
      "typeVersion": 1,
      "position": [
        340,
        500
      ]
    },
    {
      "parameters": {
        "amount": 2,
        "unit": "seconds"
      },
      "id": "4ad1f5c4-f1f6-4837-91ac-a8d1ba6e6dc9",
      "name": "Wait",
      "type": "n8n-nodes-base.wait",
      "typeVersion": 1,
      "position": [
        1460,
        700
      ],
      "webhookId": "2ca89fce-aea0-48eb-8a90-623e44a1d037"
    },
    {
      "parameters": {
        "jsCode": "// Code here will run once per input item.\n// More info and help: https://docs.n8n.io/nodes/n8n-nodes-base.functionItem\n// Tip: You can use luxon for dates and $jmespath for querying JSON structures\n\nconst curPage = parseInt($data.meta.page);\nconst totalPage = parseInt($data.meta.total_pages);\nlet newPage = \"\";\n\nif (curPage == totalPage) {\n  newPage = 1;\n}\nelse {\n  newPage = curPage + 1;\n}\n\nreturn [\n  {\n    json: {\n      curPage,\n      newPage,\n      totalPage\n    }\n  }\n];"
      },
      "id": "6442b76d-a409-419a-bd40-14da0bec117d",
      "name": "Set pages",
      "type": "n8n-nodes-base.code",
      "typeVersion": 1,
      "position": [
        1220,
        760
      ]
    }
  ],
  "connections": {
    "GET apiCall": {
      "main": [
        [
          {
            "node": "Save to Nextcloud",
            "type": "main",
            "index": 0
          },
          {
            "node": "Settings?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check if pagination?": {
      "main": [
        [
          {
            "node": "GET apiCall",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Set apiCall",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Settings?": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Set pages",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cron - Sat 12:00": {
      "main": [
        [
          {
            "node": "Set apiCall",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set apiCall": {
      "main": [
        [
          {
            "node": "GET apiCall",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "Check if pagination?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set pages": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }esql query node after which if query fails to return data, data from a earlier node is not accesible in the “code” node.
        ]
      ]
    }
  }
}

Information on your n8n setup

  • n8n version: 0.209.4
  • Database you’re using (default: SQLite): PostgreSQL
  • Running n8n with the execution process [own(default), main]: Note sure here
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Runs via docker on TrueNAS Scale

Hi @GrandMa, welcome to the community and sorry for the trouble.

Based on your description it very much sounds like you’ve hit an out of memory situation. It might be worth checking your server logs to confirm this.

Seeing your are fetching hundreds of file here you can move some of your workflow logic (such as the file download and upload parts) into a sub-workflow which would then only return a minimal item (such as the next page URL)? This would mean the memory required for each website downloaded becomes available again on the next loop.

I’ve posted an example for such sub-workflow over here. It’s not exactly a copy of your use case (but rather paginates through calendar months), but it hopefully still helps with getting started.

@GrandMa I’ve created this custom n8n image with these changes to log the memory usage after every node is executed, and also to force the garbage-collector to run after every node.

Can you please run your workflows on this image (n8nio/n8n:debug-memory-usage), and share the logs with us?

@MutedJam , thanks makes sense, took a look at your example and makes sense. Will test this out in the coming days.

@netroy I will try to get this up and running, but may be above my technical abilities on TrueNAS scale. Deploying n8n from Truecharts, who have setup the docker container to run on TrueNAS Scale already.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.