N8N workflow keeps crashing and then restarting. Log says “FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory”. I have also increased the memory from 4GB to 8GB but no luck. Recent changes include addition of simple code block to mimic switch node behaviour. Screenshots attached. This behaviour is only for workflows running longer like 5 mins or more.
What is the error message (if any)?
Error Message in the Docker Logs: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
No error message in the workflow execution.
Please share your workflow
(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)
Can’t share workflow because of characters limit here.
Thank you @barn4k ! But if I move it to a different sub workflow, it will take same amount of memory there and eventually causing same issue isnt it? I want to quanitify this, I want to see exactly which node taking how much memory and how I can optimise it. Taking almost 8 gb of memory is not efficient just a for 5-10 mins of text only data. How can I enable detailed logging to see exact steps and thier memory utilization?
Also looks like it is not even utilizing full memory, somewhere around 2.2 GB it crashes while I have allocated 7.5 GB. Is there a default setting that control how much memory it will use? And can we do something give proper warning and stop at memory full instead to just crash? Please see recordings and screenshots.
ok, but can you answer why it is taking so much memory and even if I have like 7.5 gb, why it is crashing at 2 gb? And where and how can I see detailed logging on this?
Thank you @ria, I have given more memory to as you suggested. The problem of workflow crashing at 2 GB is now resolved as I have increased the size to 6 gb, however it keeps piling on the data, for few hours of uses it has already accumulated 4.5 gb, how can i clear this and what kind of data it holds? All the workflow and execution data I belive is stored in the database, then what is this data and how can I clear it on periodic basis?
to clear the data you’ll need to use the sub-workflow. n8n keeps the memory until the execution is not finished and as you have a wait node there that can be used infinite times, the memory won’t be released at all.
Actually, you should move only these nodes into the sub-workflow
In that case your parent workflow will hold only the output of the Code node.
It is also good practice to set a hard limit for the workflow to work. E.g. no more than 4 hours. You can set it in the workflow settings.
Thank you @barn4k, I am new to sub-workflows specially on loops, do you have any suggestions or tutorial on how to do this? and what are best practices.
