Question about splitting in batches

GM

I had a questions about splitting in batches.

I have a whole workflow after this node, and generally I don’t think this workflow will be often used at the same time, however it could be a possibility of course.

My workflow is that data gets split up in batches → IF node → further workflow (let’s call it workflow ‘B’)
Now, just after the IF node, it will return to the split in batches node.

What happens if workflow B gets executed, all while another line of data is being put through the batch and the if node, which will also get into workflow B. In short: can workflow B handle two ‘runs’ at the same time?

Hope it is clear! Thanks!

So as far as I know n8n run one node at the time, so it will run as numbers say on my screenshot

Split will split the output and send lets say 1 wait till finished (wait for 4 end) and if back them send another one.

See my other WF what use it. It runs 1-3, 4 push one item run 5 then 6 make decision either back to 4 or go to 7-8


Code:

Let me know if that helps you :))

1 Like

thanks @Shirobachi ! So imagine you would have the flow above (1 to 8), but instead of a line going from 8 to 4, you have a line going from 6 back to batches (4).
I guess yours is best practice, right? but what would happen with what I described?

I think also the reason why I am thinking of doing it this way, is because I am scared of getting an error, and in that case it would just stop.

So I am not sure if I understand you ;/ Do you mean do not have like 8->4, only 6->4 (https://i.imgur.com/ZPubOrU.png)?

If yes, that would stop workflow, ones hit 8th node (actually when 8th finish job) so some items from split could be lost.

Example of an error:
image
It just stops processing. In this case the batchnode won’t get triggered anymore.
The reason here I am getting an error, is because of a timeout of 10s (yes, annoying), but it also does not give me the option to do a next step if there is an error.

thanks for your reply, that does make sense that it stops. In that case, perhaps you have a solution to trigger the next step if there is an error with a specific node (I saw there is an errornode trigger, but that would mean it triggers with any error, which would not be very helpful)
Thanks!

Be aware that HTTP node has option timeout, so that might help you:


Also, you can set in node options to continue on error that would make that only that batch of items what make error would possibly do not run correctly, others would work completely fine (unless error ofc).
I would just suggest put right next some IF node what will check if error accrue and if do nothing (idk what you’re building but let’s say adding sth to DB, you probably do not want to add item with id undefined it could potentially break your other systems)

About error node, it’s trigger node what run completely separately, so it would be actually different to execution. That only for notifying you that error accrue, not handling it.
Pro-tip: make WF with error trigger node add, i.e. discord node, so it will let you know that error accrue. Go to settings and set this WF as error WF:


Now this WF will run if in any other WF error will happen

Good luck :crossed_fingers:

3 Likes

amazing answer - full of helpful material, thanks so much!!

2 Likes