Extremely slow scenario

Hi! I have been using n8n for a while now, but I have a problem I am not capable to solve in any way. I have a scenario that goes extremely slow (I am talking 133 mins for 15 records slow) and I can’t work. I am really thinking about changing automation software because of this… It is not the biggest scenario I have (every 4 hours takes 50 records from Airtable, it is a really low number if you ask me…), nor the one that uses most nodes or writes most records in Airtable but it is an important workflow and I need it to work. There is not error message. I am running from Hetzner and the version is 1.50.1

It doesn’t let me upload the code (there is a character limitation) and it doesn’t let me upload it as a json file either… I do’t know what to do.

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

It looks like the sheets and airtable bubbles have problems but they don’t. They work, I don’t know why they appear like this.

hello @uncovercity

if the workflow is too big to upload, you can share it partially by copying part of the nodes per one post.
You can also export the workflow and place it somewhere like google docs and share it that way

Generally, it shouldn’t be so big and it’s a good practice to split the workflow into set of smaller ones

Hello, @barn4k, this is the link to the google drive json: PRUEBA_PRICE_RANGE.json - Google Drive

I don’t think is too big, and I have way bigger scenarios that work perfectly… There are things you cannot split because then it doesn’t work either (there are some issues with webhooks and such).

Thank you very much!!

That part is incorrect. They do not have a proper ending which may lead to the unpredictable behavior

That part is better to move into a sub-workflow to reduce the memory costs. Plus you can use the option “Do not wait for sub workflow completion” in the Execute Workflow node (mode: run once for each item) to update Airtable rows simultaneously

1 Like

Thank you, @barn4k ! I am going to move that last part to a subworkflow, as you say. But I don’t understand the part where you say the first three routes do not have a proper ending (the airtable record gets updated and the workflow moves on, no?). What would be a proper ending or how do i give it a proper ending?

Again, thank you so much for your time and your response.

It depends on what are trying to achieve. Are you planning to update them in the Loop as well?

@barn4k What I want is for the record to get updated on those three fields (price range, servicio, ambiente, comida y prices for type of food) and, when that is done in all those fields, then I want for the loop to proceed with the next record, and then with the next one and so on until it has no more items to pass.

Better to offload all parts to the sub workflows, so they could be executed simultaneously (if there is no interference in their fields). You can also play with the “Wait for the sub-workflow to complete” option in each to get the optimal time/performance and do not overload the Airtable (or just leave that option enabled in one of the nodes)

Best approach is to update the Airtable only once per Loop, but that may be tricky

2 Likes

Now I have it like this (Prueba_subworkflow_price_range.json - Google Drive) @barn4k, but the problem remains… I think I am not understanding then what should I move to other workflows… I tried putting each “route” of the loop in a separate workflow and send to each of them the same dataset via http request (which contains the data they need) but the time of execution was still the same…

Can you provide an example of the HTTP node output? So, that I could be able to test it

Yes, of course, this is a real example, @barn4k: dataset_reviews-task_2024-08-01_12-00-08-810.json - Google Drive

That’s how your master workflow will look:

1 Like

And how the sub-workflow will look:

2 Likes

Thank you so much for this response, @barn4k, I have been working with it for hours now. But now the problem is that, from the 20 items that arrive to the merge node, only 1 gets output and, because of that, only 1 record gets updated in airtable (instead of 20). I am trying to solve that but my fear is that, if I have to use a loop in this workflow as well, it would be as slow as my previous workflow…

The design was to work with 1 item at a time, that’s why I’ve set the Execute Workflow node to run for each item in the parent workflow. It will act as a Loop
image

You can also disable the option Wait for Sub-Workflow Completion to speed up the process, but you’ll need to test it. I don’t know the Airtable limitations

1 Like

Got it! Thank you so much for your help, I think I got it from here!

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.