Shopify big catalogue download to n8n

Hey there!

So has anyone came up to a solution of getting big Shopify catalogues to n8n (we have about 2500 sku’s, so i’m trying various pagination models, but without any success at the moment)?

Would appreciate the input…

P

Can you share your existing workflow we can see if there is a way to improve it

Here’s what i have at the moment.

Did you try using the pagination feature in the HTTP Request node?

There’s a built-in option that allows you to configure pagination directly.

You can find more details in the official documentation:

1 Like

What is the current issue?

  • Is it not working at all?
  • Is it not fetching all pages?
  • Is it too slow in completing?

Hey Wouter!

The issue atm is that Fetch Products node gets an error, when it receives a second request after first iteration of the loop is complete. And the error is “Problem in the Fetch Products node: The service is receiving too many requests from you”. So i figured from the beginning, that the requests happen too fast, hence the two Wait nodes, but they do not help. Can it be, that after the first loop is complete, Fetch Products node receives all the data that is being collected through the nodes, hence thew error? Should i filter out excess data?

Im only noticing now you’re not using a Loop node. This will avoid the http node being called multiple times. because I think on your second iteration, it’s hitting the same http call for each of the results that you returned, causing the rate limit