Shopify all products

Hello,

I am trying to get all my products in shopify but if I am saying all products it keeps running no limit.
and I am puting 4000 limit it changes automatically to 250 that is an internal limit for shopify.
Therefore how can I get all the pages one by one ?
thanks
Alex

Hey @Alexandre_Husset,

We will limit the API call a bit but if you use the Return All option it should get all of the products. Can you confirm how many products you have and what version of n8n you are using?

4000 products.
version 0.195.5 on a macbook pro M1

Hey @Alexandre_Husset,

How long do you leave it running for?

Hi! I’m working with @Alexandre_Husset .

We just got this error on the shopify node. How can i slow down the shopify node les than 2 calls per second ? (new to N!N) Thanks!

context": {
},
"name": "NodeApiError",
"cause": {
"message": "429 - {"errors":"Exceeded 2 calls per second for api client. Reduce request rates to resume uninterrupted service."}",

I attach here a screenshot of my node:

Hey @oly-dev,

Welcome to the community :raised_hands:

Have you tried setting the Retry on Fail option on the node settings that might help.

let me try this .

thanks :slight_smile: i’m hoping to become a power user

it didn’t work, i get the same error.
“Exceeded 2 calls per second for api client. Reduce request rates to resume uninterrupted service.”

Isn’t there a way to make the shopify node slow down the requests? i know there is a “wait” node but i don’t know if its relevant in this case? it seems to be useful to slow down the interactions between nodes, but not sure if it works to slow down a node itself …

Hey @oly-dev,

Sadly no way to slow down the node itself, It looks like the requests per second are set based on shopify tier as well so if you had Shopify plus it is 4/second. It looks like maybe what we should do is look for the 429 header and wait briefly before sending the next request. For now it might be a case of using the HTTP Request node to manually call the API and loop through the results that way.

hi @Jon thanks for your help, how can i look for this header? can you give me a brief step by step of wha tnodes i should use?

Hey @oly-dev,

Add an HTTP Request, Set the credentials to use your Shopify ones, Set the URL and parameters to whatever Shopify expects and a set a limit of 250 (The max shopify allows). Next add a wait node and set it to 1 second then loop back to the HTTP Request node to grab the next 250.

You will need to add in some bits for the pagination as needed but the API docs for Shopify can be found here: Product

It looks like they will give you a link to use to get the next batch as well which is handy (Make paginated requests to the REST Admin API). I am just about to finish up for the day but I think this might be enough to get started with, If you don’t get anywhere let me know and I will see if I can free up some time to pop an example together.

thanks so much i’ll try this tomorrow as i’m also finishing here.

What is the order of the different nodes then ? as i understand we have 3 nodes: shopify, HTTPRequest and wait?

OR
are we replacing the shopify node by the HTTPRequest node to ask for the data on shopify? and so it would be just using these 2 nodes: httprequest and wait ?

You would be replacing the shopify node for now with the http request node, you may also need a set node to set the cursor value, an if node to check if there is more pages and a code node to merge all the items.

There are a few examples on here on pagination with APIs that could help.

Hello @Jon !
So i advanced, and now im able to do a HTTP request, get 50 products, dump them on angoogle sheet. On the other brand of the HTTP Request i get the link to do the HTTP GET request for the next 50 products , and do the same process to dum these 50 products ont he google sheet. However the issue i have no is that i don’t know how to iterate on the pagination without everytime having to add again the same nodes , again and again … do you have any insight?

Hi @oly-dev,

That is where you would need the loop, Looking at the Shopify API documentation it will return the next URL to use so you would need to check if that exists and if it does run the HTTP Request node again with the new URL.

There is a recent example of how to do something like this here: Request of help to configure an API call - #12 by Jon

Where the Item Lists node is that is where you would want to start your workflow to work on the products.

Hi @Jon , so as you said i looked at that thread and did the following loop:

The first HTTP request is to get it initiated, then the “IF” checks if there is a link pointing towards the next page with products ( so if there are more products to be requested). If it does then it gets the link of this next page and does an HTTP request with it to get the second page, then again, the IF checks if there is another page, and on one branch goes and dumps the products found on the google sheet, and on the other cleans the reponse header with the link to the next page, to use it for the next HTTP request.

There is probably a simpler way to do it however it works yey! the problem i have now is that i’m constantly running into google sheet api limit to do 50 requests … I already limited the shopify call to 50, to make sure that we have batches of 50 products to dump into google sheet, but after around 4 or 5 loops, it spits out the error again of "The service is receiving too many requests from you! Perhaps take a break"

To try to solve it i’m trying with a WAIT node of 30 seconds … but i wonder if there is a better way to do this ? isn’t 30 seconds too much ? i previously tried with 15 seconds and it still gave me that error.

Thanks for your help!

Hey @oly-dev,

30 seconds is a very long time but if Shopify is limiting your requests the only thing to do is wait as we can’t really do anything about that although you should be falling under their limits. In the workflow screenshot above did it throw the same error?

You are also coming off of the true branch of the if node twice is that intentional?

Hello! so upda:te it has been blocked after 14 iterations of the loop … so didnt finish exporting all my products from shopify :frowning:

Regarding the If branch, yes it is intentional, as i use it to check if the link on the header resopnse contains “next” which would mean that tere is another page, and therefore i do the loop to clean the link and do another request …

i feel like there is probably a better way to do this but not sure

Hey @oly-dev,

There is no way that I can think of without making a custom node or making a copy of our node and adding in the changes.

Are there maybe other limits to the Shopify API with the amount of requests you can make?

1 Like

hello Jon!
want’ed to update you, thanks for your help you helped me make it work. i’m sure it is not the most optimum wy but it is working :slight_smile: here, feel free to lmk if there re any inmediate improvements we can do:

1 Like