Mysql pagination help needed

I am currently creating a workflow where I am to read data from one MySQL database and then insert it into another MySQL database. I have reused a couple of things from this workflow
The issue I am facing is that the input DB is rather big and I, therefore, can’t load all the data at once as the JSON file becomes too big.
What I, therefore, need to do is to have a where clause that looks something like this:

WHERE id BETWEEN [variable_number] AND [variable_number2]

[variable_number] would then be 1, to begin with and change to the last id from the previous input and the [variable_number2] would be [variable_number] *1000 for example.

Does anyone know how I could accomplish this? I am not that familiar with js to some parts of the function nodes that are troublesome for me.

As n8n keeps always all data in a workflow execution accessible, even if you would split it up, it would still get too large. So you would have to make sure that you run each “batch” in a totally separate workflow-execution.

What you would need is two workflows. One that does the actual copying of the data and expects some kind of input like “start” & “end” and a second workflow which calls the first one. That can be done with either the Execute Workflow nodes or a Webhook/HTTP-Request combination.

You can find additional information here, where another user had a very similar problem:

I hope that helps. If not simply get back to me.