Solution needed to run the SQL node in batches

Questions
1

Hi Team,

I have a use case where the sql select query has returned me a total of 89 input fields and I want to process these 89 fields in a batch of 10, each batch should reach till slack node (last node) and again it should return to second batch and again should reach to slack node so in total this should be running for 9 times in a loop, until all the output fields are run which are returned by slack and once the batches are executed it should stop execution

Tried the split batch node but this is not working as expected, any suggestions are appreciated, attaching the screen shot of my code as reference

Screenshot 2024-05-24 at 5.45.03 PM
Screenshot 2024-05-24 at 5.45.03 PM1251×361 43.1 KB

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3
  • n8n version: 0.231.3
  • Database (default: SQLite): mysql
  • n8n EXECUTIONS_PROCESS setting (default: own, main): own
  • Running n8n via (Docker, npm, n8n cloud, desktop app): docker
  • Operating system: MAC
4

hello @And112
Actually, Split in Batches (Split node) should work fine. Can you share the workflow and some example output of the SQL node?

btw I strongly suggest updating your n8n instance, as you are using too old version (that was released about 2 years ago)

6

Hi @barn4k

We have installed the new version of N8N and we are still facing the same error

This is the overall scenario

  1. Http → Fetches the user information

  2. SQL Node → Has an output of 80 fields followed by ID and channel name

  3. Http node → Invite a user to slack channel

  4. If node → If user is already present send a message to user that, you’re already present

  5. If Not present send message to user that you’re added to slack channel

Limitation here we want to add the user in a batch of 10 and sent message to user, wait for a minute and then process the second batch sent message to slack and the loop should end after all the nodes are processed

If we written the below flow but it is only processing the first batch and performing the operation for second batch

SQL Node Output for reference:

ID. Name
abc123 channelname

N8N Verison : 1.43.0

image
image930×459 26.8 KB

7

Hi @bartv

Can you please help us in resolving the above scenario

8

Hmm, using loops within loops sometimes has unexpected side effects. Can I ask why you need to post in batches of 10 and then wait 1 minute before processing the next one - can’t you just use a ‘loop over items’ with a batch size of 1 and add a wait between each operation?

9

We have a limitation of adding user to slack channels as we can add a user to 50 slack channels within a minute in order to be more specific and cautious we want to add a user to 10 slack channels per minute

As per your suggestion, we have kept only 1 loop over items and tried the execution and this works only for the 1st batch of items and then it re-runs on the first batch of items only and it doesn’t go to second batch of items

10

Can you share the workflow?

11

I do have limitations at my end and cannot share the workflow

13

I’m confused about what you’re doing exactly - these loops don’t seem to describe the functionality you require.

14

Hi @bartv

ohh… can you please suggest us what would be the best possible solution here, in order to make our worflow to work