Merge Node inside Loop Noode for dig data merge

Hi,

  • n8n version:
    Running version [email protected]
  • Database (default: SQLite):
    default: SQLite (I can change if needed)
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
    default
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
    n8n cloud
  • Operating system:
    windows

I have 2 different sources

  • orders : between 10 and 20 thousand rows
  • hs_codes : 34 thousands row

orders columns

  • NUMBER
  • HS_CODE
  • other columns (10 in total)

hs_codes columns

  • hs_code
  • complementary_unit
  • prohibited_h7

I need to have an output of hs_codes columns added to the orders.

I tried multiple things. For now, I have a Postgre not that makes a select from the hs_codes to get only the ones in the orders.

When I do a simple Merge Node, I have an error because the data is to huge.

When I try to do a loop (I dit with batch of 100 / 200 and now 1000). I have everything demultiplicated (I tried with an 8000 rowas) :

I am not sure of my approach. If you have any idea of fhat could be done better?

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
1 Like

I just updated the required information

I found the appropriate method:

Aggregate before doing the Query to the postgre.
Use a Query with SELECT … IN …

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.