Hello everyone, I have an n8n workflow where I’m making multiple HTTP requests to fetch threat intelligence data (IP address attributes). Each HTTP request returns a JSON response with a nested structure. I have a list of IP addresses that was generated using other tool so I can compare to my generated one. Right now I am working with 40k+ Ip addresses and i noticed that my list is missing some IP addresses hat are present in my other list. Going through each HTTP request node I can locate the missing IP, and in the next node (merge node) the same IP is gone. I am using append node in the merge node, and the thing is that I have identical workflow with different sources and much smaller data set, and the problem that I described above is not present.
I have 6 http requests and now I tried to group those requests to separate merge nodes with 3 inputs each and they both get fed into third merge node with two inputs. In the first group of three request, one ip appears three times- once in one node, twice in the second one. After the first merge, all three ips “survive”, but after the second merge node they get dropped.
