Large amout of data to a Code node

Questions
,
1

hey all ,
im tring to pass 10 mb of data no a JS code node , but it keeps running for too long then stops with no output , even replacing the code with a simple code it doesn’t give me any output or console.log

image
image3413×1742 384 KB

2

@0xSphinx your browser console is the tell here, those [Violation] setTimeout warnings mean the editor tab is choking trying to render 10.5mb in the input/output panels, not your code actually failing. does it still fail when the workflow runs from the trigger in production, or only when you hit Test step in the editor?

3

hey @achamm its always you , the saver ,

when it runs from the production execution , i get “the green tick”, the console.log but without getting output from the node,

even trying with much lower input , and the code you gave me from my last question , i still get no output but get my console.log normaly

image
image1279×952 107 KB

4

@0xSphinx the regex is the bug. youre matching https?:// but look at your input, its subdomain,ip lines with no http:// anywhere in them, so matchAll returns zero matches, hosts comes back empty, and an empty array out is exactly n8ns “no output data”. the console.logs before the return still fire, thats why you see logs but nothing downstream.

grab the domain sitting before each comma instead:

const text = $input.first().json.data;
const hosts = [...text.matchAll(/([a-z0-9.-]+\.[a-z]{2,})\s*,/gi)].map(m => m[1]);
return [...new Set(hosts)].map(h => ({ json: { subdomain: h } }));

that pulls api.hackerone.com, docs.hackerone.com etc, skips the ips since they end in numbers, dedups, one item per subdomain. dont bother with Always Output Data, that just emits an empty item, it wont give you the actual subdomains.

5

Hi @0xSphinx

Change your return statement to this

const uniqueHosts = [...new Set(hosts)];
return [{ 
  json: { 
    subdomains: uniqueHosts, 
    count: uniqueHosts.length 
  } 
}];

Does that help?