HELP: Accessing data across multiple nodes?

[weooo99]

I have set up a process to store search results from Google into a database. However, everything I’ve tried so far hasn’t worked because I can’t access data from previous nodes. Here’s a brief description of the problem:

1. Node “HTTP REQUEST”
   I call an API and receive a JSON response.

2. Node “Save Search”
   I save some values in the database. I receive the ID of the entry as a result.

3. Node “Split Out”
   Here, I want to split the values for “organic_results” from the JSON of the node “HTTP REQUEST” so that I can store each one individually in the database. However, I can’t access it because there is an error message:

4. Node “Save Results to DB”
   Finally, in this node, the split results should be saved in the database, and when saving, the value from the node “Save Search” containing the ID of the entry should be saved with each entry. However, I can’t access it, and it doesn’t execute.

I’m at a loss here. What am I doing wrong? Can someone please help me?

The Workflow:

[n8n]

It looks like your topic is missing some important information. Could you provide the following if applicable.

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[bartv]

Hey @weooo99, welcome to the community!

Note that the field ‘Fields to Split Out’ expects just the name of the field, you’re trying to pass it a value now. So try entering organic_results as text.

Hope that helps!

[weooo99]

Hey bartv,
thanks for your help. I am new to n8n and still didn’t figure out how some things work. I have tried that allready. This is the result:

I don’t recieve the data.

This is what my workflow looks like:

Thanks for trying to help me

[bartv]

Can you show me that the output of the first MySQL node looks like? Because that’s what this node will try to split out.

[weooo99]

Sure this is the JSON:

[
  {
    "sql": "INSERT   INTO `google_search` (`success`, `credits_used_this_request`, `time_period`, `location`, `google_domain`, `gl`, `hl`, `output`, `num`, `q`, `engine`, `total_time_taken`, `engine_url`, `html_url`, `json_url`, `location_auto_message`, `total_results`, `time_taken_displayed`) VALUES (true,1,'last_year','Germany','google.de','de','de','json','10','\\\"ruhrallee 9\\\" + dortmund + impressum','google',7.53,'https://www.google.de/search?q=%22ruhrallee+9%22+%2B+dortmund+%2B+impressum&gl=de&hl=de&uule=w+CAIQICIHR2VybWFueQ&num=10&tbs=qdr%3Ay','https://api.valueserp.com/search?time_period=last_year&location=Germany&google_domain=google.de&gl=de&hl=de&output=html&num=10&q=%22ruhrallee+9%22+%2B+dortmund+%2B+impressum&api_key=6159A6C92A0E427CA2C3A68F8E6616DD&engine=google','https://api.valueserp.com/search?time_period=last_year&location=Germany&google_domain=google.de&gl=de&hl=de&output=json&num=10&q=%22ruhrallee+9%22+%2B+dortmund+%2B+impressum&api_key=6159A6C92A0E427CA2C3A68F8E6616DD&engine=google','We have automatically set the \\'google_domain\\', \\'gl\\' and \\'hl\\' parameters to match the supplied location of \\'Germany\\'. You can stop this behaviour by setting the \\'location_auto\\' parameter to \\'false\\'.',1080,0.25)",
    "data": {
      "fieldCount": 0,
      "affectedRows": 1,
      "insertId": 7,
      "info": "",
      "serverStatus": 2,
      "warningStatus": 0
    }
  }
]

[bartv]

I think that’s the problem - this node does not output the organic_results field. What if you attach the Split Out node directly to the HTTP request node?

[weooo99]

Hey,
Yes i could attach it directly to the HTTP Request, but than it would run the Save Search Node for each Item and I need it to run only once!
And I need to use from the result the value for “insertId” (from the first sql-result) in the second sql to save for each entry additionaly to the splited data values the “insertId”.

Do you understand what I mean? To me it seems that I can’t use values in nodes that are more than one Step back.

[bartv]

I do! You can nodes in parallel though. In the (untested) example below, the first MySQL node will only be executed once, but you can use its output in the second MySQL node, which will be executed once for each split out item:

[weooo99]

Hi,
I have tested it. And its still not working. This is what my process looks now:

There seems to be a problem I don’t understand in the second SQL Node:

Any Idea?

[barn4k]

hello @weooo99

Try this one

[weooo99]

Hello barn4k,
Thanks that worked. I made all changes as you proposed, and its working. Could you please explain me why it is nessesary to add this step? I mean the data was there before to, why is it nessesary do merge all needed data in a single array before?
It would help for future solutions to know why it is that way.

Thanks again for helping.

[barn4k]

There are some key concepts here of how the n8n works.

First - if the output is an object, then it will be processed as one item (one execution per node). If the output is an array - then it will be processed for every item of that array (N executions per node, for each item of N items).

Next, it’s your design. You have an HTTP node that outputs all data as one item. Then you need to save that data into the DB and receive the insertionID for the whole results. That part was quite easy. But after that, you want to save each organic_result of the HTTP node as a separate entry in the DB. To do that, n8n must process items one by one. So how to tell it to do so? You need to output them as an array of elements (see my first paragraph). As your output of the HTTP node is not the array but an object with a nested array, you need to move that array to the top level.

For that, you have the Split Out node. However, it does not work with the absolute references (it cannot access data that is outputted somewhere in the previous nodes. It only accesses the output of the last node. So to force the proper split, we should provide the results of the HTTP node once more. That’s why we have the Edit Fields node. It acts as a proxy. Plus it has one useful ability to add the output of the previous node to the results (the same is for the Split Out node).

Actually, you can omit the insertionID field and not pull it to the last node and use the absolute reference $('nodename').last().json instead.