Recursive Json through REST API

Questions
1

It is my first approach with n8n, so if I’m asking something obvious excuse me. I was able with basic auth to access to a json table that it is only an index. By accessing to this url I have a list of customers from my ERP:

https://myurl/api/QU0gU3ZpbHVwcGkgSW1tb2JpbGlhcmk/ec37c11e-2b67-49c6-8a58-6eccb7dd75ee/index.json

I get this list

image

which is basically a list of links to the single JSON of each client.

For example the first element is accessible through https://myurl/api/QU0gU3ZpbHVwcGkgSW1tb2JpbGlhcmk/00448c03-eefd-435b-8078-edc442bad296.json

So I think that I have to create something “recursive”: I should generate a complete link from the list in the format I have shown and then open all the generated links and populate a table with the data coming from the links.

Can someone help me?

2

Hi, I think you could use the function Split in batches to do this. It depends if you need to combine the results of each call into a single table or multiple but you could certainly push this to spread sheets (local / google / airtable). It really depends what you want to do with the data in the end.

Cheers

3

I need to combine all of them in an unique table to update a MySQL table

4

As I mentioned before, you need to map that array to an array with the n8n structure, then use the HTTP node with a reference to the ids. Check the example below:

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "url": "https://mockup-pj5l0yxsjrbr.runkit.sh/",
        "options": {}
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        590,
        300
      ],
      "notesInFlow": true,
      "notes": "Mockup data"
    },
    {
      "parameters": {
        "functionCode": "const ids = items[0].json\n\n\nreturn ids.map(id => ({ json: { id } }))\n\n"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        830,
        300
      ],
      "notesInFlow": true,
      "notes": "Map array"
    },
    {
      "parameters": {
        "url": "=https://myurl/api/{{$node[\"Function\"].json[\"id\"]}}.json",
        "options": {}
      },
      "name": "HTTP Request1",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        1080,
        300
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request": {
      "main": [
        [
          {
            "node": "Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function": {
      "main": [
        [
          {
            "node": "HTTP Request1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
5

Thanks… I’m starting to understand how it works. I get this error in the last HTTP Request:

image
image1016×700 22 KB

6

Make sure the API is running in https://192.168.1.77:8080. If you test it from the web explorer, does it work?

7

Yes it does. And it works in the first http request of the workflow

8

That is quite weird. Is the first HTTP request using the same base url?

9

Yes it does. Also, if I use a static url of only one of the json file in the list I gat the data correctly.

10

is it possible that one of the ids that the first HTTP request returns is not valid and that is causing the error?

11

I will have to check. Is there a way to ignore the errors?

12

Set the continue on fail to true.

image
image678×940 39.7 KB

13

Thanks. It seems that it can get only few lines. Tomorrow I’ll check to understand the reason.