How do you guys deal with this situation?

How do you guys deal with this situation?

There is no ability to “match” one list to another… which causes the 2 lists to be “out of alignment”.

@RedPacketSec I had a similar situation with one of the flows I was working on.
I added a unique identifier number/name for each record so no matter how the data was manipulated I can merge and match the data based on the unique identifier that was never changed.

in your case, it seems like you are trying to remove an item you already have before you merge the items together…

You might want to just add a data point that says the image already exists then filter it out after merge…
Anyway, in my eyes based on the photo, you provided there are a bunch of ways to solve this.

Would be happy to help in any way I can.

2 Likes

Hi @RedPacketSec, my suggestion would be to use a common key for both branches. If you don’t have one yet, you could add it using a Set node before the Item Lists2 node from your screenshot. An expression like {{(+new Date).toString(36).slice(-5) + Math.random().toString(36).substr(2, 5)}} could add a random key to each item.

Using this key, you could then use the Merge By Key operation of the Merge node.

Example Workflow
{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        240,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "let example = [];\n\nfor (let step = 0; step < 10; step++) {\n  example.push({\n    json: {\n      message: step\n    }\n  })\n\n}\n\nreturn example;"
      },
      "name": "Generate Example Data",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        460,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "unique_id",
              "value": "={{(+new Date).toString(36).slice(-5) + Math.random().toString(36).substr(2, 5)}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Add Key",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        680,
        300
      ]
    },
    {
      "parameters": {
        "operation": "sort",
        "sortFieldsUi": {
          "sortField": [
            {
              "fieldName": "message"
            }
          ]
        },
        "options": {}
      },
      "name": "Sort",
      "type": "n8n-nodes-base.itemLists",
      "typeVersion": 1,
      "position": [
        900,
        300
      ]
    },
    {
      "parameters": {
        "operation": "sort",
        "type": "random"
      },
      "name": "Randomize",
      "type": "n8n-nodes-base.itemLists",
      "typeVersion": 1,
      "position": [
        1340,
        120
      ]
    },
    {
      "parameters": {
        "operation": "limit",
        "maxItems": 8
      },
      "name": "Remove 2 Items",
      "type": "n8n-nodes-base.itemLists",
      "typeVersion": 1,
      "position": [
        1560,
        120
      ]
    },
    {
      "parameters": {
        "keys": {
          "key": [
            {
              "currentKey": "message",
              "newKey": "message_a"
            }
          ]
        }
      },
      "name": "Rename Keys",
      "type": "n8n-nodes-base.renameKeys",
      "typeVersion": 1,
      "position": [
        1120,
        120
      ]
    },
    {
      "parameters": {
        "mode": "mergeByKey",
        "propertyName1": "unique_id",
        "propertyName2": "unique_id"
      },
      "name": "Merge By Key",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        1800,
        300
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Generate Example Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Example Data": {
      "main": [
        [
          {
            "node": "Add Key",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Add Key": {
      "main": [
        [
          {
            "node": "Sort",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sort": {
      "main": [
        [
          {
            "node": "Rename Keys",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge By Key",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Randomize": {
      "main": [
        [
          {
            "node": "Remove 2 Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Remove 2 Items": {
      "main": [
        [
          {
            "node": "Merge By Key",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Rename Keys": {
      "main": [
        [
          {
            "node": "Randomize",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Edit: Nevermind, @David_Go beat me to it :smiley:

2 Likes

interesting i will have a play around and see how that works out, thanks for the ideas.

1 Like

I am just trying to give back when I can :slight_smile:
Still, I really like that your expiration came with an example workflow! I know all the examples people like you share really help me and others get a much better idea of how to get things done in the most optimal way.

Like the this " expression like {{(+new Date).toString(36).slice(-5) + Math.random().toString(36).substr(2, 5)}} could add a random key to each item." That is a really good idea I think I might use in my flow since I have been doing this in a less automated fashion … so I guess you now helped me save some time and effort on my own flows :slight_smile:

@MutedJam Thanks again, for opening my mind to how n8n can help me save time and make a reliable scalable process.

3 Likes

fyi, just to be complete. There is also the variable $position which returns the index of the item, which could also be used here.

3 Likes

interesting thanks.

I’m going to try first with the unique id, as I think that might work well

This would result in the simplest way to reference data from a previous nodes. I’m pretty sure @RicardoE105 had a great example, but I can’t seem to dig it out. Anyway here’s an example flow simply using the $position variable:

Workflow JSON
{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        -280,
        220
      ]
    },
    {
      "parameters": {
        "functionCode": "item1  = {json: {sound: 'meow'}};\nitem2  = {json: {sound: 'woof'}};\nitem3  = {json: {sound: 'moo'}};\nitem4  = {json: {sound: 'oink'}};\n\nreturn [\n  item1, item2, item3, item4\n];\n"
      },
      "name": "Sounds",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        -60,
        320
      ]
    },
    {
      "parameters": {
        "functionCode": "item1  = {json: {name: 'Cat'}};\nitem2  = {json: {name: 'Dog'}};\nitem3  = {json: {name: 'Cow'}};\nitem4  = {json: {name: 'Pig'}};\n\nreturn [\n  item1, item2, item3, item4\n];\n"
      },
      "name": "Names",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        -60,
        120
      ]
    },
    {
      "parameters": {
        "functionCode": "outputData = [];\nfor (item of items) {\n  if (Math.random() > 0.5) {\n    outputData.push(item);\n  }\n}\nreturn outputData;"
      },
      "name": "Random Dropout",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        320,
        320
      ]
    },
    {
      "parameters": {
        "mode": "mergeByKey",
        "propertyName1": "_position",
        "propertyName2": "_position"
      },
      "name": "Merge",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        560,
        200
      ]
    },
    {
      "parameters": {
        "values": {
          "number": [
            {
              "name": "_position",
              "value": "={{$position}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Add Name Position",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        140,
        120
      ]
    },
    {
      "parameters": {
        "values": {
          "number": [
            {
              "name": "_position",
              "value": "={{$position}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Add Sound Position",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        140,
        320
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Sounds",
            "type": "main",
            "index": 0
          },
          {
            "node": "Names",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Sounds": {
      "main": [
        [
          {
            "node": "Add Sound Position",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Names": {
      "main": [
        [
          {
            "node": "Add Name Position",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Random Dropout": {
      "main": [
        [
          {
            "node": "Merge",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Add Name Position": {
      "main": [
        [
          {
            "node": "Merge",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Add Sound Position": {
      "main": [
        [
          {
            "node": "Random Dropout",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
3 Likes

well with a few extra merges here and there I think that using the unique code generator and the field merge has worked.

3 Likes