How to get new RSS item?

Question

How to get new RSS item?

When i run this code, I’m receive all items. every times.

I want receve new item only.
like this https://ifttt.com/applets/dT7vx4JY-rss-to-gmail

Code

{
  "nodes": [
    {
      "parameters": {
        "url": "http://example.com/rss"
      },
      "name": "RSS Feed Read",
      "type": "n8n-nodes-base.rssFeedRead",
      "typeVersion": 1,
      "position": [
        400,
        450
      ]
    },
    {
      "parameters": {
        "fromEmail": "[email protected]",
        "toEmail": "[email protected]",
        "text": "={{$node[\"RSS Feed Read\"].data[\"title\"]}}\n{{$node[\"RSS Feed Read\"].data[\"link\"]}}",
        "html": "="
      },
      "name": "Mailgun",
      "type": "n8n-nodes-base.mailgun",
      "typeVersion": 1,
      "position": [
        550,
        450
      ],
      "credentials": {
        "mailgunApi": "my_mailgun"
      }
    },
    {
      "parameters": {
        "triggerTimes": {
          "item": [
            {
              "mode": "everyX",
              "value": 10,
              "unit": "minutes"
            }
          ]
        }
      },
      "name": "Cron",
      "type": "n8n-nodes-base.cron",
      "typeVersion": 1,
      "position": [
        250,
        450
      ]
    }
  ],
  "connections": {
    "RSS Feed Read": {
      "main": [
        [
          {
            "node": "Mailgun",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cron": {
      "main": [
        [
          {
            "node": "RSS Feed Read",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Welcome to the community!
Yes, the node simply returns all the items it finds. The filtering out of already processed items has to be done manually. That means you have to save the date of the last processed item and then remove all the ones before that date. It is possible to save that value in multiple ways, the easiest is with the static data. Here an example workflow:

{
  "nodes": [
    {
      "parameters": {
        "url": "https://www.nasa.gov/rss/dyn/breaking_news.rss"
      },
      "name": "RSS Feed Read",
      "type": "n8n-nodes-base.rssFeedRead",
      "typeVersion": 1,
      "position": [
        450,
        450
      ]
    },
    {
      "parameters": {
        "fromEmail": "[email protected]",
        "toEmail": "[email protected]",
        "text": "={{$node[\"Function\"].data[\"emailText\"]}}"
      },
      "name": "Mailgun",
      "type": "n8n-nodes-base.mailgun",
      "typeVersion": 1,
      "position": [
        1250,
        550
      ],
      "credentials": {
        "mailgunApi": ""
      }
    },
    {
      "parameters": {
        "triggerTimes": {
          "item": [
            {}
          ]
        }
      },
      "name": "Cron",
      "type": "n8n-nodes-base.cron",
      "typeVersion": 1,
      "position": [
        250,
        450
      ]
    },
    {
      "parameters": {
        "functionCode": "const staticData = this.getWorkflowStaticData('global');\n\nlatestRead = staticData.latestRead;\n\nfor (let item of items) {\n  item.json.latestRead = latestRead || '1970-01-01';\n}\n\nreturn items;"
      },
      "name": "Latest Read",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        650,
        450
      ]
    },
    {
      "parameters": {
        "functionCode": "const staticData = this.getWorkflowStaticData('global');\n\nif (items.length > 0) {\n  staticData.latestRead = items[0].json.isoDate || staticData.latestRead;\n}\n\n\nreturn items;"
      },
      "name": "Write Latest Read",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1050,
        350
      ]
    },
    {
      "parameters": {
        "conditions": {
          "number": [
            {
              "value1": "={{new Date($node[\"Latest Read\"].data[\"latestRead\"]).getTime()}}",
              "value2": "={{new Date($node[\"RSS Feed Read\"].data[\"isoDate\"]).getTime()}}"
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        850,
        450
      ]
    },
    {
      "parameters": {
        "functionCode": "let emailText = '';\n\nfor (let item of items) {\n  emailText += `${item.json.title}\\n${item.json.link}\\n\\n`;\n}\n\nreturn [{json: {emailText}}];"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1050,
        550
      ]
    }
  ],
  "connections": {
    "RSS Feed Read": {
      "main": [
        [
          {
            "node": "Latest Read",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cron": {
      "main": [
        [
          {
            "node": "RSS Feed Read",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Latest Read": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Function",
            "type": "main",
            "index": 0
          },
          {
            "node": "Write Latest Read",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function": {
      "main": [
        [
          {
            "node": "Mailgun",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

It is, however, important to remember that the static data does not get read and written when testing via the Editor-UI. So every time you test it will use the in the node defined default date and will to always return all items. If you then, however, activate the workflow it will work like intended. You can test that easily by setting the cron-time to something short like 1 minute. You will then see that the first time it runs it will send an email with all items and the second run, it will not send an email at all (unless by accident a new item gets added in the meantime).

2 Likes

Btw. here the documentation about the static data:
https://docs.n8n.io/#/nodes?id=method-getworkflowstaticdatatype

2 Likes

I have done something similar using files to track sitemap changes - but it really gets very verbose (see below). I think this needs a dedicated node, @jan. Comparing previous and current execution is just a too common problem.

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        270,
        280
      ]
    },
    {
      "parameters": {
        "functionCode": "const urls_new = items[0].json.urls;\nconst urls_old = items[1].json.urls;\nconst difference = urls_new.filter(x => !urls_old.includes(x));\n\nreturn [{ json: {\n  \"old\": urls_old,\n  \"new\": urls_new,\n  \"diff\": difference\n} }];"
      },
      "name": "Diff",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1150,
        -150
      ]
    },
    {
      "parameters": {
        "url": "https://torstencurdt.com/tech/sitemap.xml",
        "responseFormat": "string",
        "options": {}
      },
      "name": "Get Sitemap",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        490,
        -150
      ]
    },
    {
      "parameters": {
        "functionCode": "const urls = [];\n\nfor (const item of items) {\n  for (const url of item.json.urlset.url) {\n    urls.push(url.loc);\n  }\n}\n\nreturn [ { json: { urls: urls }} ];\n"
      },
      "name": "Extract URLs",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        790,
        -150
      ]
    },
    {
      "parameters": {},
      "name": "Old and New",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        970,
        -40
      ]
    },
    {
      "parameters": {
        "filePath": "old.json"
      },
      "name": "Read Old",
      "type": "n8n-nodes-base.readBinaryFile",
      "typeVersion": 1,
      "position": [
        490,
        30
      ],
      "continueOnFail": true
    },
    {
      "parameters": {
        "fileName": "old.json"
      },
      "name": "Write Old",
      "type": "n8n-nodes-base.writeBinaryFile",
      "typeVersion": 1,
      "position": [
        1470,
        30
      ]
    },
    {
      "parameters": {
        "functionCode": "return [ items[0] ];"
      },
      "name": "Select New",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1150,
        30
      ]
    },
    {
      "parameters": {
        "chatId": "-3111",
        "text": "={{$node[\"Get Sitemap\"].parameter[\"url\"]}} has {{$node[\"Diff\"].data[\"diff\"].length}} new entries",
        "additionalFields": {}
      },
      "name": "New Entry",
      "type": "n8n-nodes-base.telegram",
      "typeVersion": 1,
      "position": [
        1470,
        -170
      ],
      "credentials": {
        "telegramApi": "some_bot"
      }
    },
    {
      "parameters": {
        "options": {}
      },
      "name": "XML to JSON",
      "type": "n8n-nodes-base.xml",
      "typeVersion": 1,
      "position": [
        640,
        -150
      ]
    },
    {
      "parameters": {
        "options": {
          "keepSource": false
        }
      },
      "name": "Data to JSON",
      "type": "n8n-nodes-base.moveBinaryData",
      "typeVersion": 1,
      "position": [
        640,
        30
      ]
    },
    {
      "parameters": {
        "mode": "jsonToBinary",
        "options": {}
      },
      "name": "JSON to Data",
      "type": "n8n-nodes-base.moveBinaryData",
      "typeVersion": 1,
      "position": [
        1310,
        30
      ]
    },
    {
      "parameters": {
        "triggerTimes": {
          "item": [
            {}
          ]
        }
      },
      "name": "Cron",
      "type": "n8n-nodes-base.cron",
      "typeVersion": 1,
      "position": [
        280,
        -150
      ]
    },
    {
      "parameters": {
        "conditions": {
          "number": [
            {
              "value1": "={{$node[\"Diff\"].data[\"diff\"].length}}",
              "operation": "larger"
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1310,
        -150
      ]
    },
    {
      "parameters": {
        "fileName": "old.json"
      },
      "name": "Clear Old",
      "type": "n8n-nodes-base.writeBinaryFile",
      "typeVersion": 1,
      "position": [
        810,
        280
      ]
    },
    {
      "parameters": {
        "functionCode": "items[0].json.urls = [];\nreturn items;"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        480,
        280
      ]
    },
    {
      "parameters": {
        "mode": "jsonToBinary",
        "options": {}
      },
      "name": "JSON to Data1",
      "type": "n8n-nodes-base.moveBinaryData",
      "typeVersion": 1,
      "position": [
        650,
        280
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Get Sitemap",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Diff": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get Sitemap": {
      "main": [
        [
          {
            "node": "XML to JSON",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract URLs": {
      "main": [
        [
          {
            "node": "Old and New",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Old and New": {
      "main": [
        [
          {
            "node": "Diff",
            "type": "main",
            "index": 0
          },
          {
            "node": "Select New",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Read Old": {
      "main": [
        [
          {
            "node": "Data to JSON",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Select New": {
      "main": [
        [
          {
            "node": "JSON to Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "XML to JSON": {
      "main": [
        [
          {
            "node": "Extract URLs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Data to JSON": {
      "main": [
        [
          {
            "node": "Old and New",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "JSON to Data": {
      "main": [
        [
          {
            "node": "Write Old",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cron": {
      "main": [
        [
          {
            "node": "Get Sitemap",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "New Entry",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function": {
      "main": [
        [
          {
            "node": "JSON to Data1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "JSON to Data1": {
      "main": [
        [
          {
            "node": "Clear Old",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
2 Likes

Yes agree that a node is needed to make that simpler in the future.

3 Likes

Thanks for sharing the example. I’m now learning some Java to understand it better. However, correct me if I’m wrong but I believe that doesn’t account for the exact time, only the date so if I use a 5 minute interval it will post again whatever RSS item was created on that day. Any suggestions?

Hey I’m running into the same issue, do you know how to solve it or does anyone know how to solve this?

I need the static data to be saved minute by minute, not day by day

The static data does get saved with the workflow after every run. So minute by minute is not a problem at all (unless the workflow execution takes longer than one minute).

The problem i have is that the minium date format is DAY, so it can’t difference between two rows created in the same hour, or minute

Where is the minimum date format day?

The above is also just an example. You can do literally everything you want. If you want you can hash the whole item and then save that.

1 Like

Sorry about my misunderstanding, but I literally don’t know how to do anything so I thought that what was done was the only option.

Where can I refer to learn how to code the “Function” node? I am really interested in using it correctly

Ah OK, in this case you can find the documentation for the Function-Node here:
https://docs.n8n.io/nodes/n8n-nodes-base.function/#node-reference

1 Like

What should I learn for a better understanding of things that are not explained in the documentation of n8n? which language does the function node use?

And sorry if those are very basic questions, I’m not trolling or anything I’m just new in all of this.

Sorry do not know how I can answer the first part of the question. By now the most important things should already be in the documentation. It is for sure not perfect and still wip but it is getting better every day. Apart from that is this forum for sure a very good resource as it contains a lot of questions and their answers already.

The language used in the Function-Node is mentioned in the documentation. It is JavaScript.

1 Like