I wanted a system to monitor website content changes and notify me. So I made it using n8n.
Especially my competitor blogs. I wanted to know how often they are posting new articles. (I used their sitemap.xml file) (The below workflow may vary)
In the Below example, I used HackerNews for example.
- First HTTP Request node crawls the webpage and grabs the website source code
- Then wait for x minutes
- Again, HTTP Node crawls the webpage
- If Node compares both results are equal if anything is changed. It’ll go to the false branch and notify me in telegram.
Here is the Telegram Response.
This use case can be expanded with libraries such as diff.
If you don’t want to use the Wait node. You can store the first HTTP Node result somewhere and compare it later using another workflow.
Hope this helps somebody.
This won’t work well for Jamstack sites. Since the DOM will be rendered after the complete page load