Help with scraping website

Questions
1

Hey everyone! I’m looking to scrape a list of people from a website that has 80 pages. Is there an agent or tool that can automatically navigate through all the pages and extract the listed individuals? Alternatively, I’d love to hear any N8N workflow ideas that could help me achieve this. I’m happy to build it out if I have a solid concept.

Describe the problem/error/question

What is the error message (if any)?

Please share your workflow

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Information on your n8n setup

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Oh. Going to try now. WIll let you know. Thanks

5

Or… in addition to what was mentioned above - Crawl4AI and Firecrawl - you could just automate crawling yourself with n8n :slight_smile:

HTTP Request node support pagination, which you can use to iterate over pages.

2 Likes
6

This scrapper on chrome store. Instant web scrapper. It did the job.

Thanks once again

1 Like
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.