Web scraping

Help me Build my Workflow
1

Hey,

For our recruitment, we tried setting up a workflow to automatically collect job vacancies from a client portal and generate a daily overview for our recruiters, but unfortunately, we weren’t able to get it working as intended. We couldn’t reach the vacancies in the portal, because of anti-bot cookies and other security methods.

Is there a way how we can still manage to do this?

2

I’ve seen similar issues when sites block direct HTTP requests. Sometimes, extracting the structured data first and then sending it into n8n works better than scraping directly inside the workflow.