First wanted to say I have followed the n8n project for quite some time now and have always wanted to play around with it. Finally got around to doing it and I love it!
So the subject line says it all, is it possible to scrape a website that requires a login?
What I would like to do:
- Scrape my works website behind my account to get my eschedule.
- Organize that data into a google sheet
- Add the events to a google calendar
- Alert me when it happens, when it is done and report what was added.
Now I know the last 3 are 100% easy peasy lemon squeezy but the first has me stumbled. I know I can use HTTP Requests to probably send the login information and then scrape the site (once logged in and holding the proper cookies) to get the schedule (still trying to learn how to even do that )… but would it be easier with something that acts like the Selenium project?
Integrating something like this could (if I’m not mistaken ?) enable web scraping on almost every web page?
Fruit for thought! Again love the project! Thank you all for the hard work!