Community node - complex web scraping with Puppeteer

Hi,

I have created an extended version of the existing Puppeteer node because I needed complex interactions features and I decided to share it with the community. If you try it let me know if you find a bug or need clarification on how to use it.

If you want to see it in action, I made a video here: https://twitter.com/i/status/1574320458245472256

Hi @hckdotng

Welcome to the community!
Always happy to see new community nodes. :slight_smile:

Have you contacted @drudge to see if you could work together on extending the existing puppeteer node? It would be very nice to see you guys working together to make one excellent node.

Hi, I’m in contact with @drudge, I hope we will have the opportunity to collaborate. I didn’t reach him for this because I had to break the compatibility with a very different logic but if he wants to modify the original node I’ll be happy to help.

1 Like

I developed a similar node to work with headless browser, that may fits your needs

https://www.npmjs.com/package/n8n-nodes-browserless

project is in the early stage so might need contributions from those who are interested in this topic.

2 Likes

this plugin works very well for me so far. https://www.npmjs.com/package/n8n-nodes-browserless

However there needs to have the ability to use a proxy, as sometimes I need to force the browser to use a specific proxy, and I think I remember a couple of spelling mistakes in some of the menus, but that wasn’t a issue for me really.

Would be good to have it do actions like “scroll to the bottom”

but I don’t want to hijack this thread.