Community node - complex web scraping with Puppeteer

Built with n8n
21

I fixed it. I had to reboot the docket container.

22

I found a case where i need to get a verification code in email and go back to puppeteer. Is it possible? How?

23

. Use a temporary email service or IMAP/SMTP

  • Option A: Use a temporary email service with an API (e.g. Mail.tm, MailSlurp, Mailinator).
  • Option B: Use your own email inbox via IMAP (e.g., Gmail, Outlook) using a Node.js email library like:
    • imap-simple
    • node-imap
    • mail-listener2

2. Wait for and extract the code

  • After triggering the email via Puppeteer (e.g., filling a signup form), your script polls the inbox every few seconds.
  • Once the message arrives, parse the body with regex or a parser to extract the code.

3. Return to Puppeteer and input the code

  • Use page.type() or page.evaluate() in Puppeteer to enter the code into the page.
24

A bit late to this thread, but this might be useful for anyone landing here looking for Puppeteer-style browser automation while keeping n8n as the main workflow orchestrator.

I’ve been building Puppetflow, which takes a slightly different approach: instead of running Puppeteer directly inside n8n, you build and run the browser automation in Puppetflow, then trigger and control it from your n8n workflows through the Puppetflow integration.

The idea is to keep n8n focused on the workflow logic, while Puppetflow handles the browser side, including more complex interactions, authenticated sessions, file downloads, 2FA, screenshots, and cases where a human may need to take over the browser temporarily.

So a typical workflow can look like:

n8n > start Puppetflow browser flow > interact with website > return data/files > continue in n8n

It might be interesting for people here who like the flexibility of Puppeteer but don’t necessarily want to manage Chromium/Puppeteer dependencies directly inside their n8n instance.

I’m the developer behind it, so definitely biased :grinning_face_with_smiling_eyes:, but happy to answer technical questions or hear about browser automation use cases you’re struggling with.

25

Hi, @puppetflow! This seems overly great and I tried the self hosted version. However I guess for the average user a bit more guidance would be helpfull.

26

Thanks a lot for trying it and for the feedback! I’ve put a lot of time and energy into Puppetflow, and I haven’t had much feedback so far, so this genuinely means a lot.

And yes, it still really helps to know Puppeteer/Playwright basics, especially selectors. I’m hoping useful blueprints will make it much easier for people to learn the logic from real examples and build their own. And feel free to join the community Discord if you need any help!