Bright Data community node for scrapping anything

Built with n8n
1

​I’m excited to introduce the n8n-nodes-brightdata node, developed by myself at n8nhackers.

This community-developed node integrates Bright Data’s Web Unlocker API into n8n, simplifying your web scraping tasks.​

Key Features:

  • Effortless Web Scraping: Easily scrape websites by specifying the target URL and desired parameters, with the node handling proxy selection, custom headers, fingerprinting, and CAPTCHA resolution.​
  • Ideal For:
    • Scraping without encountering blocks.​
    • Emulating real-user behavior.​
    • Assisting teams without proxy infrastructure.​
    • Accessing premium domains that are typically challenging to scrape.​

Installation:

To install the n8n-nodes-brightdata node:

  1. Navigate to Settings > Community Nodes in your n8n instance.​
  2. Click on Install.​
  3. In the search bar, enter n8n-nodes-brightdata.​
  4. Select the node from the list and click Install.​

For more detailed installation instructions, refer to the n8n community nodes installation guide.​

Compatibility:

This node was developed and tested with n8n version 1.81.4.​

Usage Example:

To automate the process of collecting and delivering “Deals of the Day” from MediaMarkt:

  1. User Interaction: Users submit a form selecting product categories and providing their email addresses.​
  2. Data Extraction: The workflow uses the Bright Data node to scrape the MediaMarkt deals page, retrieving data in JSON format.​
  3. Content Processing: Extracted data is processed to generate personalized deal recommendations using AI models.​
  4. Email Delivery: Tailored deal lists are sent to users via email.​

This workflow streamlines the collection and distribution of personalized product deals, enhancing user engagement and satisfaction. You can use the next workflow to scrape anything you want:

External nodes Used:

Contribution

To make this node even better, please let us know, how you use it. Commits are always welcome.

Issues

If you have any issues, please let us know on GitHub.

About

Node by n8nhackers.com. For productive use and consulting on this, contact us please.

Special thanks to N8n nodemation workflow automation by Jan Oberhauser.

5 Likes
2

That’s cool! Could you enable tool usage in your code, so we can use it with AI agents as well?

1 Like
3

I checked one of the first nodes I made lot time ago for another service I own UProc.

Node code includes usableAsTool: true

I will apply this small change.

4

Awesome!
You might need to upgrade eslint (incl. config migration to mjs, etc.) in order to get it working. Had the same issue a couple of weeks ago. Here is a working example, maybe it helps: GitHub - octionic/n8n-nodes-appflowy

1 Like
5

Hi @octionic,

It’s not working with the current changes :frowning:

I will try again during this week.

Thank you!

6

Yeah it’s a bit tricky to implement the first time. Maybe you can check the commit history of the repo I shared. There you can see that quite a few changes need to be made.

1 Like
7

Sure, I will check your commits :wink:

Thank you!

1 Like
8

Will be trying this later. If it works well I will love you forver

1 Like
9

Yes. Bright Data is a nice service that helps you avoid lots of common problems when scrapping.

Next week I will publish changes to enable use as tool in Agents :wink:

10

Hi Miquel,
Just installed the brightdata community node, and evaluated your “LinkedIn Profile Finder” workflow. Everything worked immediately, …
Very nice template, well done !
Stéphane

2 Likes
11

Hi everybody,

I have updated Bright Data to allow:

  • Access to Marketplace Dataset API endpoints (preparing some samples here). This opens accessing to more than 160 Datasets for AI processing.
  • Enabled using Bright Data node as an AI Agent tool.
    image
    image902×762 105 KB

Thank you @octionic. Only usableAsTool property is required to add as a tool in an AI Agent :wink:

2 Likes