# How do you scrape Cloudflare-protected websites in n8n?

**URL:** <https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663>\
**Category:** Questions\
**Created:** [May 24, 2026, 5:06pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663 "2026-05-24T17:06:46Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Asim\_Arman](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/asim_arman/32/165663_2.png) [@Asim\_Arman](https://community.n8n.io/u/Asim_Arman)\
**Post date:** [May 24, 2026, 5:06pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/1 "2026-05-24T17:06:46Z")

</div>

I’m building a daily digest workflow that fetches content from HVAC industry blogs. Three of my sources (HouseCall Pro, Jobber Academy, HVAC Informed) are protected by Cloudflare. When my HTTP Request node hits these URLs, it gets a 403 “Just a moment…” Cloudflare challenge page instead of the actual content.

These sites don’t have RSS feeds either, so RSS Feed Read nodes don’t work.

**What I’ve tried:**

- HTTP Request node (GET) — returns Cloudflare challenge HTML

- RSS Feed Read node — returns XML parse errors (pages aren’t RSS)

**Questions:**

1. Is there a way to bypass Cloudflare bot protection in n8n natively?

2. Should I use Apify, ScrapingBee, or BrightData for these sources?

3. Is there a community node that handles Cloudflare-protected sites?

4. Would adding browser-like headers (User-Agent, Accept-Language, etc.) help?

---

<div class="post-metadata">

**Author:** ![achamm](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/achamm/32/143734_2.png) [@achamm](https://community.n8n.io/u/achamm)\
**Post date:** [May 24, 2026, 5:10pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/2 "2026-05-24T17:10:46Z")

</div>

@Asim_Arman did a quick check on ur 3 sources before recommending anything and theres some good news for one of them — HouseCall Pro isnt actually on Cloudflare and its /feed/ works fine if u just add a real browser User-Agent header to the HTTP Request node. tested with a Mozilla/5.0 UA against [Housecall Pro](https://www.housecallpro.com/feed/) and got valid RSS XML back. ur 403 there is almost certainly because n8n’s HTTP Request node defaults to a generic UA (or none at all) and the site has a basic UA filter. switch to RSS Feed Read pointed at [Housecall Pro](https://www.housecallpro.com/feed/) with a User-Agent override and ur done for that one, no scraping needed.

Jobber ([getjobber.com](http://getjobber.com)) is genuinely on Cloudflare — cf-ray header present in responses — and theres no RSS for the academy section. HVAC Informed also doesnt expose RSS. those two genuinely need a scraping service.

on the bypass-cloudflare-natively question, no real way to do it in n8n. their challenge requires executing JS to solve proof-of-work plus behavioral fingerprinting, and the HTTP Request node is a plain http client with no browser engine. the cloudscraper-style libs from 2022-2023 are all broken against current Cloudflare too, so the few community nodes wrapping them dont work either.

browser-like headers fix basic UA filters (which is what HouseCall Pro’s 403 actually was, ironically) but do nothing against the real Cloudflare challenge — they fingerprint TLS, JA3, request timing, and JS execution beyond just headers.

for ScrapingBee vs Apify vs BrightData — for ur use case (low-volume daily digest, just 2 sites needing real scraping) ScrapingBee or Firecrawl is the cleanest. rough comparison:

```json
{
  "scrapingbee": {"per_request": "~$0.001-0.005", "fit": "low-mid volume, simple, fast"},
  "scraperapi": {"per_request": "~$0.001-0.003", "fit": "comparable to scrapingbee"},
  "firecrawl": {"per_request": "~$0.001", "fit": "scrape-to-markdown, LLM-ready output"},
  "brightdata": {"per_request": "~$0.005-0.02", "fit": "hardest targets, expensive but most reliable"},
  "apify": {"per_request": "varies", "fit": "complex multi-page flows"}
}

```

ScrapingBee called via HTTP Request looks like:

```json
{
  "method": "GET",
  "url": "https://app.scrapingbee.com/api/v1/",
  "qs": {
    "api_key": "<your_key>",
    "url": "https://www.getjobber.com/academy/",
    "render_js": "true",
    "premium_proxy": "true"
  }
}

```

end state is one RSS Feed Read for HouseCall Pro, two ScrapingBee HTTP calls for Jobber and HVAC Informed, then parse → digest. way cheaper than treating all 3 as Cloudflare-protected when only one actually is.

---

<div class="post-metadata">

**Author:** ![Asim\_Arman](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/asim_arman/32/165663_2.png) [@Asim\_Arman](https://community.n8n.io/u/Asim_Arman)\
**Post date:** [May 24, 2026, 5:16pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/3 "2026-05-24T17:16:24Z")

</div>

> [@achamm](#):
>
> apingBee or Firecrawl is the cleanest. rough comp

Thanks alot!, I am trying this and letting you know

---

<div class="post-metadata">

**Author:** ![achamm](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/achamm/32/143734_2.png) [@achamm](https://community.n8n.io/u/achamm)\
**Post date:** [May 24, 2026, 5:17pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/4 "2026-05-24T17:17:18Z")

</div>

Your welcome! Lemme know how it works!

---

<div class="post-metadata">

**Author:** ![Milan\_AutomableAI](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/milan_automableai/32/57849_2.png) [@Milan\_AutomableAI](https://community.n8n.io/u/Milan_AutomableAI)\
**Post date:** [May 25, 2026, 7:43am UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/5 "2026-05-25T07:43:20Z")

</div>

I would like to add that Apify has the **[Smart Article Extractor](https://apify.com/lukaskrivka/article-extractor-smart?fpr=9lmok3)** which can (usually) detect article links on a page automtaically, and then scrape them one by one.

And it does emulate a browser so it can bypass Cloudflare too. It’s a usage-only actor as well, so a once a day call to three blogs will probably **fit into their free tier**.

For more general (non-article) cases, you can get structured content by using their **[Website Content Crawler](https://apify.com/apify/website-content-crawler?fpr=9lmok3)** and then running them through a small AI model, such as haiku or gpt minis.

---

<div class="post-metadata">

**Author:** ![d-hippolyte](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/d-hippolyte/32/174236_2.png) [@d-hippolyte](https://community.n8n.io/u/d-hippolyte)\
**Post date:** [May 25, 2026, 7:47am UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/6 "2026-05-25T07:47:43Z")

</div>

For more complex sites like Jobber and HVAC Informed, if you use a **[self-hosted n8n instance with Docker](https://docs.n8n.io/hosting/installation/docker/)** and prefer to avoid paid solutions, an interesting alternative is to run **SeleniumBase in Undetected ChromeDriver (UC) mode** within a separate Docker container. SeleniumBase’s **UC (Undetected ChromeDriver)** mode is currently one of the most robust open source tools for bypassing Cloudflare protections.

You can then configure this container to run your scraping scripts and trigger them directly from n8n using the **[Execute Command](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.executecommand/)** node (or using a **[custom tool](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/tools-agent/)** if you’ve structured your workflow around an n8n AI Agent).

---

<div class="post-metadata">

**Author:** ![syed\_noor](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/syed_noor/32/174285_2.png) [@syed\_noor](https://community.n8n.io/u/syed_noor)\
**Post date:** [May 25, 2026, 9:33am UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/7 "2026-05-25T09:33:36Z")

</div>

One thing nobody’s mentioned yet: before reaching for a scraping service, check if these sites have a sitemap or Atom feed hiding behind a non-standard path.

Try hitting these before paying for anything:

- /sitemap.xml
- /sitemap\_index.xml
- /feed/
- /atom.xml
- /blog/feed/ or /blog/rss/

Many WordPress/HubSpot-powered blogs (which HouseCall Pro and Jobber both use) have feeds enabled but not linked in the . The RSS Read node will work fine if you hit the right URL.

For the ones that genuinely have no feed and Cloudflare blocks you:

Cheapest production-grade approach: Firecrawl or ScrapingBee via HTTP Request node.

HTTP Request → POST [https://api.firecrawl.dev/v1/scrape](https://api.firecrawl.dev/v1/scrape)  
Headers: Authorization: Bearer  
Body: { “url”: “[https://target.com/blog](https://target.com/blog)”, “formats”: [“markdown”] }

Returns clean markdown — no HTML parsing needed. Run it on a Schedule Trigger once daily. At 3 sites × 1 request/day, you’ll stay well inside any free tier.

Production hardening (the part most tutorials skip):

1. Add a retry with backoff on the HTTP Request node — scraping APIs occasionally 429 you
2. Store the last-fetched URL hash in a Postgres table so you don’t re-process the same article  
tomorrow (idempotency)
3. Add an Error Trigger workflow — if your digest silently fails on Monday, you want to know before Friday

The “just scrape it” part is easy. Making it run reliably every day for 6 months without silent failures is where most daily digest workflows break.

---

<div class="post-metadata">

**Author:** ![Asim\_Arman](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/asim_arman/32/165663_2.png) [@Asim\_Arman](https://community.n8n.io/u/Asim_Arman)\
**Post date:** [May 27, 2026, 3:02am UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/9 "2026-05-27T03:02:30Z")

</div>

Hey, really appreciate the detailed breakdown, **[achamm.](https://community.n8n.io/u/achamm)** This was super helpful and I ended up following most of it.

**What I followed exactly:**

- HouseCall Pro - confirmed it wasn’t Cloudflare, added the User-Agent header to the HTTP Request node, pointed it at`/feed/`, parsed through an XML node, then a Code node. Works perfectly now.

- Jobber and HVAC Informed - used Firecrawl instead of ScrapingBee (same price tier you mentioned, just preferred the markdown output for passing to Claude).

**What we added beyond your recommendations:**

- A Parse Code node after each Firecrawl request to normalize all sources into identical fields (title, content, link, feedSource) before merging

Thanks so much man, glad to have you in the community

---

<div class="post-metadata">

**Author:** ![achamm](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/achamm/32/143734_2.png) [@achamm](https://community.n8n.io/u/achamm)\
**Post date:** [May 27, 2026, 10:36am UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/10 "2026-05-27T10:36:07Z")

</div>

Your welcome, happy we were able to help you!

---

<div class="post-metadata">

**Author:** ![d-hippolyte](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/d-hippolyte/32/174236_2.png) [@d-hippolyte](https://community.n8n.io/u/d-hippolyte)\
**Post date:** [May 27, 2026, 3:16pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/11 "2026-05-27T15:16:21Z")

</div>

If you have the possibility to [self-host](https://docs.n8n.io/hosting/installation/docker/) a tool (for example via Docker on the same server as your n8n) and you want to avoid the quota limits of paid platforms (like Apify, ScrapingBee or Firecrawl), you should look into [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr/blob/master/docker-compose.yml).  
[It’s an open-source proxy that launches a headless browser in the background to solve the Cloudflare challenge and return you the raw HTML of the page. You just need to query it locally with a simple HTTP Request node in n8n.](https://github.com/FlareSolverr/FlareSolverr/blob/master/README.md)  
However, FlareSolverr returns you the raw HTML code. You’ll therefore need to add an [HTML](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.html/) node or a [Code](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.code/#external-libraries) node right after to parse and structure the data yourself into JSON.  
a small example:

{  
“nodes”: [  
{  
“parameters”: {  
“jsCode”: “const html = $json.solution.response;\n\nfunction getText(html, startMarker, endMarker) {\n const start = html.indexOf(startMarker);\n if (start === -1) return ‘’;\n const end = html.indexOf(endMarker, start);\n if (end === -1) return ‘’;\n return html.substring(start + startMarker.length, end).trim();\n}\n\nfunction getMetaContent(html, name) {\n const regex = new RegExp(`<meta name=\"${name}\" content=\"([^\"]+)\"`, ‘i’);\n const match = html.match(regex);\n return match ? match[1] : ‘’;\n}\n\nlet title = getText(html, ‘\<h1 class="story-title"\>’, ‘’);\n// cleans the title of tags\ntitle = title.replace(/\<a[^\>]_\>([^\<]_)\</a\>/, ‘$1’).trim();\n\nconst description = getMetaContent(html, ‘description’);\n\nlet content = getText(html, ‘\<div class="articlebody clear cf" id="articlebody"\>’, ‘\<div class="stophere" id="hiddenH1"\>’);\n// cleans the content\ncontent = content\n .replace(/\<script[^\>]_\>[\s\S]_?\</script\>/gi, ‘’)\n .replace(/\<style[^\>]_\>[\s\S]_?\</style\>/gi, ‘’)\n .replace(/\<[^\>]+\>/g, ’ ')\n .replace(/\s+/g, ’ ')\n .trim();\n\nreturn [{\n title: title,\n description: description,\n content: content\n}];”  
},  
“type”: “n8n-nodes-base.code”,  
“typeVersion”: 2,  
“position”: [  
-912,  
4480  
],  
“id”: “a6275095-c4ed-427f-a791-620bcd83710b”,  
“name”: “Extract from HTML”  
},  
{  
“parameters”: {  
“method”: “POST”,  
“url”: “=[http://flaresolverr:8191/v1](http://flaresolverr:8191/v1)”,  
“sendHeaders”: true,  
“headerParameters”: {  
“parameters”: [  
{  
“name”: “Content-Type”,  
“value”: “application/json”  
}  
]  
},  
“sendBody”: true,  
“bodyParameters”: {  
“parameters”: [  
{  
“name”: “cmd”,  
“value”: “request.get”  
},  
{  
“name”: “url”,  
“value”: “={{ $json.url }}”  
},  
{  
“name”: “maxTimeout”,  
“value”: 90000  
}  
]  
},  
“options”: {  
“timeout”: 120000  
}  
},  
“type”: “n8n-nodes-base.httpRequest”,  
“typeVersion”: 4.4,  
“position”: [  
-1120,  
4480  
],  
“id”: “4c884f8a-09c7-4c13-b7e8-ac38098b00f6”,  
“name”: “FlareSolverr”,  
“retryOnFail”: true  
}  
],  
“connections”: {  
“Extract from HTML”: {  
“main”: [  
  
]  
},  
“FlareSolverr”: {  
“main”: [  
[  
{  
“node”: “Extract from HTML”,  
“type”: “main”,  
“index”: 0  
}  
]  
]  
},  
“pinData”: {},  
“meta”: {  
“templateCredsSetupCompleted”: true,  
“instanceId”: “03428eda2f2a76af33393f7fa59419dea9e2e74e0d630500b36df8cf5b8e161d”  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://dub1.discourse-cdn.com/n8n/user_avatar/community.n8n.io/system/32/195824_2.png) [@system](https://community.n8n.io/u/system)\
**Post date:** [June 3, 2026, 3:17pm UTC](https://community.n8n.io/t/how-do-you-scrape-cloudflare-protected-websites-in-n8n/296663/12 "2026-06-03T15:17:20Z")

</div>

Closed on June 3, 2026, 3:17pm UTC

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.