I’ve been experimenting with Apify for web scraping and connecting the scraped data with n8n for further processing and automation.
My workflow:
Website → Apify Actor → Structured Data → n8n → Cleaning/Processing → Output
So far, I’ve been exploring:
• Apify Actors
• Structured data extraction
• Pagination & rate limits
• Data cleaning and transformation
• Connecting scraped data to n8n workflows
The combination of Apify + n8n makes it possible to turn web data into useful automated workflows.
Would love to hear how others in the n8n community are using Apify in their workflows!
Nice one, mine’s the same shape. Two things that saved me money: set max total charge on the Apify node, and put Remove Duplicates right after it keyed on post id, so scheduled runs only hand you new rows.
The example I built follows that pattern for historical Reddit exports into a Sheet, with a Markdown field for retrieval. Disclosure: I maintain the paid Archive Actor used in this example.
One detail to add to the deduplication point: if the dataset mixes posts and comments, use type:id as the record key. Posts store their text in selftext; comments use body. Reading only selftext produces empty comment text even when the run succeeds.
For repeated exports, the template uses Google Sheets’ Append or Update Row, matching on record_key, with raw cell values. That preserves literal text and updates existing records. It doesn’t prevent repeat Actor charges when date windows overlap.
The workflow JSON and setup guide start with 25 posts and up to three comments each, a $0.50 charge cap and 512 MB. Archive coverage varies; the small limit can miss records in a busy period. A three-post/six-comment cloud check cost $0.04205 at the current Free-tier event prices.
The formatter was checked against those real rows. Destination credentials are absent, so test your Sheet before enabling the schedule. The n8n gallery correction is under review; use the Gist version for now.
@dami1 Setting a max total charge limit on the Apify node and placing Remove Duplicates right after it is a game changer for cost optimization—definitely going to implement that in my scheduled runs. Thanks for sharing your template too!
Thanks for the detailed explanation and practical tips, @benthepythondev!
The distinction between selftext for posts and body for comments is super helpful—using type:id as the record key makes complete sense to avoid empty fields during deduplication. Also, setting up Append or Update Row matched on record_key in Google Sheets is a great way to handle duplicate entries cleanly.
Thanks for sharing your template workflow and cost breakdown as well!
Exploring Web Scraping with Apify + n8n