Keyword-driven profile scraper across Skool, Reddit, Upwork, Twitter/X

Need a keyword-driven scraper that finds people across multiple platforms based on search terms I provide.

  • Platforms: Skool communities, Reddit, Upwork, Twitter/X
  • Input: keywords I define
  • Output: name, platform, profile URL, contact info if available, short description
  • Volume: 200+ profiles per run
  • Reusable: I change keywords, it runs again for different searches

Tools: your choice. DM me if you’ve built something similar.

5 Likes

Hey :waving_hand:,

I’m Milan, with 8 years of experience in Business Automation and AI. Including 2 years at Apify working on enterprise-level browser automation.

Currently specializing in n8n, but also proficient in Python & Javascript.

Find out more about my work here:

If you think I might be a match, please:

Book a call here with me

Or reach out at hello@smoothwork.ai

Looking forward to hearing from you!

Hi Darius, thanks for reaching out.

This is very doable — I’ve built similar keyword-driven scraping + enrichment workflows across multiple platforms.

Typical setup I use: • Keyword input → trigger (manual or scheduled)

• Scraping layer (Apify / custom APIs / HTTP nodes)

• Data cleaning + deduplication

• Enrichment (profiles, contact data where available)

• Structured output to Airtable / Google Sheets

Handles repeat runs easily — just swap keywords and re-run.

Relevant work: • Multi-platform lead generation + enrichment system

• Automated scraping + CRM pipeline

Portfolio:

https://www.upwork.com/freelancers/~0122761e4734295f4b⁠�

Quick questions:

• Do you need real-time scraping or batch runs?

• Any priority platform to start with?

Happy to show a quick working structure.

Book: https://cal.com/folafoluwa-olaneye-osrofp/30min⁠�

Or DM here / email: folafoluwaolaneye@gmail.com

Best regards,

Folafoluwa Stephen

your instinct is off again—you’re trying to dump info, not win attention. but fine, here’s a version that answers him + includes your links without looking desperate:


comment:

Hey, I’ve built similar keyword-based scraping + lead enrichment systems across multiple platforms, so this fits well.

For your use case:

  • I can pull profiles from Reddit, twitter/x, Upwork, and Skool based on your keywords

  • structure output as: name, platform, profile url, short bio, and available contact signals

  • handle 200+ profiles per run with clean, deduplicated data

  • set it up as a reusable workflow (just change keywords → run again)

Approach would be:

  • APIs where possible (more stable)

  • scraping with rotation where needed (to avoid rate limits/blocks)

  • An optional enrichment layer if you want better lead quality

I’ve already built systems like:

  • automated lead generation pipelines (1000+ leads)

  • multi-platform scraping + enrichment workflows

  • n8n-based reusable automation systems

You can check my work here:
:globe_with_meridians: https://www.muhammadz.fun/
:brain: https://www.notion.so/muhammad-ai-automations/AI-Solutions-Automation-Showcase-2026-2f8a292a24138082acece2ccbb1c3a3b

Happy to map the exact setup for your case if needed :+1:

I’ve built keyword-driven scrapers across multiple platforms and can create a reusable workflow for you—are you available to discuss further?

sent you DM as well Looking forward to hearing from you.

Hey Darius —

Most people replying to this will hand you a scraper that pulls names and URLs.

You’ll run it, get 200 rows, and realise half the profiles are ghost accounts with no contact info and zero context on why they match your keyword.

The real problem isn’t scraping. It’s relevance filtering.

What I build is different — every profile gets scored against your keyword intent, so the 200 you get are actually the 200 you’d want to reach out to.

One question before I send you a breakdown:

Are these keywords niche-specific — like a skill or job title — or are you targeting a pain point people are talking about?

Changes everything about how I’d set this up.

Best
Hamza
itsameerhamza203@gmail.com

Hey Darius,

A reusable, keyword-driven scraper across Skool, Reddit, Upwork and Twitter/X with structured profile output and 200+ results per run — that is exactly the kind of pipeline I build.

I recently completed a multi-platform lead generation pipeline for a client using Apify that pulled 92 qualified contacts across multiple platforms in a single run, with structured output to Google Sheets including name, profile URL, contact info and a short bio. Before that I built the Imperial Engine, a scraper processing 500+ profiles per day with proxy rotation, anti-detection headers and clean JSON/CSV output. The reusability pattern you described — swap keywords, re-run — is how I architect every scraper: keyword inputs drive the search queries, not the scraper structure.

For your four platforms: Reddit has a usable API where keyword search is native and I would pull it via n8n HTTP nodes. Upwork’s public search is scrapeable through HTTP requests. Skool doesn’t expose a public API so I’d use an Apify actor with session rotation to pull community member profiles. Twitter/X is the trickiest since API access has tightened, but Apify’s Twitter actor still works reliably on keyword searches and returns usernames, bios and contact details where available.

The output would be a clean, deduplicated spreadsheet (Google Sheets or CSV) with columns for name, platform, profile URL, available contact info and a short description. 200+ profiles per run is comfortably achievable on Reddit and Upwork alone for most keywords. All four platforms in parallel would exceed that easily.

A couple of quick questions before I scope it: do you want all four platforms running in one workflow, or phased by source so you can review quality per platform? And should cross-platform deduplication happen at run time so the same person found on Reddit and Upwork is counted once?

You can see my work at Goodnews-code (Egbewale Jesutofunmi Goodnews) · GitHub . Email: goodnewscode@gmail.com. Available immediately and happy to DM a quick breakdown or show you a sample output structure first.

JT

Hey Darius — concrete sketch of how I’d build this so you can compare against the other replies:

Per-platform extractor — separate n8n sub-workflow per source (Skool / Reddit / Upwork / X), each with its own auth + rate-limit handling. Skool and Upwork have meaningful bot detection, so those need residential proxy + headless browser; Reddit and X have official APIs that are the cheaper and more reliable path at volume.

Identity dedup across platforms — same person often shows up on Reddit + X with different handles. A canonical-profile resolver (display name + bio fingerprint + linked-website match) merges duplicates before they hit your output. Otherwise 200+ profiles/run becomes 200+ duplicates over a few keyword cycles.

Enrichment pass — name + URL is the easy part. Email finder (Hunter / Anymail) + role/seniority guess from bio gets it from “scraped row” to “outreach-ready record.”

Structured output — single Airtable or Notion target with platform/keyword/run-id columns, so re-runs deduplicate against history rather than spamming you with the same person twice.

Happy to take it private — DM me here or via the contact links in my profile.

— Priyanshu
Portfolio: Work — Priyanshu Kumar

Just sent you a dm with all the details and a loom for the sample I have created. Looking forward to speaking more regarding the same.
Calendar- https://axonyx.framer.ai/
Pmediaaryan@gmail.com

Hi Darius!

I’ve built several custom scraping engines for lead generation, and I can deliver exactly what you’re looking for. Instead of a basic scraper that breaks when a UI changes, I build keyword-driven systems designed for volume and reliability.

How I’ll build this for you:

  • Multi-Platform Logic: I’ll use a combination of n8n for orchestration and specialized scraping APIs (or custom Python/Playwright scripts) to bypass rate limits and anti-bot measures on X and Reddit.

  • Skool & Upwork Integration: I’ll implement specific logic to navigate Skool communities and extract profile data, ensuring the output includes the bio and social links where available.

  • Keyword Flexibility: The system will be “input-ready.” You just update a Google Sheet or a simple interface with your keywords, and the scraper triggers a fresh run for all platforms.

  • Data Enrichment: For the “contact info” part, I can integrate enrichment tools to find emails or LinkedIn profiles linked to the handles we find.

Why me?

I’ve already shipped a “Lead Factory” for a similar project that classifies intent and handles lead extraction at scale. I understand how to manage the data flow so you get a clean CSV or database with 200+ profiles per run without hitting blocks.

Relevant Projects:

  • Fintech & Real Estate Scrapers: Built custom lead extraction workflows to target specific business accounts and realtor data.

  • AI Content Factory: Managed massive data extraction pipelines that feed into AI content generators.

  • Portfolio: https://mikedevai.netlify.app/

Contacts:

I’m ready to show you a demo of my previous scraping builds. Let’s get this running.

Best regards,

Mikhail

hey Darius - we actually run exactly this in production right now. keyword-driven scraping across Skool, Reddit, Upwork, X, LinkedIn. our system handles the full loop: keywords in, profiles out with enrichment (name, URL, contact where available, relevance score).

the tricky part most people miss is deduplication across platforms and keeping output clean when you’re doing 200+ per run. we solved that with an AI scoring layer that filters noise before it hits your sheet.

happy to show you a working demo if you want to see how it handles your use case. you can grab 15 min here: https://cal.com/princeps/poly-digital-workforce

or just DM me and I can share a sample output from a recent run.