I’ve been building a pair of research workflows and figured the method might be useful to folks here — it’s a good example of n8n handling a genuinely messy multi-source pipeline.
What it does:
Give it a niche (currently a catalogue of “boring” single-purpose tools — calculators, converters, generators), and it:
- Expands seed queries via a keyword-suggestion API (Twinword) — one HTTP request per seed, with batching tuned to the provider’s rate limit (2 req/10s — needed a 7s batchInterval, 5s sat exactly on the boundary and intermittently failed)
- Filters suggestions to tool-shaped queries, drops branded/app-shaped keywords (regex sets in Config), canonicalizes aliases (
bmi calculator≈calculator bmi) - Scores on demand (log volume), monetisation (CPC), build simplicity, cluster strength — with per-category quotas so one pillar can’t dominate the shortlist
- Checks live SERP top-10 per shortlisted keyword (Serper) against a curated strong-domain list — presence of calculator.net/nerdwallet/etc. penalizes the score; weak incumbents give it a bonus. Also runs a Chrome Web Store check via Apify for extension competition
- Pulls real market comps (Flippa scraper) for revenue anchoring
- Sends the scored cluster to OpenAI with
response_format: json_schema(strict) to name the site and write per-tool briefs — then a validation node rejects any tool whoseprimary_keyworddoesn’t exist in the real scored data (cheap grounding layer, works well) - Writes everything to Notion with dedup keys (
utl:{domain}:{date},tool:{kw}:{date}) so reruns don’t duplicate rows
Numbers: ~$5/run, ~20 min, produces a ranked blueprint of 40+ tools with real volumes/CPCs/evidence strings.
What I learned building it:
continueRegularOutput+ retries per node saved the run repeatedly — one dead source shouldn’t kill 45 minutes of upstream work- Keep the LLM at the end for synthesis only — everything quantitative is deterministic code. The model can’t hallucinate numbers it never sees.
- Field-name validation matters more than expected: Twinword returns
"search volume"with a literal space, and"paid competition"is an ad-density index, not SEO difficulty — almost shipped a scoring bug on that one - Dedup keys in the destination DB (queried before write) make the whole thing safely re-runnable
The other workflow (same architecture) scans Product Hunt/HN/Reddit/AppSumo/app stores for launch traction — same normalize→score→enrich→LLM-brief→Notion shape.
Happy to share the JSON or answer questions about any part — the filtering/scoring code nodes are where most of the interesting decisions live.