Short tip: keep lead enrichment workflows cheap and re-runnable

Tutorials English 🇬🇧
1

A pattern that has saved me a lot of API credits when enriching leads in n8n:

1. Normalize first. Before any enrichment call, lowercase the email, extract the domain with a Set node, and drop free mail domains (gmail, outlook and so on) with a Filter node. Most enrichment APIs work at company level anyway.

2. Cache by domain. Keep a Data Table (or a Postgres table) with domain, enrichedAt and the fields you care about. Look the domain up first and only call the paid API when there is no row or the row is older than, say, 90 days.

3. Batch and pace. Use Loop Over Items with a small batch size plus a Wait node so you stay inside the provider’s rate limit, and turn on Retry On Fail in the HTTP Request node settings for 429 responses.

4. Write back with an upsert keyed on email, so you can re-run the whole workflow safely after a failure without creating duplicates in the CRM.

5. Store a status field (pending, enriched, not_found). Leads marked not_found are skipped next time instead of costing you another call.

The result is a workflow you can re-run any time, and the cost grows with new companies rather than with total lead volume.

2

:high_voltage: HIGH-THROUGHPUT CONCURRENCY HARDENING FOR LEAD ENRICHMENT PIPELINES

Exceptional architectural breakdown, @stackshortlist! Moving away from naive linear API enrichment loops to domain-level caching and out-of-band pre-filtering is a critical prerequisite for building economically sustainable automation infrastructure. Dropping freemail domains early and implementing explicit state tracking (not_found, enriched) significantly reduces third-party provider billing waste.

However, when scaling this exact enrichment graph inside enterprise-level multi-tenant platforms or high-velocity D2C marketing funnels handling severe traffic surges, relying purely on sequential database lookups or stock CRM upsert keys introduces an infrastructure vulnerability. When hundreds of leads hit your webhook intake nodes within the exact same millisecond window, asynchronous lookup delays create database write-race conditions. Two identical domain records will clear the not_found check simultaneously, causing duplicate mutations, double-billing calls, and database index locks.

To harden this blueprint into an un-crashable enterprise engine, I wrap this lead-enrichment loop inside a strict 2-step write boundary directly at the ingestion tier:

:one: Distributed Mutex via Redis Lock Primitives: Before letting a lead clear the normalization node, route the parsed domain string or email hash through an isolated custom Python or code node backed by native Redis locks (specifically INCR + TTL gates). If a concurrent thread attempts a duplicate enrichment call for an active domain within a 45ms window, the system intercepts the collision at the ingestion boundary, cleanly queue-stacking or discarding the rogue payload before it can trigger an API cost or strain your downstream PostgreSQL ledger.
:two: Rigid Schema Boundaries: Wrap your non-deterministic data enrichment outputs inside rigid, hardcoded Pydantic validation schemas. If an external API returns a corrupted payload or a drift parameter during an escalation loop, your workflow flags the schema exception out-of-band instead of quietly writing partial anomalies to your CRM tables.

:laptop: You can directly audit how I structure these containerized Docker microservices, secure proxy layers, and state-locking sub-graphs straight from my verified repository:
:backhand_index_pointing_right: https://github.com

Fusing your domain-caching pattern with memory-layer concurrency control is the absolute gold standard for production-grade engineering. Fantastic tutorial share, stackshortlist!