Hi @Secure_Growtech — answering your 4 properly since you asked specific questions and said you’re willing to pay for guidance. I’ve built the same stack you’re describing (scheduled scrapes, Telegram/email alerts, ElevenLabs tool-call + post-call), so here’s what actually breaks at scale:
1. Workflow refinement — what hurts reliability at scale
The 3 I see weekly in client-facing scrapes:
- Silent half-failures: workflow shows green but scraper returned 0 rows or notification failed. Fix: never end with empty result silently — add IF node after scrape (count=0 → route to Error handling, not success). For notifications, use separate Execute Workflow for alerts so failure is visible.
- No idempotency: re-run sends duplicate Telegram/email or double-writes. Fix: key every write/notification by a stable id (listing_id, lead_id) + check Postgres/Supabase exists before insert, and use Wait + Code dedupe on conversation_id for ElevenLabs path.
- HTTP retries missing: external site/API times out once and run dies. Add in HTTP Request: Retry on fail = ON, 3 retries, exponential backoff (1s, 3s, 9s), continueOnFail OFF so it actually retries instead of falling through.
2. Error handling patterns — central Error Workflow that actually helps
One central Error Workflow, set as Error Workflow in every client workflow’s Settings:
- Inputs it receives: executionId, workflow name, node name, error message, timestamp
- Inside: Code node formats, Postgres node logs to error_logs table, IF node splits retriable (timeout, 429, ECONNRESET) vs fatal
- Final: Telegram alert to one channel you actually read — include workflow name + node + error + last 2 execution ids, not generic “workflow failed”. That way you don’t open n8n to triage.
- In main workflows: use continueOnFail ONLY where partial result is useful (e.g., 1 of 5 pages fails but you still want 4). Always route failed branch to a visible Stack/Set + alert, don’t let it fall off canvas.
- Trap to avoid: global handler that catches everything and says nothing — put enough context in alert that you know if it’s scraper selector vs ElevenLabs vs Telegram token.
3. ElevenLabs + n8n — deeper than webhook receiver
Your webhook-receiver approach is right. Make it two workflows:
Workflow A — Tool Call Receiver (must be fast <2s):
Webhook → Code node: extract conversation_id, tool name, args → Postgres: INSERT raw payload with ON CONFLICT (conversation_id, event_type) DO NOTHING (this is your dedupe for ElevenLabs retries) → Respond to Webhook immediately with tool result JSON. Done. No CRM work here.
Workflow B — Post-call Processor (async):
Postgres trigger / Schedule (every 30s check new rows) → Fetch ElevenLabs transcript via HTTP Request GET https://api.elevenlabs.io/v1/convai/conversations/{conversation_id} → Store transcript + summary in transcripts table → IF intent = booking/support → HTTP Request to CRM (HubSpot/whatever) with retry → IF needs follow-up → Execute Workflow for Telegram/email.
Node combo that works: Webhook + Respond to Webhook + Postgres + Code (dedupe) in A, and Postgres (poll) + HTTP Request (ElevenLabs transcript) + HTTP Request (CRM) + Execute Workflow (notifications) in B. This way slow CRM never blocks call, and if CRM fails you replay from stored payload.
4. Polish / maintainability across clients
- Naming: [CLIENT]-verb-noun-env e.g., GT-scrape-listings-prod, GT-notify-telegram-prod
- Sticky Note at top of every workflow: purpose, trigger schedule, what it assumes (e.g., “expects listing_id unique”), owner
- Credentials: per client, not shared — prevents token rotation breaking 5 clients
- Reusable: make sub-workflows for error alert, telegram format, CRM upsert — fix bug once
- Versioning: export JSON to git (folder per client) after any change — use n8n tags for prod vs dev, and add version note in workflow description
If you want to do the paid review you mentioned, I’d keep first session concrete: you send one sanitized workflow export + 2-3 sanitized executions (redact client data, keep node structure), I review ack/dedupe path, what belongs in main vs Error Workflow, transcript → CRM pattern, naming/versioning, and return a prioritized checklist + error-workflow template you can import + updated workflow JSON where fix is straightforward. No prod credentials needed. Can do async over DM/email or 1:1 call — whichever you prefer.
Happy to do that as fixed-scope if useful — just send export + 2 runs + what you want workflow to do, and I’ll outline what I’d change first.
Igor