Confidence-Gated B2B Lead Qualifier & Reply Triage Engine (AI Scoring + Human-in-the-Loop)

Built with n8n
, , , , ,
1

Hey everyone,

Inbound lead intake usually suffers from two extremes: either every submission gets dumped directly into the CRM and floods the sales reps with spam, or rigid regex filters accidentally reject viable clients with non-standard phrasing.

I built this end-to-end qualification and routing architecture at Quiet Commerce HQ. It uses LLMs for semantic qualification and reply sentiment, paired with deterministic confidence gating and human-in-the-loop approval.

1. Inbound Lead Qualification Pipeline

  • Ingestion & Normalization: Inbound webhooks catch raw submission payloads and clean/normalize data via JavaScript before feeding the AI agent.
  • LLM ICP Evaluation: An AI agent compares the prospect’s profile, budget, timeline, and project scope against our ICP rubric and outputs a structured confidence score.
  • Confidence Gate:
    • High Confidence: Bypasses manual review, immediately enrolls the lead in the onboarding email sequence, constructs the CRM payload, and syncs directly to the CRM.
    • Borderline / Low Confidence: Triggers a human-in-the-loop branch (Wait for Human Approval via an interactive approval email). If an admin approves, it routes right into the CRM/email sequence; otherwise, it logs as rejected without cluttering sales queues.
  • Alerting: Admin notification node dispatches instant alerts for high-value routing events.

2. Bi-Directional Reply Sentiment Triage

  • Reply Ingestion: An incoming webhook captures prospect replies from the email campaign.
  • Intent Analysis: A secondary chat model parses the sentiment and intent of the reply.
  • Routing: Positive replies flag an admin and park in an asynchronous review state for personalized closing, while negative or uninterested responses cleanly route to an archive path with no further automated pings.

Canvas Architecture

Screenshot 2026-10-25 131237
Screenshot 2026-10-25 131237844×366 38.6 KB

Tech Stack & Resiliency

  • Self-hosted n8n in Docker
  • Caddy reverse proxy for path allowlisting and webhook endpoint protection
  • Local LLM / inference models for evaluation
  • Deterministic fallback paths so webhook payloads are never silently dropped if an external API rate-limits

Curious how others are handling human-in-the-loop approval thresholds for high-volume intake!