Main goal: The agent should remember everything (tasks, notes, meetings, emails, ideas) and be able to answer context-based questions like “What’s the status of project X? What are meeting insights from customer xy?” with full awareness of what’s in Notion, etc.
Current setup:
Main Orchestrator Agent (routes input to sub-agents)
I am looking to build something similar pulling from my vast knowledge base and active info. My framework includes a vector database to store the context which you can segment into various segments based upon context. The vector database solution is better tailored to AI models being able to more accurately find then utilize the data since everything is stored into text. That also brings me to the next point of moving into a more text based format of communication. Instead of relying upon simple memory, it might be better to use a more robust memory storage solution or self-hosted AI model that has local memory available to it rather than just using the native N8N. Imagine the memory being appended to an up-to-date knowledge file (also stored in a vector database) to be more accurate in the info. Have you looked at any solutions that involve managing the data that way?
Hi Luca, thanks for sharing your setup — really interesting use case and solid architecture! You’re definitely on the right track with the orchestrator + sub-agent pattern, but I agree it can get brittle quickly as the system scales or when edge cases start popping up.
A few thoughts and suggestions:
1. Agent Orchestration & Routing:
To improve routing accuracy, consider implementing a lightweight intent classification layer before input hits the orchestrator — even a simple LLM prompt chain or embedding-based classifier can drastically improve routing accuracy by reducing ambiguous handoffs.
2. Sub-Agent Reliability:
Inconsistencies often stem from state mismanagement or context loss. One approach is to wrap each sub-agent in a middleware layer that handles input validation, retries, and fallbacks. This way, your orchestrator gets more predictable responses and can handle failures gracefully.
3. Context & Long-Term Memory (n8n / General):
In n8n, for context over time, you can use a combination of:
PostgreSQL/Firestore (or any external DB) to store vectorized summaries or structured logs
Memory nodes in n8n flows, coupled with embeddings to retrieve past tasks/notes dynamically
A scheduled agent that refreshes summary memory periodically from Notion to keep things current and reduce latency when answering context-based queries
4. Making It More Conversational & Intuitive:
Introduce “meta-awareness” prompts (e.g., “You are aware of Notion, Gmail, and GCal. Always check relevant context before responding.”) to improve agent alignment
Add tool usage logging + user feedback loops for continuous fine-tuning
Implement a clarification prompt mechanism when confidence is low to reduce errors from incorrect assumptions
5. Alternative Architectures:
You might want to look into architectures like:
ReAct-style agents (reasoning + acting loops)
Function-calling LLMs that dynamically select which “tool” (sub-agent) to call
Using frameworks like LangGraph or AutoGen if you’re open to Python — they offer more robust orchestration patterns with memory persistence
I’m also building an AI Agent like this. My orchestrator runs differently from yours in terms of workflow design. From my view, your long-term memory is too basic — you should look into using RAG or Knowledge Graphs to structure and retrieve context more effectively.
