Sharing something I built that might be useful here. It grades how reliable an n8n workflow is before it bites you in production.
You paste the workflow JSON or upload the .json file, and it gives back a reliability score out of 100 plus a plain-language list of weak spots. The four things it looks at: whether failures actually alert you or pass silent, whether external calls retry when a service hiccups, whether any secrets are hardcoded, and whether the workflow is documented and named well enough to maintain.
It runs entirely in memory, no signup or account connection, and the workflow and any secrets are discarded the second the grade is done. Nothing stored.
If you try it, I’d genuinely like to know what score you get and whether the issues it flags match what you already knew was shaky.
3 Likes
96 is a strong score, nice one, and thanks for actually running it. Curious what the one flagged thing was. Even on a clean workflow it’s usually something small, a node with no retry on an external call, or an alert path that would not actually fire if the run just stopped. Did the fix it pointed at match something you already half-knew was there?
And genuinely, if any of the wording felt off or confusing, tell me. I would rather hear it now while it is easy to change. The grader is a one-off snapshot for now, the thing I am building it toward is watching that flagged spot continuously so you hear about it if it ever trips, but the snapshot is the part I want to get right first.
1 Like
Tested it on a production AI agent workflow - the hardcoded secrets check and the silent-failure flag both fired correctly. One thing I’d love to see added: a check for HTTP Request nodes that have no timeout configured. Those are a common source of workflows hanging indefinitely in production without any alert.
3 Likes
This is exactly the feedback I was hoping for, thanks for running it on a real production agent. Good to know the secrets and silent-failure checks held up on something live.
The no-timeout HTTP node is a great call, and honestly a perfect example of the whole category: it does not error, it just hangs forever while everything looks fine, so nothing alerts. That belongs in the grader. Adding it to the list now. Out of curiosity, were the ones you saw hang usually external API calls, or internal sub-workflow hops? I want the check to flag the cases that actually bite, not just any node missing the setting.
1 Like
Tested this on a production AI agent workflow - solid grader. It caught the missing error handler on the main agent node which I had actually noticed but hadn’t fixed yet.
On your question about which nodes hang in practice: in my experience it’s almost always external API calls, not sub-workflow hops. Sub-workflow calls fail or complete within a predictable window. External HTTP calls with no timeout are the real problem - especially LLM APIs and webhook endpoints that queue requests but take 5+ minutes to respond. The check is most valuable when flagging HTTP Request nodes calling third-party APIs (anything in the URL that’s not the n8n instance itself).
One check worth adding: whether workflows that use the AI Agent node also have an “On Error” connection or a sibling Error Workflow configured - agent nodes can swallow errors silently and that’s easy to miss in static analysis.
1 Like
Appreciate you running it on something live, and it is useful to hear it surfaced the missing error handler. That “I knew but had not fixed it yet” nudge is exactly what the grader is for.
The third-party-URL heuristic is sharp: scoping the no-timeout flag to HTTP Request nodes hitting anything that is not the instance itself. That cleanly separates the calls that actually hang (LLM APIs, queueing webhooks that sit for minutes) from the predictable internal hops, so I will build it that way rather than flagging every node that is missing the setting.
The AI Agent On-Error / Error-Workflow check is a smart one too. Agent nodes swallowing errors quietly is a perfect static-analysis target, the kind of silent gap you never see until it bites. Adding both to the list. This makes the grader sharper, thank you.
That bridge is the whole point, and you have drawn it more precisely than most. The static grade catches the design risk before launch, but the drift you listed, a response shape that changes, a credential scope that narrows, an HTTP node that starts timing out, an AI tool returning empty while the agent talks over it, none of that shows up in the structure of the flow. It only shows up at runtime, from outside the run, which is a different check entirely.
Your five-point breakdown is close to how I think about it. The two that carry the most weight for agency work are the owner (who gets told, and does the client hear it or do you) and whether it lands in a client report. That last one is the difference between a tool you use and a thing you can put your name on in front of a client.
That runtime layer is exactly the part we built alongside the grader: the grader is the free static snapshot, and connecting a flow read-only gives you the ongoing check and a plain-language brief, with the client-report side for the agency case. Curious which of your five you would automate first, and which you would keep as a human judgment call.
