Hi folks, I’ve been working on this testing tool for n8n workflows and would greatly appreciate your feedback on what’s correct and incorrect.
The tool was born out of my frustration when clicking Execute Workflow and seeing the happy path pass, and assuming everything was fine. This can make sense if the workflow has no logic, but as soon as there’s an IF or Switch, a loop, or error paths, it’s not that useful.
Some of the issues that bit me in production were
• A value being unexpectedly null
• An error output going nowhere
• A Postgres node killing the whole run due to a deadlock
The tool is two fold.
flowproof lint some-workflow.json checks an exported workflow for possible issues without executing it. It doesn’t require n8n to be running, or any credentials. It checks for unreachable nodes, error outputs that go nowhere, leftover pinData, unguarded expressions such as $json.body.user.profile.email, hardcoded tokens/local URLs, and nodes that can fail without an error path.
The second, and more interesting part is
flowproof coverage some-workflow.json
which looks at the IF and Switch conditions and tries to find which input would cause each branch to be taken. To do that, it replaces external nodes with fixtures, executing no actual API calls . It then runs the generated tests and reports which branches were actually reached
For example, the sample workflow in the repo has 7/10 possible branches exercised. For the remaining 3, FlowProof does not say they failed, but that the Switch was using output keys to choose the branch, and therefore there was insufficient information to generate an input.
I also try to be precise about the provenance of each piece of information.
If an IF branch was taken, it’s marked as observed. For loops, n8n does not expose the number of items, so FlowProof has to infer it from the execution data. This is marked as inferred rather than observed.
While writing this tool I’ve had one major false positive, which taught me to be careful. Every LangChain sub-node was marked as unreachable, because the connections used the ai_ fields rather than the main ones. This was easy to fix, but the fact that it happened means there are probably more subtle n8n-specific cases I’m not handling.
This is why I’m asking for your help. If you try this tool and it reports something obviously wrong, please let me know! You don’t have to share the worflow, but knowing the node type and what FlowProof wanted to say would already help.
Repo/install
It’s MIT licensed, doesn’t send any data anywhere, and doesn’t require an account. The only caveat is that while linting works from the JSON, the coverage tests need a local n8n instance to be run.
Finally, this is not really about testing AI workflows. n8n’s own Evaluations feature is much better at deciding whether an LLM’s answer is good. What I’m really interested in are regular business workflows, and whether their author actually tested all the possible failure scenarios. category for this