A data table has 4 rows with a simulated user question and an expected output value
I am forcing structured output
When I click execute workflow, I can see the execution for each of the 4 rows, and values are added to the data table. This works as expected.
After the logging of the output to the data table, you can see that I am using a Set Metrics node. This is a custom metric that is the length of the intermediate steps. Again, a silly test, but I wanted to see the data, as suggested by the node (shown below)
I have run the evaluation path of the workflow a few times, yet nothing shows.
Perhaps I am missing something obvious, but why wouldn’t I see data in this tab? From the page here: Use metrics to measure quality | Build | n8n Docs there is a reference to run test, but that is also not shown.
Any help and/or guidance will be much appreciated.
Executing from the canvas doesn’t count as an evaluation run. Set Metrics only records results when the run is started from the Evaluations tab with Run Test, which is why your data table fills in fine but the tab stays empty. The workflow is doing the right thing, it’s just not being logged as a test.
As for Run Test not appearing, the first thing I’d check is your plan. Metric based evaluation is Pro and Enterprise, and community or Starter accounts get it for one workflow only. If you’ve already used evaluations on a different workflow, that would explain the button not being there on this one.
One other thing worth knowing once it’s running: you can put the metric logic behind a Check If Evaluating node so it only runs during evaluations and doesn’t add cost to production executions.
Thanks @Hammad_gaming . I wasn’t aware of that; can you point to where that is documented?
I am in the lowest paying tier, and didn’t realize that was a limitation (the single workflow). How would I even find if/where I may attempted to associate a workflow to evaluations to show the “Run test” flow.
It’s in the info box at the top of that same page, above the first heading. Pro and Enterprise, with community and Starter getting it for a single workflow.
To be straight though, the docs state the limit but don’t say the Run Test button hides once you’re over it. That bit was me joining dots, so treat it as a lead.
I don’t know of anywhere in the UI that shows which workflow already claimed it, which is a fair gap. Look for any other workflow with an Evaluation Trigger node in it.
If there isn’t one, my theory’s wrong and it’s worth raising with support.
Logging in case others stumble across this. I ran a self-hosted n8n instance, added a community key, and I can see that custom metrics are blocked on the 2nd workflow.
The version that is running is 2.33.7. The version on my (lowest) paid account is 2.33.4. I don’t know if the slight minor version difference would be culprit, but it is a source of variation.
On the first workflow, I did see the guided checklist above and was able to run in the editor. I will ask additional questions in other threads.
Nice, that message confirms it and states it better than the docs do. Thanks for coming back with the screenshot, that’ll help the next person who hits this.
One detail worth noting from the wording: it says delete the workflow with existing evaluation runs. So the slot is taken by a workflow that has actually run evaluations, not just one containing the node. That’s a better thing to search for than what I suggested earlier.
I wouldn’t worry about 2.33.7 vs 2.33.4. This is a plan gate rather than anything version specific, so the minor difference shouldn’t come into it.
Hi @datanerdery3
The block lands on step 3 of that checklist, which is marked Optional, so Skip next to See plans lets you continue on the second workflow. You still get the dataset run and the outputs written back to the data table, you just don’t get scores in the Evaluations tab. Only the metric scoring is capped at one workflow, light evaluation itself is available on every registered and paid plan.
