I’m setting up an evaluation flow using Trigger, Set Output, and Set Metrics nodes. I configured the Set Metrics node with the metric Correctness (AI-based) and filled in all the required fields: Expected Answer, Actual Answer, Prompt, and AI Model.
However, when I run the evaluation, I get the error:
“Set Metrics node is not configured.”
Using the exact same node but configuring it with a String Similarity metric instead results in a successful evaluation run.
What is the error message (if any)?
Set Metrics node configuration resulting in an error during runs:
Set Metrics node configuration resulting in a successful run:
##Alternative solution
After posting the first version of this topic, I come up with an alternative solution taken from this template Evaluation metric example: Correctness (judged by AI) | n8n workflow template
The alternative consist in adding an Open AI node that fullfill the role of evaluating the correctness of the output, using the exact same prompt template thet shows up in the set metrics node when select the metric option correctness (ai-based). Then connect a set metric node configured with “custom” metric, where the value of the metric is the correctness score outputs from the former open AI node.
This is a snap of the alternative eval flow
This is the prompt and user prompts in the OpenAI nodes
This the configurarion of the set metric node
Information on your n8n setup
- n8n version: 1.103.2
- Database (default: SQLite): N/A
- **n8n EXECUTIONS_PROCESS setting (default: own, main): n/a
- **Running n8n via (Docker, npm, n8n cloud, desktop app): cloud
- **Operating system: windows 10