I have a workflow that involve Basic LLM Chain with Structured Output. Sometimes the output from LLM is not exactly the same as the structured output, which will cause the error Model output doesn't fit required format. This is expected due to the undeterministic nature of LLM.
My question is can I add retry for Basic LLM Chain? Or should I choose the AI Agent node because I think that node allow few iterations with the max_iterations parameter.
Addtional Info:
Basic LLM Chain node version 1.7 (Latest)
For Basic LLM Chain, AI Agent and AI Tools, its no need to use retry on fail. I’ve experience with that and the problem is around with your prompt or your Structured output parser (mostly).
I have a tips with the structured output parser, you can use Generate from JSON Example, and paste the exact JSON shape you expect from the Basic LLM Chain. This significantly improves consistency.
hey @fahmiiireza , thanks for your reply! That’s what I did, instructing the LLM to output the result in JSON format. However, it will still output the result in the wrong format occasionally. Hence a error handling and retry mechanism is needed.
This is how I instruct the LLM: You must return the result in JSON format with the following structure: { “title”: “string”, “summary”: “string”, “confidence_score”: "number" }
Okay, that’s good, but I suggest giving an example. So, something like this:
At your prompt, use like this:
You must return the result in JSON format with the following structure:
{
“title”: “Mouse”,
“summary”: “This mouse will help other people instead of using a mousepad”,
“confidence_score”: “98”
}
In the structured output parser, choose Generate from JSON Example and then input something like this:
{
“title”: “Mouse”,
“summary”: “This mouse will help other people instead of using a mousepad”,
“confidence_score”: “98”
}
That should work fine. Just let me know if you encounter any further issues.
