Gemini 2.0 how to pass structured output

gustaff-weldon · February 24, 2025, 4:34pm

Describe the problem/error/question\

Hi, I’m trying to enforce a structured json format for Gemini 2.0 Flash via specifying response schema.

I cannot find an option to pass responseSchema to api call in Google Gemini Chat Model node

When I try to connect structured output parser via AI Agent node, the whole process fails.

According to google docs Generate structured output with the Gemini API | Google AI for Developers there is support on API to pass schema as an option via responseSchema

What is the error message (if any)?

Bad request - please check your parameters
GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:streamGenerateContent?alt=sse: [400 Bad Request] 
Invalid JSON payload received. 
Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[0].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[0].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[1].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[2].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[3].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[4].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[5].value': Proto field is not repeating, cannot start list. 
Invalid JSON payload received. Unknown name "type" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[6].value': Proto field is not repeating, cannot start list. [{"@type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":[{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[0].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[0].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[0].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[0].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[1].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[1].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[2].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[2].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[3].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[3].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[4].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[4].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[5].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[5].value': Proto field is not repeating, cannot start list."},{"field":"tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[6].value","description":"
Invalid JSON payload received. Unknown name \"type\" at 'tools[0].function_declarations[0].parameters.properties[0].value.properties[1].value.items.properties[6].value': Proto field is not repeating, cannot start list."}]}]

Please share your workflow

I cannot share the code of the full workflow, I can create a sample one if needed

The schema:

{
  "type": "object",
  "properties": {
    "bill_of_lading_number": {
      "type": "string",
      "nullable": true
    },
    "containers": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "pickup_location": {
            "type": "string",
            "nullable": true
          },
          "container_number": {
            "type": "string",
            "nullable": true
          },
          "container_type": {
            "type": "string",
            "nullable": true
          },
          "pickup_reference": {
            "type": "string",
            "nullable": true
          },
          "drop_off_location": {
            "type": "string",
            "nullable": true
          },
          "drop_off_reference": {
            "type": "string",
            "nullable": true
          },
          "closing_date": {
            "type": "string",
            "nullable": true
          }
        },
        "required": [
          "pickup_location",
          "container_number",
          "container_type",
          "pickup_reference",
          "drop_off_location",
          "drop_off_reference",
          "closing_date"
        ]
      }
    }
  },
  "required": ["bill_of_lading_number", "containers"]
}

Share the output returned by the last node

The output is the error above

Information on your n8n setup

n8n version: 1.78.1
Database (default: SQLite): postgresql
n8n EXECUTIONS_PROCESS setting (default: own, main):
Running n8n via (Docker, npm, n8n cloud, desktop app): k8s
Operating system: Linux

jksr · February 24, 2025, 5:02pm

Hey @gustaff-weldon I guess this is just confusing schema stuff. This should work:

{
  "properties": {
    "bill_of_lading_number": {
      "description": "The bill of lading number for the shipment",
      "type": "string"
    },
    "containers": {
      "description": "List of containers in the shipment",
      "items": {
        "properties": {
          "pickup_location": {
            "description": "Location where the container is to be picked up",
            "type": "string"
          },
          "container_number": {
            "description": "Unique identifier for the container",
            "type": "string"
          },
          "container_type": {
            "description": "Type/size of the container",
            "type": "string"
          },
          "pickup_reference": {
            "description": "Reference number for pickup",
            "type": "string"
          },
          "drop_off_location": {
            "description": "Location where the container is to be delivered",
            "type": "string"
          },
          "drop_off_reference": {
            "description": "Reference number for drop-off",
            "type": "string"
          },
          "closing_date": {
            "description": "Closing date for the container",
            "type": "string"
          }
        },
        "required": [
          "pickup_location",
          "container_number",
          "container_type",
          "pickup_reference",
          "drop_off_location",
          "drop_off_reference",
          "closing_date"
        ],
        "type": "object"
      },
      "type": "array"
    }
  },
  "required": [
    "bill_of_lading_number",
    "containers"
  ],
  "type": "object"
}

gustaff-weldon · February 25, 2025, 8:18am

hey @jksr thanks, the error is gone in Gemini but the output parser returns nothing. The updated schema fails on OpenAI with an error:

I wonder what was confusing about that schema as it was a valid JSON API schema? I have noticed you dropped nullable and formatted it differently.

The previous schema with nullable worked fine in OpenAI and gave me the output, was not working with Gemini.

jksr · February 25, 2025, 8:49am

No, I meant that it’s confusing to me. I had similar issues with Gemini model before. However, it does work with both models for me. See attached:

Tested with a generated text:

SHIPPING NOTIFICATION
Bill of Lading: MEDU7654321

Container Details:
1) Container: CGMU1234567
   Type: 40ft High Cube
   Pick-up: Port of Rotterdam
   Pick-up Reference: ROT-2025-001
   Delivery: Hamburg Terminal
   Delivery Reference: HAM-2025-123
   Closing: 2025-03-15

2) Container: TRLU9876543
   Type: 20ft Standard
   Pick-up: Port of Rotterdam
   Pick-up Reference: ROT-2025-002
   Delivery: Berlin Depot
   Delivery Reference: BER-2025-456
   Closing: 2025-03-17

gustaff-weldon · February 25, 2025, 9:31am

I see

I think the reason it works is because you have all data, try this with the same schema (even with OpenAI):

SHIPPING NOTIFICATION
Bill of Lading: MEDU7654321

Container Details:
1) Container: CGMU1234567
   Type: 40ft High Cube
   Pick-up: Port of Rotterdam
   Delivery: Hamburg Terminal
   Delivery Reference: HAM-2025-123
   Closing: 2025-03-15

2) Container: TRLU9876543
   Type: 20ft Standard
   Pick-up: Port of Rotterdam
   Pick-up Reference: ROT-2025-002
   Delivery: Berlin Depot
   Delivery Reference: BER-2025-456

With my prompt this causes output parser to complain about the output not meeting the schema.

jksr · February 25, 2025, 9:54am

I mean, it makes sense because we say that all fields are required. There is even a test for it. So if everything is nullable, it doesn’t have to be required, if that makes sense.

github.com/n8n-io/n8n

packages/%40n8n/nodes-langchain/nodes/output_parser/OutputParserStructured/test/OutputParserStructured.node.test.ts

35c00d0c8

import { mock } from 'jest-mock-extended';
import { normalizeItems } from 'n8n-core';
import {
	jsonParse,
	type IExecuteFunctions,
	type INode,
	type IWorkflowDataProxyData,
} from 'n8n-workflow';

import type { N8nStructuredOutputParser } from '@utils/output_parsers/N8nStructuredOutputParser';

import { OutputParserStructured } from '../OutputParserStructured.node';

describe('OutputParserStructured', () => {
	let outputParser: OutputParserStructured;
	const thisArg = mock<IExecuteFunctions>({
		helpers: { normalizeItems },
	});
	const workflowDataProxy = mock<IWorkflowDataProxyData>({ $input: mock() });

This file has been truncated. show original

“should throw error for null values in non-nullable fields”

gustaff-weldon · February 25, 2025, 10:03am

Yeah, so my original question was that it seems with n8n I cannot really pass a schema to Gemini the way I can do with API calls:

const genAI = new GoogleGenerativeAI(process.env.API_KEY);

const schema = {
  description: "List of recipes",
  type: SchemaType.ARRAY,
  items: {
    type: SchemaType.OBJECT,
    properties: {
      recipeName: {
        type: SchemaType.STRING,
        description: "Name of the recipe",
        nullable: false,
      },
    },
    required: ["recipeName"],
  },
};

const model = genAI.getGenerativeModel({
  model: "gemini-1.5-pro",
  generationConfig: {
    responseMimeType: "application/json",
    responseSchema: schema,
  },
});

As nodes do not support it. And if you use structured output parser it comes with its own set of problems.

That’s a real shame.

openmymai · April 8, 2025, 8:06am

Gemini is so strict if you need the complicated JSON, you need to add Basic LLM Chain to generate output JSON for you.

system · July 7, 2025, 8:07am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.