Chat and LLM Support for OpenAI Proxy Server?

dkindlund · February 20, 2024, 6:32pm

The idea is:

I would love to create an n8n workflow that uses my OpenAI Proxy Server, which supports the same API as standard OpenAI, but is reachable from a different URL/endpoint.

In Flowise, this functionality is supported in their ChatLocalAI node type.

My use case:

If I’m going to use n8n workflows that leverage LLMs, I need to be able to add in load-balancing, failover, and recovery support. Rather than bake that logic into each/every n8n workflow, I’d rather like to have that logic centralized in the OpenAI Proxy Server for consistency.

I think it would be beneficial to add this because:

It allows users to create n8n workflows that use LLMs in a more resilient manner.

Any resources to support this?

github.com

FlowiseAI/Flowise/blob/main/packages/components/nodes/chatmodels/ChatLocalAI/ChatLocalAI.ts

import { OpenAIChatInput, ChatOpenAI } from '@langchain/openai'
import { BaseCache } from '@langchain/core/caches'
import { BaseLLMParams } from '@langchain/core/language_models/llms'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'

class ChatLocalAI_ChatModels implements INode {
    label: string
    name: string
    version: number
    type: string
    icon: string
    category: string
    description: string
    baseClasses: string[]
    credential: INodeParams
    inputs: INodeParams[]

    constructor() {
        this.label = 'ChatLocalAI'

This file has been truncated. show original

Are you willing to work on this?

Yes. Happy to contribute testing/verification to this.

dkindlund · March 19, 2024, 8:57pm

Update: This is already supported in n8n. Just use the standard OpenAI Chat Model and point it to your LiteLLM Proxy server.

maugernot · November 26, 2024, 10:17pm

Hi, I’ve tried to do that, but how should I authenticate? because LiteLLM uses Autorization header and the OpenAI Chat Model only allow to use OpenAI credentials format

dkindlund · November 26, 2024, 10:35pm

LiteLLM supports the same credential format as OpenAI

TAN_YONG_SHENG · March 20, 2025, 8:28am

Although litellm is openai-compatible, I can’t find ways to add metadata to openai node for litellm and langfuse’s logging

For example, we need a LiteLLM node so that we could pass in extra metadata we want to log into self-hosted litellm proxy and langfuse

dkindlund · March 21, 2025, 12:19pm

Agreed. @bartv , is there any chance the devs working on the LLM nodes could add the ability to pass custom metadata through to the LLM provider? More and more LLM vendors are offering unique features that can only be enabled if we have the ability to pass custom metadata through from n8n.

Jehangir_Chaudhry · April 29, 2025, 12:49am

Any chance we could get an update if this will be possible @bartv, I am using LiteLLM similar to google ADK. Any hosting an OpenAI style /chat/completions fastapi through google Apigee. Apigee requires me to pass certain headers for authentication when using the openai sdk to interact. Can we add the ability to pass metadata or headers to OpenAI model node?