Support for LLama.cpp

Feature Requests
1

Hi, I think it is a good idea to add support for llama.cpp
I myself use this backend for hosting llm models.
Especially that it allows you to improve performance on not necessarily high hardware.

Integration will allow wider use of n8n for all

2

Yes, that would be great, but you can still use llama.cpp in server mode.
Start llama.cpp in server mode with a model loaded.
Connect a OpenAI LLM model node and use the server url (local - started by llama.cpp) as the base url and you should be able to use it,

1 Like
3

@StackCV Thanks for your reply!

i am running into the same issue but cant seem to get your fix working. Would you care to elaborate on it?
My use case is using the AI agent and use my llama.cpp server as the chat model. Is there a way?

Thanks in advance!

1 Like