Opensource LLMs to use

Questions
1

Hi, I recently added a private, selfhosted n8n next to my enterprise account (which exposes all possible LLMs). I had a feeling that its not gonna be easy with open models but did not expect such failure. So far on my 8gb vram:

  1. new thinking models like gemma3, deepseek r1 do not support tooling

  2. llama 3.1, 3.2 cant parse a JSON output

  3. qwen3 has some success but is not consistent

Is there any guideline that would save me the effort? Could you post what you use as general purpose llm?

1 Like
2

HI @Jurek_Kopacz Welcome!
The concept of opensource LLM is kind of too shiny, first there is no model good enough to parse data and use tool calls AND is small enough to run on a desk size machine, i have tried multiple models like GPT-OSS-120B and many known models like that but none of these are actually good to use even in basic tasks and they cost too much of computational power that using APIs is more economical, i recommend using Openrouter, and if you are worried about data security ofc that is why you got interested in Local LLM, you can give this a try:

1 Like