I get it. AI is cool. AI agents are even cooler. An AI agent that reads your calendar and your emails and syncs your todo list with the open tasks you left on Jira is THE BEST.
But you know what’s not cool? Knowing that your data isn’t yours anymore. That’s the price you pay when you use third party LLMs in your workflows.
That, and COSTS. Be careful folks,they do ramp up fast. You start with a simple LLM call to check whether a data point is valid, then you build an agent that calls a couple of tools, and next thing you know your bill has skyrocketed. But it’s too late now, you’re already in too deep to pull out and go tell Martha from HR that she needs to go back to the way you used to work last month.
INTRODUCING: SELF-HOSTED MODELS
Self-hosting lets you cut your AI bill entirely, and finally get rid of that tingling feeling you get every time sensitive data passes through one of your automations.
So how do you do it? Personally, what I like to use is Ollama. It’s a pretty well known framework that lets you download and run open source models locally on your machine.
Ollama has been around for a while, it was already popular a couple of years ago, but back then consumer hardware wasn’t at the level where it could host decent models. And open source models weren’t at the level where you’d feel safe handing them any real work (for most users, at least).
So what changed? Open source models got better, consumer hardware got better, and now some of the newer models are genuinely safe to run on your own machine.
The reason I think this matters is that we’re moving fast in this space. If you’re trying to keep up with AI, you know that. You blink, and a new model is out, a new paradigm has shifted, and so on. Learning how to serve your own large language models now means you can bank on that knowledge later.
In n8n this is pretty straightforward. You connect your n8n to your Ollama instance, you select the model, and you’re good to go. To do that, head to Download Ollama on Linux, download the version compatible with your machine, run the setup, pull the model you picked and try running it.
Don’t panic if the fans on your PC spin up like crazy. That’s normal, inference is heavy on your CPU.
Keep in mind that if you want to try open models without sacrificing your machine, or if you don’t have a workstation powerful enough to run bigger models, you can still use cloud providers that run open source models for you. GreenPT, for example, has servers in Europe, so it’s GDPR compliant. Or you can use OpenRouter and pick an open source model there.
I’ve had a lot of success with open source models, especially in my work as an automation consultant, where I sometimes get clients with very particular needs.
One example: a client who needed to analyze a huge corpus of sales documents for water supplies. The documents were full of personal data, from the seller and from the customers, so they couldn’t afford to send anything to a third party model. And here’s the thing, and keep in mind this could be you: they didn’t need a powerful model. They needed a small model that was very good at one single thing.
In our case that model was IBM Granite, 7 billion parameters, small enough to run on a modest machine. So we went with a self-hosted n8n instance with Ollama on the side, and n8n calling Ollama to run inference on the documents.
The result? A slower analysis rate, one document every two to three minutes instead of one every ten seconds with a closed source model. But the inference was completely free, electricity aside, and it was entirely private. For them that meant the world, and for me that meant success.
So how are you running LLM workflows locally on self-hosted n8n? And what models are giving you the most bang for your buc… oh wait, it’s free.