N8n queue mode with worker

Describe the issue/error/question

Hi,

First of all, as I’m new and have discovered n8n since a few days, I would like to congratulate the n8n team for this greal tool. The approach is really interesting and is kind of: exactly the tool we need :slight_smile:

We’ve started using n8n on our preprod environment to test and start building data workflows. Our setup is:

  • install on debian server with pm2 + apache reverse proxy. using mysql database.
  • configured queue mode with the worker on another debian server. Fine too.

My questions are multiple:

  • you’re talking a lot about docker. Would you recommand using Docker instead of manual / old fashioned installation?

  • for the worker, what do you recommand to launch it as a daemon + in case of failure to restart it, etc. There is no info on this in the documentation. And same as first question, would you also recommand docker to setup the worker(s) instance(s) ?

  • Finally what do you recommend to monitor all of this?

Thanks again for this great tool!

Jean

Information on your n8n setup

  • n8n version: 0.181.2
  • Database you’re using (default: SQLite): Mysql
  • Running n8n with the execution process [own(default), main]: main
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: debian hosted + PM2

Hi @Jean_Lamy, welcome to the community!

Glad to hear you like n8n so far. As your your questions:

you’re talking a lot about docker. Would you recommand using Docker instead of manual / old fashioned installation?

That’s really a matter of personal preference. Docker takes a lot of the hassle out of having several different applications on a single webserver. Each container is nicely isolated from each other. If you browse the forums you can for example see occasional permissions problems with Node.js or npm which wouldn’t happen using Docker.

That said, if your machine is running fine there is no actual need to go down the docker route. Especially if you’re familiar with pm2 it might be much easier and faster for you to manage your n8n setup instead of learning a new tool.

for the worker, what do you recommand to launch it as a daemon + in case of failure to restart it, etc. There is no info on this in the documentation. And same as first question, would you also recommand docker to setup the worker(s) instance(s) ?

I suppose the answer from above applies to this question too. Docker can restart failed containers automatically (by setting the corresponding restart policy) and I find using these policies to be the easiest way, but it’s certainly not the only way. Other approaches are totally fine, depending on what you feel comfortable with.

Finally what do you recommend to monitor all of this?

I’ve used Graylog in the past as a central place to manage logs. n8n also comes with a Prometheus metrics endpoint and you could visualize these in tools like Grafana. There is also an open feature request for a dashboard which you might want to leave your vote on: A nice dashboard for the home screen

So tl;dr, I am a fan of docker (and docker compose because it uses easy to read YAML-files to describe complex application environments). But my suggestion would be to use whatever you’re comfortable with. Nothing wrong with bare metal (and it often makes things a bit easier).

1 Like