Issue with collecting metrics from the worker and webhook n8n instances?

[RobertR]

Describe the problem/error/question

Have a n8n split out into the three processes for the web, worker, and webhook. Collecting metrics from the web instance works however the webhook and worker doesn’t. Does n8n support collecting metrics from all three?

What is the error message (if any)?

2023-12-09T00:31:38.692Z warn internal/transaction.go:123 Failed to scrape Prometheus endpoint {"kind": "receiver", "name": "prometheus", "data_type": "metrics", "scrape_timestamp": 1702081898687, "target_labels": "{__name__=\"up\", instance=\"localhost:5678\", job=\"prometheus\"}"}

Information on your n8n setup

• n8n version: 0.233.0
• Database (default: SQLite): external postgres
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):. Docker
• Operating system: Linux

[MutedJam]

Hi @RobertR, welcome to the community!

You’re quite right, the /metrics endpoint is not available on the workers or webhook processors (this also goes for more recent n8n versions than your current version 0.233.0). I believe this is intentional, but perhaps @krynble can confirm this with certainty?

For a simple health check you could consider using the /healthz endpoint instead which is available on workers and webhook processors as well. It would, however, not provide full metrics.

[RobertR]

Thanks @MutedJam, @krima can you confirm and if so, is there a reason? If not, is there a plan to implement it on all n8n containers running as a webhook or worker service?

[krima]

Hi @RobertR,

I think you referenced me by mistake. I think you meant to include @krynble.

[krynble]

Indeed workers and webhook processes don’t support the /metrics endpoint - the metrics it exposes and collect makes sense when running in non-queue mode.

We have plans to improve observability and monitoring of queue mode but it requires collecting and aggregating metrics from multiple services, which require careful planning.

We have plans to make improvements to reliability and scalability of n8n and i’ll be adding this topic to our list of items to prioritize.