Hello!
I am working to integrate prometheus in n8n to send stats from workflow nodes. I have n8n running as a pod on a kubernetes cluster. I am using node js promethues client library https://github.com/siimon/prom-client.
prom-client works by registering and recording metrics in a global registry by default which can be polled by prometheus server by an exposed endpoint. You can check example here prom-client/server.js at master · siimon/prom-client · GitHub. But this is not suitable for n8n use case because each workflow run works in a separate process. This means that all the metrics recorded in workflow execution will be saved in WorkflowRunnerProcess global registry instead of the main process that actually exposes the endpoint. To deal with this scenario prom-client provides an AggregatorRegistry implementation which only works if processes are spawned with cluster.spawn() or cluster.fork().
Proposed Solution:
I am planning to send these metrics through process messages to the parent process to record them which is similar to how workflow hooks work. This will require a lot of changes at my end. Thats why before proceeding in this direction I want to ask if there is a better way to do it?
