Production Experience: Staging / Filter Failed / Long Lists

Questions
#1

First off. Really great tool.
Loving it as a self hosted integration solution.

I wanted to share some production experience. Perhaps someone has found a good solution.

Filter for failed runs:
In production I’m counting 45,000 executions. If an execution fails it’s not easy to find the failed execution unless with this workaround:

  • set up general alert email and include the execution-id in the mail.
  • Load workflow executions (which loads slowly given 45’000 runs)
  • open any execution
  • change the execution-id in the URL directly and load

A way to filter for failed executions or set date filters would be great.

This brings me to my 2nd question
Loading workflow executions is relatively slow and I guess it wont get better as the number of executions increases.
Is there a way to improve that?
Different backend db?

Dev / Staging / Production: Deployment via code.
I was missing a simple way to be able to deploy nodes like code to different environments (dev/staging/production).
There is an import function but it imports the nodes into a new workflow.
It’s also hard keep track of the workflow-id.

Once in production and you encounter issues, it’s very tempting to live adjust the workflow.
That would be ok, but I couldn’t find an easy way to “downcopy” the changes from prod to staging/dev.

Thanks.

Just wanted to share some experiences from prod.

PS: Oh if in doubt if a change has been applied to the live workflow. Restart the n8n service.

1 Like
#2

Thanks a lot @leonardlin for sharing your experience. It is very appreciated!

Yes, I can imagine that the Execution-List loads slow with 45k. Was actually never intended to have that many. The max I hoped for was like 1k. Normally the only ones that should be saved and stay around are the ones that did error and they should also be deleted once they got “fixed”. Is there a reason why you want to keep all of them around? Especially the successful ones? Because I would propose to set as a default (or at least in the workflow-settings) that successful executions should not be saved at all.

Improving how executions can be filtered is on the ToDo-List for a long time. So totally agree also there.

In a next step, it is actually planned to add an option to auto-delete past executions after X days.

About finding the failed execution. What you can do is to simply construct the deep-link to the failed execution already in your error-workflow. You can then save all the hassle you are currently going through and you can directly jump to it from the email.

Yes, support for multiple environments is sadly still missing as are many other important features like for example multi-tenancy and user-management. It will all improve over time but will sadly still take some time as the project is still very young. So it will come :wink: