Autosave for workflows

More often than once I have been forgetting to save the state of the project and then my nice process or the logic I entered was partially or fully lost.

I was wondering whether an autosave (be it temporary or similar) would make sense to prevent that.

Possible implementation scenario:

  • On new Workflow creation directly ask the user for name (and store it with the id)
  • Register a onNodeAttributeChange and fire an event
  • Listen to that event and automatically save the workflow

Yes, that would be great to have. But it can not be an actual auto-save. The problem is when an active workflow gets changed. If somebody would work on it, it means that all changes are directly live and would so very likely break current executions. So it should probably best save it only in the local storage of the browser. There it can save as much as it wants without breaking anything. And if the user leaves and comes back it could check if one is saved in the local storage. If one is available it could simply ask the user if he wants to load the one from the server or the one from local storage.

so is working together on one workflow at the same time a use-case or would it rather make sense to display a warning, that it is read-only because someone else is editing it. Because in case both save it you’d run into the same problem anyways. So imho if you have multi-user workflow editing in mind, n8n would need some kind of workflow-based or node-based locking, which then again would make it possible to centrally auto-save. What are your thoughts on this?

It would for sure be nicer and very helpful if multiple people could work on the same workflow at the same time and it would update it live. Would have been handy on multiple occasions for me already. But for sure would be much more complicated than implementing a simple locking. But even with a locking, there would need to be something in place to be able to overwrite it. Because how do we know that somebody is working on it? If somebody did open it and has an active connection for example. But what if he has internet connection issues or stops working and his computer goes in sleep mode? Would it then be blocked indefinitely or unblock after X minutes? With live-updating it would be similar but even more complicated if they get out of sync.

That are all issues a proper solution can be found for sure, however, not sure how much time it would take especially considering what we gain. Think maybe something like saving history/revisions would be more important. That would be very helpful and important by itself and could also help here. At least with people overwriting the work of others. That would also be much easier to implement and offer a bigger gain.

With the available resources, I have to look at how to improve the project the most with the least resources. So like the low hanging fruit.

Interested to hear what you think about that.

I also favor the “quick-wins”.
Which is why I asked whether we’d need to assume that it is edited by multiple users at the same time.

What I like about approaching it by saving revisions is:

  • your JSON-data structure of the workflow is easily diffable, hence if a diff exists a revision can be created

What I find difficult:

  • how would you synchronize between users?

In Git we can merge conflicts because it is text-based. This is extremely difficult in a graphical interface. But storing revisions already improves things. So it doesn’t solve the sync-problem between multiple users, but with storing each change, it’s at least possible to trace back stuff.

Ah yes, you are right it would not solve the sync problem. But the main goal here would be actually to first not lose any work anybody did because it got over saved by somebody else.

In the beginning, it would also not have to display anything. Just the different times it got saved. As a second step displaying what changed between revisions should not be too hard. The Workflow-JSON format is quite simple and t displaying the differences (what nodes got added/remove, what parameters changed, …) in a reasonably nice fashion should be doable.

This could be solved by separating save and live.

The live version is immutable. When someone starts editing this will be saved - but does not immediately go live.
A little indicator could show where one is looking at the live version. Then there should be a button to go live or to roll back.

But I am not sure it really is feasible as the software architecture would need to support sharing trigger events across multiple instances.

Once proper versioning is in place it would be possible to simply define which one of the versions should be the production one. Then people can then create as many versions as they want without interfering with the production one and would be able to easily up and downgrade them.

3 Likes

I lost a few workflows as well. I think the user perception is that since there is no save button before execution, you forget that you have to save before you leave, —because you just ran it and everything worked!

1 Like

Sorry to hear @levensailor! Yes, can understand that. If you are interested in that feature please upvote it, that we know what is important for the community and what not.

Lost a workflow today because I hit the wrong shortcut in Safari that closed all but the current tab (which was not n8n). If there is a draft workflow open, could we at least get a standard “you’re navigating away from a site with entered data” warning?

So, here is a crazy idea. How difficult would it be to implement an option at the workflow level that would let you choose if you want to autosave the workflow before it executed? It could be a trinary setting with the following options:

  • Save before running
  • Don’t save before running
  • Prompt to save before running

Thoughts?

2 Likes

What I see in this post and requests simply defines a “Staging” environment / behavior in the n8n solution.

Those below could be a long list of tasks and effort to do, but will definitely be a level up in the development experience.

  • Setting up maker / checker roles to the builder interface. Instead of a single global user, developer and deploy op roles added.
  • Workflows now follows a staging pipeline (test, staging and prod).
  • Interface preferences can be as suggested: Auto-save, Save Only with a Button and Prompt with an " Are you sure?" message at all times when navigating out.
  • Similar to the webhook url structure, we can see workflow-test, workflow-staging and workflow/ entry points in the deployed workflows.
  • test/staging and prod workflows could use different thread pools to isolate development mistakes.
  • Bonus: pushing the json representation of workflows to a github account could be really good.

Regards

2 Likes

With [email protected] we did the first step by warning in cases user move away from a workflow with unsaved changes like press backspace, opening another workflow, …

2 Likes

Thanks for all your hard work on this devs!! I also had an issue with pasting an expression in the editor and the whole instance crashed causing me to lose my work. Looking forward to seeing any automatic implementation of saving automatically, either change based, or time based! Thanks!

I see this issue has been open for a while now and I’ve lost some work again!

Can we get a quick hack together even just saving the workflow to the file system?

An idea for a quick and dirty hack would be:
If the workflow is open then it would simply save it to a json file every min

workflowID_user_datetime.json

and have it delete anything older then an hour. Basically a workflow log with the code in it.

Solves the problem of multiple users, solves the problem of having the ability to recover data and should be pretty easy to implement without affecting other things:)

A simple backup occasional automatic backup of current work to local storage or to a user-backup on the server is the easiest win in my opinion, and an important feature. A simple refresh of the tab or connection issue can mean hours of lost work.

As far as the system is concerned the backup is simply a copy of the workflow so there are no issues unintentionally saving or overwriting someone else’s work. This is a common and safe approach - like a text editor, auto-saving a copy of your work occasionally to a backup folder.

If a backup is found for a workflow that is newer (or different) than the last production version a notification can be shown for the user to review them. But even that is not necessary, even if I had to go to a menu and browse through my recovery files that would be fine.

+100 for this, just lost 35 mins of work, because of an out-of-memory error that crashed the page on the flow I was working on and thus lost all of the work because I wasn’t manually remembering to save every few mins :frowning:

1 Like

Hmmm. Takeaways.

  1. You cannot just autosave because save = publish in the current setup.
  2. It would be cool if a workflow could exist in a draft state, if it did then changes could be made on the draft, saved on the draft, or autosaved, and those changes would not affect live code.
  3. If functionality were added to support a draft state, then that draft could be synced up between multiple sessions allowing multiple people to work on one workflow.
    3b. The use case described above more simply works when trying to describe the experience of working on one workflow and then accidentally opening that workflow in a new tab or on a different computer you own. How awesome would it be for changes made in one session to reflect through to the other session?

In the past I know I have lost lots of work because I accidentally opened a workflow in multiple browsers.

1 Like

Something is happening in the background that blows out memory and pumps the CPU when it really should jsut be at idle, if anyone has an idea of how to dig deep into whats going on it would be nice. I never had these issues until a few months ago