[n8n-workbench-syncer] Syncing local n8n workflows with remote host

yavetal · December 26, 2022, 9:46pm

I’ve made it for myself, but as i’ve seen some similar questions regarding this, I decided to share it.

The idea is to develop n8n workflows locally, and then deploy them to the host with as small effort as possible. I use private bitbucket repo as a storage, so I did not bother with creds encryption yet, but will deal with it some moment later on.

Feel free to give it a shot and share feedback\suggestions:

yavetal · December 28, 2022, 1:36pm

I’ve pushed v 0.2.0 - now it supports encryption and other minor improvements.

Now the flow is even easier (the setup is the same: npm i and fill the .env with your repo data):
On the local machine, run npm run export-push - that will, well, export and push workflow+creds to the private repo.
On the host (i use remote RDP), run npm run pull-import - that will pull workflows+creds and import to the host n8n
Then just npm run n8n

There are still minor issues (i want to make it cross-platform), but in overall its looking good

yavetal · December 30, 2022, 2:07pm

Just pushed 0.3.0 and tested the whole flow - now its looking good. Will play a bit more for a week and will release 1.0.0.

Now the flow is probably the simplest possible:

Upload the repo (with .env) to the host and run npm i on both local and host - one-time installation step
Do changes in local n8n
Run npm run ex
Switch to the host
Run npm run im
Run npm run n8n

Profit

yavetal · December 30, 2022, 9:08pm

Pushed 0.5.0 - now preserving the private git repo history (in previous versions it was overwritten).

Well, my todo and bugs lists for this project are clean now - have no idea what can be improved/fixed further

If anyone is up for it, please do a test run and share the feedback.

yavetal · January 13, 2023, 9:24pm

Just pushed 1.0.0.
The only change is i’ve added “npm run imr” which means “import”+“run”, just a shortcut.

in overall, i’m amazed how simple workflow is now: i do the change locally, run “npm run ex”, watch how the changes are exported and are pushed to the repo, then i switch to the host, run “npm run im” and “npm run n8n” (tbh i didnt test “imr” yet - bad practice, but no one watches ) - changes are pulled and imported, and that’s freaking it!

Okay, just tested “imr” - works like a charm. For the continous workflow, “imr” is pretty useful, because it just takes you to stop the current n8n (Cmd+c) on the host, press up button to receive the previous “imr” command, and press enter to update and start n8n.

Just 2 things to note:

Sometimes there’s an error “resource is busy” during the n8n run. Just re-run the command, and it fixes the issue.
Workflows statuses are exported as well. So before the export, it’s important to enable the ones you wish to run on the host. Or enable them manually on the host afterwards, whatever you prefer.

yavetal · April 1, 2023, 2:46pm

Released 1.1.0.

Moved all n8n env variables from package.json to the .env. It’s a win-win, because package.json is looking much cleaner, also all envs are stored in a single place: .env file. Ah, also dropped some dependencies. So technically, its win-win-win

yavetal · June 27, 2023, 6:16pm

It seems like this package is extremely popular

Anyways, just pushed version 1.3.0 (version 1.2.0 had some minor changes).

I’ve worked with this script for a while (and i found it extremely useful!) and noted something i can automate even more. I have a difference between the local workflows and the host: i need all local workflows to be disabled (i run them manually during the development), and some of the deployed workflows to be enabled (because they are finished and prod-ready).
At first, i used to enable them manually, right after the deployment. So I’ve added “[A]” to the name of the workflow, so i can easily find the ones i want to enable. Recently, i’ve got an idea to enable the workflows that starts with “[A]” just before starting the n8n on the host automatically.
Today i had some time, so i’ve implemented it. The logic is pretty straightforward, and it is fired by the new command “imar” (IMport-Activate-Run).

github.com

Systemify-me/n8n-workbench-syncer/blob/main/scripts/activate.js

const shell = require('shelljs'),
    fs = require('fs'),
    path = require('path');

const workflowsPath = path.join(__dirname, '..', 'project', 'workflows'),
    files = fs.readdirSync(workflowsPath);

files.forEach(function (file) {
    const fullFilename = path.join(workflowsPath, file);
    const content = fs.readFileSync(fullFilename, 'utf8');
    const jsonContent = JSON.parse(content);
    const workflowName = jsonContent.name;

    jsonContent.active = workflowName.startsWith("[A]");

    fs.writeFileSync(fullFilename, JSON.stringify(jsonContent, null, 2), 'utf8');

    console.log(file + " - '" + workflowName + "' - Active is set to: " + jsonContent.active); 
});

yavetal · June 27, 2023, 6:42pm

Pushed 1.3.1

Realized the obvious thing: the sequence should be “pull-activate-import-run”. Fixed that, but i guess i’ll keep the “imar” naming.

yavetal · July 17, 2023, 1:51pm

Version 1.3.2
Updated the n8n to the current latest (0.236.2)
Updated the action script, so now it runs n8n update:workflow --id= --active=true for each workflow that starts with “[A]” (see previous message) right after the import.

yavetal · July 17, 2023, 1:54pm

@jan How can i update the original post? I re-read it, and find the description of this package has a lot of space for improvement and i’d like to edit it, if possible.

Jon · July 17, 2023, 2:35pm

@yavetal is there not an edit icon?

Try not to tag specific members of the team, We see when there are new posts and if you want an answer to a question it is better to just ask it that way the rest of us won’t wait to see if the tagged person replies.

yavetal · July 17, 2023, 4:18pm

Okay, sure, thanks!

Yes, there’s no edit icon for older posts.
E.g. there’s edit icon for this post (created 20 days ago), and no edit icon for this one and all earlier posts (created 3.5+ months ago).

Here’s edit button:

No edit button:

Jon · July 18, 2023, 9:34am

Ah we must lock it out after a while so I guess you would need to add another post instead as it is probably a system wide setting.

The good news is we have this exact thing coming out soon so you can make workflows on one instance, then push the workflow, credential and variables to a git repo then on your main instance pull them in.