"n8n import:workflow --separate" does not import all .json files

Describe the issue/error/question

I have built 24 workflows, and I exported all of them with n8n export:credentials --backup --output=backups/latest/, which correctly results in 24 json files.
Then I start a new n8n container (with an empty database) and try to import the same files with n8n import:workflow --separate --output=backups/latest/.
Even though I get a message saying “Successfully imported 24 workflows” I only see 16 of them in my db and the n8n UI. When I run the above command (n8n import:workflow) a second time the remaining 12 workflows appear.
Is this a bug with the cli or am I missing something?

Thanks in advance!!

What is the error message (if any)?

No error messages, the message is: “Successfully imported 24 workflows”

Information on your n8n setup

  • n8n version: 0.204.0
  • Database you’re using (default: SQLite): postgres
  • Running n8n with the execution process [own(default), main]: own
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: docker

Hi @Georgina, welcome to the community!

I am sorry you’re having trouble. I just exported 85 workflows from my own production n8n instance using docker exec -it -u node n8n-n8n-1 n8n export:workflow --backup --output=/storage/export/ and also got the Successfully exported 85 workflows. message as expected.

I have then started a new n8n instance, imported all workflows and retrieved them through n8n’s REST API - the count was identical, all 85 workflows were imported.

So unfortunately, I was not able to reproduce your problem so far :slightly_frowning_face:. Could you share example workflow files that won’t import as they should using the CLI? These workflows won’t have to actually do anything, so anything confidential (such as URLs or auth information) can be redacted or removed, but it’d be great to see if there might be any duplicate IDs or anything else that would cause this problem.

Thank you!

Hi @MutedJam ,

Thanks for the reply.

I was able to reproduce the problem with a rather simple setup, I am describing the steps here:

  1. Start n8n with postgres
  2. Create 10 workflows with a single activation node each.
  3. Export the workflows with docker exec -u node -it n8n_n8n_1 n8n export:workflow --backup --output=/home/node/workflows
  4. Close the n8n (and the db), remove all mounted volumes and restart (with a clean db)
  5. Import the workflows with docker exec -u node -it n8n_n8n_1 n8n import:workflow --separate --input=/home/node/workflows
  6. Inspect the workflow-entity table in the db and see that only 9 workflows exist :slight_smile:

My assumption is that the cli tool imports the workflows in alphabetic order, so when it tries to import workflow 10.json the id (10) is not available, so it imports the workflow with id 2 and when 2.json is imported overwrites the previous one. In this setup we loose only 1 workflow, but with more workflows we end up loosing more.

I hope this helps,
Georgina.

1 Like

This is super helpful, thank you so much! I was able to reproduce the problem and have added it to our engineering backlog for a fix.

2 Likes

Hey, first of all thank you really, really much for the Awesome tool you are creating and updating every day, and thank you for the support you’re constantly giving to your community.

I came here just to report on this issue too. This also doesn’t only affect Workflows, it happens with credentials too. Is this, by any chance, already fixed in a newer version of N8N?

We couldn’t update and test it yet, since it seems that text and functions inside expressions were broken for us (e.g. instead of “Hello Martin” the Gmail-Node would send Hello {{$json[“Name”]}} after the update to the latest version, which isn’t really ideal) and large “texts” (HTML-Mails with some dynamic content inside the expression field of the Node) got also cut off (both happened in the Gmail-Node). Was this a bug, or an intended change, and is threre another way to migrate to postgres, after installing N8N with postgres from the beginning but forgetting to set the DB_TYPE=postgresdb variable. I am an idiot sometimes :smiling_face_with_tear:

EDIT: Double Importing looks like it caused some duplicates, and since a lot of our >350 workflows have webhook triggers, and nodes to execute another workflow by id, it would be very painful to manually go through every workflow again and decide which one to keep, to update or to change.

Hi @one_juru, thanks so much for sharing your details. I’ve just checked the bug ticket we have created for this issue, but it seems this is still awaiting prioritization I am afraid. Perhaps @sirdavidoff can confirm when this functionality will be looked at?

As a workaround, you could consider transferring your database tables instead of using the CLI. I did this once years ago (before I joined n8n and before the import/export CLI command existed) and have described the steps taken over here. They are most likely no longer be 100% accurate after all this time but might still provide a starting point.

As for the Gmail node causing trouble in more recent versions of n8n perhaps you can share an example workflow using which this can be reproduced in a new forum topic? I just tried using the expression you have described but got the expected message:

n8n:

image

Gmail:

image

Hey @MutedJam,

Thank you for the quick reply. I will definitly take a look at migrating to postgres this way :slight_smile:.

I tried updating again today, but again with no luck. I updated from Version 0.204.0 to the latest version on dockerhub and the bug persists. The content, which previously was in the “HTML-Message” field of the gmail node got cut (the node only displayed 3753 instead of the 24529 characters after the update). Since this is a production enviroment, I didn’t want to test it any further, and reverted back to 0.024.0, which revealed the entire content of the node again. If you want to see if you’re able to reproduce the behaviour, you can test it with the following workflow:

I initially wanted to post it here, but because of the content of the gmail node (the entire html-mail) beeing to large, I coudn’t post the workflow here. However, if you still want to take a look at it, here is a link to it on my Nextcloud: JONE.IO CLOUD

I’ve done some further testing and can now say that this bug occurs while updating from 0.204.0 to 0.205.0 and that copying the workflow into a newer version of n8n works without any problems, just updating causes issues of content missing. Thank you for your time :slight_smile:

Hi @one_juru, I am sorry for the trouble. This sounds like a different problem than the failing workflow import tbh: [BUG] Not seeing complete information in expression editor

So you might want to check if your data is still shown as truncated in the current version 0.211.1 of n8n. If so, perhaps you can open a new topic with a reproducible example?

1 Like

Hi all, sorry for the late update. This was resolved by fix(core): Upsert credentials and workflows in the import:* commands by netroy · Pull Request #5231 · n8n-io/n8n · GitHub, but it seems the automated post here on the forum failed. @Georgina, in case you haven’t imported your real world data yet, could you re-test this using the latest version of n8n? Your test case worked fine for me using a PostgreSQL database.

Hi @MutedJam ,

Thanks for this fix! I can confirm that after the fix we can export/import our real workflows as expected.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.