What is needed that binary data is saved multipe times?

Questions
1

Describe the problem/error/question

I have a workflow in n8n which calls couple of sub workflows. The main workflow starts with a form upload containing a binary file.

I store the files locally on the file System and the file always gets stored twice with different names. Is this related to the sub workflows and is there any way I can get rid of this?

Thanks for your information.

Information on your n8n setup

  • n8n version: 1.60.1
  • Database (default: SQLite): postgres
  • n8n EXECUTIONS_PROCESS setting (default: own, main): main
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • Operating system: Linux
  • Settings: N8N_DEFAULT_BINARY_DATA_MODE=filesystem
2

Hey @Andreas_Pointner , why the files are saved twice you probably need to check the logic of your workflows. It’s hard to comment here not knowing how your workflows were built and what they are doing.

As for the files retention, you can control it with data pruning.

3

Thanks it is quite a complex worklfow with lots of sub-flows.

I figured out that it is stored twice in

binaryData/workflows/EzOODKkjPqWODhAU/executions/[EXECUTION_NUMBER]

and one more time in

binaryData/workflows/EzOODKkjPqWODhAU/executions/temp/binary_data
where I have an additional .metadata file as well.

The first directory automatically gets cleaned up (as you mentioned). The temp directory does not.

So basically it is stored 3 times.

I’m working in queue mode maybe that is the reason?

Here are my ENV settings:

- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- QUEUE_HEALTH_CHECK_ACTIVE=true
- EXECUTIONS_DATA_PRUNE=true
- NODE_FUNCTION_ALLOW_EXTERNAL=lodash
- DOMAIN_NAME=${HOST}
- WEBHOOK_URL=https://${HOST}/
- NODE_ENV=production
- N8N_DEFAULT_BINARY_DATA_MODE=filesystem
- N8N_PAYLOAD_SIZE_MAX=512
- N8N_FORMDATA_FILE_SIZE_MAX=1024
4

It sounds like it.

If you’re using queue mode, keep this to default. n8n doesn’t support filesystem mode with queue mode.
Scaling binary data in n8n | n8n Docs

5

Ok, it looks like as if i’ve figured out the storage mechanism.

Everytime a binary data gets passed to another workflow the file is stored individually.

The cleanup mechanism works as well and deletes the duplicate data.

I changed my workflows to only pass the needed data to the sub-workflow without the binary data and then it looks way better.

Thanks for your support.

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.