Context:
We are an enterprise customer of n8n, currently operating on the Enterprise Plan. Our setup involves hundreds of workflows created and managed by multiple teams, with millions of workflow executions running in queue mode.
Problem Statement:
Recently, one misconfigured workflow generated 40–50 MB of execution data per run. This rapidly consumed our PostgreSQL storage, leading to database saturation and widespread workflow failures.
While we acknowledge that enabling auto-scaling for storage can provide temporary relief, it is not a sustainable solution; especially given the unpredictable nature of workflow data volume.
Proposed Solution:
To ensure stability, scalability, and cost efficiency, we propose that workflow execution data be stored in external object storage (e.g., Amazon S3) rather than in PostgreSQL.
Specifically:
-
PostgreSQL should retain only execution metadata (e.g., workflow ID, execution status, timestamps, duration, etc.).
-
S3 should store the actual execution data, including:
-
Node inputs and outputs (in JSON or text format)
-
Binary data (already supported by n8n’s existing functionality)
-
This architecture would:
-
Prevent database overload from large execution logs
-
Improve performance and reliability of the system
-
Allow for cost-effective, long-term data retention (e.g., 1 year of execution data in S3)
Rationale:
Given that multiple users contribute to workflow configuration, operational resilience should not rely solely on human discipline (e.g., asking users not to “break production”). Instead, the system should be designed to be fault-tolerant by default, capable of handling misconfigurations gracefully without jeopardizing platform stability.
My use case:
We embed N8N in our dashboard and let clients create workflows and attach them to AI Agents.
I think it would be beneficial to add this because:
Scaling and Data Retention.
Any resources to support this?
Are you willing to work on this?
Yes