[N8N & BASEROW] Excessive Processing Delay in my n8n HR Data Insertion Process in Baserow

Micka_Rakotomalala · November 14, 2023, 11:01am

Hello Team,

I am encountering a major issue with my n8n process, which involves inserting data from HR source files into a Baserow database. The process includes extracting data from an Excel file, inserting it into Baserow, and initiating a process that generates an automatic report with qualitative and quantitative statistics on this data.

The challenge I’m facing is the excessive processing time of this workflow, particularly at the dataset level. Currently, the execution takes about two days, which is clearly abnormal. However, the structure of the table in Baserow is somewhat complex: it retrieves field names and value types, inserting each column as a separate row, with another pivot table to link rows and columns.

I have analyzed the n8n script and Baserow configuration, but I cannot identify the source of the problem. I suspect that the issue lies either in my platform or in the process itself.

Do you have any suggestions for optimizing this process and significantly reducing the execution time at the dataset level? Any expertise would be greatly appreciated.

Thank you in advance for your assistance!

Here are my configurations:

.env
QUEUE_BULL_REDIS_HOST=redis

COMPOSE_HTTP_TIMEOUT=3600

Set the logging level to ‘debug’
N8N_LOG_LEVEL=debug

Set log output to both console
N8N_LOG_OUTPUT=console

Set a 50 MB maximum size for each log file
N8N_LOG_FILE_MAXSIZE=500

EXECUTIONS_DATA_SAVE_ON_ERROR=all
EXECUTIONS_DATA_SAVE_ON_PROGRESS=false
EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=false

EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=168
EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000

EXECUTIONS_DATA_SAVE_ON_ERROR=all
EXECUTIONS_DATA_SAVE_ON_PROGRESS=false
EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=false

EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=168
EXECUTIONS_DATA_PRUNE_MAX_COUNT=500

EXECUTIONS_TIMEOUT=-1
NODE_OPTIONS=“–max-old-space-size=22528”

docker-compose.yml
version: ‘3.1’

services:

postgres:
image: postgres:11
container_name: postgres
restart: always
ports:

5435:5432
env_file:
.env
volumes:
/opt/n8n-XXX/data/database/postgresql:/var/lib/postgresql/data
networks:
servernet
redis:
image: redis:6-alpine
container_name: redis
restart: always
volumes:

redis_storage:/data
healthcheck:
test: [“CMD”, “redis-cli”, “ping”]
interval: 5s
timeout: 20s
retries: 10
mongo:
image: mongo:4.4
env_file:

.env
ports:
“27018:27017”
networks:
servernet
volumes:
my-mongo-volume:/data
n8n:
image: n8nio/n8n:1.15.2
labels:

io.portainer.accesscontrol.teams=developpement
restart: always
env_file:
.env
ports:
5678:5678
extra_hosts:
“DNS.XXX.XXX:IP.XX.XXX.XXX”
links:
postgres
redis
mongo
labels:
traefik.enable=true
traefik.http.routers.n8n.rule=Host(${SUBDOMAIN}.${DOMAIN_NAME})
traefik.http.routers.n8n.tls=true
traefik.http.routers.n8n.entrypoints=web,websecure
traefik.http.routers.n8n.tls.certresolver=mytlschallenge
traefik.http.middlewares.n8n.headers.SSLRedirect=true
traefik.http.middlewares.n8n.headers.STSSeconds=315360000
traefik.http.middlewares.n8n.headers.browserXSSFilter=true
traefik.http.middlewares.n8n.headers.contentTypeNosniff=true
traefik.http.middlewares.n8n.headers.forceSTSHeader=true
traefik.http.middlewares.n8n.headers.SSLHost=${DOMAIN_NAME}
traefik.http.middlewares.n8n.headers.STSIncludeSubdomains=true
traefik.http.middlewares.n8n.headers.STSPreload=true
traefik.http.routers.n8n.middlewares=n8n@docker
volumes:
/opt/n8n_XX:/home/node/
/opt/sftp-n8n/data/uploads:/home/data
command: “start”
depends_on:
postgres
redis
mongo
networks:
servernet
networks:
servernet:
driver: bridge

volumes:
n8n_storage:
redis_storage:
my-mongo-volume:
external: false

Information on your n8n setup

**n8n version: 1.15.2
**Database (default: SQLite): mongoDB, postgres
**third-party database: Baserow
n8n EXECUTIONS_PROCESS setting (default: own, main):
**Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
**Operating system: RedHat Linux serveur

Micka_Rakotomalala · November 14, 2023, 11:02am

Jon · November 14, 2023, 5:33pm

Hey @Micka_Rakotomalala,

Are you always running this workflow from the UI with the execute button or do you have it scheduled as that can also impact the speed.

One thing I would maybe do is break that workflow up into smaller workflows then use the execute workflow options so that you can make the most of the workers if you are using them.

Micka_Rakotomalala · November 15, 2023, 7:18am

Hi @Jon

Thank you very much for your feedback

Eventually, when we deliver to production, we’re going to plan with a trigger that will be triggered in the background every week.

The old version of dataset is separated by independent workflows, however if I retrieve a file of 6800 lines in the database, it takes a long time, even several minutes, it seemed that it even crashes.
So maybe I need to split the files too, or not?

That’s why I put the switch by company there, but it’s frozen. What other option is there for retrieving a large list from a node?

Micka_Rakotomalala · November 15, 2023, 10:22am

on a baserow node, to retrieve 29 lines, is it normal that it takes 35 seconds to obtain 29 lines of data in baserow?

I have a redhat server, which hosts baserow and n8n in the same server separated just by containers. Is it a good idea to put them together?

Just for test

Jon · November 15, 2023, 11:31am

Hey @Micka_Rakotomalala,

35 seconds is a long time, On my setup of n8n and Baserow both in different containers it is taking under 3 seconds to get 468 items although this is only 10 columns so I suspect if you have a lot more it will take longer.

Have you done a test against the Baserow cloud offering to see if that is also slow?

Micka_Rakotomalala · November 17, 2023, 7:37am

Hi @Jon

On the other hand, it’s just two baserow nodes that I’m using, so it shouldn’t cause 2 days of execution if it’s 28 seconds to access, because afterwards I used http request node for batch insertion of the element into the database.

Locally, I tested with baserow cloud, it was fast, about 2 seconds. However, I don’t have access to baserow cloud on my server.

Is using several code nodes also a problem for this process?

Jon · November 17, 2023, 9:50am

Hi @Micka_Rakotomalala,

The code nodes are likely to be ok, It sounds like if it is 2 seconds using Cloud and 28 seconds locally there is something local in your environment causing a delay.

Could your server be overloaded or maybe something like SELinux is getting in the way?

Micka_Rakotomalala · November 18, 2023, 12:38am

Thanks for your help, I’ll check with the teams for the operating system configuration and investigate server side, but for network flows it’s open to everything and no specific blocking, Cloudflare for security then reverse proxy to access via url.

Just one question: given the load, which is set to increase still further, with both projects on the same server, can we use docker swarm for docker installation of n8n?

Jon · November 20, 2023, 11:37am

Hey @Micka_Rakotomalala,

If you are comfortable using Docker Swarm you should be able to use it, We don’t have any guides on that and as an advanced deployment option you would need to rely on the skills of your team to maintain and deploy it.

Out of interest are you connecting to your Baserow instance directly or are you using the url and going through cloudflare?

Micka_Rakotomalala · November 20, 2023, 11:41am

Hey @Jon

I used to connect with n8n’s baserow node, but I compared the latency between baserow node and http request so I changed to http request via url passed through cloudflare.

Micka_Rakotomalala · November 20, 2023, 11:43am

Sucuri is replaced by cloudflare, an older architecture

Micka_Rakotomalala · November 20, 2023, 12:26pm

And also how do you connect directly to baserow in n8n, I’ve used the baserow IP but it doesn’t work either with baserow nodes or with http request?

Jon · November 20, 2023, 12:39pm

Hey @Micka_Rakotomalala,

I would recommend getting the IP working as there is no sense in using the URL and possibly routing the traffic outside of your network then back in to make the call unless you have your local DNS configured to keep the traffic internal to you.

Micka_Rakotomalala · November 20, 2023, 1:13pm

I don’t understand, I configured my url just by my IP and port, IP baserow and equal to IP n8n, differentiate just by ports, but I can’t get the tables in baserow? should I add the ip in docker-compose like this :
n8n_image:
extra_hosts:
- “baserow.pulse.mg:101.X.X.X”

Micka_Rakotomalala · November 20, 2023, 1:37pm

And also I’ve already optimized my process, limited the logs, inserted the entries with the number of rows in entries to the number of rows inserted in the base, it still took more than 5 hours my process.

There’s the limitation of nodes and scripts.

For the rest of the investigation, other than the direct database connection, I don’t know what else to do to limit my process execution time.

would limiting the size of log writes on the docker have any impact on n8n?

Micka_Rakotomalala · November 20, 2023, 1:52pm

2023-11-20T13:45:49.905Z | debug | Executing hook on node “Value splitInBatch1” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:45:49.905Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:45:49.905Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘Value splitInBatch1’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:45:49.942Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:45:50.399Z | debug | Running node “GET Raw Batch1” finished with error “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:46:28.878Z | debug | Running node “GET Raw Batch1” started “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:46:28.882Z [Rudder] debug: no existing flush timer, creating new one

2023-11-20T13:46:28.924Z | debug | Add editor-UI session “{ sessionId: ‘qv7sg3f0k4b’, file: ‘abstract.push.js’, function: ‘add’ }”

2023-11-20T13:46:44.564Z | debug | Start processing node “Switch” “{\n node: ‘Switch’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

11-20T13:46:44.565Z | debug | Executing hook on node “Switch” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:46:44.566Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:46:44.566Z | debug | Running node “Switch” started “{\n node: ‘Switch’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:46:44.568Z | debug | Running node “Switch” finished successfully “{\n node: ‘Switch’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:46:44.569Z | debug | Executing hook on node “Switch” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:46:44.569Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:46:44.569Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘Switch’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:46:44.585Z | debug | Found no executions to hard-delete (pruning cycle) “{ file: ‘pruning.service.js’, function: ‘hardDeleteOnPruningCycle’ }”

2023-11-20T13:46:44.586Z | debug | Scheduling hard-deletion for next 15.00 min (pruning cycle) “{ file: ‘pruning.service.js’, function: ‘scheduleHardDeletion’ }”

2023-11-20T13:46:44.589Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:46:44.604Z | debug | Running node “GET Raw Batch1” finished with error “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:25.270Z | debug | Running node “GET Raw Batch1” started “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:25.271Z [Rudder] debug: in flush

2023-11-20T13:47:25.271Z [Rudder] debug: cancelling existing timer…

2023-11-20T13:47:25.271Z [Rudder] debug: cancelling existing flushTimer…

2023-11-20T13:47:25.271Z [Rudder] debug: batch size is 4

2023-11-20T13:47:25.274Z | debug | Wait tracker querying database for waiting executions “{ file: ‘WaitTracker.js’, function: ‘getWaitingExecutions’ }”

2023-11-20T13:47:42.050Z | debug | Start processing node “Loop Over Items9” “{\n node: ‘Loop Over Items9’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:42.051Z | debug | Executing hook on node “Loop Over Items9” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:47:42.051Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:47:42.051Z | debug | Running node “Loop Over Items9” started “{\n node: ‘Loop Over Items9’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:42.053Z | debug | Running node “Loop Over Items9” finished successfully “{\n node: ‘Loop Over Items9’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:42.054Z | debug | Executing hook on node “Loop Over Items9” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:47:42.054Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:47:42.054Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘Loop Over Items9’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:47:42.063Z | debug | Add editor-UI session “{ sessionId: ‘eh9hzcuwlw5’, file: ‘abstract.push.js’, function: ‘add’ }”

2023-11-20T13:47:42.066Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:47:42.074Z | debug | Running node “GET Raw Batch1” finished with error “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:47:42.075Z | debug | Executing hook on node “GET Raw Batch1” (hookFunctionsPush) “{\n executionId: ‘32173’,\n sessionId: ‘igj0nue36j’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:47:42.076Z | error | The session “igj0nue36j” is not registered. “{ sessionId: ‘igj0nue36j’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:47:42.076Z | verbose | Workflow execution finished with error “{\n error: {\n context: {},\n severity: ‘warning’,\n name: ‘NodeApiError’,\n timestamp: 1700488062073,\n node: {\n parameters: [Object],\n id: ‘e9131ffa-96c5-44f9-b548-325bf75f335c’,\n name: ‘GET Raw Batch1’,\n type: ‘n8n-nodes-base.baserow’,\n typeVersion: 1,\n position: [Array],\n retryOnFail: true,\n maxTries: 5,\n executeOnce: false,\n alwaysOutputData: true,\n credentials: [Object],\n continueOnFail: true\n },\n httpCode: ‘ECONNREFUSED’,\n description: ‘connect ECONNREFUSED 127.0.0.1:8085’,\n message: ‘The service refused the connection - perhaps it is offline’,\n stack: ‘NodeApiError: The service refused the connection - perhaps it is offline\n’ +\n ’ at Object.getJwtToken (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Baserow/GenericFunctions.js:64:15)\n’ +\n ’ at processTicksAndRejections (node:internal/process/task_queues:95:5)\n’ +\n ’ at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Baserow/Baserow.node.js:123:26)\n’ +\n ’ at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:670:19)\n’ +\n ’ at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:652:53’\n },\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.verbose’\n}”

2023-11-20T13:47:42.077Z | debug | Executing hook (hookFunctionsSave) “{\n executionId: ‘32173’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘workflowExecuteAfter’\n}”

2023-11-20T13:48:20.927Z | debug | Executing hook (hookFunctionsPush) “{\n executionId: ‘32173’,\n sessionId: ‘igj0nue36j’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘workflowExecuteAfter’\n}”

20T13:48:20.928Z | debug | Save execution progress to database for execution ID 32173 “{\n executionId: ‘32173’,\n workflowId: ‘qkp1ObK2jApdiSDp’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘workflowExecuteAfter’\n}”

2023-11-20T13:48:20.928Z | error | The session “igj0nue36j” is not registered. “{ sessionId: ‘igj0nue36j’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:48:37.331Z | debug | Wait tracker querying database for waiting executions “{ file: ‘WaitTracker.js’, function: ‘getWaitingExecutions’ }”

2023-11-20T13:48:37.332Z | debug | Starting soft-deletion of executions (pruning cycle) “{ file: ‘pruning.service.js’, function: ‘softDeleteOnPruningCycle’ }”

2023-11-20T13:48:37.334Z [Rudder] debug: no existing flush timer, creating new one

2023-11-20T13:48:38.031Z | debug | Start processing node “Init File” “{\n node: ‘Init File’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:48:38.032Z | debug | Executing hook on node “Init File” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:48:38.032Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:48:38.033Z | debug | Running node “Init File” started “{\n node: ‘Init File’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:48:38.050Z | debug | Running node “Init File” finished successfully “{\n node: ‘Init File’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:48:38.051Z | debug | Executing hook on node “Init File” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:48:38.051Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:48:38.052Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘Init File’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:48:38.065Z | debug | Found no executions to soft-delete (pruning cycle) “{ file: ‘pruning.service.js’, function: ‘softDeleteOnPruningCycle’ }”

2023-11-20T13:49:17.825Z [Rudder] debug: in flush

2023-11-20T13:49:17.825Z [Rudder] debug: cancelling existing timer…

2023-11-20T13:49:17.825Z [Rudder] debug: cancelling existing flushTimer…

2023-11-20T13:49:17.825Z [Rudder] debug: batch size is 2

Error fetching feature flags [CanceledError: canceled] { code: ‘ERR_CANCELED’ }

2023-11-20T13:49:34.306Z | verbose | User ced81ffc-6bf1-49d1-9cd9-d678a35de8e9 attempted to query non-shared workflow qkp1ObK2jApdiSDp “{ file: ‘executions.service.js’, function: ‘getExecutionsList’ }”

2023-11-20T13:49:35.013Z | debug | Start processing node “GET Raw Batch1” “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:49:35.014Z | debug | Executing hook on node “GET Raw Batch1” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:49:35.014Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:49:35.015Z | debug | Running node “GET Raw Batch1” started “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:49:35.036Z | verbose | User attempted to access a workflow without permissions “{\n workflowId: ‘qkp1ObK2jApdiSDp’,\n userId: ‘ced81ffc-6bf1-49d1-9cd9-d678a35de8e9’,\n file: ‘workflows.controller.js’\n}”

2023-11-20T13:49:35.043Z | debug | Remove editor-UI session “{\n sessionId: ‘eh9hzcuwlw5’,\n file: ‘abstract.push.js’,\n function: ‘remove’\n}”

2023-11-20T13:49:35.062Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:49:35.448Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:49:35.629Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:49:36.020Z | debug | Running node “GET Raw Batch1” finished successfully “{\n node: ‘GET Raw Batch1’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:49:36.022Z | debug | Executing hook on node “GET Raw Batch1” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:49:36.022Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:49:36.023Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘GET Raw Batch1’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:49:36.102Z [Rudder] debug: no existing flush timer, creating new one

2023-11-20T13:50:18.531Z | debug | Wait tracker querying database for waiting executions “{ file: ‘WaitTracker.js’, function: ‘getWaitingExecutions’ }”

2023-11-20T13:50:18.532Z [Rudder] debug: in flush

2023-11-20T13:50:18.532Z [Rudder] debug: cancelling existing timer…

2023-11-20T13:50:18.532Z [Rudder] debug: cancelling existing flushTimer…

2023-11-20T13:50:18.532Z [Rudder] debug: batch size is 1

2023-11-20T13:50:33.215Z | debug | Start processing node “No Operation, do nothing” “{\n node: ‘No Operation, do nothing’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:50:33.216Z | debug | Executing hook on node “No Operation, do nothing” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:50:33.216Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:50:33.216Z | debug | Running node “No Operation, do nothing” started “{\n node: ‘No Operation, do nothing’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:50:33.217Z | debug | Running node “No Operation, do nothing” finished successfully “{\n node: ‘No Operation, do nothing’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:50:33.217Z | debug | Executing hook on node “No Operation, do nothing” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:50:33.217Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:50:33.217Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘No Operation, do nothing’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:51:24.577Z | debug | Wait tracker querying database for waiting executions “{ file: ‘WaitTracker.js’, function: ‘getWaitingExecutions’ }”

2023-11-20T13:51:25.288Z | debug | Start processing node “GET DATA BATCH” “{\n node: ‘GET DATA BATCH’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:51:25.289Z | debug | Executing hook on node “GET DATA BATCH” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteBefore’\n}”

2023-11-20T13:51:25.289Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:51:25.289Z | debug | Running node “GET DATA BATCH” started “{\n node: ‘GET DATA BATCH’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:51:25.291Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:51:25.292Z | debug | Proxying request to axios “{ file: ‘LoggerProxy.js’, function: ‘exports.debug’ }”

2023-11-20T13:51:26.047Z | debug | Running node “GET DATA BATCH” finished successfully “{\n node: ‘GET DATA BATCH’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”

2023-11-20T13:51:26.049Z | debug | Executing hook on node “GET DATA BATCH” (hookFunctionsPush) “{\n executionId: ‘32171’,\n sessionId: ‘ie7oh93hzg’,\n workflowId: ‘2RuMSv4YkXEzPaKl’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

2023-11-20T13:51:26.049Z | error | The session “ie7oh93hzg” is not registered. “{ sessionId: ‘ie7oh93hzg’, file: ‘abstract.push.js’, function: ‘send’ }”

2023-11-20T13:51:26.050Z | debug | Save execution progress to database for execution ID 32171 “{\n executionId: ‘32171’,\n nodeName: ‘GET DATA BATCH’,\n file: ‘WorkflowExecuteAdditionalData.js’,\n function: ‘nodeExecuteAfter’\n}”

Micka_Rakotomalala · November 20, 2023, 1:59pm