Unable to execute workflow on random in queue mode

Questions
,
1 
(node:1255) UnhandledPromiseRejectionWarning: Error: Cannot read property 'id' of undefined
0|npm run start:default  |     at Queue.onFailed (/node_modules/bull/lib/job.js:516:18)
0|npm run start:default  |     at Queue.emit (events.js:412:35)
0|npm run start:default  |     at Queue.emit (domain.js:470:12)
0|npm run start:default  |     at Redis.messageHandler (/node_modules/bull/lib/queue.js:444:14)
0|npm run start:default  |     at Redis.emit (events.js:400:28)
0|npm run start:default  |     at Redis.emit (domain.js:470:12)
0|npm run start:default  |     at DataHandler.handleSubscriberReply (/node_modules/ioredis/built/DataHandler.js:80:32)
0|npm run start:default  |     at DataHandler.returnReply (/home/ubuntu/dev_workflow/node_modules/ioredis/built/DataHandler.js:47:18)
0|npm run start:default  |     at JavascriptRedisParser.returnReply (/home/ubuntu/dev_workflow/node_modules/ioredis/built/DataHandler.js:21:22)
0|npm run start:default  |     at JavascriptRedisParser.execute (/home/ubuntu/dev_workflow/node_modules/redis-parser/lib/parser.js:544:14)
0|npm run start:default  |     at Socket.<anonymous> (/home/ubuntu/dev_workflow/node_modules/ioredis/built/DataHandler.js:25:20)
0|npm run start:default  |     at Socket.emit (events.js:400:28)
0|npm run start:default  |     at Socket.emit (domain.js:470:12)
0|npm run start:default  |     at addChunk (internal/streams/readable.js:290:12)
0|npm run start:default  |     at readableAddChunk (internal/streams/readable.js:265:9)
0|npm run start:default  |     at Socket.Readable.push (internal/streams/readable.js:204:10)
0|npm run start:default  |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
0|npm run start:default  | (node:1255) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 169)
  • n8n version: 0.163.1
  • Database you’re using (default: SQLite): MySQL
  • Running n8n with the execution process [own(default), main]: own
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: npm

I have used two separate EC2 instances to run n8n main on one instance and n8n worker on the other one. Also using AWS managed Redis and Mysql on RDS.

Please let me know what could be the error here.

2

Hi @sulabhsuneja, welcome to the community!

I am very sorry to hear you’re having trouble. [email protected] contains a fix for the behavior you have reported. Could you try upgrading and confirm if you’re still seeing this issue afterwards?

1 Like
3

Thanks @MutedJam I will upgrade it and let you know if the issue still persists.

4

@MutedJam I updated the n8n version to 0.192.2 and When I am trying to run the n8n, it is not able to initialize the DB giving me the error during migrations DB

query is slow: ALTER TABLE user ADD CONSTRAINT FK_f0609be844f9200ff4365b1bb3d FOREIGN KEY (globalRoleId) REFERENCES role(id) ON DELETE CASCADE ON UPDATE NO ACTION
execution time: 3435
Migration "CreateUserManagement1646992772331" failed, error: Can't write; duplicate key in table '#sql-4246_13ce5'
2022-09-15T05:56:48.423Z | error    | There was an error initializing DB: "Can't write; duplicate key in table '#sql-4246_13ce5'" "{ file: 'start.js' }"

I have some previous workflow data which I cannot afford to lose. Can you please provide a solution to run migrations successfully?

Thanks

5

Hey @sulabhsuneja, [email protected] contains a fix for migrations on MySQL, so as a first step I’d try upgrading to the current version of n8n (which is 0.194.0 at the time of writng this). Are you also having trouble upgrading to this version?

6

Hey @MutedJam, thanks for the quick repsonse. I will try updating it to the latest version and let you know.

7

Hey @MutedJam
I am getting this error while building the docker image.

image
image797×512 65.9 KB

Can you please look into that?
Thanks

8

Hey @sulabhsuneja, are you trying to build the n8n-custom docker image?

This has changed quite a lot. Can you make sure you pull the latest code from the n8n repo and add your modifications to the new Dockerfile/entrypoint before trying to build the image?

9

Hey @MutedJam ,
Yes I am building the n8n-custom docker image.
I am using latest code from the repo. Dockerfile and docker-entrypoint.sh are as follows:
DockerFile

ARG NODE_VERSION=16

# 1. Create an image to build n8n
FROM n8nio/base:${NODE_VERSION} as builder

RUN \
  apk --no-cache add git && \
  npm install -g run-script-os turbo

COPY turbo.json package.json package-lock.json tsconfig.json ./
COPY packages ./packages

RUN chown -R node:node .
RUN npm config set legacy-peer-deps true

USER node

RUN \
  npm install && \
  npm run build && \
  # TODO: removing dev dependecies is deleting `bn.js`, which breaks the Snowflake node
  npm prune --omit=dev && \
  npm i --omit=dev bn.js && \
  find . -type f -name "*.ts" -o -name "*.js.map" -o -name "*.vue" -o -name "tsconfig.json" | xargs rm &&\
  rm -rf node_modules/.cache packages/*/node_modules/.cache packages/*/.turbo .config .npm /tmp/*


# 2. Start with a new clean image with just the code that is needed to run n8n
FROM n8nio/base:${NODE_VERSION}
COPY --from=builder /home/node ./
COPY docker/images/n8n-custom/docker-entrypoint.sh ./

RUN \
  mkdir .n8n && \
  chown node:node .n8n
USER node
ENV NODE_ENV=production
ENTRYPOINT ["tini", "--", "./docker-entrypoint.sh"]

docker-entrypoint.sh

#!/bin/sh
if [ "$#" -gt 0 ]; then
  # Got started with arguments
  COMMAND=$1;

  if [[ "$COMMAND" == "n8n" ]]; then
    shift
    (cd packages/cli; exec node ./bin/n8n "$@")
  else
    exec node "$@"
  fi

else
# Got started without arguments
cd packages/cli; exec node ./bin/n8n
fi

Please let me know what do I need to update.
Thanks

10

Hm, I am afraid I don’t know exactly why the dependency resolution here might fail. @netroy could you kindly take a look as to what might be failing here?

11

Hey @MutedJam , @netroy any update on the above issue?
Were you able to replicate it? I could really use some help here.

Thanks

1 Like
12

Hey @sulabhsuneja, Thanks for reporting this.

The error screenshot you posted suggests that your package-lock.json file is not the same as the one from the n8n repo.
Is it possible that you ran npm install locally with an older version of npm, and then that file is getting picked up by the docker build? I don’t see any other reason why the build would complain that package-lock.json was created with an old version of npm

Are you using the n8n repo directly, or do you have a fork? if you are using a fork, please make sure that you are using an up-to-date package-lock.json.

13

Hey @netroy . Thanks for the update.
I am using node 16.17 & npm 8.1.2. Locally, npm install is working fine.

I am using a fork of n8n repo and package-lock.json is already updated and it is still failing.

Thanks

14

Hey, is the fork public? is there a branch that I could look at?

15

Hey @netroy
The fork is not public as we have made some customizations according to the requirement of project but thanks for the help. I was able to figure out that it happened due to latest node 16 minor version and npm version which has strict peer dependency check.
Please check this comment npm ERR while resolving [email protected] · Issue #7095 · vuejs/vue-cli · GitHub

So I created a custom base image with node version and rebuilt docker image for n8n and it worked.

Thanks

16

Hey @netroy @MutedJam
thanks for the last help. I am stuck on another queue issue.
It would be really great if anyone can please take a look into it.

return new Error('Missing lock for job ' + jobId + ' ' + command);

2022-09-26T17:33:13.953+05:30	^

2022-09-26T17:33:13.953+05:30	Error: Missing lock for job 898 failed

2022-09-26T17:33:13.953+05:30	at Object.finishedErrors (/home/node/node_modules/bull/lib/scripts.js:189:16)

2022-09-26T17:33:13.953+05:30	at Job.moveToFailed (/home/node/node_modules/bull/lib/job.js:342:19)

2022-09-26T17:33:13.953+05:30	at processTicksAndRejections (node:internal/process/task_queues:96:5)

I have googled that it is related to the queue but haven’t been able to find a solution for the same.

Thanks

17

Hey @sulabhsuneja,

What are the resources looking like for the Redis server?

18

Hey @Jon

I am using AWS managed Redis clusters and metrics are showing just fine.(utilization below 10 percent)

Thanks

19

Hey @sulabhsuneja,

That was the only thing I could find on the Bull side, Is this an error you are always seeing or is it intermittent?

20

@Jon . thanks for the quick response.
This is the only error I am getting right now. I have enabled logs and all the nodes are working perfectly and finishing successfully but workflow’s executions are just not getting updated I guess. I’ll also take a look into database if anything is logged there.

Meanwhile, I also found some links on the same issue.

Please check if you can figure something out from it.

Thanks