Docker creating zombie processes

brunoamaral · November 9, 2019, 11:59am

I’m running ubuntu server and whenever I leave my workflows running I return to see Zombie processes that go away once I restart.

It’s not an issue for me, but I would like to help figure out if it’s a bug. How can I do that?

jan · November 9, 2019, 12:08pm

Yes, I also saw on my server that a few zombie processes were there and wanted to investigate. Sadly did not get to it yet. So it would definitely be great if you could check that out! Honestly, do not know much how a process turns into a zombie (and not even what the definition is).

However, the interesting code would be this:

This code starts the workflow processes:

github.com

n8n-io/n8n/blob/master/packages/cli/src/WorkflowRunner.ts

/* eslint-disable no-restricted-syntax */
/* eslint-disable no-console */
/* eslint-disable @typescript-eslint/no-non-null-assertion */
/* eslint-disable @typescript-eslint/no-unsafe-member-access */
/* eslint-disable @typescript-eslint/restrict-template-expressions */
/* eslint-disable @typescript-eslint/no-shadow */
/* eslint-disable @typescript-eslint/no-floating-promises */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
/* eslint-disable @typescript-eslint/prefer-optional-chain */
/* eslint-disable no-param-reassign */
/* eslint-disable @typescript-eslint/explicit-module-boundary-types */
/* eslint-disable import/no-cycle */
/* eslint-disable @typescript-eslint/no-unused-vars */
import { BinaryDataManager, IProcessMessage, WorkflowExecute } from 'n8n-core';

import {
	ExecutionError,
	IDeferredPromise,
	IExecuteResponsePromiseData,
	IRun,

This file has been truncated. show original

This code gets executed in the new process and executes the new workflow:

github.com

n8n-io/n8n/blob/master/packages/cli/src/WorkflowRunnerProcess.ts

/* eslint-disable @typescript-eslint/no-unsafe-member-access */
/* eslint-disable consistent-return */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
/* eslint-disable @typescript-eslint/no-shadow */
/* eslint-disable @typescript-eslint/no-non-null-assertion */
/* eslint-disable @typescript-eslint/no-use-before-define */
/* eslint-disable @typescript-eslint/unbound-method */
import {
	BinaryDataManager,
	IBinaryDataConfig,
	IProcessMessage,
	UserSettings,
	WorkflowExecute,
} from 'n8n-core';

import {
	ExecutionError,
	ICredentialType,
	ICredentialTypeData,
	IDataObject,

This file has been truncated. show original

Maybe it would be better to not create a new process and instead start it in a worker thread.
There is some discussion about that going on here:

That could maybe at the same time solve this problem, reduce CPU and memory consumption and reduce the time that it takes new workflows to start to execute (which is currently horribly slow with around 1 second).

brunoamaral · November 13, 2019, 12:27pm

From what I managed to read and ask around, the Zombie is a process that didn’t receive a kill signal from it’s parent.

And a quick look at the files you showed, I saw there are some subprocess calls. Shouldn’t there be some process to cleanup any process left after a while?

jan · November 13, 2019, 12:56pm

The workflow processes should kill themselves once they are done. The killing processes after a while could be a problem. Because when should they be killed? I created once a workflow that was running 3 days executing SQL queries to migrate data. So would need a way to check if the workflow is still running or if something went wrong.

Just looked up what a zombie process is:

Sounds like something is maybe going wrong in the script which creates the child processes?!?!!

cierre · January 13, 2020, 9:55pm

has this issue been resolved yet? concerned about the CPU usage…

jan · January 13, 2020, 11:17pm

No, because it does not seem to happen for me, I can not reproduce it any nobody else did report it. Are you experiencing this issue?

romain · February 20, 2020, 10:33pm

I’m not experiencing the issue, but just noticed the thread.

I noticed the Alpine-based Docker image uses tini as the init process, whereas the Ubuntu-based Docker image doesn’t. Its main purpose is to reap zombies on shutdown, so maybe it’s needed for the Ubuntu image too?

Tini is now baked into Docker as well, so the docs could specify to run with --init if that’s needed.