Running cron every 1 second makes server unresponsive

Hi!
I have a workflow (well, at least 2 of them) which I need to run every second during a couple of hours each day.

Everything works fine when I run them ever 1 minute, even when I run them every 5 seconds but as soon as I set either one to 1 second the server becomes unresponsive and I need to reboot it.

I am not sure if this is an issue that can be solved by purchasing a bigger server (currently I am running on a 2GB RAM -before that I was testing on a 1GB-) or if there is anything I can do to prevent the server to crash while at the same time have continuous execution of those 2 (eventually more) workflows.

Any help or direction will be really appreciated,
Thanks a lot!

Edit: I forgot to mention that when I check server logs I see CPU utilization at 100% when setting the cron every 1 second.

Depends on what you do and how you have n8n set up. First, make sure that you do run n8n in “main” mode by setting the environment variable EXECUTIONS_PROCESS to main. If that is not enough, you either have to make the workflow you execute simpler or get a larger server.

2 Likes

Thanks @jan ! I will try setting the EXECUTION_PROCESS and see if that helps.
In case it doesn’t would you suggest a bigger server or setting the mode to queue and using workers?

Thanks again!

Depends on how heavy your workflows are. I would start with a larger server as that is much simpler and cheaper.

Anyway, setting it to “main” should make a huge difference and one ore multiple executions a second should be no problem for n8n at all, unless the workflows run a long time or are very heavy.

1 Like

Thanks! testing it right now! And agree I believe that should get it to work as the workflows run super fast (around 0.4ms)

I’ll let you know! Again thanks SO much for your time!

Yes that makes a huge difference as creating an own process for each execution has a lot of overhead both memory and CPU.

Guess you mean 0.4 seconds not ms unless you really have some amazing hardware :wink:

You are welcome. Have fun!

lol :smile: , yes you are right, I meant 0.4s!
By the way, your solution worked perfectly!
image
This used to went up all the way to 100%

Thanks a lot!

1 Like

@yakito I am curious, what are you doing that needs to run every second?

2 Likes

Hello @jon , sorry for the late reply, it was late over here.

Yes, my project involves checking things happening real time via some video sources using OpenCV and I need to move the results to several places as they happen so I thought of n8n to move things around and perform some calculations. I love the results so far!

2 Likes

That makes sense, Just wanted to make sure it isn’t something that could be done with a bigger wait just to ease up the load a bit in the future.

1 Like