Triggering Cron workflow from Other workflow

Questions
,
1

I have two distinct workflows I’m using to create a queueing workflow for bulk message processing leveraging a SQL db for message storing/execution:

  1. one workflow that consumes webhooks from an upstream service, stores them to a queue then “turns on” a second workflow using the N8N activate workflow node; and
  2. a secondary workflow that pulls down batches of messages from the queue and processes them triggered on a cron basis every minute

I have a check in place to not turn on the secondary workflow if it’s already on, but because of the high volume of messages coming into the webhook, I’m running into a condition where the activate workflow node is being called multiple times before it can register that it’s already on. The outcome of this is that the processing service is unnecessarily/unintentionally running in parallel when I want it to run in serialized batches using the cron schedule.

Does anyone have any ideas on how I can get around this limitation? I’ve copied both the queue creation flow and the processing flow below for reference. Please let me know if any other info would be helpful.

Queue Creation Workflow:

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Queue Processing Workflow:

4

Hi @cs_lec ,

Have you tried adding a wait node before the conditional check on the 1st workflow to give the follow up executions some time to “register” that the 2nd workflow is already active?

5

Hey @mariana-na - Thanks for the response! I just tried installing a wait node pausing for 1 second before proceeding forward with the IF step then the activation step. On a set of 100 inbound messages I still got 9 unintentional calls of the enablement step when I’d expect one so that doesn’t seem to have solved it.

Updated flow:

6

I figured this out. Since there’s no inherent configuration on the activate workflow node to only allow for a single instance of turning it on within a certain period of time, it’s going to call multiple times with high frequency, high volume message streams.

To get around this I installed a status table in a sql database that has the service status (e.g. on/off) and am doing the check against that table. I’m running a single SQL statement to check for whether the service is on or off and if it’s off, change the status to on. Within the same check i’m also outputting the number of records that were changed as part of the call. Since the SQL can’t modify the same field in a table while it’s being modified (processing is serialized) then there is no risk of duplication.

Updated flow:

2 Likes
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.