BullMQ Node for n8n, why not?

Built with n8n
, , ,
1

Hey n8n Community! :wave:

I’m excited to introduce a new node that could be a game-changer for anyone looking to queue jobs and process them asynchronously in n8n—the BullMQ Node.

Please check it out:

Why BullMQ?

As many of you know, BullMQ is a fast, simple, and reliable job queue library built on Redis. It’s perfect for handling everything from background jobs to rate-limited tasks. While working with n8n, I saw a clear need for better job queue management, and BullMQ was the ideal solution to fill this gap.

So I went ahead and built a set of BullMQ nodes that seamlessly integrate with n8n, enabling users to efficiently manage and process jobs in their workflows.

What can you do with the BullMQ Node?

Here are some of the key features supported by the BullMQ Node:

  • Queueing Jobs: Easily add jobs to be processed asynchronously without blocking your main workflow.
  • Rate Limiting & Throttling: Control the number of jobs processed at a time to prevent system overload.
  • Job Retries: Automatically retry jobs that fail, ensuring they eventually succeed.
  • Job Scheduling: Delay jobs to run after a specified time, perfect for reminders or future tasks.
  • Event-Driven Execution: Trigger actions based on job events like completion or failure.
  • Concurrency Management: Optimize resource usage by controlling the number of concurrent jobs per worker.

image
image628×911 21.5 KB

image
image708×899 17.9 KB

Notes

This is still in active development, and I need your feedback to make it even better. Whether it’s feature suggestions, bug reports, or ideas for new nodes, every bit of input is valuable. Here are some upcoming features that are currently being worked on:

  • Flow Management: Better control over job flows and dependencies.
  • Improved Error Handling: More robust and user-friendly error management.
  • Bug Fixes: General improvements and refinements.

Thanks all

2 Likes
2

This is very interesting, as I’ve thought about many of these issues while making workflows myself–especially the rate limiting/throttling issue.
Great work!

2 Likes
3

Hey @minhlucvan, thanks for sharing this with the community! We’re just starting our n8n journey, and your BullMQ node looks like it’ll be a cornerstone for our microservices architecture.

Do you have any:

  • Lessons learned from using it so far?
  • Standout use cases where it shines?
  • Pitfalls to avoid or known bugs the community could help tackle?

Excited to test it out—great work!

4

I’m using this node! First off, congrats on developing it. I think it would be really helpful to have a way to override job delays. For example, if we’re building a debounce mechanism for an AI agent in WhatsApp or Telegram, people often send fragmented messages. If we don’t have a way to combine these into one, the AI agent will respond to each message separately. So, having the option to override the job delay parameter or even cancel a job by its ID would be super useful.

In my current setup for handling multiple messages from a single WhatsApp number, I’ve implemented a 15-second delay. When the BullMQ trigger node fires, I check if the incoming message from the main workflow matches the one stored in Redis within the last 15 seconds. If it matches, I let the workflow proceed. If not, I do nothing, as it indicates there’s another new message scheduled in a job that will expire soon.

Another suggestion for this node would be to include a datetime-based delay option for follow-up workflows. I’ve already built this functionality by creating a logic that returns a future date and then calculates the diffFromNow to get a delay in milliseconds. However, if this were a built-in feature, it would make the workflow much more readable and easier to understand.