Check email for specific attachments once a month and only for the last month

IMAP node works great, can I schedule it once a month and retrieve specific attachments from the last month only?

Problem Description

I am trying to get PDF receipts from our IMAP email for a monthly account reconciliation. I can get the attachments as they come in and filter any I do not need using an IF node and a REGEX match. When I execute the node manually, the workflow retrieves all emails and attachments based on my filter criteria and downloads them into a local folder. As new emails come in, the attachments download automatically (I am sure this is the intended function) but I would like to execute this workflow once a month to retrieve all emails and download those that meet my criteria (attachments that match regex and are within the last month).

I have tried to use Cron node to do this and reference the workflow but since the IMAP node is a trigger node it doesn’t seem to work with the Cron node. Making a separate workflow and referencing my IMAP workflow does not work either since it triggers the START node and not the IMAP workflow itself.

IMAP workflow

Output from last node in IMAP workflow

Output of the last node downloads the pdf to a local file and is working as expected and does nothing with emails without an attachment or one that is not recognized by the match function. This part works great.

Cron Workflow that references the IMAP workflow

Cron workflow output

This workflow runs, but does not execute the IMAP node, I am assuming because it is initiating the START node, but nothing is connected to it and since IMAP is already a trigger node, then there is nothing to execute.

I am mostly OK with having the PDFs being downloaded one at a time as they come in but it would be nice to batch this process into one or two workflows that execute once a month and download the PDFs into a folder for each month. That would be amazing. I believe this would require some changes to the IF node to increment its filter date by a month after each execution (like the option in Cron to repeat every month). Maybe what I am talking about is some kind of ‘inbox scanner’ and not the IMAP node I am using.

Note: I have been testing this with a separate email address to test the filter. I am not 100% sure if the date filter I am using in the IF node will work in filtering out emails within my chosen dates (i.e. one month ago).

Please let me know if I am missing some simple thing, like a node or link or if this is a more complicated function that would need some special code. I am very new to coding so the simpler, the better. I could just be overthinking this and maybe just altering my IF node and executing it manually once a month is a far simpler solution for me lol.

Information on n8n setup

  • n8n version: 0.182.0
  • Database you’re using (default: SQLite): Not sure
  • Running n8n with the execution process [own(default), main]: Using Desktop version and running it manually (for now).

Hey @MattiFish, welcome to the community!

Unfortunately, this wouldn’t work with the IMAP node as it’s a trigger node, running whenever a specified event happens (in this case a new email arriving). Meaning you can’t just have it search for messages once a month.

Are you using Gmail or Outlook by any chance? These two services have regular nodes which you can run whenever needed.

Thanks for the reply.

I figured as much. I am using another company to host our email. We use Outlook in the office but I am using the same details to login to the IMAP node as I do with Outlook.

I will have a look at the Outlook node and see what I can do with that, at a glance it seems like it has more options and can be triggered? So maybe there is some way to schedule that and still get filtered contents.

It is my first time interacting with Azure but the instructions on n8n got me through the connection. Still need to play around a bit more to get the connection going (running into a ’ ERROR: Bad request - please check your parameters. Id is malformed’). Could be because I have not used Outlook or Azure on this account or my Message ID in the Outlook node could be wrong.

I will keep on fiddling and get back with any results I get. I might be a while as this is a little side project.

Hi @MattiFish, I was using the Outlook node just fine this morning and have also just tried reconnecting the underlying OAuth credentials.

Can you confirm where exactly you are getting this error (perhaps share a screenshot)?

HI @MutedJam. I initially thought the problem was with me using a free MS Office account that did not have access to Outlook, but I switched over to another account, connected it up and I am running into the same error. I believe it has something to do with the syntax of my request, likely message and attachment ID. I am not sure what ID to use here so I just named them ‘Message’ and ‘Attachment’ respectively, but this probably wrong? Any idea how to label these properties correctly? Also the Outlook inbox is empty if that makes a difference.

See screenshot

1 Like

Hi @MattiFish, thanks so much for the screenshot, this helps a lot! What would happen with the request from your screenshot is that n8n asks Outlook’s API for a message with an ID of Message (which isn’t what Outlook expects).

Seeing want to search for specific messages once a month you’d probably want to use the Get All operation instead (which doesn’t need additional parameters in its simplest form):

Could you give this operation a go on your side and see if it works (might want to put a dummy message in the inbox so you can see some data)? Once it works you can look into building the query finding the right messages, but let’s take one step at a time.

That worked like a charm, thanks. One thing now is that I am using my Microsoft account for this test and it is from my university. The way it is set up, is that it is using ’ ****.onmicrosoft.com’ as a domain. Since the email I am using has been used for Gmail and Microsoft, I am only receiving emails into my Gmail inbox. I will look into this and see what I can find. I might need to use/make another account on the company microsoft account to do these tests.

Not so familiar with the Microsoft environment at the moment but I guess this is where the learning curve is.

Really appreciate the prompt responses on this as well.

1 Like

Hi @MattiFish, no problem at all.

The Gmail node would work in a similar fashion btw. You can execute it whenever needed and search/filter messages like you can in the Gmail UI.

Hi @MutedJam, thanks for the suggestion, I am trying it now and I am getting a Error 400: redirect_uri_mismatch (screenshot below). There were some problems with spaces getting pasted into the ID and secret fields, but I have checked those and they seem fine. Copied the redirect OAuth Redirect URL from the node and generated the ID and secret fine. Not sure if it is my OAuth Consent screen on Google that is the issue here. Still looking for answers on the documentation but no luck so far.

EDIT: Checking a solution posted here

Hi @MattiFish, this seems like a problem with the tunnel URL used by the desktop application. The problem you are seeing is because the tunnel URL used by n8n doesn’t match the redirect URL stored with Google. So you would need to update the URL you have registered with Google here.

On a general note, this tunnel URL is subjected to change and not the most stable. We actually suggest avoiding this tunnel in production environments.

So you might want to consider setting up n8n on a small VPS with a static domain instead if you’re testing your workflows for an extended period of time.

Ok I think I am understanding you. I will look at setting up a VPS. I might have a dedicated machine that I could prepare that could run linux or windows and then setup a n8n instance on that. Or maybe I could run a virtual machine on my existing desktop. Docker and Kubernetes could also be an option. Which do you think would be the easiest to setup and manage for a new user such as myself?

Any advice or tutorials would be welcome to take me step by step in setting up and accessing a VPS locally (and maybe even remotely) would be appreciated.

Hey @MattiFish,

I would go for whatever option you have experience with when it comes to k8s or Docker… I went with Docker on my own setup as I know how to use it but we also have documentation available for that approach which can be useful.

https://docs.n8n.io/hosting/server-setups/docker-compose/

Thanks @Jon, I will give that a try. Still very new to servers and associated services so I might be awhile getting used to them and hosting these services. I will let you know how it goes at some point :stuck_out_tongue:

1 Like

Reviving an old thread here…

I noticed this thread is similar to an issue/challenge I have with the IMAP node. With the IMAP node you can get all the email of Yesterday by adding this to the custom email rules:
2022-11-30 12_18_09-n8n - ▶️ IMAP Test

["ALL",["ON", "{{new Date(new Date().getTime() - 24*60*60*1000).toDateString()}}"]]

This is pretty awesome, because now I can validate if all incoming message have been correctly processed before, with the regular IMAP setup that triggers for incoming messages. I experience some issues from time to time, so this way I can check if everything went okay or if I need to reprocess certain emails.

Now the issue is that the IMAP node is a trigger only node, so it does not work when you try to schedule it with cron or do a ‘execute workflow’. Also, it’s not possible to connect another node as input.

But it is possible to manually trigger it by pressing this ‘execute workflow’ button. And as you can see I get 40 emails from yesterday’s date:
2022-11-30 12_21_58-n8n - ▶️ IMAP Test

In my case using the outlook or gmail nodes is sadly not possible, due to where the email is hosted.

I did find the code of this IMAP node here n8n/packages/nodes-base/nodes/EmailReadImap at master · n8n-io/n8n · GitHub and I can clearly see it is a trigger only, node.

Is making a custom copy of this node the only way to get it to work as an action node, instead of a trigger node? If so does anyone have any pointers on how to get this done? Or would there be other options on how to get yesterdays email by a schedule?

Thanks in advance for your help.

bg

Jelle

1 Like

So, how do you know you can just use $binary stuff? it does not appear in the variable selector in the UI. It is mostly a guesswork