Partial Workflow Data Fetched on Sheet with Update or Append Operation

[Subhayan_Chanda]

Hello, I am working on auto-filter workflow through which 10k emails/day are passed, with the trigger being of every hour.

The workflow has certain keywords to identify specific emails. These emails then are passed through a ChatGPT node to further classify into certain categories. I have two branches below with loops,
Loop 1 - Directly adds the email data

1. Sender 2. Reciever 3. Date & Time(of email) 4. Subject Line 5. Body 6. Message ID 1 7. Keyword 8. Date Data Fetched(of workflow) 9. Time Data Fetched (of workflow)

Loop 2- Passes the email through ChatGPT to make certain classification

1. Email Type 2. Escalation? 3. Experience Category 4. Severity 5. Experience Detail 6. Message ID 2

Now Loop 1 has append operation on a google sheet, while loop 2 has Append or update operation where the ‘Message ID 1’ is used to map column. ‘Message ID 1’ and ‘Message ID 2’ are same data point just being fetched from different nodes which I have added to validate further if the output from ChatGPT is of the correct email

Challenge

While for most emails this workflow works perfect for certain emails the row is either filling data of Loop 1 or Loop 2. I have validated that data is missing for those emails from either loop, and not that the loop are adding data in different rows, as the Message ID would match for both rows.

My requirement is to get the data for the email along with the ChatGPT output. I am completely fine if this can be done in a single loop, instead of two.

(I am avoiding adding a part in my prompt asking ChatGPT output to have loop 1 data, as my current input token size for each email is just below 1000, and output token is below 50. Workflow identifies around 500 emails each day, hence I don’t want double my API cost)

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[barn4k]

hello @Subhayan_Chanda

Why you need the top Loop? Is that the whole part of the workflow?

The issue may be related to the Filter node and two lines from it that lead to separate Loop nodes. So it better to use only one Loop node or even use a sub-workflows

[Subhayan_Chanda]

The reason for the loop was in every hour window multiple emails comes, and I require a new row to be added for each email, hence the two loop approach.

Currently as an band aid fix where I am using a single branch loop(deleted the top loop), where the ChatGPT node also returns the input it is receiving along with the output in JSON, and then the output from ChatGPT node is directly maped to the sheet.

I would be interested to know if there is a way to do it without the above process, as it is costing me 2000 tokens for each email instead of 1000 tokens including input and output tokens.

(Also I am super new to n8n, so I maybe not aware of some nodes which can solve the issue, would love to know them)

[barn4k]

Well… I didn’t quite catch what you mean, but try something like that.