Best practices for stateful/diff-like workflows

Questions
,
1

Describe the problem/error/question

Some integrations don’t offer granular triggers (eg webhooks) and only provide a way to do a cron-based bulk request. I want to do this repeatedly and do something only if the data has changed.

I just installed n8n today and while looking for some guidance, I came up with nothing (I tried queries along the lines of “n8n stateful”, “n8n stateful diff”, “n8n cache between invocations|executions”, but the closest I got was Merging | n8n Docs which talked about “Merge items of different executions”, which seems to be actually about merging items from different nodes in the same execution, not across executions). I was hoping to find a node that would take the response (“Microsoft To Do” node in the example below) and would return two outputs: current response (optional, a simlple pass-through) and the previous response (that the node itself would save and look up between workflow executions). Instead, I had to add 7 nodes to compare the “Microsoft To Do” node response to a previous execution (also, not sure if what I did was actually the best practice, e.g. there is a small risk of the Write Binary File node running before the Read Binary File in the same execution and the whole diff yielding nothing).

Thanks for building a great tool!

Please share your workflow

Information on your n8n setup

  • n8n version: 0.228.2
  • Database (default: SQLite): default
  • n8n EXECUTIONS_PROCESS setting (default: own, main): default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • Operating system: Ubuntu 22.04 LTS
2 Likes
2

I was hoping to find a node that would take the response (“Microsoft To Do” node in the example below) and would return two outputs: current response (optional, a simlple pass-through) and the previous response (that the node itself would save and look up between workflow executions).

Hi @driib, welcome to the community :tada:

I am afraid this isn’t a trivial task in n8n. That’s because an execution can only access its own data and has no knowledge of previous executions. So you’d have to inject this during the current execution as needed which based on your description you are doing. I don’t really have a best practice for this, but writing them to the file system temporarily seems like a suitable approach to me.

Alternatives you could consider here would be using a database you have access to or the built-in “static data” of a workflow. The latter is a bit tricky to use though as it won’t work for manual executions.

You can also read past execution data through n8n’s REST API (provided you are saving it for your workflow) which can also be consumed through the n8n node. It requires additional parsing though as you’d get all data for a past execution, not just the piece of information you’re interested in.

1 Like
3

I like the idea of reading the previous execution data. Is this also possible via n8n cloud?

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.