How to trigger manually a workflow to get sample data?

[jb8n]

Hi,

One of the things that slows me down a lot with n8n is debugging, and especially getting data to debug. Looking forward to your inputs!

1. Getting data structure from previous nodes

For example, if I have a node in the middle of the workflow for which the input depends on previous nodes, the left pane of n8n will show “No input data yet - Execute previous nodes - (From the earliest node that has no output data yet)”

Even if I have multiple execution of this workflow, I cannot access sample data when testing a node. For example, with this screenshot (OpenAI node replying on Telegram), if I work on a later node and want to select one parameter from a previous node, I am puzzled: Was it {{ $('SwitchWelcome').prompt }} ? Was it something else? Maybe I should write {{ $('SwitchWelcome').[prompt] }} instead?

So I end up going to the Execution pane, looking at a previous execution, noting down the JSON structure, and then inputing it again in my node on Editor mode. I’ve sometimes seen a “drag and drop” feature to drop a parameter from a previous node in the left pane of the node editor, but it only works if data is already present!

How do you do it?

2. Getting sample data from a Trigger node

On that example again, the workflow starts with a TelegramTrigger. It means that if I want to obtain test data to see if a node, in the middle of the workflow, process it correctly, I have to manually send data in webhook-test. If I type “Execute Workflow” at the bottom if the screen, on that screenshot, it will not receive data from the actual telegram channel.

My workaround so far was to have a “double trigger” workflow, with a “When clicking Execute Workflow” node, as seen in screenshot, which sets mock data, so that the workflow can be executed manually without a need to fiddle with Telegram and -test webhooks. However, today, when I clicked the “Execute Workflow” button, the only thing that happened was a tooltip “Waiting for you to create an event in Telegram” appearing above the “TelegramTrigger” node (and it would wait forever).

So, with these 2 use cases, I feel that my debugging is very slow and laborious, and when I see complex workflows share here, I’m thinking there must be a better way to build with n8n. Is it? Or do you actually succeed at your first try?

Thanks for your suggestions.

–
n8n: Docker image v.0.219.1

[FelixL]

Hi @jb8n,

I always pin my data, so it’s available even though I did not execute the workflow.
You can even just copy the JSON from a node in an old execution and pin that JSON in your editor.

Afaik there is also a new feature in the pipeline to more easily debug already run Executions, but I do not know when this gets released

Some links to the documentation about these features:

Data pinning - n8n Documentation

Data editing - n8n Documentation

[Jon]

I use a mix of pinned data or a code node depending on if it is a flow I am working on for myself (I have credentials for the service) or if I am answering a community question and I have some sample data.

The approach I took before was the trigger node and a start / manual execution node to a code node that would mock the data then I would disable the main trigger and press run then enable it once finished

[jb8n]

Thanks!

I always pin my data, so it’s available even though I did not execute the workflow.

I’ve done “Pin data” a few times. Overall I think I lost as much time as saved with it, for forgetting to unpin data from a node, and wondering why my subsequents executions behave strangely . Also, copying and pinning data often takes more time than just running the workflow from the start and get a “fresh” batch of data through the nodes, since I’m impressed they execute really fast. I use it more when on nodes that write data externally, and for which deleting the impact of test executions would be more time consuming than using pinned data.

I think I’ve been enjoying so much how fast it is to build with n8n, that I get more impatient with debugging phase

Afaik there is also a new feature in the pipeline to more easily debug already run Executions, but I do not know when this gets released

That’d be terrific! Knowing how much time I spend going back and forth between Execution view and the Editor view, being able to edit and save a node from the Execution view would be great. Especially when it contains custom javascript: I like the view that interprets expressions in real time and show green/red outputs, and wish I could use it on previously ran executions.

[jb8n]

Thanks! I’m starting to insert Code nodes inside my workflow be able to return what I want (debug), or just pass the input with no modification (pass-through).

Talking about Pinning, it is also not working in many situations.

• For Example1, I cannot pin the output of a Switch node: the PIN icon is not present on the right pane of the editor.

• For Example2, that Switch node that has 2 inputs, including one being pinned, will not run until it also receives data from the other input. I guess that’s expected behavior. I can debug my disabling the previous nodes of the input line I don’t expect data from.

• For Example3, and this is really weird to me, starting with TelegramTrigger, if I:

  • pin the output of a subsequent node, say node6, to a given JSON
  • connect node6 to node7
  • node 7 being receiving input from only 1 connection, node6

Then I cannot execute node7 on its own:

• on the left pane of the editor, I select node6, which shows the pinned JSON
• I click on “Execute node”
• The left pane is replaced by “Executing previous nodes”
• A tooltip appears with “Waiting for a Trigger node to execute. Close this view to see the Workflow Canvas.”

So pinning JSON on the previous node doesn’t help when there’s a TelegramTrigger upstream. Is that the expected behavior of n8n?

[Jon]

Hey @jb8n,

I have not seen an issue with pinning data from the Telegram node, There are some nodes where pinning is not an option typically ones with multiple branches (If / Switch) as there is rarely a need to pin those as the data will always take the same route if pinned earlier.

When you did the test did you also pin the data of the telegram node? When you pin data it doesn’t mean the workflow will start from that point. My approach when it comes to pinned data is if it is a node that will make a network call I will pin the data this will probably solve a lot of the issues you are seeing.

[jb8n]

Ah I see, thanks!

Given the node(N) I was about to execute only needed data from the node(N-1), I thought that pinning data from node(N-1) would let me debug node(N) quickly.

So, if I have pinned data in the TelegramTrigger node (node (0)), and want to execute a manual call to OpenAI node(node(10)), what’s the best way to do so?

For now, to avoid seeing “We didn’t find any data in Code to simulate an event. Please create one in Code and try again”, when clicking on the Execute Node button of the Editor for node(10), I click on the Play icon of all nodes between node(1) and node(9).

[Jon]

Hey @jb8n,

Assuming Telegram is pinned… Just hit the execute workflow button or press play on the OpenAI node and it should be good.

If it isn’t share your workflow and I will take a look.

[maxT]

@jb8n thanks for your feedback on our pinning feature. As you’ve pointed out, there’s a lot of scope for improving it and making’s it more efficient.

Of course, there’s all manner of ways to improve it but I’d love to try and understand if there is a lower effort change that could make the pinning flow more efficient for you (as then it’s faster to prioritize and release).

In short, if we added a basic history functionality to Pinning feature that allowed you to access the previously pinned data (i.e. after you unpin); would that help? To keep scope light, I’d envision the initial version only has 1x “slot” so can keep only the most recently used pinned data for that specific node. That way you can toggle back and forth between Live and Pinned. Of course, from there we could evolve the feature to keep a handful of these; perhaps with a custom label - this way you could keep a few test cases handy like “Subscription created”, “Subscription Cancelled”.

Any other solution I can envision falls into medium / large feature territory (UI to pin data from previous exec; or backend effort to store numerous previous pinned data in memory) and thus would take longer to build and make it on roadmap. So please do lmk if the single slot solution would alleviate your biggest pains with Pinning.

[jb8n]

It did now! Thanks.
Are nodes of different types behaving differently when it comes to the past data flow, and if so, how?

For example, on the same workflow as above, if

• I don’t pin the initial Telegram node,
• I click “Execute workflow” and trigger the Telegram node
  Then
• I can observe the flow of data going through all my nodes, the connections appear in green
• I can open a Code node, edit the code, and run “Execute node”: it will run immediately and takes in the input gathered at the last full run, which is displayed on the left pane
• I cannot do this with all the nodes, as for some, the left pane will be empty and show “No input data”

Sometimes the past execution will be there, sometimes I have to pin data or re-run the whole workflow to have sample data, and I can’t figure out what makes the difference.

[jb8n]

Hi @maxT ,

Thanks for running suggestions publicly.

Pinning feature that allowed you to access the previously pinned data (i.e. after you unpin); would that help?

As I’ve not used Pinned data this much, I am not 100% confident that I’d use this feature. Other community members could comment better? What would help is if the “previously pinned data” is actually data coming from:

• the previous executions from the execution view,
• the previous execution in the editor view, that shows propagating data and greenering lines in real time

so I don’t have to go to the Executions pane and copy it from there. If you have only 1 “slot”, could this slot automatically remember the data that went through the node when I last clicked “Execute workflow” in the editor mode, even if I didn’t pin it at that time? As I commented above, sometimes the previous data is still there in the Editor view, sometimes it is not.
Also, if the previously pinned data was pinned before the node was changed, and the node was edited in a way the structure of the output has changed, then that previously pinned data is no longer relevant, is it?

Quickly toggling back and forth between Live (i.e. testing the whole workflow with live data and live API/webhook calls) and Pinned (i.e. editing the javascript of a node code, the regex in a Switch node, etc, without needing to start the workflow from scratch) definitely has value, and can maybe be seen as an equivalent between integration tests and unit tests.

The use case that triggered this thread is debugging node-by-node. Before inserting the next node, I want to make sure that the current node is “reasonably stable” and will be able to process the data I expect it to receive. For this, I need data on the left pane. So, what would help is:

• Ability to quickly pin a valid data sample. Quickly means rather “Fetch from previous executions, from last successful execution, from last failed execution, etc” than “manual input”. Something that would replace the current default template [ { "name": "First item", "code": 1 }, { "name": "Second item", "code": 2 } ]
• Ability to edit/pin data in the left hand pane. I understood that to be able to immediately execute Node(n), I have to first go to node(n-1), and pin data on the right pane, correct? I think it would be faster to pin the input data, based on previous executions, from the left hand pane of Node(n), instead of having the “No input data yet Execute previous nodes (From the earliest node that has no output data yet)” message.
• Global unpinning feature: once I feel I’m done debugging, I need to unpin data in all nodes, so that they do not replace live execution data, correct? Now, I do it manually for each node. There could be a feature to unpin all nodes. Or a feature that, when a workflow is turned from inactive to active, a confirmation shows up asking “You are making with WF active with pinned data, do you want to unpin all nodes?” (I am suggesting this because it seems there’s a bug that sometimes makes active workflows unresponsive after editing them, so I always toggle it active->inactive->active after saving)

Apologies for the long non-answer and I hope it helps understanding my habits more than a yes/no answer.

[maxT]

On the contrary @jb8n thank you very much for the detailed reply chock full of insights! I’m not working directly on pinning right this moment but have added this thread to our internal notes so your response can help inform future efforts there.