Hello,
I am an intensive user of dagster for dataflow processing (in python) but I really like the API approach from n8n with it’s UI.
I wonder if I can have best of both world. One use case would be n8n is managing the API connections and pass the data (json) to dagster for complexe dataflow then come back to n8n to make whtaever we need.
For this I see 3 solutions:
1- Full pipeline with node entry → dagster graphql call + execution → get results from dagster
2- break in two n8n workflow one for the left part and one for the right part. the 1st triggers dagester and finished. dagster process the data and trigger the 2nd n8n worflow.
3 - dagster call n8n worflow and push to another n8n worflow.
My feeling is that the 1st is easier to maintain (2 workflow) but requires a node that waits for dagster process to finish.
Any insights are welcome !