Frustrations and Future

I think n8n is a great project. It’s been already a while since I started playing with it for the first time - unfortunately I still find it quite frustrating. I never feel productive using it.

Now I hope I can give this post a positive spin and it does not come across as bitching.

It’s great that the number of nodes/integrations is increasing at such pace :tada:, but IMO some of the older nodes also need some love.

  1. Google Sheets

Appending works fine. Updating works fine - if a key exists. But do I really need to try and model an upsert manually as workflow? IMO it would be great if this could be handled by the node automatically when a certain option/mode is set. Search for the key, update if it exists, otherwise append.

  1. Speadsheet File

Trying the same with a local spreadsheet file I read the docs on the Binary Property and…

Binary Property field: Name of the binary property in which to save the binary data of the spreadsheet file.

…if this was a movie you now would hear crickets :wink:
Binary data of a spreadsheet file? What? Why? How?

This note in the docs clears things up:

You will need to use an additional node such as the Read Binary File node […] to pass the image file as a data property to the Spreadsheet File node.

But that makes the name of the node very awkward.
Why not name it as some kind of conversion node it really is?

And wouldn’t it be nice to have a more similar interface to the e.g. Google Sheets?

  1. SQL

No support for sqlite? Bummer. Because of the native part?
Searching the nodes for “sql” does not return “Postgres”.
(Similar thing with Email and Amazon SES)
Maybe also index descriptions or tags?

  1. MongoDB

MongoDB also natively supports upsert - but the node does not expose that feature. I guess it would also be nice to also allow freeform queries for Insert and Update.

  1. Node State

This is my biggest pain point. We talked about this before. Some node operations are just in dire need for this. The “tracking changes” workflow I posted a while back was just horribly complex for the simple task it should be. And doing this natively could also pave the way to workflows like:

<do A> => <wait for 1 hour> => <do B>

That would be so awesome. And as a cheery on top it would be sweet to be able set the states inside this state machine.

<A> => <wait for 24 hour> => <B>
<webhook> => <goto B>

Conceptually quite easy - no idea about the implementation side with the current architecture.

  1. Scalability

TBH I am really afraid to use n8n for anything serious because I am unsure how a clustered setup would look like. This somewhat ties into 5), too. Having the config and state separated out, running multiple instances should be no problem. But are there concrete plans for that? I read that you are working on a hosted version. I assume that would happen for that anyway.

Let me know if I should turn some of those points into e.g. github issues or feature requests here.

Other than that I am happy the project is making so much progress. :tada:

2 Likes

Hi @tcurdt,

Thanks for your detailed feedback - I’m responsible for product design at n8n so knowing your pain points is very helpful for us to make n8n better. Line item replies:

  1. Totally see how upserts could be invaluable in Google Sheets node - I recommend creating a “Feature Request” post for that functionality. The more votes it gets in the community, the faster something like this will bubble up to the top of our backlog for implementation.

  2. Some of our older nodes definitely need some love/ standardization, as does how binary files/ data is used. Our dev rel team is currently working on various educational resources to make binary data usage in n8n more clear. I do not have a time estimate on when the Read Binary File node would be improved BUT we are assessing how to go about standardizing the experience of similar nodes. Separately, we’re also going to be spec’ing out various transform nodes to reduce dependency on Function node and I expect some renaming/ information architecture love to occur on binary data related nodes as well.

  3. sqlite - I similarly recommend creating a specific “Feature request” for it (but first searching if already exists). As far as finding nodes more easily, I am happy to say that we are working on adding a codex to each node (standardized json file with metadata like aliases and categories). Once that is completed, we plan to refactor the nodes list with aliases (alt tags), categories etc.

  4. Same as above, please create a “Feature request” for freeform queries.

  5. and 6. As you aptly pointed out, this is a non-trivial task but one we recognize as very important. Behind the scenes, we are actively working on this and are dedicating a great deal of our overall resources to solving the scalability and node state issue (our solution should solve both). To underscore, this is the biggest/ most important body of work actively on our agenda atm. Our approach here is “do it properly the first time” so this will take some time but will result in a robust, very scalable n8n.

From everyone on the n8n team, just wanted to say a big thank you for your thoughtful points :heart:. It’s this sort of discourse that will improve n8n and guide where we place resources!

4 Likes