ETL - We have the extract, we have the Load as visual elements but we lack the Transform. Understandable with the age of the platform.
Relying on js to do this limits the audience of the tool. implement the transformation as js behind the scenes though as boomi and mule and jitterbit and workato have there own proprietary scripting languages which always felt limiting as ‘everyone’ knows js to a point
https://community.n8n.io/search?q=transform
Transformation is really where all the bugs and work is done within these platforms so make it easier
Hi Geoff,
This is something we have on the roadmap to address soon, so would be interested in how you see it working.
For example, what level of granularity would you have this at? Different nodes for every operation, or some master ‘transform’ node? If the latter, how would you see the UI looking?
George…
Let’s chat about it in our meeting in an hour
It is a MASSIVE topic
BUT there are no excuses for you not to investigate and borrow from Jitterbit (two products), Mulesoft (3 Products), ̶D̶e̶l̶l̶ Boomi, Workato, Integromat, Zapier as they offer trial tiers and they have already invested millions on the decisions you have to make.
First question is who is the audience…develoeprs, low-coders, no-coders…look at mule with their 3 anypoint products anypoint studio, flow designer, orchestrator…all using the same underlying tech but presented differently…
Secondly design time testing is soooooooooooooo important. Be able to test the transformation at design time instead of running it and hitting and hoping…this keeps n8n support lean as transforming data is a huge drag on support calls, secondly introduces self learning at a different level. I cant emphases this point more.
Massive topic
Important to remember Transformation is a misleading name…these objects can do many roles such as
Enterprise Integration Patterns - Introduction to Message Transformation
Enterprise Integration Patterns - Introduction to Message Routing
I have to agree, 90% of the issue I come here for are related to debugging javascript related to transforming or iterating over data.
I reckon you could cut down a huge chunk of repeatable forum queries with better transform nodes that rely less on custom written javascript function nodes.
Don’t get me wrong they are powerful as hell and can do basically anything (if you have the knowledge) but most things i think can be done via a node
I agree.
I know little JavaScript and try to learn more in my free time but i feel bad come here asking about simple things for data transformations.
The thing is i have the idea in my head for a project, i know the logic and how to solve it BUT i get stuck because i don’t know how to translate it to JavaScript.
And the bad thing i feel is i know for example, map function i know the logic how it behaves and the theory but i can’t write the damn thing correctly to run successfully when i want to manipulate some data. Sorry to bother you and come here asking for solutions.
<3
@Mulen have you seen the (newish) Item Lists node? Granted it doesn’t cover all cases but helps with various transform utilities related to arrays <> n8n items. By using multiple Item Lists nodes, it’s possible to achieve things like “Output a single random item from a bigger input list”.
Yes indeed. But for example check my post from today.
A very basic concept to check if a value from a node array is contained in another’s node array.
It required some custom JS for a basic check.
I’m glad MutedJam helped me, really appreciate it.
I synced with @MutedJam and there is a way to do this with the Merge node in a no-code way. More on that in your other post: IF Node Values Exist - #5 by MutedJam
Cheers!