I have my ETL done in Power BI, but MS Power BI has limitations and I need to reconstruction this elsewhere. Does n8n have ETL functionality? If no…can anyone recommend an ETL? Data sources are JSON, Sheets / Excel, web scraping, HTML. Data is very “messy” and needs extensive cleanup (which Power BI) is able to handle.
It looks like your topic is missing some important information. Could you provide the following if applicable.
- n8n version:
- Database (default: SQLite):
- n8n EXECUTIONS_PROCESS setting (default: own, main):
- Running n8n via (Docker, npm, n8n cloud, desktop app):
- Operating system:
Welcome to the community @mcs24 !
Tip for sharing information
Pasting your n8n workflow
Ensure to copy your n8n workflow and paste it in the code block, that is in between the pairs of triple backticks, which also could be achieved by clicking </>
(preformatted text) in the editor and pasting in your workflow.
```
<your workflow>
```
That implies to any JSON output you would like to share with us.
Make sure that you have removed any sensitive information from your workflow and include dummy or pinned data with it!
Sure, you can use n8n as ETL pipeline depending on the available API that you need to utilize.
Thanks for the reply. Is there training info on ETL Pipeline?
There are some courses on n8n that help you understand how to use n8n.
What to build exactly and how to do it for your ETL pipeline of course depends on a couple of factors. This mostly comes down to experience in data enginering.
If you have experience with that, you should be fine after getting to know n8n through the courses.
If you do not have experience with that, you will probably need to get some more help on that. Probably best to hire someone with experience to help you with that or do it for you.
I used to be a BI consultant, my first goal with n8n was to replace the ETL pipeline built in Qlikview/Qlik Sense with n8n, that worked beautifully as soon as I understood how n8n works.
So yeah it can definitely be done, you just need a bit of experience to know what to do.
Thank you for the reply.
I have experience with Excel/Sheets, data engineering, and I used Power BI to build out and document my “steps” and problems with data. I just cannot use Power BI for my actual project.
I will look again for n8n ETL training but I am not finding any —mostly automation videos assuming the data is clean and “ready to go”.
Hi @mcs24
Not sure about Power BI, as I have never used it. To my knowledge you cannot do actual data enginering with Power BI as it is just for visualisation of the data.
n8n is an automation platform, so yeah most of it will be on that. Except for the AI hype now which also is very much on everything n8n.
Data enginering is all about getting data and transforming it to what you need, which can be done with n8n. You just need the experience and knowhow on how to do it. I have done it, it is not difficult. But yeah there will probably not be a tutorial on how to do it, as most of it is just experience with data enginering. And then of course know how to use n8n to do it.
I like Power BI because it was very similar to something I am familiar with…Excel functions. I was able to practice ETL and workout the problems nut Power BI ETL has limitations for my project. I just need to recreate the ETL in a new platform.
I like n8n automations and will need some AI functionality.
awesome, then yes you can of course use n8n for it.
Start with the courses to understand how n8n works. and then start thinking of what you want/need to do and just do it.
Of course if you are stuck with something then you can ask on the forum or hire an expert to help you out and guide you through it.
Hey @mcs24 , you could use workflow and pipeline interchangeably. Learning how to build workflows with n8n is the same as learning how to build n8n pipelines. Here are the official n8n courses:
Also there are tons of n8n tutorials created by the community members on YouTube.
n8n is indeed a great tool for ETL processes. The community and official resources, like the n8n courses and YouTube tutorials, are very helpful for getting started. By the way, the n8n team is growing and hiring, so if you’re interested in working with n8n, it’s a great time to get involved!
Thanks for the reply. We slowly figured out the steps with the help of documents and videos. “dirty” data from multiple sources is now cleaned out. We have a little more work to do then work on dashboards.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.