Technologies used in n8n, workflow execution and database

I was not sure if I should separate out the questions so putting them all at one place. It may help others looking for similar information too.

  1. What all technologies are being used in n8n on client side and server side and are thus prerequisites for writing code, building, installing and running n8n?

    I could not find a single page that lists all these, I am finding it in bits and chunks spread across docs and I am just starting with it!

    For e.g., npm - n8n Documentation mentions that Python 2.7 is also needed but so far I have not come across any pages, specially the home or such most visited pages, where Python or other technologies are mentioned as part of tech stack. Even that page does not mention why Python is needed. On github also, only Typescript and javascript languages are mentioned along with a few more client side techs.

    My impression so far was n8n is written in typescript/javascript so it can run both on server and client side. Is that true only for the UI?

    If python is used for server side code for running workflows, it just messes up my entire analysis of open source options of workflow tools! If that is true, where can I find the Python code?

    Also, is there a separate github repo that contains the n8n execution engine code? Where is it?

  2. When a workflow runs, it runs on server side and not client side, right? The user need not stay logged into the n8n UI in order for a workflow to run, right? So say a workflow starts with a trigger node. This trigger/event as configured on this node would be received on the server and the various action nodes that follow would be executed on the server automatically?

  3. Docs mention that nodes are loaded on start up - does it mean when a user loads n8n UI? That means everytime each user loads n8n UI say by refreshing the page? Does it mean workflows cannot be created programmatically because the nodes have not been loaded as there is no UI being used to create the workflow?

  4. How does a workflow execute i.e., synchronously so that next task in the flow does not start until previous task has finished and the thread waits until the entire workflow has finished? Or asynchronously i.e. next task does not start until previous task has finished but the thread returns immediately without waiting? Does the workflow state get saved in DB and when the previous task finishes, next task execution gets triggered? If so, how does this trigger work - is there any message queue or some other event/messaging mechanism used?

  5. Whether message queue is used or not, whether synchronous or async, what happens if a workflow fails because an error occurred in the execution of any of the nodes? Would the workflow state be saved in DB and next execution would resume from the point of failure? Would it re-run the entire workflow causing duplicate execution of nodes prior to point of failure?

  6. How to install n8n from code? What I am looking for is to download the source from github and build it with nodejs commands etc locally and then run with nodejs commands. This is to keep the local development, build and run steps as close to actual deployment steps on our Openshift clusters as possible as described below.

    In other words, to deploy the code, I want to use the same commands to build and run that I use for local development (ofcourse not the debugging and testing commands but actual prod build commands). This will ensure that I am developing in line with deployment steps.

    In our organization, we have Openshift clusters where our apps a deployed. We use Jenkins pipelines to run various steps of deployment like download the code from source repo, build it, run tests, build docker images, push to docker repo, download the docker image on cluster and install it.

    So for local development all steps till running the tests and running the UI and execution engine is what I am looking for. There will not be any docker on my local machine thus those steps are not needed for local working.

  7. How to install the DB? Are there any DB scripts for various supported DBs specially MySql?

  8. Which classes/modules in code deal with DB operations?

  9. Where can I find the DB documentation, the DB table/view schemas and their relationships, stored procs and other details about DB like where are workflows saved when created, where is the audit trail stored when workflows are modified, where is the execution information saved, whatever else that n8n saves in DB?

  10. How do I configure n8n to use a particular DB? Are there separate DAL classes for each type of DB or the same code runs based on configuration?

  11. Another question popped up in my mind when I saw this article. It says the workflow created by one user is not sharable with another user. The entire JSON of the workflow will need to be shared with the user and that other user will need to create the same workflow again.
    That is less than ideal actually. I am still getting started with n8n and trying to understand it all. I had a look at IFTTT and what is possible over there is something I was looking for in n8n too. When there is a workflow already present publicly, other users can find it and just connect i.e., activate it for themselves configuring the parameters to suit their needs. This way it ensures maximum reusability and less rework for the same thing. Is it not possible to do that in n8n?

Hey @Atul_Lohiya, that’s a lot to unpack here.

  1. I believe Python is used during the installation of one of the n8n dependencies and not used directly by n8n.
  2. Yes, workflows will run on the server in the background (as long as they are active), there is no need for a user to be logged in.
  3. Now, this refers to the application itself, not to users navigating the n8n UI.
  4. Executions are synchronous so only one node within a single execution will run at a given time. Multiple different executions can take place at the same time though.
  5. By default, execution data is only stored at the beginning and end of an execution. You can configure this behaviour on a workflow level or globally by setting the EXECUTIONS_DATA_SAVE_ON_PROGRESS environment variable.
  6. I strongly suggest using docker for a pain free prod deployment. However, if you’d like to build n8n yourself, the steps are documented here: n8n/CONTRIBUTING.md at master · n8n-io/n8n · GitHub (starting in the “Development setup” section)
  7. No need to do this manually. n8n creates the required database schema automatically on launch.
  8. I don’t think there’s a comprehensive list of all the db interactions on the code. n8n uses Typeorm for database interactions though, so you could start by searching for such references.
  9. Database structure - n8n Documentation
  10. Via environment variables: Environment variables - n8n Documentation
  11. Easier workflow sharing is something we’re currently working on. You can, however, easily download and import workflows on different n8n instances or alternatively just copy and paste workflows using Ctrl+C and Ctrl+V directly on the n8n canvas.
4 Likes