Running Python with n8n

Hi n8n community,

We have some Python files that we would like to run on a regular basis, and I was thinking of using n8n for this: A Cron node followed by an Execute command node to run a specific Python file.

Is this possible, and if so does anyone have any insights that could help us deploy such a solution? We have deployed n8n on Heroku, maybe there are some specifics? Such as how to install Python and all necessary libraries.

Also, once we have the Python file that we have developed locally, should we use an FTP to send it to our n8n database or can this be done directly via n8n?

Many thanks,
Maxim

2 Likes

Hi @maximpoulsen , that’s totally doable as you can see in the screenshot. Basically the execute command node is a bash instance where you can execute any available command.

To achieve what you are aiming for, you need to have python installed on your instance with the required libraries you will be using.
Also you can use the execute command node to install them, just make sure you have the right user (with the right privileges) when starting then n8n instance.

Concerning the files, it is better to save them directly on the file system, rather than using n8n database, also you can use a combination of FTP and writeBinaryFile node to retrieve and save the files before executing them.

3 Likes

Btw. if you want to build a docker-image with python you can extend the default n8n image and extend it.

For that create a file name Dockerfile with this content:

FROM n8nio/n8n
RUN apk add --update python3 py3-pip

And then run in the same folder the following command to build it:

docker build -t n8n-python .

If you then use this image, you will be able to run python scripts (be aware that you would have to use the command python3 and not python).

10 Likes

Thanks @jan,
Is it also possible to add python to a docker-compose file? I use docker-compose to deploy n8n and traefik but would love to add python support as well.

Yes, @chris! The docker-compose file just describes how what docker images to use and with what parameters. So you would do exactly what I did explain above and then change the docker-image you use in the docker-compose file.

So from something like:

...
  n8n:
    image: n8nio/n8n
...

to:

...
  n8n:
    image: n8n-python
...

Thanks @jan.

To understand completely:
image: n8nio/n8n uses official n8n image from the docker registry, while
image: n8n-python uses the image I locally build (which uses official n8nio/n8n, just with python)?

I know this is a Docker specific question, so appreciate your support here.

1 Like

Yes, that is 100% correct.

It is however important to remember that you would have to build a new custom image every time a new n8n version gets released.

So would be best to use a Dockerfile like this:

FROM n8nio/n8n:0.119.0
RUN apk add --update python3 py3-pip

in which you specify the exact version of the n8n image you want to use. After that has then be build you would have to do:

# Stop the currently running containers
docker-compose stop 
# Remove them so that it has to recreate them and does not reuse the old ones
docker-compose rm
# Startup n8n again
docker-compose up -d
4 Likes

Is there a way to add external libraries permanently? Every time I restart my docker container, the requests library gets lost and I’m greeted with this error.

After I successfully install the library again with pip3 install requests in an execute command node, I get the following warning,

WARNING: The script normalizer is installed in '/home/node/.local/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available. You should consider upgrading via the '/usr/bin/python3.8 -m pip install --upgrade pip' command.

I think they are related? So how do I fix this? Thanks :slight_smile:

1 Like

Hey @d4vidsha,

You may need to make a storage volume to store the Python modules and map it to the module path on the container, that should then make them available.

I have not tested it though so it is just theory.

I think so too @jon. I’m not sure how to do this though.

Have you throught about using a requirements file that you load in your docker file to install the modules needed when the container is built? Could be easier and more familiar if you know your way around Python.

Is there an update on how to install the python image and have libraries stored?

How could i do that? create a storage or folder i got but where you make them available?

I explain my process in full here: How to automatically update n8n-python

If there’s anything in this which doesn’t make sense, hit me up :smile:

3 Likes

Thats cool info, will work on it.

By the way do you know on how to install Anaconda?

Is there a way to get some custom data from previous node and give that as a input to this python node, to transform the data and then later output it to other node ?
I cant find any node in n8n that would support advanced data transformation , like pandas , or once the data is fetched from diff source, no sql like query option to transform that data to my liking.
I was thinking if i can do this in python

Hello Jan, can you help me?

The DockerFile i can create anywhere? Or it already exists?
I pretty new with Docker and a little confused :sweat_smile:

Old link now does not work, here is the new link.

3 Likes

How about when running it on the npm global. I have been having an error

“ERROR: exitCode: 1 Traceback (most recent call last): File “C:\Users\user\AppData\Local\Temp\36107e071cc1c15332690c0d325e0c13.py”, line 1, in import fire ModuleNotFoundError: No module named ‘fire’” yet I have al modules installed