Start a Python script with external libraries - via API or Command Execution?

Questions
, ,
1

Hi!

I have a question: I am making an app that allows a person to scrape some data via a Python library. I have a Python script that needs to be triggered after certain user actions.

What is the best way to:

  1. Send a parameter to this script (e.g., a site URL that needs to be scraped)?
  2. Execute this script?

I see three possible options:

  1. Using FastAPI/Django/Flask—just create API endpoints and trigger data collection via requests.
  2. Using a Command Execute node by sending additional parameters through the command.
  3. Somehow adding the necessary libraries to the Docker image. But should I create a new separate image for n8n or modify the existing one?

Could you please suggest what to choose? Are there any other possible options?

Thanks!

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Hi there! It basically comes down to preference. You seem to be aware of the options, but just for thoroughness, I’ll reiterate below:

  • You can use Execute Command with python scrape.py {{args}} and then print out your answer at the end of the python script, then capture it from $json.stdout in the next node.

  • If you want to use a separate API endpoint you can, and then use HTTP Request to receive the JSON, which is a little more professional, but functionally the same thing as above.

  • You can create your own node in a custom folder without messing with the n8n setup. Follow the community nodes tutorial, it’s pretty simple to setup and link. But then of course you must work in node.js, not python.

  • You can use the Code node with python, but that is a subset of python built for webassembly and not everything is compatible for it. In that situation you would have to add new modules into the docker image using pip.

2 Likes
4

JayF, thanks for your answer!

  • You can use the Code node with Python, but that is a subset of Python built for WebAssembly, and not everything is compatible with it. In that situation, you would have to add new modules to the Docker image using pip.

I added libraries via pip in the Docker image, and I can see them using python3 -v and similar commands. However, Code node doesn’t recognize the additional libraries. The error message indicates that some “Python sandbox” is involved.

Do you have any information on how to modify the Docker image so that the Code node can execute these libraries?

5

It’s possible that Pyodide doesn’t support the packages you are using.
For more information please read the reply in this thread:

1 Like
6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.