IBM WatsonX.AI models for agents

Feature Requests
1

The idea is:

Add a node for models hosted on WatsonX.AI to use in LLM-chains and agents.

My use case:

We host our custom models on WatsonX.AI, which we want to use for inference within N8N. Since they are not completely conform OpenAI spec, there is no way to just change the base_url of that node.

I think it would be beneficial to add this because:

IBM provides data-zone specific LLM’s in the EU, which is valuable for the customers situated there.

Any resources to support this?

https://ibm.github.io/watsonx-ai-python-sdk/fm_model_inference.html

Are you willing to work on this?

We don’t have the development skills for this.

2

Hey, we at Bitovi ended up making a watsonx llm node and a custom ai agent node that allows it to connect to the node which may help with your setup. Here is a link to npm package which should have links to the github if wanted. Link

1 Like
3

Hi Manuel @Manuel_Preston thanks for sharing custom node for watsonx - I tried it and got it working, however sometimes some mystic errors come out from model and all of sudden it was not able to produce solid answer, did you find anything similair?

4

How strange. I haven’t seen anything like that myself. A lot of AI host services, like Ollama or Watsonx, have system prompts and backend processing. My experience with IBM is that their service is a bit less battle-tested, which can lead to unusual results.

For example, when using JSON structured output, IBM API docs directly say:

“when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit.”

If you are getting something like this, it may just require some more system prompt tuning on your end, unfortunately. The node itself is really just a langchain wrapper, so not much complexity on the node side of things.

5

Thanks! And that might be exactly the case, I remember when using IBM watsonx promptlab there was those settings to handle things like ’stop/end stream’ etc and if not ’nicely’ set model might produce nonsense charachters or empy spaces etc… meaning that handling response properly need proper guideline for expected (json) format in n8n prompt itself - I’ll try that approach .