Local AI issue with accessing binary image

I’m testing out local AI installed on my server, and Llama 3.1 doesn’t seem to see the binary file.

I have followed the manual to the letter https://www.youtube.com/watch?v=xz_X2N-hPg0

LLamma is accessible and If I put the URL to the external resource, it works, but if I pass an image as binary (following the manual) LLAMA refuses to see it.

What is the error message (if any)?

Please share your workflow

Share the output returned by the last node

text:Unfortunately, I’m a large language model, I don’t have the ability to see or access images. The [img-0] notation suggests that you’re trying to reference an image, but without more context or information about what the image is supposed to be, it’s impossible for me to know what’s on it.\n\nIf you’d like to describe the image or provide more context, I’d be happy to try and help!

It should have seen it

Information on your n8n setup

  • n8n version: - Version 1.55.3
  • Database (default: SQLite): - PostgreSQL 16.4
  • n8n EXECUTIONS_PROCESS setting (default: own, main): main
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • Operating system: - Ubuntu 24.04 LTS

Hi there, what Ollama Chat Model are you running currently?

Edit: My apologies, I think I see now you said Lama 3.1.

Correct it’s 3.1 that came with local AI installation

@djangelic any suggestions. This issues makes this local AI feature useless.

Hi, I’m still working on figuring this out, thanks for your patience.

bump, following, this is quite interesting

Were you able to reproduce this bug and confirm the issue is there?

Having DNS issues unrelated to this while trying to spin up the instance. Should have a response by Friday.

Got the dns issues sorted, but now I’m having trouble getting ollama to work at all. On Monday I’ll talk to our AI team to walk me through so I can work with the docs team to put together better documentation on it.

Hi @ramblingnate,

The issue you’re seeing is due to llama3.1 not supporting image input; it’s a text-only model. In the video you posted, I’m using a different model — lava-llama3, which does support image input. You could also use llava or any other vision model from Ollama; just make sure it has that Vision tag.
CleanShot 2024-09-17 at 16.09.57

Using llava:7b I’m able to process images and get sensible output:

I hope that helps!

Oleg

2 Likes

Oleg, thanks for getting back to me; however, the issue is the same with HTML, pdf files, etc. It’s not the vision part. Something in your AI package doesn’t see it.

@ramblingnate Doesn’t see what? Again, you can only pass text to text only models. Trying to pass pdf, HTML or any other document file wouldn’t work because those are binaries. So you’d need to convert them to text first and then pass that text to LLM. Here’s an example with a PDF and llama3.1:

The workflow fetches PDF using the HTTP Request node, extracts the text from PDF, and then passes the text to the Basic LLM Chain for summarization.

What version of N8N are you running? My Basic LLM node looks a bit different. The model says there is no attached PDF, but the file is attached via chat.

It finally worked, but the hosted chat doesn’t produce any result in the chat even though the the execution is completed inside n8n. What could be causing it? Hosted chat doesn’t produce output.

image

It would really help if you could provide the workflow JSON, it’s really hard to debug just based on a few screenshots. Also, make sure you’re using the latest versions of the nodes(you can check in the node settings) as some features might not be available in previous versions.

Sure totally understandable. I’m on 1.55.3 hosted version.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.