Binary and json output with llm (to apply some code on the ouput of OpenAI or Anthropic node llm)

Describe the problem/error/question

Hello
I’d like totest if an image is rotated or not.
I’ve tried multiple solutions, but still stuck as I can identify if an image is not correctly oriented, but I can’t apply some code (JS or Python) to rotate it, then save it (I’m not sure where yet, but I could use a SFTP node, a temporary execute command node, then to send it in a directory)

I can’t make the binaries available and / or also loosing the URL passed to the llm.

I have tested the OpenAI llm , with URLs binaries, or binaries, and also Anthropic node (that needs biaries only).
Last issue, with Anthropic I may only pass JPG files, as the Mime Type is not recognized correctly except for JPG. ( and they claim to accept webp, png etc… also)

What is the error message (if any)?

Please share your workflow

I’m going to add the workflows after posting as I have 403 errors.

Share the output returned by the last node

Information on your n8n setup

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Here’s the Anthropic workflow

Here’s the OpenAI workflow (first part as the full version returns a 403 error while posting)

I can’t post the OpenAI with the advanced AI node as I have a 403 error on posting :frowning:

Hey @NiKoolass,

I am not sure if AI is smart enough to know if an image is rotated or not, It isn’t really something I have ever tried to do.

What have you found out from your testing so far?

1 Like

It’s an interesting problem because without an original to compare, who is to say if an image has really been “rotated”? And seriously, what does “up” and “down” even mean for an AI? :slight_smile:

I created a quick test below which is able to identify if the image is “rotated” but is unable to correctly provide the params to rotate it. I would imagine specialised AI vision models to do a lot better than Multimodal but I haven’t experimented enough to say for certain.

1 Like

Yes it does work !
I think I’ve mixed up a bit the versions I was working on, but in the prompt if you stipulate that it has to be easily human readeable horizontally, it takes it ! :slight_smile:
I have successfully tested this approach.
What’s important is to take things step by step, with multiple llm calls, with the lowest tokens possible, produce a json, sorts file that needs to be rotated, then rotate them (here I fail atm to use a tool for that) then keep going on analysis…etc…
Bascially, reducing the “noise” at each step.
I’ll share some stuff very soon, I guess we’ll find out very soon :slight_smile:

1 Like

Here it is, with OpenAI and also Anthropic Claude 3 Haïku

1 Like

That is some nice work.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.