Supabase error on different vector dimensions

Questions
1

Describe the problem/error/question

I have a Supabase knowledge base where all embeddings are stored with 3072 dimensions (created with the text-embedding-3-large model). I connect this to an AI agent in n8n to ask questions.

When I query the Vector Retrieve tool in n8n with the text-embedding-3-large model and 3072 dims set as option, I get the following error:

Error searching for documents: 22000 different vector dimensions 1536 and 3072 null

What I have verified

  • Running:

    SELECT vector_dims(embedding) 
FROM knowledgebase 
LIMIT 1;

    confirms my stored embeddings are 3072 dims.

  • When I query the same Supabase RPC directly via the Supabase API, I get a correct and relevant response (no error).

  • This makes me suspect that n8n may be generating or passing 1536-dim vectors for the query, even though I have configured it to use text-embedding-3-large.

My question

Could it be that, despite explicitly selecting text-embedding-3-large (3072) in n8n, the embeddings node is actually generating 1536-dim vectors?

Or is there something in n8n’s Supabase / Vector Retrieve tool implementation that could be defaulting to the wrong dimension?

Please share your workflow

Share the output returned by the last node

Last node = vector retrieve tool. Only the error as stated above

Information on your n8n setup

    • n8n version: 1.102.3

    • database: PostgreSQL 14

    • EXECUTIONS_MODE: queue

    • Running n8n via: Docker in k8s

    • Operating System: Amazon Linux EKS 1.28.5-20240227

2

Hey @Anais_van_Asselt hope all is good.

Please show us your documents table and your match function in supabase.

3

Hi @jabbson I am, I hope you are as well, thanks for your quick reply! My documents table (name = knowledgebase) looks like this:

id (bigint), content (text), metadata (jsonb), embedding (vector(3072))

Match function (based on LangChain | Supabase Docs , referred to from Supabase Vector Store node documentation | n8n Docs ):

CREATE OR REPLACE FUNCTION public.match_knowledgebase(query_embedding vector, match_count integer DEFAULT 5, filter jsonb DEFAULT NULL::jsonb)
 RETURNS TABLE(id bigint, content text, metadata jsonb, similarity double precision)
 LANGUAGE sql
AS $function$
  SELECT
    d.id,
    d.content,
    d.metadata,
    1 - (d.embedding <=> (query_embedding::vector(3072))) AS similarity
  FROM public.documents d
  WHERE (filter IS NULL OR d.metadata @> filter)
  ORDER BY d.embedding <=> (query_embedding::vector(3072))
  LIMIT match_count;
$function$

LMK if you need more info! KR, Anais

4

Hey @Anais_van_Asselt thank you for your answer.

If I am reading this right, your match function is still querying from original documents table instead of the one you created with a different name:

image
image713×312 34.2 KB

I assume documents and knowledgebase tables have different dimentionality and this is where the issue could be coming from.

6

hi @jabbson, apologies, that’s not the correct match function.
When I printed the function I was messing around with the match function to try and fix it by explicitly mentioning the 3072 dims.

So you can disregard the last function, this is the right function (where I can query on query with 3072 dims through Supabase API, just mention EMBEDDING_MODEL=text-embedding-3-large there)

CREATE OR REPLACE FUNCTION public.match_knowledgebase(query_embedding vector, match_count integer DEFAULT 5, filter jsonb DEFAULT NULL::jsonb)
 RETURNS TABLE(id bigint, content text, metadata jsonb, similarity double precision)
 LANGUAGE sql
AS $function$
  SELECT
    kb.id,
    kb.content,
    kb.metadata,
    1 - (kb.embedding <=> (query_embedding::vector(3072))) AS similarity
  FROM public.knowledgebase kb
  WHERE (filter IS NULL OR kb.metadata @> filter)
  ORDER BY kb.embedding <=> (query_embedding::vector(3072))
  LIMIT match_count;
$function$

based on SELECT pg_get_functiondef(‘public.match_knowledgebase(vector, integer, jsonb)’::regprocedure)

7

Apologies, I am on a vacation with limited access to the internet. Do you still need assistance with this or have you figured it out?

8

then enjoy your holiday! I’m still facing the same issue I’m afraid, but it can wait :slight_smile: (I have a workaround in LangGraph, but it would be great if I can set this bot up in N8N so my non tech colleagues can tweak as well!)

9

Hey @Anais_van_Asselt hope all is good. I am back. Could you try it with the default table and function definitions?

create table knowledgebase (
  id bigserial primary key,
  content text,
  metadata jsonb,
  embedding vector(3072)
);

create function match_knowledgebase (
  query_embedding vector(3072),
  match_count int default null,
  filter jsonb DEFAULT '{}'
) returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
  return query
  select
    id,
    content,
    metadata,
    1 - (knowledgebase.embedding <=> query_embedding) as similarity
  from knowledgebase
  where metadata @> filter
  order by knowledgebase.embedding <=> query_embedding
  limit match_count;
end;
$$;
10

hi @jabbson , now I’m back from a long holiday :sweat_smile:
I tried with a new table using your queries, but still getting ‘Error searching for documents: 22000 different vector dimensions 3072 and 1536 null‘ (I tried embedding 3-large with and without 3072 dimensions option), what’s the alternative, switch to another embedding model?