Why is my OpenAI Embedding not working with Pinecone Vector Store in n8n?

Hi everyone :waving_hand:

Iโ€™m working on an automation in n8n where I:

  1. Trigger with a schedule
  2. Use the Apify actor to fetch scraped data
  3. Format the result in a Code node
  4. Then pass the content to OpenAI Embeddings
  5. Finally, store the vectors in Pinecone

:white_check_mark: Code inside my Code node:

js

CopyEdit

const data = $input.item.json;

if (!data || !data.document || !data.url) {
  return {
    json: {
      pageContent: '',
      metadata: {}
    }
  };
}

const content = data.document;
const url = data.url;

if (typeof content !== 'string' || content.trim().length < 10) {
  return {
    json: {
      pageContent: '',
      metadata: {}
    }
  };
}

const maxLength = 6000;
const finalContent = content.length > maxLength
  ? content.substring(0, maxLength) + '...'
  : content;

const result = {
  pageContent: finalContent,
  metadata: {
    url: url,
    source: 'apify',
    title: url.replace(/^https?:\/\//, '').split('/')[0],
    originalSize: content.length,
    processedSize: finalContent.length,
    timestamp: new Date().toISOString()
  }
};

return { json: result };

:firecracker: PROBLEM:

The embedding isnโ€™t working โ€” Pinecone receives no vector or fails to store the data.
I suspect that the format passed to the OpenAI Embedding node or Pinecone might be incorrect.


:red_question_mark: MY QUESTION:

How should I structure my nodes after the Code node to ensure:

  • OpenAI receives only pageContent for embedding
  • Pinecone receives the proper id, embedding vector, and metadata

Any working example or clean structure would help me a lot :folded_hands:

Hello,
This is my Recommended Fix:

Add a Set node right after your Code node
โ†’ In this node, extract and structure:

  • pageContent โ†’ for embedding
  • metadata โ†’ any extra fields you want to store in Pinecone

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.