This is another quick tutorial in Langchain Code node series. This time we’ll be using the Langchain Code Node to build a custom retriever which will use Cohere’s Rerank API to give us signficantly better results in our RAG-powered workflows.
If you haven’t already, please also check out the previous post in the series!
Edit 1: Replaced hardcoded vector store code with vectorstore subnode.
Edit 2: Add prerequisite to set NODE_FUNCTION_ALL_EXTERNAL
Background
This tutorial is in response to this questions post. At time of writing, there isn’t much documentation on building your own custom retrieval code so here we are.
Prerequisites
- Self-hosted n8n. The Langchain Code Node is only available on the self-hosted version.
- Ability to set
NODE_FUNCTION_ALLOW_EXTERNALenvironmental variable. For this tutorial, you’ll need to enable the@langchainpackage.- set
NODE_FUNCTION_ALLOW_EXTERNAL=@langchain
- set
- Pinecone API Key & VectorStore. I’m using Pinecone out of convenience but if Pinecone is not your thing, feel free to swap it out for any n8n supported vectorstore.
- Cohere API Key. We’ll need this to access the Rerank API. Grab one from the Cohere developers portal.
Step 1. Add the Langchain Code Node
The Langchain Code Node is an advanced node designed to fill in for functionality n8n isn’t currently supporting right now. It’s a really powerful tool but requires coding know-how; only use it if you know what you’re doing!
- In your workflow, open the nodes sidepanel.
- Select Advanced AI → Other AI Nodes → Miscellaneous → Langchain Code
- The Langchain Code Node should open in edit mode but if not, you can double click the node to bring up its editor.
- Under Inputs
- Add an input with type “Main”, max connections set to “1” and required set to “true”
- Add an input with type “Language Model”, max connections set to “1” and required set to “true”
- Add an input with type “Vector Store”, max connections set to “1” and required set to “true”
- Under Outputs, add an output with type “main”.
- Go back to the canvas.
- On the Langchain code node you just created, add Language Model and Vector Store Subnodes. I’m using OpenAI for both LLM and embedding but you can just any you like (embeddings need to match with what you used to insert the data however).
Step 2. Writing the Langchain Code
For this tutorial, I’ve tried to keep the implementation as close to a standard chat template chain as I can. The introduction of Rerank does introduce a few more steps but I hope it’s not too bad… if you know how to improve it, let me know!
- Open the Langchain Code Node in edit mode again.
- Under Code → Add Code, select the Execute option.
- Tip: “Execute” for main node, “Supply Data” for subnodes.
- In the Javascript - Execute textarea, we’ll enter the following code.
- Be sure to change
<MY_COHERE_API_KEY>before running the code!
- Be sure to change
// 1. Get Inputs
const { chatInput } = this.getInputData()[0].json;
const llm = await this.getInputConnectionData('ai_languageModel', 0);
const vectorStore = await this.getInputConnectionData('ai_vectorStore', 0);
const systemPrompt = 'You are a helpful assistant.';
// 2. Setup Cohere Rerank
const { CohereRerank } = require("@langchain/cohere");
const cohereRerank = new CohereRerank({
apiKey: '<MY_COHERE_API_KEY>', // Default
model: "rerank-english-v2.0", // Default
});
// 3. Construct a chain using a ContextualCompressionRetriever
const { createStuffDocumentsChain } = require("langchain/chains/combine_documents");
const { ChatPromptTemplate } = require("@langchain/core/prompts");
const { StringOutputParser } = require("@langchain/core/output_parsers");
const { ContextualCompressionRetriever } = require( "langchain/retrievers/contextual_compression");
const retriever = new ContextualCompressionRetriever({
baseCompressor: cohereRerank,
baseRetriever: vectorStore.asRetriever(),
});
const prompt = ChatPromptTemplate.fromMessages([
["system", systemPrompt +'\nYou have the following context:\n{context}\n\n'],
["human", "{question}"],
]);
const ragChain = await createStuffDocumentsChain({
llm,
prompt,
outputParser: new StringOutputParser(),
});
// 4. use our chatInput in our chain
const output = await ragChain.invoke({
context: await retriever.invoke(chatInput),
question: chatInput,
});
return [ { json: { output } } ];
Step 3. We’re Done!
All there is left to do is add the Chat Trigger to the left side of your Langchain Code node and Congratulations! You’ve now successfully built a custom Q&A Chain using Cohere Rerank.
Honestly speaking, this was quite a challenge to put together! Here are my thoughts on the whole process:
- Developing in the Langchain Code Node is difficult because of lack of autocomplete and debugging tools. To keep your sanity, it’s recommended to setup tighter feedback loops such as using pin data and the manual trigger.
- Langchain docs are a maze and I ended up relying heavily on google to figure out what I needed. If you’re finding yourself in the same spot, keep at it you’ll get there eventually
Cheers,
Jim
Follow me on LinkedIn or Twitter.
If you like n8n & AI topics, be sure to check out my other posts in the forum.
