Building a RAG System with LangChain and Pinecone

RAG (Retrieval-Augmented Generation) systems combine the power of large language models with your own data. Here's how to build one.

What is RAG?

RAG enhances LLM responses by retrieving relevant information from your knowledge base before generating answers.

Architecture

1. Document Processing: Split documents into chunks 2. Embedding: Convert chunks to vectors 3. Storage: Store in vector database 4. Retrieval: Find relevant chunks 5. Generation: Generate answer with context

Implementation

Step 1: Install Dependencies

npm install langchain @pinecone-database/pinecone openai

Step 2: Set Up Pinecone

import { Pinecone } from '@pinecone-database/pinecone'
const pinecone = new Pinecone({
  apiKey: process.env.PINECONE_API_KEY,
})
const index = pinecone.index('my-index')

Step 3: Process Documents

import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter'
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
})
const chunks = await splitter.splitText(document)

Step 4: Create Embeddings

import { OpenAIEmbeddings } from 'langchain/embeddings/openai'
const embeddings = new OpenAIEmbeddings()
const vectors = await embeddings.embedDocuments(chunks)

Step 5: Store in Pinecone

await index.upsert(
  chunks.map((chunk, i) => ({
    id: chunk-${i},
    values: vectors[i],
    metadata: { text: chunk },
  }))
)

Step 6: Query System

const queryEmbedding = await embeddings.embedQuery(question)
const results = await index.query({
  vector: queryEmbedding,
  topK: 3,
  includeMetadata: true,
})
const context = results.matches
  .map(match => match.metadata.text)
  .join('\n\n')

Step 7: Generate Answer

import { ChatOpenAI } from 'langchain/chat_models/openai'
const llm = new ChatOpenAI()
const answer = await llm.call([
  {
    role: 'system',
    content: 'Answer based on this context: ' + context,
  },
  { role: 'user', content: question },
])

Use Cases

- Customer Support: Answer questions from documentation - Internal Knowledge Base: Company wiki search - Research Assistant: Query research papers - Code Documentation: Search codebase

Best Practices

1. Chunk Size: Experiment with different sizes 2. Overlap: Prevent context loss 3. Metadata: Store useful information 4. Caching: Cache frequent queries 5. Monitoring: Track performance

Conclusion

RAG systems unlock the power of LLMs with your own data. Start small and iterate.

Need a custom RAG system? I build AI-powered solutions. [Get in touch](/contact)