What I learned building my first RAG pipeline

Retrieval-augmented generation (RAG) sounds simple: find relevant text, give it to a language model, get an answer. Building one during my internship showed me where the real work is.

1. Chunking decides what the model can see. Chunks that are too small lose context; chunks that are too large drown the relevant sentence. Overlap between chunks helped a lot.

2. Measure retrieval on its own. Before judging answers, check whether the right chunk is even in the top results for each test question.

3. Citations build trust. Asking the model to answer only from the retrieved text, and to say which passage it used, made wrong answers easy to spot.

chunks = split(text, size=800, overlap=120)
hits = index.search(embed(question), k=5)
answer = llm(prompt(question, hits))  # answer + [source] markers

Next I want to try hybrid search (keywords + vectors) and re-ranking.