RAG ingestion

1 min read
On trails:AGRAGstep 4/8

Ingestion is the work done beforehand and offline, before any user asks anything: take the data, chunk it, embed the chunks, save them in a vector database.

Offline ingestion: documents are split into chunks, each chunk goes through an embedding model to become a vector, and the vectors are saved in a vector database. OFFLINE, BEFOREHAND Documents PDFs, code files Chunks smaller parts Embedding model text to vectors Vector DB saved for search
Run once, and again whenever the source changes.

The steps

Chunk: a PDF or code file is cut into smaller parts, usually a few hundred tokens each, so that each part can be retrieved on its own. Embed: each chunk goes through an embedding modelEmbeddingsData turned into an array of numbers, placed so that similar meaning sits close together.Opens in a new tab and comes out as a vector. Save: the vector is stored in a vector databaseVector databasesStores for embeddings that find the nearest ones fast, by trading a little accuracy for speed.Opens in a new tab with the original text and metadata such as the source, so the text can go back into a prompt and be cited.

In LangChain

Python
from langchain_core.documents import Documentfrom langchain_core.embeddings import DeterministicFakeEmbeddingfrom langchain_core.vectorstores import InMemoryVectorStorefrom langchain_text_splitters import RecursiveCharacterTextSplitter docs = [Document(page_content=open("handbook.md").read(), metadata={"source": "handbook.md"})] splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)chunks = splitter.split_documents(docs) # Placeholder embeddings; swap in a real embedding model.embeddings = DeterministicFakeEmbedding(size=256)store = InMemoryVectorStore(embedding=embeddings)store.add_documents(chunks)

The overlap repeats text across neighbouring chunks, so context at a boundary is not lost. The embeddings are a placeholder so the snippet needs no API key, and a real embedding model replaces it.

The catch

Chunk size is a trade. Too large and a chunk carries unrelated text and eats the window. Too small and it loses the context that gave it meaning. And the index goes stale: when the source changes, ingestion has to run again, which is the sync work that long contextRAG versus long contextPassing the whole file avoids retrieval, until the cost, the attention and the window limit show up.Opens in a new tab avoids. The other half is the query sideRAG at query timeA one-way chain: embed the query, search, stuff the chunks into the prompt, generate.Opens in a new tab.

Sources