Chunking

You can't embed a whole file as one unit — well, you can, but retrieval gets bad fast. An embedding is a fixed-size summary of meaning; squeeze three unrelated topics from one long document into a single vector and it points at none of them. And even when a big chunk is retrieved, you pay to stuff the entire thing into the prompt when only one paragraph mattered.

So stage one of indexing is chunking: slice each document into pieces small enough to have one meaning each, big enough to still carry context. Two knobs control it:

• size — how many characters per chunk. Hundreds of characters is the usual ballpark: a paragraph or two.
• overlap — how many characters each chunk shares with the previous one. Without overlap, a sentence that straddles a boundary gets cut in half and neither half embeds well; with it, the boundary region appears intact in both neighbors.

Overlap must be smaller than size — otherwise a chunk starts at or before where the previous one started and the loop never advances. Your function should reject that loudly (ValueError), because it's a config bug, not a data condition.

This is precisely what the fancy splitters in production frameworks do — LangChain's RecursiveCharacterTextSplitter, LlamaIndex's node parsers. Theirs try to cut on paragraph and sentence boundaries before falling back to raw characters, but the core loop — walk the text, emit size-character windows, step forward by size - overlap — is exactly the function you're about to write. Add it to rag.py:

def chunk_text(text, size, overlap):
    if size <= 0:
        raise ValueError("size must be positive")
    if overlap < 0 or overlap >= size:
        raise ValueError("overlap must be >= 0 and smaller than size")
    if not text:
        return []
    chunks = []
    start = 0
    while start < len(text):
        chunks.append(text[start:start + size])
        if start + size >= len(text):
            break
        start += size - overlap
    return chunks

The break matters: once a chunk reaches the end of the text, stop. A naive range(0, len(text), step) loop happily emits one more chunk that lies entirely inside the previous one — pure duplication that pollutes retrieval (the tests below catch that bug specifically).

Now chunk your real corpus. Each chunk gets an id like notes.md#3 — source file plus chunk index — which is what your app will cite in its answers later. Add:

def build_corpus(docs, size=800, overlap=200):
    corpus = []
    for source, text in docs:
        for i, chunk in enumerate(chunk_text(text, size, overlap)):
            corpus.append((f"{source}#{i}", chunk))
    return corpus


if __name__ == "__main__":
    docs = load_documents("docs")
    corpus = build_corpus(docs)
    print(f"{len(docs)} documents -> {len(corpus)} chunks")
    for chunk_id, chunk in corpus[:3]:
        print(f"  {chunk_id}: {chunk[:60]!r}")

Run it. Eyeball a few chunks: do they read like coherent passages? Try size=200 and watch them turn into confetti; try size=5000 and watch whole files collapse into single chunks. There's no universal right answer — this knob is one of the highest-leverage tuning parameters in any real RAG system, and now you own it.