Why RAG

Ask a model about your stuff — your notes, your team's runbooks, last month's design docs — and it will either admit it has no idea or, worse, answer confidently and wrong. Two hard limits cause this:

1. The model wasn't trained on your data. Your meeting notes from last Tuesday are not in any training set.
2. You can't just paste everything in. Context windows are large but finite, you pay per input token on every call, and models get measurably worse at using facts buried in the middle of enormous prompts.

Retrieval-Augmented Generation (RAG) is the standard fix, and right now it's one of the most practical, in-demand skills in AI engineering — nearly every "chat with your docs" product, support bot, and internal knowledge assistant is a RAG pipeline under the hood. The idea fits in one sentence: store your documents in a searchable form, and when a question arrives, retrieve the few most relevant passages and hand only those to the model, with instructions to answer from them alone.

Every RAG system is the same five-stage pipeline:

documents --> chunk --> embed --> store        (indexing, done once)
question  --> embed --> retrieve top-k --> prompt the model   (per query)

By the end of this course that pipeline will be running on your machine, over your own files, with real embeddings and a real vector database. Each lesson adds one stage to the app; each challenge unit-tests the one pure function at the heart of that stage — string and list logic, no network, no heavyweight libraries. That split is honest: the pure pieces you'll be graded on are exactly what the big libraries do for a living.

Pick your corpus. Choose a folder of your own plain-text or Markdown files — personal notes, a blog, a project's docs directory, saved articles. A few dozen files is plenty; anything you'd actually like to ask questions about. Then set up the project:

mkdir ragapp && cd ragapp
python -m venv .venv && source .venv/bin/activate
pip install sentence-transformers chromadb
mkdir docs        # copy your files in here (or symlink your notes folder)

sentence-transformers downloads a small embedding model (~90 MB) on first use and runs it locally — no API key needed. If you'd rather use a hosted embeddings API instead, the next lessons show that variant too.

The first stage is the least glamorous: loading. Create rag.py:

from pathlib import Path


def clean_text(text):
    return " ".join(text.split())


def load_documents(folder):
    docs = []
    for path in sorted(Path(folder).rglob("*")):
        if path.is_file() and path.suffix.lower() in {".md", ".txt"}:
            text = clean_text(path.read_text(encoding="utf-8", errors="ignore"))
            if text:
                docs.append((str(path.relative_to(folder)), text))
    return docs


if __name__ == "__main__":
    docs = load_documents("docs")
    print(f"loaded {len(docs)} documents")
    for source, text in docs[:3]:
        print(f"  {source}: {text[:60]!r}")

Run python rag.py and confirm your files show up.

Don't skip past clean_text. Raw files are full of formatting whitespace — indentation, hard-wrapped lines, runs of blank lines — that carries no meaning but eats characters. In the next lesson we slice text into fixed-size chunks, and every wasted character is retrieval budget spent on nothing. Every production document loader does some version of this normalization before splitting (many text splitters strip and collapse whitespace for exactly this reason). It's also our warm-up: small enough to write in one line, real enough that your pipeline is already running it.