Embeddings and Similarity

An embedding turns text into a vector — a plain list of floats, a few hundred long — with one magic property: texts that mean similar things get vectors that point in similar directions. "How do I reset my password?" and "steps to recover account access" share almost no words, but their embeddings are near-parallel. That property is the entire trick behind semantic search; everything else in RAG is plumbing around it.

Add local embeddings to rag.py with sentence-transformers:

from sentence_transformers import SentenceTransformer

_model = SentenceTransformer("all-MiniLM-L6-v2")


def embed_texts(texts):
    return _model.encode(texts, normalize_embeddings=True).tolist()

all-MiniLM-L6-v2 maps each string to 384 floats — that's the "few hundred long" vector made concrete. It also has an input limit: text past 256 word pieces gets truncated, and a truncated chunk embeds only its first part. This is why the 800-character default from the chunking lesson isn't arbitrary — it comfortably fits under that limit with room to spare, so every chunk gets embedded in full instead of silently losing its tail. Swap in a model with a shorter limit (or hand it much bigger chunks) and this stops being true — worth checking whenever you change either knob.

Prefer a hosted embeddings API? Same shape, swap the body (and pip install openai, export OPENAI_API_KEY=...):

from openai import OpenAI

_openai = OpenAI()  # reads OPENAI_API_KEY

def embed_texts(texts):
    resp = _openai.embeddings.create(model="text-embedding-3-small", input=texts)
    return [item.embedding for item in resp.data]

Either way, embed_texts takes a list of strings and returns a list of vectors — the rest of the course doesn't care which one you picked. That's the same injectable-callable seam you used for the LLM client in the CLI chatbot course, and it's what will keep the graded challenges offline.

How do we compare two vectors? Cosine similarity: the cosine of the angle between them.

cos(a, b) = (a . b) / (|a| * |b|)

Dot product over the product of lengths. It ranges from 1.0 (same direction — same meaning), through 0.0 (unrelated), to -1.0 (opposite). Dividing by the lengths makes it care only about direction, so a long document and a three-word query can still match. Add it to rag.py:

import math


def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(y * y for y in b))
    if norm_a == 0 or norm_b == 0:
        return 0.0
    return dot / (norm_a * norm_b)

(The zero-vector guard avoids a ZeroDivisionError on degenerate input; by convention a zero vector is similar to nothing.)

Now the payoff — semantic search over your own corpus, brute force:

if __name__ == "__main__":
    corpus = build_corpus(load_documents("docs"))
    vectors = embed_texts([chunk for _, chunk in corpus])

    query = "how do I restore a backup"   # ask something YOUR docs answer
    qvec = embed_texts([query])[0]

    scored = [
        (cosine_similarity(qvec, vec), chunk_id, chunk)
        for (chunk_id, chunk), vec in zip(corpus, vectors)
    ]
    scored.sort(reverse=True)
    for score, chunk_id, chunk in scored[:5]:
        print(f"{score:.3f}  {chunk_id}  {chunk[:70]!r}")

Run it a few times with different queries. Spend real time here — this is the eyeball test every RAG engineer runs constantly. Do the top hits actually answer the question? Try a query using words that appear nowhere in your docs but mean the same thing as something that does; watch it match anyway. That's embeddings earning their keep.

One note before the challenge: normalize_embeddings=True asked the model for unit-length vectors, which makes cosine similarity collapse to a plain dot product. Vector databases lean on that constantly. Your graded function takes arbitrary vectors, so it does the full formula — the one line every vector database on earth evaluates a billion times a day. No stub needed; it's pure arithmetic.