Conversation Memory

Your chatbot now remembers everything — which is also its bug. Every turn appends two messages, and every call re-sends the whole list. Three things degrade as the list grows:

1. Cost. Input tokens are billed per call. A 100-message history is paid for again on every turn.
2. Latency. More input means more to process before the first output token appears.
3. The context window. Every model has a hard cap on input size. Blow past it and the API returns an error — your app crashes mid-conversation, always at the worst time, because it only happens in long chats.

The classic fix is trimming: before the history gets too long, drop the oldest turns. But not the first message — if history[0] is the system message, it carries your bot's persona and rules, and silently losing it changes behavior in confusing ways ("why did it stop being concise after 20 minutes?"). So the policy is: keep the system message, keep the most recent messages, drop the middle.

We'll measure the budget in message count. Real apps often count tokens instead, but the shape of the function is identical — count is simpler and plenty for a CLI bot. Add this to chatbot.py:

def trim_history(history, max_messages):
    if len(history) <= max_messages:
        return list(history)
    if history and history[0]["role"] == "system":
        keep = max_messages - 1
        tail = history[len(history) - keep:] if keep > 0 else []
        return [history[0]] + tail
    return history[-max_messages:]

It returns a new list rather than mutating — the caller decides what to do with it. Wire it into your loop by reassigning after each turn:

        print(chat_turn(history, user_input, anthropic_client))
        history = trim_history(history, max_messages=30)

Try it with a tiny budget like max_messages=5 and watch the bot forget the start of the conversation while keeping its persona — exactly the trade you asked for. This is pure list logic, so the challenge needs no fake client at all.