Context windows are a design constraint, not a spec number

When context windows grew, a lot of people concluded that retrieval was dead: just stuff everything into the prompt. I built both ways, and the stuffing approach loses — not because of window size, but because of attention quality and cost.

A million tokens of context is not a million tokens of free comprehension. Models, like people, get worse at finding the needle when the haystack gets bigger. Cost scales with every token, every call. And latency grows until users leave.

How I design with context now

• the system prompt stays small and stable — it's architecture, not content
• the conversation carries what's recent and relevant
• a retrieval layer fetches what's old and relevant, on demand
• summarisation compresses what matters into what fits

The deeper principle

Context is a budget, and every token competes for the model's attention. The skill that separates good AI systems from mediocre ones is curation: deciding what deserves to be present. That's an editor's skill, not a packrat's.

Big windows are a gift. Pouring everything in anyway is how you turn a gift into a landfill.