Module: 3 · Duration: ~90 min (presentation-pace + compaction-building lab)
[SLIDE 1 — Title] Module Three: Context Management. Ninety minutes. The hardest engineering problem in production harnesses. Modules 1 and 2 gave you the loop and the tools; this module is about what fills the space between them — context — and why it degrades unless you fight it actively.
[SLIDE 2 — The hardest problem] Context is the model's working memory. When it fills, the agent loses track of its task. This is measurable, not theoretical. The anchor from Module 0.3: tool outputs are sixty-seven point six percent of context, history is eighteen percent, and history is the only layer that grows unboundedly with turns. By the end of this module you'll know why context rots, the five strategies to fight it, the priority stack that governs attention, and you'll have built a compaction function.
[SLIDE 3 — 3.1 section] Sub-section 3.1: Why Context Rots.
[SLIDE 4 — Lost in the Middle] Stanford's Lost in the Middle study — Liu et al., 2023, arXiv 2307.03172 — found a thirty-percent-plus performance drop when critical information is buried mid-context. Even in models with million-token stated limits. The U-shape: strong attention at the start, where the system prompt lives. Strong attention at the end, where the latest user message lives. And a desert in the middle, where history piles up. The implication: WHERE you place information matters as much as WHAT the information is.
[SLIDE 5 — 256k cliff] There's often a hard degradation threshold around two hundred fifty-six thousand tokens, regardless of the model's stated context limit. A model that supports a million tokens may perform measurably worse above two-fifty-six-k. The stated limit is a ceiling, not a quality target. Effective context is smaller than nominal context. The harness must keep EFFECTIVE context below the degradation cliff — managing proactively, not just fitting within the nominal limit.
[SLIDE 6 — Token breakdown] Restate the breakdown with the key insight: history is the only layer that grows every turn. Tool outputs are the largest share but compressible — Module Two's truncation and observation masking handle them. History is smaller but UNBOUNDED. A fifty-turn session has fifty turns of history piling into the middle of context, where attention is weakest. This is context rot. Compaction targets history.
[SLIDE 7 — 3.2 section] Sub-section 3.2: Compaction and Summarization.
[SLIDE 8 — Preserve vs compress] The compaction decision. Preserve verbatim: the original task, key decisions and rationale, current state, recent three-to-five turns, errors and resolutions. Compress or discard: verbose tool outputs already acted on, redundant reasoning chains, confirmations like ok and done, exploratory dead-ends. The ACON approach: twenty-six to fifty-four percent token reduction with ninety-five-plus percent accuracy preservation. Compaction done well does not meaningfully degrade performance.
[SLIDE 9 — Observation masking] A specific technique Claude Code uses. Keep the reasoning chain — the model's Thought — visible. Hide old raw tool outputs — the Observation. Before: a Thought followed by a twenty-four-hundred-token file dump. After: a Thought followed by a forty-token summary. The model sees its decision history without the noise. The cost: if raw data is needed later, it's gone — unless retained in a side store for JIT retrieval.
[SLIDE 10 — Sync vs async] Synchronous compaction happens in the loop, blocks the next call. Simple, adds latency on the compaction turn. Async runs in the background while the loop continues — no latency, but race conditions. Most production harnesses use synchronous compaction triggered at a threshold — around fifty percent of the context window. Tunable: too low wastes tokens on compaction calls, too high lets rot set in.
[SLIDE 11 — 3.3 section] Sub-section 3.3: JIT Retrieval and Memory Integration.
[SLIDE 12 — 3-tier design] Claude Code's industry-benchmark memory design. Three tiers. Tier one: the index — lightweight summaries, about a hundred fifty chars each, always loaded, low cost. Tier two: topic files — detailed content, loaded on demand when the model requests, medium cost. Tier three: raw logs — full interaction history, search only, never bulk-loaded, high cost. The principle: like a database index. You don't load the whole table; you load an index, then fetch rows on demand. The model never pays Tier three's cost unless it explicitly searches.
[SLIDE 13 — Three retrieval methods] Keyword search — ripgrep — for exact code. Vector retrieval — embeddings — for semantic concepts. File read — for known paths. Provide all three; let the model choose. The tool surface from Module Two determines retrieval capability.
[SLIDE 14 — 3.4 section] Sub-section 3.4: Prompt Assembly Engineering.
[SLIDE 15 — Priority stack] The standard priority stack. System message first — highest attention. Tool definitions. Memory, the Tier one index. Conversation history — weak attention, the middle. User message last — high attention. Core rule: anything placed early gets higher effective attention. This is Lost-in-the-Middle operationalized. The original task, placed in the system prompt, survives better than the same task in turn-one history.
[SLIDE 16 — Thin vs dense] Ultra-thin system prompts — Pi, under a thousand tokens — co-evolve with the model. Dense — Claude Code, about forty thousand — control behavior explicitly, reliably, but fight capability growth. Thickness spectrum applies to prompt assembly. How much behavior do you encode versus delegate?
[SLIDE 17 — Caching] Anthropic's cache_control. System prompt and tool definitions are fixed across turns, so caching them turns a recurring cost into a one-time cost. Reward stability; dynamic prompts invalidate the cache.
[SLIDE 18 — Anti-patterns] Four. The sliding window — drop oldest when full, catastrophic, loses the task; cure is compaction. No context management — works short-term, fails long; cure is any of the five strategies. Unbounded subagent return — cure is the one-to-two-k bound. Premature compaction — threshold too low; cure is tuning, with ACON's twenty-six-to-fifty-four percent as the target.
[SLIDE 19 — Takeaways] Context rots measurably — thirty percent mid-context drop, two-fifty-six-k cliff, history grows unboundedly. Compaction: preserve task, decisions, recent, errors; compress verbose outputs. JIT retrieval: index always, topic files on demand, raw logs search-only. Priority stack: early equals high attention. Cache stability. Next: Module Four — Memory Architecture. Continuity across sessions.
End of Module 3 teaching script. ~2,300 scripted words; remaining module time is the compaction-building lab.
# Teaching Script — Module 3: Context Management **Module**: 3 · **Duration**: ~90 min (presentation-pace + compaction-building lab) --- [SLIDE 1 — Title] Module Three: Context Management. Ninety minutes. The hardest engineering problem in production harnesses. Modules 1 and 2 gave you the loop and the tools; this module is about what fills the space between them — context — and why it degrades unless you fight it actively. [SLIDE 2 — The hardest problem] Context is the model's working memory. When it fills, the agent loses track of its task. This is measurable, not theoretical. The anchor from Module 0.3: tool outputs are sixty-seven point six percent of context, history is eighteen percent, and history is the only layer that grows unboundedly with turns. By the end of this module you'll know why context rots, the five strategies to fight it, the priority stack that governs attention, and you'll have built a compaction function. [SLIDE 3 — 3.1 section] Sub-section 3.1: Why Context Rots. [SLIDE 4 — Lost in the Middle] Stanford's Lost in the Middle study — Liu et al., 2023, arXiv 2307.03172 — found a thirty-percent-plus performance drop when critical information is buried mid-context. Even in models with million-token stated limits. The U-shape: strong attention at the start, where the system prompt lives. Strong attention at the end, where the latest user message lives. And a desert in the middle, where history piles up. The implication: WHERE you place information matters as much as WHAT the information is. [SLIDE 5 — 256k cliff] There's often a hard degradation threshold around two hundred fifty-six thousand tokens, regardless of the model's stated context limit. A model that supports a million tokens may perform measurably worse above two-fifty-six-k. The stated limit is a ceiling, not a quality target. Effective context is smaller than nominal context. The harness must keep EFFECTIVE context below the degradation cliff — managing proactively, not just fitting within the nominal limit. [SLIDE 6 — Token breakdown] Restate the breakdown with the key insight: history is the only layer that grows every turn. Tool outputs are the largest share but compressible — Module Two's truncation and observation masking handle them. History is smaller but UNBOUNDED. A fifty-turn session has fifty turns of history piling into the middle of context, where attention is weakest. This is context rot. Compaction targets history. [SLIDE 7 — 3.2 section] Sub-section 3.2: Compaction and Summarization. [SLIDE 8 — Preserve vs compress] The compaction decision. Preserve verbatim: the original task, key decisions and rationale, current state, recent three-to-five turns, errors and resolutions. Compress or discard: verbose tool outputs already acted on, redundant reasoning chains, confirmations like ok and done, exploratory dead-ends. The ACON approach: twenty-six to fifty-four percent token reduction with ninety-five-plus percent accuracy preservation. Compaction done well does not meaningfully degrade performance. [SLIDE 9 — Observation masking] A specific technique Claude Code uses. Keep the reasoning chain — the model's Thought — visible. Hide old raw tool outputs — the Observation. Before: a Thought followed by a twenty-four-hundred-token file dump. After: a Thought followed by a forty-token summary. The model sees its decision history without the noise. The cost: if raw data is needed later, it's gone — unless retained in a side store for JIT retrieval. [SLIDE 10 — Sync vs async] Synchronous compaction happens in the loop, blocks the next call. Simple, adds latency on the compaction turn. Async runs in the background while the loop continues — no latency, but race conditions. Most production harnesses use synchronous compaction triggered at a threshold — around fifty percent of the context window. Tunable: too low wastes tokens on compaction calls, too high lets rot set in. [SLIDE 11 — 3.3 section] Sub-section 3.3: JIT Retrieval and Memory Integration. [SLIDE 12 — 3-tier design] Claude Code's industry-benchmark memory design. Three tiers. Tier one: the index — lightweight summaries, about a hundred fifty chars each, always loaded, low cost. Tier two: topic files — detailed content, loaded on demand when the model requests, medium cost. Tier three: raw logs — full interaction history, search only, never bulk-loaded, high cost. The principle: like a database index. You don't load the whole table; you load an index, then fetch rows on demand. The model never pays Tier three's cost unless it explicitly searches. [SLIDE 13 — Three retrieval methods] Keyword search — ripgrep — for exact code. Vector retrieval — embeddings — for semantic concepts. File read — for known paths. Provide all three; let the model choose. The tool surface from Module Two determines retrieval capability. [SLIDE 14 — 3.4 section] Sub-section 3.4: Prompt Assembly Engineering. [SLIDE 15 — Priority stack] The standard priority stack. System message first — highest attention. Tool definitions. Memory, the Tier one index. Conversation history — weak attention, the middle. User message last — high attention. Core rule: anything placed early gets higher effective attention. This is Lost-in-the-Middle operationalized. The original task, placed in the system prompt, survives better than the same task in turn-one history. [SLIDE 16 — Thin vs dense] Ultra-thin system prompts — Pi, under a thousand tokens — co-evolve with the model. Dense — Claude Code, about forty thousand — control behavior explicitly, reliably, but fight capability growth. Thickness spectrum applies to prompt assembly. How much behavior do you encode versus delegate? [SLIDE 17 — Caching] Anthropic's cache_control. System prompt and tool definitions are fixed across turns, so caching them turns a recurring cost into a one-time cost. Reward stability; dynamic prompts invalidate the cache. [SLIDE 18 — Anti-patterns] Four. The sliding window — drop oldest when full, catastrophic, loses the task; cure is compaction. No context management — works short-term, fails long; cure is any of the five strategies. Unbounded subagent return — cure is the one-to-two-k bound. Premature compaction — threshold too low; cure is tuning, with ACON's twenty-six-to-fifty-four percent as the target. [SLIDE 19 — Takeaways] Context rots measurably — thirty percent mid-context drop, two-fifty-six-k cliff, history grows unboundedly. Compaction: preserve task, decisions, recent, errors; compress verbose outputs. JIT retrieval: index always, topic files on demand, raw logs search-only. Priority stack: early equals high attention. Cache stability. Next: Module Four — Memory Architecture. Continuity across sessions. --- *End of Module 3 teaching script. ~2,300 scripted words; remaining module time is the compaction-building lab.*