Lab Specification — Module 3: Context Management

interface HistoryEntry { role: string; content: string; turn: number; type: "thought" | "observation" | "decision" | "task" | "error" }

function compact(history: HistoryEntry[], keepRecent = 5): HistoryEntry[] {
  const task = history.filter(e => e.type === "task");           // preserve verbatim
  const decisions = history.filter(e => e.type === "decision");  // preserve verbatim
  const errors = history.filter(e => e.type === "error");        // preserve verbatim
  const recent = history.slice(-keepRecent);                     // preserve recent
  const old = history.slice(0, -keepRecent);

  const compressible = old.filter(e => e.type === "observation"); // compress these
  const summarized = compressible.map(e => summarize(e));         // LLM call per segment

  return [...task, ...decisions, ...errors, ...summarized, ...recent];
}

async function summarize(entry: HistoryEntry): Promise<HistoryEntry> {
  const summary = await model.complete([
    { role: "system", content: "Summarize this tool output into 1-2 sentences: what was found, what was decided, what state. Discard raw contents." },
    { role: "user", content: entry.content }
  ]);
  return { ...entry, content: summary.content, type: "decision" };
}

const before = history.reduce((sum, e) => sum + tokenCount(e.content), 0);
const after = compacted.reduce((sum, e) => sum + tokenCount(e.content), 0);
const reduction = ((before - after) / before * 100).toFixed(1);
console.log({ before, after, reduction: reduction + "%" });

function loadContext(budgetTokens: number, index: Tier1Index, topicFiles: Map<string, Tier2File>): LoadedContext {
  let remaining = budgetTokens;
  const loaded: LoadedContext = { system: "", tools: "", index: "", history: "", topics: [] };

  // Priority order
  loaded.system = SYSTEM_PROMPT;     remaining -= tokenCount(loaded.system);
  loaded.tools = TOOL_DEFS;          remaining -= tokenCount(loaded.tools);
  loaded.index = formatIndex(index); remaining -= tokenCount(loaded.index);
  loaded.history = recentHistory;    remaining -= tokenCount(loaded.history);

  // Fill remaining with highest-priority topic files
  for (const [topic, file] of topicFiles) {
    const cost = tokenCount(file.summary);
    if (cost <= remaining) {
      loaded.topics.push(file);
      remaining -= cost;
    }
  }
  return loaded;
}

# Lab Specification — Module 3: Context Management

**Module**: 3 · **Duration**: 75–90 min · **Environment**: Codespace, Node.js 18+. Optional OpenAI key.

---

## Learning objectives

1. **Build a compaction function** that preserves task/decisions/recent and compresses verbose outputs.
2. **Measure** the ACON reduction (target: 26–54%) and verify accuracy preservation.
3. **Implement a JIT context loader** that fills a 100k-token budget in priority order.
4. **Diagnose context rot** from token logs and prescribe the correct intervention.

---

## Phase 1 — Build the compaction function (25 min)

```typescript
interface HistoryEntry { role: string; content: string; turn: number; type: "thought" | "observation" | "decision" | "task" | "error" }

function compact(history: HistoryEntry[], keepRecent = 5): HistoryEntry[] {
  const task = history.filter(e => e.type === "task");           // preserve verbatim
  const decisions = history.filter(e => e.type === "decision");  // preserve verbatim
  const errors = history.filter(e => e.type === "error");        // preserve verbatim
  const recent = history.slice(-keepRecent);                     // preserve recent
  const old = history.slice(0, -keepRecent);

  const compressible = old.filter(e => e.type === "observation"); // compress these
  const summarized = compressible.map(e => summarize(e));         // LLM call per segment

  return [...task, ...decisions, ...errors, ...summarized, ...recent];
}

async function summarize(entry: HistoryEntry): Promise<HistoryEntry> {
  const summary = await model.complete([
    { role: "system", content: "Summarize this tool output into 1-2 sentences: what was found, what was decided, what state. Discard raw contents." },
    { role: "user", content: entry.content }
  ]);
  return { ...entry, content: summary.content, type: "decision" };
}
```

**Verify**: create a 30-entry history with verbose observations. Run compaction. Confirm task, decisions, errors, and recent 5 are verbatim; old observations are summarized. Measure token reduction.

---

## Phase 2 — Measure ACON reduction (15 min)

```typescript
const before = history.reduce((sum, e) => sum + tokenCount(e.content), 0);
const after = compacted.reduce((sum, e) => sum + tokenCount(e.content), 0);
const reduction = ((before - after) / before * 100).toFixed(1);
console.log({ before, after, reduction: reduction + "%" });
```

**Target**: 26–54% reduction (ACON range). If yours is <20%, your summarize function isn't aggressive enough. If >60%, you're losing too much detail — check accuracy.

**Accuracy check**: after compaction, can the model still answer "what file did we read at turn 5 and what did it contain?" If yes, accuracy preserved. If no, over-compaction.

---

## Phase 3 — JIT context loader (20 min)

Build a loader that fills a budget in priority order:

```typescript
function loadContext(budgetTokens: number, index: Tier1Index, topicFiles: Map<string, Tier2File>): LoadedContext {
  let remaining = budgetTokens;
  const loaded: LoadedContext = { system: "", tools: "", index: "", history: "", topics: [] };

  // Priority order
  loaded.system = SYSTEM_PROMPT;     remaining -= tokenCount(loaded.system);
  loaded.tools = TOOL_DEFS;          remaining -= tokenCount(loaded.tools);
  loaded.index = formatIndex(index); remaining -= tokenCount(loaded.index);
  loaded.history = recentHistory;    remaining -= tokenCount(loaded.history);

  // Fill remaining with highest-priority topic files
  for (const [topic, file] of topicFiles) {
    const cost = tokenCount(file.summary);
    if (cost <= remaining) {
      loaded.topics.push(file);
      remaining -= cost;
    }
  }
  return loaded;
}
```

**Verify**: with a 100k budget, confirm the load order (system → tools → index → history → topics) and that topics fill the remaining headroom.

---

## Phase 4 — Diagnose context rot (15 min)

Given a token log from a 50-turn session (provided in the lab repo or generated):

1. Plot tokens-per-turn. Identify where the growth accelerates.
2. Identify the dominant contributor (tool outputs? history? both?).
3. Prescribe: which intervention (truncation, masking, compaction, JIT) and at what threshold.
4. Estimate the post-intervention effective context size.

**Deliverable**: a one-paragraph "context-rot diagnosis" naming the cause and the cure.

---

## Deliverables

- [ ] Phase 1: compaction function code
- [ ] Phase 2: before/after token counts + reduction % (in ACON range?) + accuracy check result
- [ ] Phase 3: JIT loader code + load-order verification
- [ ] Phase 4: context-rot diagnosis paragraph

---

## Stretch goals

1. **Add observation masking**: before compaction, replace acted-on observations with one-line summaries; measure the additional reduction.
2. **Implement cache_control**: mark the system prompt + tool defs as cacheable; measure latency reduction across 10 turns.
3. **Build the full 3-tier store**: a real T1 index (JSON file), T2 topic files (one per topic), T3 raw log (append-only JSONL). Have the model request T2 files by topic name during a task.