Compaction & Branch Summarization

Context always fills up — the cut point decides what stays

Learning Objectives

After this chapter you will be able to:

• Explain why compaction must trigger proactively before overflow
• Describe findCutPoint's selection strategy and its impact on coherence
• Explain why file tracking (readFiles/modifiedFiles) must survive compaction
• Distinguish the responsibilities of agent-layer pure-function compaction vs coding-agent-layer compaction

Key Source Files

File	Lines	Responsibility
packages/agent/src/harness/compaction/compaction.ts	880	Pure-function compaction: findCutPoint + summarize
packages/coding-agent/src/core/compaction/compaction.ts	969	Coding-agent compaction: adds file tracking, idle trigger
packages/agent/src/harness/compaction/branch-summarization.ts	275	Compresses severed branch history on /fork

Design Motivation: Why Not Wait for Overflow?

Problem: Model context windows are finite (128K-200K tokens). The messages array grows monotonically with conversation and will eventually overflow.

Naive approach: Wait for the API to return "context_length_exceeded" then compact.

Why that fails:

• At overflow the user's prompt is already sent — this interaction inevitably fails
• Emergency compaction has no room for choice; may cut critical context
• User experience: sudden error → wait for compaction → retry, breaking flow

pi's approach: Trigger proactively at ~92% window utilization — the user never notices.

Analogy: Don't wait until the tank is empty to find a gas station; pull in automatically at 1/8 — you don't even need to know it happened.

shouldCompact: Lead-Time Calculation

// Simplified logic
function shouldCompact(messages: AgentMessage[], model: Model): boolean {
  const used = calculateContextTokens(messages)   // real-time token count
  const limit = model.contextWindow
  return used > limit * COMPACTION_THRESHOLD      // default 0.92
}

AgentSession checks at two moments:

1. After each turn ends: if over threshold, compact immediately
2. During user idle: quietly pre-compact during pauses, zero wait on next interaction

findCutPoint: Where to Cut

function findCutPoint(messages: AgentMessage[]): number {
  // Walk backward from the end to the nearest user turn boundary
  for (let i = messages.length - 1; i >= 0; i--) {
    if (messages[i].role === "user") return i
  }
  return 0
}

Why cut at user turn boundaries?

• Cut mid-assistant-reply → model sees "half a sentence," coherence collapses
• Cut mid-tool_result → tool call separated from result, model confused
• Cut before a user message → preserves complete "question-answer-tool-answer" units

Analogy: Like film editing — you don't cut a scene mid-sentence; you wait for a complete dialogue beat to end.

summarize: Summary + Original Mixed

async function compact(messages: AgentMessage[]): Promise<AgentMessage[]> {
  const cut = findCutPoint(messages)
  const old = messages.slice(0, cut)          // history to be compacted
  const recent = messages.slice(cut)          // recent original preserved

  const summary = await generateSummary(old, SUMMARIZATION_SYSTEM_PROMPT)

  return [
    { role: "system", content: summary },     // summary replaces old history
    ...preserveFileTracking(recent),          // recent original untouched
  ]
}

Critical design: The original text remains in the JSONL file (→ Chapter 4). Compaction only changes "what the model sees" — history is never truly lost, always recoverable.

File Tracking: No Amnesia After Compaction

interface CompactionDetails {
  readFiles: string[]       // file paths the model has read
  modifiedFiles: string[]   // file paths the model has modified
  summary: string           // history summary
}

Why must this survive? Imagine the model read auth.ts in turn 3, and turn 15's compaction summarized that history. Without preserving readFiles, the model reads the same file again — wasting tokens and appearing "forgetful."

With file tracking preserved, the system prompt can inject: "You have read these files: [...], modified these files: [...]" — no redundant operations.

Branch Summarization: /fork Housekeeping

When the user executes /fork (→ Chapter 4), the severed branch history also needs compaction:

main: A → B → C → D → E (current)
                 \
fork:             F → G (new branch)

branch-summarization.ts generates a summary of A→B→C→D as the fork branch's "previously on..." — the new branch doesn't need the full trunk history, but needs to know "what happened before."

Engineering Insights

1. Two-layer compaction: The agent package's is a pure function (messages in, compacted messages out); coding-agent's adds trigger strategy, file tracking, idle scheduling. Separation lets the pure-function layer be tested independently.

2. SUMMARIZATION_SYSTEM_PROMPT is hidden prompt engineering: Summary quality directly determines post-compaction "memory" accuracy. pi's summarization prompt requires preserving: decision rationale, file paths, unfinished tasks.

3. Compaction itself costs tokens: generateSummary calls the model once. If you only compact at the overflow edge, there may not be enough tokens for the compaction itself. The 92% lead time reserves this space.

Cross-Chapter Links

Direction	Chapter	Relationship
Prerequisite	Ch01 Agent Loop	messages growth is the root cause of compaction
Prerequisite	Ch02 AgentHarness	Harness invokes compaction on overflow
Next	Ch04 Session Tree	Compaction entries written to JSONL, originals preserved
Next	Ch11 AgentSession	Idle compaction scheduling lives in AgentSession

Check Your Understanding

1. If findCutPoint cut mid-assistant-message (instead of at a user turn boundary), what problem would the model's next response exhibit?
2. Why are originals retained in JSONL after compaction rather than deleted? Which features critically depend on this?
3. If COMPACTION_THRESHOLD were changed from 0.92 to 0.99, what risks would that introduce?

Key Takeaways

• Proactive trigger (~92%), not emergency firefighting — reserves token space for compaction itself
• Cut at user turn boundaries, preserving complete "question-answer" units
• File tracking prevents post-compaction "amnesia" — no re-reads, no forgotten edits
• Originals never deleted; compaction only changes the model's view

---

This article is based on source analysis of earendil-works/pi main branch.