Dynamic context cleaner — implementation plan

Status: Planned — no implementation started
Author: Claude (Fable), for Christophe
Date: 2026-07-09
Repo: north-os
Related: docs/superpowers/specs/2026-07-09-dynamic-context-cleaner-design.html (the design this implements — read it first; §-references below point there)

Five phases, each independently shippable and each ending with bun run typecheck + bun run lint green from root, the named tests passing, and a status flip when the phase changes the doc's state (bun docs/superpowers/status.ts set dynamic-context-cleaner in-progress in the Phase 0 commit; … implemented when Phase 4 lands). Phase 0 and Phase 2 change no live behavior (telemetry; dormant read path). The first behavior change is Phase 1's inline cleaning, and its rollout posture is explicit: active by default on deploy, with CONTEXT_CLEANER_DISABLED=1 as the opt-out kill switch (spec §4.5) — there is no "ships off" state.

Phase 0 — Baseline instrumentation (measure before touching anything)

Goal: the before/after in spec §5 is measured, not estimated. No behavior change.

• Add @workspace/observability as a dependency of packages/agent-runtime (it emits no metrics today) and instrument graph.ts:
  • Registry change first: add an optional per-metric buckets parameter to observeHistogram/the registry (additive — the shared HISTOGRAM_BUCKETS_MS default stays untouched). Required: the global buckets top out at 60,000 (milliseconds), so char-scale observations would all land in +Inf and the spec §5 median would be unmeasurable.
  • context_prompt_chars{stage="raw"} histogram with char-scale buckets (e.g. 10k → 1.5M) — total prompt chars entering compactMessagesForModel per llmCall.
  • llm_call_duration histogram around the model invoke — the turn-latency baseline (none exists yet).
  • context_compaction_events_total — counter when drop-oldest compaction fires.
  • context_toolcap_events_total{kind=per_result|budget_exhausted} — counters when capToolResult truncates.
• Expose the registry from apps/web: a minimal internal GET /api/internal/metrics route returning serializeProm() (localhost/dev-gated) — apps/web has no metrics surface today; workers already get /metrics from worker-chassis.

Files: packages/agent-runtime/src/agents/legal-assistant/graph.ts (metric calls), packages/agent-runtime/package.json, small apps/web metrics route.
Tests: extend packages/agent-runtime graph tests — graph-metrics.test.ts: a fake-model turn over an oversized fixture increments the compaction + cap counters; a small turn increments neither (uses resetMetricsForTest).
Shippable: yes — pure telemetry. Run a week of dev + preprod traffic through it to fix the baseline numbers.

Phase 1 — assembleModelView + deterministic inline cleaning (no LLM, no worker, no DB)

Goal: the safe subset of the junk taxonomy (spec §1 categories 2 + 4) cleaned at prompt-build time; the seam for digest substitution exists.

• New packages/agent-runtime/src/context/:
  • keep-rules.ts — the never-junk predicates (spec §1) as pure functions: isKeepProtected(message, ctx), recency-wall computation.
  • inline-clean.ts — exact-duplicate dedupe (same tool + normalized args + identical content ⇒ the earlier occurrence stubbed, latest stays raw — spec §1 cat. 2 direction, load-bearing for re-reads) and error/retry collapse (error ToolMessage with a successful same-tool sibling ⇒ stub). Stub texts per spec §2.
  • assemble.ts — assembleModelView(messages, digests, opts): recency wall → digest substitution (Phase 2 fills this; empty map for now) → inline cleaning → compactMessagesForModel moved here from graph.ts unchanged. (capToolResult and the toolNode caps stay in graph.ts — they run at result-creation time and their output is what gets checkpointed; they cannot move to assembly.)
• llmCall switches from compactMessagesForModel(state.messages) to assembleModelView(state.messages, digestsFromConfigurable, …).
• Kill switch: CONTEXT_CLEANER_DISABLED=1 ⇒ assembleModelView reduces byte-for-byte to today's compactMessagesForModel behavior.

Files: packages/agent-runtime/src/context/{keep-rules,inline-clean,assemble}.ts + graph.ts wiring.
Tests (all in packages/agent-runtime/src/context/):

• keep-rules.test.ts — every never-junk category from spec §1 (human messages, offset=0 ledger pages, interrupts/ask_user, provenance blocks, recency window, recently-referenced file_ids) is protected; fixtures include a paginated read chain.
• inline-clean.test.ts — dedupe and error/retry collapse fire on the right fixtures and nothing else; tool_call_id pairing preserved (every tool_call keeps exactly one ToolMessage — the Anthropic-API invariant from graph.ts).
• assemble.test.ts — disabled flag ⇒ identical output to legacy compaction (golden fixture); figure-provenance haystack unaffected (guard still built from raw state.messages); backstop caps still fire on oversized fixtures.
• Extend the existing packages/agent-runtime/src/tools/no-graph-mutations.test.ts (it lives in src/tools/, not src/context/) — assembly never mutates state.messages.

Shippable: yes — deterministic-only cleaning, kill-switched. Watch Phase 0 metrics for the first real chars-saved.

Phase 2 — context_digest schema + hash-gated substitution (table empty ⇒ no-op)

Goal: the read path is complete and proven safe before any producer exists.

• New schema packages/db/src/schema/context_digest.ts per spec §3.2 — org-scoped, pgPolicy() + .enableRLS(), fk cascade to chat_thread, unique (thread_id, tool_call_id). Migration via bun db:generate (never hand-written; check the journal when watermark). GRANTs in provision-roles.ts (app_role read/write).
• packages/chat-runtime/src/run.ts: load the thread's digests once per turn (single indexed query) and pass as configurable.context_digests (map tool_call_id → {digest, content_sha256}), alongside memory_block.
• assemble.ts: implement substitution — hash match ⇒ substitute, mismatch ⇒ keep raw + context_digest_stale_total.

Files: packages/db/src/schema/context_digest.ts + generated migration + packages/db/scripts/provision-roles.ts; packages/chat-runtime/src/run.ts; packages/agent-runtime/src/context/assemble.ts.
Tests:

• packages/db — RLS test on the isolated-test-DB pattern: org A cannot read org B's digests; cascade delete with the thread.
• assemble.test.ts additions — substitution on hash match; raw + metric on mismatch; keep-protected messages never substituted even when a digest row exists (defense in depth against a buggy producer).
• packages/chat-runtime — run.test.ts addition: digests load once per turn and ride configurable; empty table ⇒ turn byte-identical to Phase 1.

Shippable: yes — dormant read path; production behavior unchanged until a producer writes rows.

Phase 3 — apps/worker-context + the digest producer (the parallel cleaner goes live)

Goal: background digesting per spec §3.4/§3.5 — this is the phase that delivers "runs en parallèle des sessions".

• @workspace/agent-runtime/context-queue subpath export mirroring memory-queue: createContextDigestProducer/Consumer, debounced enqueue helper.
• packages/chat-runtime/src/turn-lock.ts — export a read-only isTurnInFlight(threadId) helper (the lock key is module-private today; the worker peeks through this, never acquires).
• packages/agent-runtime/src/context/digest.ts — the producer pipeline: turn-lock peek via isTurnInFlight (skip + requeue if held) → trigger threshold → classify via keep-rules + taxonomy → deterministic stubs (cat. 3 superseded-search included here, it needs thread-wide sight) → batched LLM digests for cat. 1/5 (injectable model, DIGEST_CHAR_CAP, figures-verbatim prompt contract per spec §2) → cost controls (CONTEXT_DIGEST_MAX_CALLS_PER_JOB, per-org daily budget, content-hash reuse) → upsert.
• apps/worker-context — chassis app modelled on apps/worker-memory: port 4011, healthz/metrics, consumer wiring, graceful shutdown, boot role-probe (app_role). Register in root bun run dev + turbo like the other workers.
• packages/chat-runtime/src/run.ts — post-turn fire-and-forget enqueue next to the memory-synthesis enqueue (run.ts:1425 posture: queue hiccup never touches the turn). The payload carries the thread's raw tool-output char total (run.ts walks the turn's messages already) so the worker's trigger check (spec §3.4 step 1) never has to load a below-trigger checkpoint.
• Digesting is restricted to idempotent document reads in v1 (spec §1 cat. 5 restriction); superseded-search stubs (cat. 3) keep hit ids verbatim and carry the "re-running returns current data" warning.
• Budget refund (spec §3.3 "The budget refund"): toolNode's usedBudget switches to effective chars (digest_chars for hash-matched digested messages, raw otherwise), guarded by the new CHECKPOINT_RAW_CHAR_CEILING (default 2,000,000) above which raw accounting resumes. This is what makes the tool-budget cliff actually move — without it the cleaner never extends thread length.
• Worker respects CONTEXT_CLEANER_DISABLED (consume-and-drop); the refund also disables with the kill switch (accounting reverts to raw).

Files: packages/agent-runtime/src/context/digest.ts + context-queue export; apps/worker-context/*; run.ts enqueue; root scripts/turbo registration.
Tests:

• digest.test.ts — fake-model unit tests: taxonomy classification on golden thread fixtures (paginated-read chain, duplicate reads with earlier-stubbed direction, error/retry, superseded searches with verbatim hit ids); keep-rules honored (incl. empty-tool_call_id exclusion); time-varying sources never LLM-digested; cost caps stop the batch; content-hash reuse skips the LLM call.
• Budget-refund tests in graph.ts's test suite — effective accounting with digests present; raw accounting past CHECKPOINT_RAW_CHAR_CEILING; raw accounting when the kill switch is set.
• digest.integration.test.ts — modelled on worker-memory/synthesize.integration.test.ts: real DB (isolated clone), fake model — enqueue → digest rows land org-scoped; turn-lock held ⇒ job requeued, no rows; below-trigger thread ⇒ skipped.
• End-to-end (fake model): heavy thread → worker pass → next turn's assembled prompt is smaller, answers still grounded — asserted on the fixture.

Shippable: yes — the full hybrid loop, kill-switched, with metrics. Enable on dev first, then preprod (morning window).

Phase 4 — Eval battery + tuning (the proof)

Goal: spec §4.6/§5 — no-regression proof wired into the standing gate, thresholds tuned on real data.

• packages/agent-runtime — eval:context-cleaner: golden long-thread fixtures (built from anonymized heavy dev threads + synthetic HSE-shaped ones) → full clean → assert: legally-material probe set answers 100%; figure-provenance guard passes for artifact calls citing condensed sources; interrupt/resume toolCallId matching intact; chars-saved ≥ fixture thresholds. Deterministic parts run on fake models; one small real-digest smoke set (cents).
• Wire into root eval:gate in package.json; confirm retrieval / router / memory-recall batteries unchanged.
• Tune CONTEXT_CLEAN_TRIGGER_CHARS, RECENCY_KEEP_EXCHANGES, DIGEST_CHAR_CAP against Phase 0 baselines; record before/after in the eval report.
• Flip bun docs/superpowers/status.ts set dynamic-context-cleaner implemented in the landing commit.

Files: packages/agent-runtime eval scripts + fixtures; root package.json.
Tests: the battery itself is the deliverable; bun run eval:gate green is the exit criterion.
Shippable: yes — this is the "cleaner is trusted" milestone.

Phase 5 — Follow-ups (explicitly out of v1, tracked, not started)

• Digest-informed history compaction — replace drop-oldest 300-char snippets with a compaction that consumes existing digests, so even the backstop is semantic.
• read_original(tool_call_id) — a checkpoint-backed read tool (org-authorized through chat_thread, same doctrine as the worker) that would make time-varying tool output honestly re-readable and unlock LLM-digesting it (spec §1 cat. 5 restriction lifts).
• Assembled-view figure sourcing (spec §6.3) — decide with Phase 4 data.
• UI transparency marker (spec §6.1) and per-firm opt-out (spec §6.2) — pending Christophe's decisions.

Rollout + risk notes

• Deploy order is free: every phase is backward-compatible; the digest table can exist with no worker, the worker can run with no readers. Preprod promotion follows the morning-window rule and asks Christophe first, as always.
• Recovery from any cleaner bug: set CONTEXT_CLEANER_DISABLED=1 (instant, no deploy) and/or DELETE FROM context_digest WHERE … — raw checkpoints were never touched.
• Schema discipline: one migration, generated, journal watermark checked, GRANTs via provision — per the prime directive. Only one agent touches the DB schema at a time.