Async research delegation — implementation plan

Status: Planned · Author: Claude (with Christophe) · Date: 2026-07-24 · Repo: north-os
Related: docs/superpowers/specs/2026-07-24-async-research-delegation-design.html (design), docs/reviews/2026-07-24-retrieval-speed-research.md (research memo)

Ships the design in dependency order. The rule of the whole plan: quality gate before latency — a phase lands only when its correctness/RLS/coverage bar passes; turn-count and latency are measured second. Schema-touching phases are flagged [SCHEMA] and must be serialized (one bun db:generate at a time).

Phase 0 — Measurement first

You cannot optimize turns you do not count. Extend packages/agent-runtime/scripts/eval-cowork-bench.ts to record and reward turn count as a primary metric (external Cowork calls, host turns, internal graph calls, native tool calls, decision depth, time-to-first-progress, time-to-answer, citation/coverage/conciseness). Add the SHF cross-matter scenario as a fixture. Stand up the already-committed monitoring stack on preprod (currently not running).
Gate: SHF baseline recorded (turns + latency + quality); dashboard live. → Linear issue 1

Phase 1 — Tier 2: batch evidence extractor [feeds Phase 2]

north__matter_documents_extract({ matter_ids | targets[], intent }) — read K documents server-side (reuse the matter-drive-documents.ts fan-out + document-deep-read.ts), return compact quote-bearing evidence with strict matter/document/section binding, full/partial coverage semantics, 6-wide shared admission, and no write behavior.
Gate: 25 sequential reads → 1 call on the SHF fixture; cross-org + forged-locator decoys refused; partial coverage reported honestly. → Linear issue 2

Phase 2 — Read-only research graph

In packages/agent-runtime: a plan → extract → synthesize → verify → stop graph with an explicit read-only tool allowlist (no ask_user, no writes, no confirmations), the same RLS gates and methodology as the interactive runtime, a 5–8 internal-round budget, coverage accounting, and the supported-negative answer shape. Runs headless (callable outside a request). Uses the Phase-1 extractor and existing search tools.
Gate: SHF arm non-inferior on correctness, citation quality, and coverage vs the current Cowork run; read-only allowlist physically enforced; no tool calls after the completion condition. → Linear issue 3

Phase 3 — Durable job lifecycle + worker-research [SCHEMA]

research_job table (RLS, org-scoped: question, scope, status, progress, covered/pending, result, budget, ownership). A research queue; a dedicated worker-research consumes it, runs the Phase-2 graph, writes progress/result, and emits a completion event to clicky.events. Evaluate reusing the worker-automations headless-agent chassis vs a new worker (lean new, per the one-worker-per-domain rule).
Gate: start enqueues + returns immediately; status/result/cancel work; job survives a client disconnect; cancellation stops queued + in-flight work. → Linear issue 4

Phase 4 — MCP surface + report-back

Expose research_start / research_status / research_result / research_cancel on apps/mcp-server (OAuth-gated, RLS). Wire the completion event to a desktop/Polaris notification ("Your SHF research is ready") off the existing clicky.events seam. Non-blocking end to end: Cowork fires start (optionally from a subagent), the lawyer keeps working, gets notified, then the answer is surfaced.
Gate: a real Cowork session runs the SHF task non-blocking and the lawyer is notified on completion; 1 external Cowork turn measured on the Phase-0 harness. → Linear issue 5

Phase 5 — Tier selection + stop discipline

Update cowork/briefing.ts, legal-assistant/prompt.ts, apps/mcp-server/src/tools/briefing.ts and the tool descriptions: batching-first affordances (steer to matter_ids[] / tier 2), the tier-selection rules (when to delegate to tier 3 vs stay fine-grained vs delegate only the packet), and the supported-negative stop rule (bottom line first; a supported negative is complete; offer one follow-up, don't pad).
Gate: on the Phase-0 eval the model selects the smallest correct tier and stops at the completion condition; SHF answer is calibrated (no "trop étalé"). → Linear issue 6

Phase 6 — Enabling speed layers (parallelizable; land underneath)

Independent of the delegation surface; they make tier 3 feel instant and shave the interactive path too. Order the [SCHEMA] ones behind Phase 3.

• 6a — Whole scored-pool knowledge cache [SCHEMA] (DB-backed corpus epoch). Cache the full reranked pool, rehydrate under RLS on hit, reapply boosts deterministically. Gate: exact top-8 identity vs uncached, incl. after supersession/deletion/policy change; real non-eval hit-rate ≥ ~10%. → Linear issue 7
• 6b — Delta-synced matter metadata catalog [SCHEMA]. Fix Graph @odata.nextLink completeness first; RLS + org-keyed indexes; live fallback until completeness proven. Gate: certified snapshot search 20–100ms; an incomplete catalog never returns authoritative absence. → Linear issue 8
• 6c — Parse-once extraction artifacts [SCHEMA] (immutable extracted text + manifest keyed by ETag + parser version). Gate: repeat reads 10–100ms; version mismatch refills; deletion revokes. → Linear issue 9
• 6d — Allowlisted concurrent read-only tool execution. The native graph runs tool calls serially (for … await tool.invoke); parallelize only provably-independent read-only calls, preserving order + coverage accounting. Gate: independent multi-read stages sum→max; no reordering/coverage drift. → Linear issue 10

Explicitly deferred (rejected as priorities, with evidence)

• Local reranker/encoder — no GPU on host (likely slower than cloud); shadow-only research track, not a production bet.
• Matryoshka / halfvec / binary ANN tuning — Postgres is already ~18ms; the floor is Voyage network, not the DB.
• Raising Graph concurrency alone — removes zero model turns, risks 429 tails.

Sequencing

0 → 1 → 2 → 3 → 4 → 5 is the critical path (each depends on the prior). Phase 6 items land in parallel once their [SCHEMA] slot is free; 6a and 6d are independent shaves that can ship any time and don't block the delegation surface. Only one schema-touching phase generates a migration at a time.