Async research delegation — North-in-Cowork orchestration speed — design spec

Status: Planned · Author: Claude (with Christophe) · Date: 2026-07-24 · Repo: north-os (apps/mcp-server, packages/agent-runtime, packages/knowledge, packages/connector-onedrive, a new worker-research)
Related: docs/reviews/2026-07-24-retrieval-speed-research.md (research memo), docs/reviews/2026-07-24-preprod-search-benchmark.md (benchmark), docs/superpowers/specs/2026-07-09-cowork-plugin-design.html (MCP surface), docs/superpowers/plans/2026-07-18-mcp-partial-state.html (partial-state contract)

Problem

Per-tool latency is already good (preprod: knowledge search ~1.5s, matter search 87ms, memory 3ms). The felt slowness on real work is turn count, not tool latency:

total latency ≈ Σ(per-tool latency)  +  turns × (model re-planning + MCP round-trip)

A real task — "across all our Settlement Housing Fund matters, did we get the Clusters 3 Freddie modifications anywhere else, cite each source" — ran on Claude Cowork over our MCP as ~30+ sequential round-trips: find 25 matters → read Clusters 3 → sweep matter-by-matter → read docs one at a time → then spawn a compare subagent. Every round-trip pays MCP-HTTP RTT + Cowork re-planning. The same task on our own heynorth.dev harness finished faster and answered tighter, because our tools run in-process and our graph owns orchestration and stopping.

Two conclusions: (1) collapsing turns beats shaving any single tool — raising server-side concurrency removes zero model turns; (2) the answer also over-delivered ("the investigation became the answer") — a calibration failure downstream of weak tool-use reasoning.

Goals

• Let a lawyer run heavy multi-matter research without blocking their chat — fire it, keep working, be reported back to.
• Collapse the dominant cost (Cowork turns) for self-contained research by moving orchestration in-process behind a small number of tools.
• Keep a graduated surface: fine-grained tools for small asks, batch tools for mid-size asks, an async subagent for big research. Not one 25→1 mega-tool for everything.
• Preserve every correctness / RLS / grounding invariant. Quality is non-negotiable; latency is counted only once quality is non-inferior.

Non-goals

• Not a universal north__do_anything(task) — unbounded, unsafe, un-evaluable.
• Not an SHF-specific tool.
• No new retrieval scoring; reuse the proven pipeline (Voyage rerank stays authoritative).
• Not removing the fine-grained tools — they stay for interactive, ambiguous, and write flows.

The granularity spectrum (the balance)

25→1 is the right collapse for a genuine cross-matter research question, but wrong for a two-document lookup. The surface is three tiers, and the briefing + tool descriptions steer the agent to the smallest tier that fits:

The tiers compose: for a mixed task, delegate only the research packet to tier 3, then let Cowork draft fine-grained from that packet. Tier 2 is the deliberate middle — it collapses turns without handing the whole task to a black box. That is the "25→1 is sometimes too much" answer.

Tier 3 — the async research subagent

Shape

north__research_matters is a domain-scoped agent task (one conceptual action: produce a grounded, cited cross-matter research answer) — not a workflow, not a generic executor. Internally it runs a read-only agent-runtime graph — plan → extract → synthesize → verify → stop — with an explicit tool allowlist (no ask_user, no writes, no confirmations), the same RLS gates and methodology as the interactive runtime, and a 5–8 internal-round budget.

Non-blocking lifecycle (the core of "run it in the background")

The tool is a durable job, not a long synchronous call — so it survives disconnects, streams progress, is cancellable, and can report back after the lawyer has moved on:

• research_start({ question, matter_scope?, doc_type?, budget? }) → returns { task_id, status: "running" } immediately. Enqueues; does not block.
• research_status({ task_id }) → { status: running|done|failed|cancelled, progress, covered[], pending[], partial? } — bounded status states ("Resolving matters", "Searching 25 matter folders", "Reading 6 documents", "Verifying sources"), never pre-rerank hits or provisional conclusions.
• research_result({ task_id }) → the final cited answer once done.
• research_cancel({ task_id }) → stops queued and, where supported, in-flight work.

Where it runs: a dedicated headless worker (worker-research, per the one-worker-per-domain rule) consumes a research queue and runs the graph, writing progress/result to a research_job table (RLS, org-scoped). The MCP _start only enqueues. This is the same headless-agent pattern worker-automations already uses for automation.run (evaluate reuse vs a new worker in the plan).

How the lawyer keeps working + gets reported back:

• Cowork fires research_start (optionally from a Cowork subagent so the main thread never even waits on the poll), gets a task_id, and the lawyer continues.
• Completion pushes an event to clicky.events → desktop/Polaris notification ("Your SHF research is ready"), the same seam worker-automations uses for run events. The polling agent (or a fresh turn) then surfaces the cited answer.
• If Cowork's async / MCP-Tasks capability proves out, expose the job through MCP Tasks too; the durable lifecycle is the substrate either way.

Inputs / outputs

Input is intent + scope (a natural-language question + optional matter/doc-type scope), not resolved IDs — the graph resolves matters itself (no cheating). Output is the calibrated answer: bottom line first, minimum supporting analysis, source docs with operative quotes and matter/doc/section binding, and an explicit coverage statement ("25 resolved SHF matters; 24 fully searched, 1 unreadable") so a negative is honestly scoped.

Tier 2 — batch evidence extraction

north__matter_documents_extract({ matter_ids | targets[], intent }) reads K documents server-side (6-wide shared admission) and returns compact quote-bearing evidence with strict matter/document binding and full/partial coverage semantics — no write behavior. This is the 25-sequential-reads → 1-call collapse, available to Cowork directly (not only inside tier 3), for asks where the agent wants to keep control of the reasoning.

Enabling speed layers

These make tier 3 feel instant rather than "one long opaque call"; specified fully in the research memo, summarized here:

• Whole scored-pool knowledge cache keyed by a DB-backed corpus epoch → exact repeat knowledge query 20–100ms, byte-identical ranking (today we cache only candidate IDs and re-rerank, so warm is still 706ms).
• Delta-synced matter metadata catalog (RLS, org-keyed) → deep/fan-out folder search 20–100ms instead of 8–11s live Graph walks. Blocker: the current walk uses $top=200 without @odata.nextLink follow — incomplete enumeration must never produce a firm-wide negative.
• Parse-once extraction artifacts (immutable extracted text + manifest keyed by ETag + parser version) → repeat reads 10–100ms.

Answer-shape / stop discipline

Add a supported-negative stop rule to the briefing (cowork/briefing.ts, legal-assistant/prompt.ts, apps/mcp-server/src/tools/briefing.ts): answer the actual question first; bottom line in sentence one; a supported negative is a complete answer; don't add sections because research was extensive; offer one specific follow-up instead of padding. Applies to both the tier-3 graph output and the interactive runtime. It fixes the "trop étalé" SHF answer and removes 0–2 post-answer exploration turns.

Correctness / RLS acceptance gate (must pass before latency counts)

• All requested matters accounted-for or explicitly pending; no source crosses org boundaries; same-org user/task ownership respected.
• Every positive conclusion entailed by cited returned text; correct matter/doc/section binding.
• No firm-wide negative from capped, unread, truncated, or incomplete coverage; "exactly 25 discovered" ≠ complete enumeration; nothing silently omitted past Graph's first 200 children.
• Source changes between fetch and extraction detected; cache/artifact existence never grants authorization (RLS rehydrate on every hit).
• Read-only allowlist physically enforced in the delegated graph; no tool calls after the completion condition; cancellation stops queued + in-flight work.
• The SHF arm is non-inferior on correctness, citation quality, and coverage before latency or turn count is even considered.

Telemetry

Extend eval-cowork-bench.ts to record and reward turn count as a primary metric (today it records model calls + wall-clock but not turns): external Cowork calls, host model turns, internal graph calls, native tool calls, decision depth, time-to-first-progress, time-to-final-answer, citation/coverage/conciseness. Stand up the already-committed monitoring stack on preprod so we can tell whether a bet survives real traffic.

Open decisions

1. worker-research as a new domain worker vs reusing the worker-automations headless-agent chassis. (Lean new worker per the domain rule; confirm.)
2. MCP Tasks vs a bespoke durable start/status/result/cancel lifecycle. (Lean bespoke-durable first; add MCP Tasks once Cowork capability is verified.)
3. Report-back channel priority: desktop/Polaris notification vs surfacing on next chat turn vs a North-side task record. (Lean all three off the one clicky.events completion event.)
4. Default budget ceiling per research job (matters × docs × model calls) and per-org fairness.