Chat pre-turn parallelization + real tool duration_ms — implementation plan

Status: Approved
Author: Claude (NOS-212 orchestrator)
Date: 2026-07-23
Repo: north-os
Related: design spec; Linear NOS-212

Step 1 — pre-turn parallelization (`packages/chat-runtime/src/run.ts`)

Extract the memory-load block (~:2073) into const loadMemoryBlock = async (): Promise<string> — keeps its internal try/catch and the fire-and-forget recordMemoryUse.
Extract the ledger-load block (~:2107) and the digest-load block (~:2204) into one loadLedgerThenDigests closure returning { ledgerBlock, ledgerEntries, ledgerCoverageAvailable, attachmentOrigins, contextDigests }. Digest keeps its !isContextCleanerDisabled() && ledgerCoverageAvailable gate and its own try/catch.
const [memoryBlock, ledgerResult] = await Promise.all([...]); downstream code (normalizeThreadDocumentCoverage, ledgerCoverage, threadHot, graph config) reads from the result — output-identical modulo the documented digest-freshness window.
New deterministic concurrency test in run.test.ts: hold the memory load unresolved via a deferred promise, assert the ledger load has started; hold the ledger unresolved, assert the digest load has not started; resolve both, assert the graph config carries the loaded blocks.

New src/audit/tool-timing.ts: AsyncLocalStorage<ToolTimingScope> where ToolTimingScope = { startedAt: number; pendingAudit: PendingAuditTee | null }. Exports invokeWithToolTiming<T>(toolName: string, fn: () => Promise<T>): Promise<T> (emits histogram always + deferred product_event on settle; rethrows GraphInterrupt without emitting) and a scope accessor for recordToolAudit.
src/audit/tool-audit.ts: inside an ALS scope with no explicit durationMs → write the audit row, stash pendingAudit (last wins), skip the immediate tee. Outside a scope or with explicit durationMs → emit immediately (mcp-server path), plus a histogram sample when an explicit duration was given. Rewrite the stale agentId/durationMs doc comments.
Wrap the seven invoke sites:
- agents/legal-assistant/graph.ts:1126
- agents/screen-companion/graph.ts:102
- agents/desktop-ack/graph.ts:84
- agents/proactive-glance/graph.ts:77
- agents/skill-author/graph.ts:124
- agents/glance-analyst/graph.ts (runToolCalls)
- workflows/executor.ts:138
Export invokeWithToolTiming from the package index only if a consumer outside agent-runtime needs it (none known — keep internal).

src/audit/tool-timing.test.ts: full-span emit (audit early, keep working, duration covers the whole invoke); throw → histogram sample with outcome: "error" and no product_event when nothing was audited; GraphInterrupt → rethrown, nothing emitted; nesting → innermost scope wins; exactly one event per invocation when a tool audits twice (last outcome wins).
src/audit/tool-audit.test.ts: mock @workspace/analytics; assert (a) explicit durationMs passes through and emits immediately with a histogram sample, (b) wrapped call defers — no immediate emit, wrapper emits real elapsed > 0, (c) unwrapped call emits 0 immediately with no histogram sample. Histogram assertions go through the observability registry (see packages/observability/src/metrics.test.ts patterns) with a registry reset between tests.
Run bun --filter=@workspace/agent-runtime test and bun --filter=@workspace/chat-runtime test.

bun run lint + bun run typecheck from root.
Scratchpad probe (never committed): time the three pre-turn loads serial vs Promise.all against the dev DB (read-only), 5 runs each, plus a 10-concurrent variant (pool-pressure check — report total latency under concurrency, not just sequential medians) — before/after table for the PR body.
Tool-duration proof: invoke real read-only tools wrapped in invokeWithToolTiming against the HSE org (timing-probe pattern) so recordToolAudit emits live rows; then run the p50/p95 SQL from the spec (with the created_at filter — historical rows are all 0) against product_event — rows must show real durations.

One omp backend worker: Steps 1–3 (single package-boundary-respecting change set, no file overlap with other NOS issues this wave; NOS-210/211 rebase cleanly — this branch touches only run.ts pre-turn block + agent-runtime audit/graph invoke lines).
Orchestrator: docs, probes, gates, PR.