NetDocs tips loop — mail intake, firm wiki, newsletters — design spec

Status: Planned
Author: Christophe
Date: 2026-07-22
Repo: north-os (apps/web, packages/agent-runtime, packages/knowledge, packages/db, apps/worker-notifications, apps/worker-automations)
Related: docs/superpowers/specs/2026-07-20-knowledge-retrieval-precision-design.html; weekly digest apps/worker-notifications/src/loops/weekly-digest.ts; automations tools packages/agent-runtime/src/tools/automations.ts

Context

Gill (HSE) asked for a test run of a multi-step agent loop: HSE staffers email NetDocs tips and tricks (text, screenshots, video links) to a dedicated intake address (netdocstips@hseny.com); an agent curates them into a coherent, referenceable resource; a periodic email goes to staff (every 2 days initially, weekly once settled). Explicit goals: nobody "manages" the intake, nobody's inbox gets flooded, and the loop demonstrates that agentic automation actually saves time. Gill notes many HSE administrative processes could follow the same rubric — this is the low-stakes pilot for that pattern.

This spec locks the architecture discussed on 2026-07-22. Three deliverables:

The tips loop itself — intake mailbox → curated wiki → knowledge base → digest.
A firm wiki surface in apps/web — new product surface, reusable beyond tips.
Newsletters as a first-class capability — entity + agent tool family + UI, reusable beyond tips. This is the largest net-new piece.

The pattern being productized: North watches a source, keeps a living record, and tells the right people at the right moment. The next instance (new intake address, new wiki section, new periodic email) must be agent-doable configuration, not a project.

Decisions (locked)

D1 — Intake: licensed account, not a shared mailbox

connector-outlook is delegated per-user OAuth with every Graph call /me-anchored (subscriptions build me/mailFolders('Inbox')/messages; message/attachment fetch, folder listing — all /me). Scopes have no .Shared variants and there is no app-only (client_credentials) path anywhere in the codebase. True shared-mailbox support would mean new scopes (fresh tenant consent) plus parameterizing /me → /users/{mailbox} across ~6 files in the package and worker-mail.

Decision: HSE creates netdocstips@hseny.com as a regular licensed account. We create a North service user for it and enroll through the standard connector OAuth signed in as that account (completeAuth binds whatever tenant account signs in to the initiating North user — no UPN match required). The entire existing machinery (subscription, webhook fan-out, attachment ingest, recent-mail import) works unchanged because the token is the mailbox. Christophe performs the connection himself given the account credentials.

The .Shared refactor is explicitly deferred; revisit only if HSE refuses a licensed seat.

Follow-up: exclude the service mailbox from the mail-intelligence realtime pipeline so tips mail doesn't pollute matter signals.

D2 — The pipeline is a prompt, not a system

No bespoke worker, no new pipeline code. A scheduled automation (existing automations + worker-automations automation.run executor) runs the agent with existing tools: read new mail in the tips mailbox since last run (search_email / read_email_thread as the service user), extract each tip, enrich via web_research, write wiki pages, ingest to knowledge. The next admin workflow is a new mailbox + a new automation prompt + a new category.

D3 — Reading surface: a wiki inside `apps/web`

Not a new app, not a Word document (unless Gill insists — asked in the reply; a doc export can be generated as a projection later). A route group in the existing shell ((app)/wiki, "Firm knowledge"): same sidebar, same Better Auth session, same org scoping. One login — North is the basecamp.

wiki_page table: org-scoped RLS (firm-wide visibility), slug, category, markdown body, updated_by provenance.
v1 canonical store = wiki pages. Tips extracted from mail are written directly into wiki pages; no separate tips table until dedupe pressure demands one.
Screenshots → R2, served via an authenticated route.
Video links → format-agnostic link cards (Loom, Stream, YouTube, anything) with the agent's step summary. No dependency on which recording tool staff use; no iframe embedding (tenant frame policies). If a platform exposes a transcript or the email describes the steps, the summary gets richer — but nothing requires it.
Agent tools: create_wiki_page / update_wiki_page / list_wiki_pages — same tool-family pattern as automations.
Synthesis behavior (the openwiki pattern, implemented in-house): the agent periodically re-writes coherent themed pages as tips accumulate — reorganizes, merges duplicates — never an append-only log. openwiki itself (single-user local CLI, no serving/auth/tenancy) was evaluated and rejected as a dependency.

D4 — Knowledge ingestion: dedicated corpus category

Every wiki page is ingested into @workspace/knowledge via the existing pipeline/ingest-single-file.ts seam, under a dedicated corpus_category (e.g. firm-know-how / netdocs-tips), scope firm_global. This makes tips answerable in chat and over MCP (north__knowledge_search) — "how do I do X in NetDocs" returns the tip with its screenshot and video link.

Gaps to close (small): retrieval currently ignores corpus_category — add a category filter, expose it on the MCP search tool, and ensure firm-procedural content is excluded or down-ranked in matter contexts (firm know-how must never outrank matter documents in a matter thread — see the 2026-07-20 retrieval precision work).

Strategically this creates the second knowledge domain: firm operational know-how, alongside matter knowledge.

D5 — Newsletters: first-class entity, automation-backed via a facade

Verified posture: the agent cannot send mail autonomously today — mail-send.ts is the human-click path behind the ReplyDraftWidget ("never reached by the agent itself"), and the shipped weekly digest is a code-driven loop in worker-notifications. Autonomous newsletter send is therefore a deliberate, guardrailed posture change — a first-class capability, not a raw send tool.

Split of state:

Scheduling state (cron, timezone, next-run mint, headless execution) — delegated to a hidden automation row provisioned by create_newsletter. The automation is an implementation detail: not shown in tasks/automations (or shown only as "managed by newsletter X"); pausing the newsletter pauses it through the service layer so the two can never desync.
Newsletter state — own tables: newsletter (name, audience, sender mailbox, content-source prompt, cadence, status: active | paused, approval mode gated | autonomous), plus recipients and per-issue send records.

Agent tool family (mirrors automations): create_newsletter, list_newsletters, update_newsletter (pause/resume, recipients, cadence), delete_newsletter. The agent can spin up a newsletter from chat — the tips digest is simply the first newsletter created through the tool.

Guardrails (what makes autonomous send acceptable):

Newsletters send only from enrolled service mailboxes (e.g. netdocstips@) — never from a lawyer's own mailbox. The "agent never sends as you" invariant stands.
Audience must be an explicit managed list or distribution-list address — no free-form recipients.
Mandatory unsubscribe footer (route already exists: apps/web/app/api/newsletter/unsubscribe).
Every newsletter starts gated: each issue pauses for one-click approval; promoted to autonomous explicitly after the first sends look right.

Sending rides worker-notifications (already the outbound-email domain, owns the weekly-digest precedent and the unsubscribe route). Composition rides worker-automations (headless run per issue). No new worker.

D6 — Newsletter UI: issue step timeline

A page alongside tasks/automations: newsletter list (next send, last send, recipients, pause/disable), and a per-issue detail view making the pipeline visible:

Issue #4 · due Jul 24, 9:00
  ● Gather      3 new tips since issue #3
  ● Compose     draft preview (rendered email)
  ◐ Approve     waiting  [Approve] [Edit] [Skip this issue]
  ○ Send        → 47 recipients ("HSE All Staff")
  ○ Report      delivered / bounced / unsubscribed

The approval gate sits between Compose and Send. Promoted newsletters show Approve as auto-passed and the timeline becomes a send log. Failures (sender token expired, Graph throttling) surface as a red step instead of a silently dead automation.

D7 — Loop niceties

Submitter acknowledgment: each processed tip triggers a short confirmation email to the submitter ("your tip is in the library" + link) — cheap, drives adoption.
Digest content: only what's new since the last issue, with links into the wiki. Every 2 days initially, weekly once settled.

D8 — Privacy

Staff screenshots of NetDocs routinely show client/matter names. Screenshot content is never sent to web search — enrichment searches on tip text only.
The wiki and all media are strictly firm-internal (RLS + authed R2 route).
Reply to Gill asks staff to crop screenshots where possible (soft mitigation).

Architecture

HSE staffer ──mail──▶ netdocstips@hseny.com (licensed acct, enrolled via std OAuth)
                            │  worker-mail (existing, unchanged)
                            ▼
              scheduled automation (automation.run, headless)
                "process new tips since last run"
                            │
        ┌───────────────────┼──────────────────────┐
        ▼                   ▼                      ▼
  web_research        create/update           knowledge ingest
  (enrich, text       wiki pages              (ingest-single-file,
   only)              (canonical store)        corpus_category)
                            │                      │
                            ▼                      ▼
                     (app)/wiki in web       chat + MCP answers
                            │
                            ▼
              newsletter "NetDocs tips digest"
              (entity → hidden automation → compose run
               → gated approve → send via worker-notifications)

Single source of truth: wiki pages. Knowledge index, digest issues, and any future doc export are projections.

Net-new build (the honest shortlist)

#	Piece	Size
1	North service user + mailbox enrollment	config, not code
2	`wiki_page` schema + RLS + `(app)/wiki` route + 3 agent tools	medium
3	Knowledge: category filter in retrieval + MCP param + ingest tool wrapping `ingest-single-file`	small
4	Newsletter domain: tables + service + 4 agent tools + facade over automations + send seam in worker-notifications + UI with issue timeline	largest piece
5	Automation prompts (intake processing, synthesis) + tips digest newsletter	content, not infra
6	MI exclusion for service mailbox	small

No new worker. No third-party wiki engine. Migrations via bun db:generate as always.

Open questions (pending Gill)

Licensed account OK (vs shared mailbox)? Credentials shared with Christophe?
Wiki in North confirmed vs. an explicit Word document requirement?
Access rollout: all staff get North logins, or small group + digest carries content meanwhile?
Digest audience: all staff or a distribution list HSE controls? Who holds the approval click during the gated phase?

Out of scope (v1)

.Shared / app-only shared-mailbox support (deferred; only if licensed account refused).
Word/doc export of the wiki (projection; add later if asked).
Video transcript fetching per platform (summaries rely on email text + whatever transcript is trivially reachable; no per-platform integrations).
Separate structured tips table (wiki pages are canonical until dedupe pressure).
Generalizing newsletters to per-matter/per-client audiences (future).