Clause taxonomy normalization — design spec

Status: Planned
Author: Claude Opus 4.8
Date: 2026-07-20
Repo: north-os
Related: Cowork demo remediation; typed-clause retrieval quality incident

Problem

The typed-clause tools treat classifier-produced free text as a stable taxonomy. Read-only preproduction diagnostics confirm 27,974 knowledge_clauses rows across 1,714 labels, including 1,103 singleton labels. The playbook has 1,245 rows across 488 labels, including 315 singletons. Both handlers currently require exact SQL equality, so semantically equivalent labels are invisible to one another.

The failure is material: limitation_of_liability returns 25 rows even though the corpus contains many equivalent spellings and nearby liability concepts. The same fragmentation affects assignment, indemnification, notices, jury-trial waivers, conflicts, and other recurring concepts. Embedding ranking cannot repair the miss because it runs only after the exact-label filter.

Goals

• Define one reviewable, typed vocabulary of 135 canonical clause types derived from the observed corpus distribution.
• Preserve every raw classifier label while adding a canonical classification beside it.
• Resolve agent free text deterministically when possible and use corpus embeddings for the unresolved long tail.
• Retrieve every clause assigned to the requested concept, then rank within that complete candidate set.
• Return exact playbook positions when present; otherwise return clearly labelled nearest canonical positions without implying an exact firm rule.
• Expose extractor-recorded clause party-role context without inferring direction, action, client side, or which party a clause protects.

Non-goals

• Deleting or rewriting clause_type.
• Weakening RLS, scope filters, matter walls, source status checks, or evaluation checkers.
• Using an LLM to remap production rows during retrieval or making a nondeterministic backfill.
• Inferring a protected party, client side, or clause favorability from party_role.
• Silently reranking precedents based on presumed client representation.

Evidence and representation boundary

Three firm retainer letters in the corpus state that Hirschen Singer & Epstein represented borrower/developer entities in permanent-loan closings: The Balton LLC, Site 8 Apartments, L.P., and East Harlem MEC Parcel C L.P. That evidence supports a borrower-side reading for those engagements, but it does not prove the firm's role on every HFA transaction. Kingsbrook clauses also record mixed actors—mortgagor, mortgagee, guarantor, and other—under the same broad clause labels.

Decision: expose the extractor-recorded party_role as named-party context only. It is not evidence of who acts, who is protected, or who the client is. Protected-side semantics remain a separate, evidence-backed classification problem.

Approaches considered

The hybrid centroid approach is selected. It uses explicit aliases for known fragmentation and existing clause embeddings for unresolved labels, but persists centroid assignments only when calibrated similarity and runner-up-margin thresholds both pass.

Canonical taxonomy

packages/knowledge/src/taxonomy.ts owns the exact ordered corpus-derived vocabulary, reviewed alias map, normalization rules, and exported union type. The 135 entries are the highest-volume observed concepts after obvious spelling/plural aliases are collapsed. This code artifact is the reproducible taxonomy reviewed by the measurement tests. Wire metadata remains validated strings in @workspace/knowledge-contracts so the lower-level contracts package does not depend on @workspace/knowledge.

The module also owns:

• token normalization: trim, lowercase, punctuation-to-underscore, repeated-underscore collapse;
• an explicit alias-to-canonical map for reviewed head labels;
• a bounded lexical nearest match for free-text spelling and word-order drift, with no forced low-confidence result;
• a canonical prompt list so new extraction is constrained to the controlled vocabulary.

Storage and rollout

Add nullable canonical_clause_type text columns to knowledge_clauses and knowledge_playbook_rules. Nullable expansion keeps deployment safe before the backfill and preserves exact-label fallback for unmapped or not-yet-backfilled rows. The existing table RLS and grants remain unchanged.

1. Edit the Drizzle schema and generate the migration with bun db:generate.
2. Deploy code that writes canonical values for new clauses and rules while tolerating null historical rows.
3. Run the Bun backfill through an owner connection, first applying reviewed aliases and then assigning only high-confidence unresolved label centroids to the nearest seeded canonical centroid per organization. Low-confidence labels stay null.
4. Use the measurement script in SELECT-only projection mode against preproduction. After deployment, an authorized operator may run the writable backfill with an owner DSN and then verify persisted counts.

The backfill updates only null canonical columns. A second run therefore reports zero changed rows and is a safe retry of the initial operation. Raw labels remain unchanged for audit. Reclassifying non-null rows after a vocabulary change requires a separate dry-run remap workflow; this initial backfill never overwrites them.

Retrieval behavior

Precedent clauses

1. Normalize and resolve the requested clause_type.
2. Match the resolved canonical column across all synonym labels.
3. Include the normalized raw exact label as a null-column deployment fallback.
4. Rank the complete candidate set by the existing clause embedding distance and preserve all authorization predicates.
5. Return raw type, canonical type, and nullable recorded party role in metadata.

Playbook rules

Query the resolved canonical type plus reviewed raw aliases on null/unmapped rollout rows first. If resolution fails, try normalized raw exact matching. Only when no exact canonical or raw rule exists, compare the requested canonical clause centroid to centroids for canonical playbook types that do have active firm-global rules. Return the closest bounded set as nearest matches, including each matched canonical type and similarity. The result contract explicitly distinguishes exact from nearest; rendering states that no exact position is on file and never presents neighbours as the requested rule.

Failure handling

• An unresolvable free-text type falls back to normalized exact raw matching rather than forcing a false canonical assignment.
• A missing or low-confidence canonical centroid leaves the row null and yields exact raw fallback or the existing honest absence result.
• Embedding or database failures propagate through the existing recoverable tool error path.
• Rows with null canonical values remain reachable by exact raw fallback during rollout.

Verification

• Resolver tests cover case, punctuation, spaces, aliases, reordered words, and unrelated low-confidence input.
• Precedent handler tests prove a synonym request reaches rows whose raw label differs but canonical type matches, while exact null-column fallback still works.
• Playbook tests prove exact canonical rules win and empty exact results return honestly labelled nearest canonical positions.
• Backfill tests prove reviewed alias seeding, high-confidence centroid fallback, low-confidence null retention, raw-label preservation, and zero updates on a second run.
• Party-role tests prove extractor-recorded named-party metadata is surfaced without directional, action, protected-side, or client-side claims.
• Read-only measurement reports old exact versus canonical counts for limitation of liability, indemnification, assignment, and one additional fragmented concept.

Success criteria

• Typed precedent lookup is concept-complete for reviewed aliases and accepted high-confidence centroid assignments, while unresolved rows retain exact raw fallback.
• Playbook absence is actionable but honest: exact position or explicitly non-exact neighbours.
• New ingestion writes canonical values and no longer encourages new free-text taxonomy labels.
• Migration, backfill code, tests, package checks, root typecheck, and lint pass without RLS or eval-checker changes; persisted production/preproduction backfill execution is a post-deploy operator step.