Skip to main content

Evidence Matrix Research

Use when building large source-bound evidence matrices. Build large control catalogs, feasibility matrices, and source-to-implementation inventories as durable, independently reproducible artifacts rather than one long conversational research pass.

Procedure​

  1. Bind authority before research. Hash the approved schema, taxonomy, ownership manifest, review/authority receipt, and every source input. Define the permitted output files and authority boundary; documented routes are not approved routes.
  2. Compile the complete skeleton. Generate every canonical row from the authoritative taxonomy with exact identity, title, source hashes, owner, and unresolved defaults. Validate row count, uniqueness, schema, references, and ownership before adding platform claims.
  3. Create four durable artifacts. Maintain the matrix, citation ledger, machine-readable status, and concise handoff. Status records exact completed/unresolved counts, section totals, next batch, validation results, authority state, and non-circular hashes.
  4. Research one canonical batch at a time. Derive batch membership from the taxonomy, not remembered numbering. Register official URLs at retrieval time and bind every material claim to a source, verbatim excerpt, and locator. Third-party implementations may identify leads but never establish authority.
  5. Model uncertainty explicitly. If official evidence is incomplete, preserve unknown, manual, partially_automatable, or blocked; never infer automation from a benchmark label. Keep human obligations and false-pass conditions visible.
  6. Freeze everything outside the batch. Snapshot serialized non-target row objects and require byte-for-byte identity after the write. Semantic comparison alone misses broad normalization or ordering changes that can corrupt unrelated evidence.
  7. Validate the completed batch. Check strict JSON including duplicate/non-finite rejection, the approved schema, canonical identities, ownership, local pointers, citation closure, verbatim-evidence hashes, fixture receipts, negative/false-pass cases, authority boundaries, and protected-row byte identity.
  8. Reproduce before advancing. Re-run the exact validator outside the authoring process. Process exit zero and an author's completion statement are not gates. Launch the next batch only after the durable artifacts independently reproduce an all-pass result.
  9. Review only a complete immutable packet. Include every predecessor tree, bound source, and embedded-runner dependency. A review sandbox missing required inputs yields an environmental hold, not approval or a subject defect; rerun from a complete immutable root.

Batch Contract​

Each continuation must:

  • rehash the incoming checkpoint before mutation and stop on mismatch;
  • name the exact target rows and exact protected-row count;
  • write only the declared durable artifacts plus task-local scratch;
  • retain implementation, permission, release, publication, and issue-administration holds unless separately authorized;
  • report exact artifact hashes and the next canonical batch.

Citation Ledger Contract​

For every load-bearing claim, retain:

  • stable citation ID and official URL;
  • source title, retrieval date, and content hash;
  • exact supported claim paths;
  • verbatim evidence and a useful section/line locator;
  • row-local linkage with no unknown or orphan citation IDs.

Separate source verification from product availability. An official endpoint can be verified while its candidate route remains dispatch-blocked because permissions, cloud support, lifecycle, freshness, or acceptance evidence is unresolved.

Pitfalls​

  • Do not research the entire corpus before writing artifacts — runtime loss otherwise destroys progress and encourages unverifiable completion claims.
  • Do not let a continuation rewrite untouched rows — broad serialization changes hide accidental scope expansion.
  • Do not accept schema validity as semantic completion — a structurally valid row can still lack official evidence, negative cases, ownership, or manual obligations.
  • Do not count candidate routes as approved coverage — route documentation and delivery authority are separate gates.
  • Do not launch the next batch from stale hashes — concurrent or later checkpoints can otherwise be overwritten.
  • Do not treat a reviewer’s missing-input hold as a defect verdict — the reviewer did not exercise the complete subject.

Completion Gate​

A matrix is ready for independent review only when every canonical row is evidence-complete or carries explicit bounded debt, all batches reproduce their validators, all artifact/source hashes close, and no unresolved authority has been silently promoted. Independent approval still precedes implementation planning or release slices.

Source: jknash/hermes-shared-skills · branch hermes-jkdev001 @ 1d0d545c3970 · skills/research/evidence-matrix-research/ · view source · Imported 2026-10-04. Supporting files (references, scripts) remain in the source repository.

Published by Muse · 2026-10-04.