# Evaluation corpus Milestone 3 ships a small, reviewed characterization corpus under `tests/fixtures/evaluation`. Its text was authored for this repository and is covered by the MIT license. Labels name observable configured signals, never a presumed author or generation method. With the embedded schema-version 1 catalog, the corpus currently reports: | Rule | True positive | False positive | False negative | Precision | Recall | | --- | ---: | ---: | ---: | ---: | ---: | | assistant-self-identification | 1 | 0 | 0 | 100% | 100% | | assistant-capability-disclaimer | 1 | 0 | 0 | 100% | 100% | | assistant-answer-preface | 1 | 0 | 0 | 100% | 100% | | knowledge-cutoff-disclaimer | 1 | 0 | 0 | 100% | 100% | | prompt-response-scaffolding | 1 | 0 | 0 | 100% | 100% | | assistant-follow-up-offer | 1 | 0 | 0 | 100% | 100% | | assistant-revision-preface | 1 | 0 | 0 | 100% | 100% | | templated-conclusion | 1 | 0 | 0 | 100% | 100% | | templated-filler | 1 | 0 | 0 | 100% | 100% | | repeated-transition | 1 | 0 | 0 | 100% | 100% | | editorial-insertion-placeholder | 1 | 0 | 0 | 100% | 100% | | stock-helpful-closing | 1 | 0 | 0 | 100% | 100% | | excess-vocabulary-cluster | 1 | 0 | 0 | 100% | 100% | | source-placeholder-todo | 1 | 0 | 0 | 100% | 100% | | source-tutorial-narration | 1 | 0 | 0 | 100% | 100% | | source-omitted-code-placeholder | 1 | 0 | 0 | 100% | 100% | | source-markdown-fence | 1 | 0 | 0 | 100% | 100% | | unicode-ascii-replacement | 2 | 0 | 0 | 100% | 100% | | unicode-invisible-control | 1 | 0 | 0 | 100% | 100% | Aggregate labeled-rule counts are 20 true positives, 0 false positives, and 0 false negatives. The clean-document false-positive rate is 0/5 (0%). The test suite recomputes these values from the manifest and fails on drift. The same manifest pins evidence scores for regression characterization: | Corpus case | Score | | --- | ---: | | six assistant disclaimer/preface rules | 64 | | one prompt/response marker | 33 | | style signals | 25 | | source-code signals | 33 | | typography signal | 17 | | one ASCII lookalike and one invisible character | 28 | | each of five negative documents | 0 | These perfect values describe a tiny purpose-built regression set, not expected performance on real documents. They must not be used as an accuracy or authorship claim. Broader, independently sourced data and domain-specific review are required before tuning default weights or thresholds. ## Default-policy sign-off Each shipped rule has at least one explicitly labeled realistic near miss. The initial 0.1.0 review kept all schema-version 1 rules, weights, and thresholds unchanged. The prompt-leakage review accepted lighter `0.90ⁿ` correlation damping for distinct prompt-leakage rules while retaining `0.75ⁿ` for other categories. The Milestone 22 review makes printable-ASCII UTS #39 mappings replaceable by default, retains a non-mutating `review` policy, and keeps the Gitea-compatible invisible/control profile diagnostic-only, with explicit multilingual limitations. The Milestone 25 review gives one-to-one fullwidth ASCII compatibility folds precedence over visual skeletons under every policy. The Milestone 26 review adds narrow response and unfinished-editing residue, keeps population-level vocabulary and stock closings deliberately low-weight, and excludes language-proficiency proxies. The checked-in `signoff.toml` records those decisions and contains the SHA-256 digest of `config.toml`, `heuristics.toml`, and `unicode.toml`. This binds the reviewed scoring behavior to both rule catalogs and the default category policy. Any default rule, matcher, pattern, weight, occurrence cap, scope, or threshold change invalidates the sign-off test. Renewal requires reviewing positive and near-miss cases, reporting changed per-rule/aggregate metrics, explaining the false-positive tradeoff, updating the digest and decision text, and obtaining explicit maintainer approval in the change review. A green test alone is not approval to tune defaults. Runtime diagnostic catalogs are deployment policy and are not part of the embedded-catalog digest. Their rules use the same validation and scoring formula, but teams must evaluate their own added patterns and near misses for their corpus rather than treating the embedded sign-off as validation of custom rules.