From 283a0542b91a299ddde89ebcc8a4cad3f19b46a1 Mon Sep 17 00:00:00 2001 From: Dandaedre Date: Tue, 29 Sep 2026 15:32:39 +0200 Subject: [PATCH] docs: define detector limitations --- README.md | 11 +++++-- docs/README.md | 2 ++ docs/limitations.md | 76 +++++++++++++++++++++++++++++++++++++++++++++ docs/usage.md | 10 +++--- tasks.md | 2 +- 5 files changed, 93 insertions(+), 8 deletions(-) create mode 100644 docs/limitations.md diff --git a/README.md b/README.md index 0b5b51c..f9502f6 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,11 @@ heuristics for human review; it does not prove AI authorship or identify a model. Today, the detector reads UTF-8 files, directories, ZIP archives, and common -ZIP-based office documents. Its first conservative rule family identifies -configured non-ASCII hyphen-like characters and exposes safe ASCII replacement -edits. Broader data-driven heuristics, terminal diagnostics, a formatter, and an +ZIP-based office documents. Eleven conservative, configurable rules cover +copied assistant scaffolding, generic style residue, source-code placeholders, +tutorial narration, and configured non-ASCII characters. Rules retain exact +source spans, apply path/content scopes, and feed a bounded, explainable +confidence score. Source-highlighted terminal diagnostics, a formatter, and an LSP are planned in [tasks.md](tasks.md). ## Quick start @@ -33,6 +35,9 @@ signals, not the probability that AI wrote the content. - [Command-line usage and current capabilities](docs/usage.md) - [Architecture and public library API](docs/architecture.md) - [Layered configuration](docs/configuration.md) +- [Heuristics, confidence, and applicability](docs/heuristics.md) +- [Evaluation corpus and current metrics](docs/evaluation.md) +- [Limitations and responsible interpretation](docs/limitations.md) - [Development, tests, features, and coverage](docs/development.md) - [Design decision records](docs/decisions/) - [Background reading](docs/background.md) diff --git a/docs/README.md b/docs/README.md index 7eca845..f02d8ce 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,6 +13,8 @@ the details needed to use, embed, and develop John Slop: applicability, evidence damping, normalization, and score interpretation. - [Evaluation corpus](evaluation.md) records fixture provenance, per-rule precision/recall, aggregate false positives, and benchmark caveats. +- [Limitations](limitations.md) covers false positives, evasion, domain shift, + score semantics, evaluation limits, and operational boundaries. - [Development](development.md) covers the supported Rust version, feature matrix, snapshots, coverage, badges, and CI quality gates. - [Background](background.md) collects external context on observable signs of diff --git a/docs/limitations.md b/docs/limitations.md new file mode 100644 index 0000000..0afd64e --- /dev/null +++ b/docs/limitations.md @@ -0,0 +1,76 @@ +# Limitations + +John Slop is a review aid for observable low-care signals. It is not an +AI-authorship detector, plagiarism detector, fact checker, security scanner, or +quality oracle. A finding says only that configured text matched at a particular +source span. Human context remains decisive. + +## False positives + +Legitimate material can contain every shipped signal. Documentation may quote a +chat transcript, an article may discuss knowledge cutoffs, a tutorial may use +step-by-step comments deliberately, and a tracked source-code TODO may be valid. +Unicode punctuation can be correct typography even when a downstream format +requires ASCII. Generic transitions are weak style evidence, not defects by +themselves. + +Use path scopes, rule disabling, and project-specific weights to reflect local +policy. Do not automatically reject, shame, or attribute a document based on a +score. Review the visible findings and original context. + +## False negatives and evasion + +Exact and regular-expression rules are inherently easy to evade through +paraphrasing, case changes, spelling variation, inserted characters, or removal +of obvious scaffolding. Careless work need not contain any known signal, while +well-reviewed AI-assisted work may contain none. John Slop deliberately avoids +opaque statistical authorship guesses, so a clean result is not evidence of +human authorship or high quality. + +Adversarial inputs can also target applicability assumptions: misleading file +extensions, unusual generated-directory names, or stdin without a filename may +select a different scope. Archive analysis uses the inner member path but does +not infer semantic language from content. + +## Domain and language shift + +The starter prose rules primarily encode English phrases. Other languages, +specialized editorial styles, historical corpora, transcripts, generated API +documentation, notebooks, and uncommon programming languages are +underrepresented. File-extension inference covers a conservative common set and +defaults unknown paths to prose. Teams should validate rules against their own +reviewed material before changing enforcement thresholds. + +## Confidence is not probability + +The score is a deterministic compression of configured weights, occurrence +caps, and category damping. Those values are policy choices and have not been +calibrated against population-level authorship data. A score of 75 does not mean +there is a 75% probability of AI authorship, low care, or an error. Thresholds +name review bands only, and changing the catalog changes their meaning. + +Every score is traceable to findings, but that explainability does not make the +underlying heuristic universally valid. Related signals can remain correlated +even after category damping. + +## Evaluation limits + +The checked-in evaluation corpus is small, purpose-built, and intended to catch +behavioral regressions. Its current perfect precision and recall reflect fixture +construction, not real-world accuracy. It contains only one clean document and +must not justify deployment claims or default-weight tuning. Representative, +licensed, domain-specific samples and independent human review are required for +such decisions. + +## Input and operational boundaries + +Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8, +encrypted archives, unsafe archive members, and configured resource-limit +violations are errors rather than clean results. Generated, vendor, dependency, +and minified paths are skipped by default. The current command-line summary does +not yet render source-highlighted diagnostics; formatter and LSP binaries remain +scaffolds for later milestones. + +Analysis is local and performs no network lookup. Consequently, it cannot check +citations, current facts, provenance, licensing, or similarity to external +sources. diff --git a/docs/usage.md b/docs/usage.md index 452896e..af18bd4 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -14,9 +14,9 @@ archive reading stream one result at a time, so callers do not need to retain an entire input tree in memory. The current human-readable output contains the source identity, bounded evidence -confidence, and finding count. Findings currently cover configured non-ASCII -hyphen-like characters. More heuristic families and source-highlighted `miette` -diagnostics are planned. +confidence, and finding count. Findings cover the embedded or overridden +heuristic catalog and retain exact UTF-8 byte spans for later presentation. +Source-highlighted `miette` diagnostics are planned for the next milestone. Exit statuses are stable: @@ -25,7 +25,9 @@ Exit statuses are stable: - `2`: arguments are invalid or an operational/input error prevents completion. Confidence is a summary of observable configured evidence. It is not an -AI-authorship probability. +AI-authorship probability. See [Heuristics and confidence](heuristics.md) for +the formula and [Limitations](limitations.md) before using thresholds in review +workflows. Configuration layers can be selected with `--config-name NAME` and `--config FILE`; `--max-document-bytes BYTES` is the current highest-precedence input diff --git a/tasks.md b/tasks.md index accaf89..1b03cc5 100644 --- a/tasks.md +++ b/tasks.md @@ -129,7 +129,7 @@ configuration is explainable, and all precedence/validation tests pass. - [x] Build a reviewed evaluation corpus that is legally distributable and labels observable low-care signals, not presumed author identity. Report per-rule precision/recall and aggregate false-positive rates before tuning defaults. -- [ ] Document limitations, adversarial evasion, domain shift, and why the score +- [x] Document limitations, adversarial evasion, domain shift, and why the score is not an AI-authorship probability. Acceptance: every score decomposes into visible findings; no high-confidence