docs: define detector limitations

This commit is contained in:
Dandaedre 2026-09-29 15:32:39 +02:00
parent f1fe9f5d8d
commit 283a0542b9
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
5 changed files with 93 additions and 8 deletions

View File

@ -10,9 +10,11 @@ heuristics for human review; it does not prove AI authorship or identify a
model.
Today, the detector reads UTF-8 files, directories, ZIP archives, and common
ZIP-based office documents. Its first conservative rule family identifies
configured non-ASCII hyphen-like characters and exposes safe ASCII replacement
edits. Broader data-driven heuristics, terminal diagnostics, a formatter, and an
ZIP-based office documents. Eleven conservative, configurable rules cover
copied assistant scaffolding, generic style residue, source-code placeholders,
tutorial narration, and configured non-ASCII characters. Rules retain exact
source spans, apply path/content scopes, and feed a bounded, explainable
confidence score. Source-highlighted terminal diagnostics, a formatter, and an
LSP are planned in [tasks.md](tasks.md).
## Quick start
@ -33,6 +35,9 @@ signals, not the probability that AI wrote the content.
- [Command-line usage and current capabilities](docs/usage.md)
- [Architecture and public library API](docs/architecture.md)
- [Layered configuration](docs/configuration.md)
- [Heuristics, confidence, and applicability](docs/heuristics.md)
- [Evaluation corpus and current metrics](docs/evaluation.md)
- [Limitations and responsible interpretation](docs/limitations.md)
- [Development, tests, features, and coverage](docs/development.md)
- [Design decision records](docs/decisions/)
- [Background reading](docs/background.md)

View File

@ -13,6 +13,8 @@ the details needed to use, embed, and develop John Slop:
applicability, evidence damping, normalization, and score interpretation.
- [Evaluation corpus](evaluation.md) records fixture provenance, per-rule
precision/recall, aggregate false positives, and benchmark caveats.
- [Limitations](limitations.md) covers false positives, evasion, domain shift,
score semantics, evaluation limits, and operational boundaries.
- [Development](development.md) covers the supported Rust version, feature
matrix, snapshots, coverage, badges, and CI quality gates.
- [Background](background.md) collects external context on observable signs of

76
docs/limitations.md Normal file
View File

@ -0,0 +1,76 @@
# Limitations
John Slop is a review aid for observable low-care signals. It is not an
AI-authorship detector, plagiarism detector, fact checker, security scanner, or
quality oracle. A finding says only that configured text matched at a particular
source span. Human context remains decisive.
## False positives
Legitimate material can contain every shipped signal. Documentation may quote a
chat transcript, an article may discuss knowledge cutoffs, a tutorial may use
step-by-step comments deliberately, and a tracked source-code TODO may be valid.
Unicode punctuation can be correct typography even when a downstream format
requires ASCII. Generic transitions are weak style evidence, not defects by
themselves.
Use path scopes, rule disabling, and project-specific weights to reflect local
policy. Do not automatically reject, shame, or attribute a document based on a
score. Review the visible findings and original context.
## False negatives and evasion
Exact and regular-expression rules are inherently easy to evade through
paraphrasing, case changes, spelling variation, inserted characters, or removal
of obvious scaffolding. Careless work need not contain any known signal, while
well-reviewed AI-assisted work may contain none. John Slop deliberately avoids
opaque statistical authorship guesses, so a clean result is not evidence of
human authorship or high quality.
Adversarial inputs can also target applicability assumptions: misleading file
extensions, unusual generated-directory names, or stdin without a filename may
select a different scope. Archive analysis uses the inner member path but does
not infer semantic language from content.
## Domain and language shift
The starter prose rules primarily encode English phrases. Other languages,
specialized editorial styles, historical corpora, transcripts, generated API
documentation, notebooks, and uncommon programming languages are
underrepresented. File-extension inference covers a conservative common set and
defaults unknown paths to prose. Teams should validate rules against their own
reviewed material before changing enforcement thresholds.
## Confidence is not probability
The score is a deterministic compression of configured weights, occurrence
caps, and category damping. Those values are policy choices and have not been
calibrated against population-level authorship data. A score of 75 does not mean
there is a 75% probability of AI authorship, low care, or an error. Thresholds
name review bands only, and changing the catalog changes their meaning.
Every score is traceable to findings, but that explainability does not make the
underlying heuristic universally valid. Related signals can remain correlated
even after category damping.
## Evaluation limits
The checked-in evaluation corpus is small, purpose-built, and intended to catch
behavioral regressions. Its current perfect precision and recall reflect fixture
construction, not real-world accuracy. It contains only one clean document and
must not justify deployment claims or default-weight tuning. Representative,
licensed, domain-specific samples and independent human review are required for
such decisions.
## Input and operational boundaries
Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8,
encrypted archives, unsafe archive members, and configured resource-limit
violations are errors rather than clean results. Generated, vendor, dependency,
and minified paths are skipped by default. The current command-line summary does
not yet render source-highlighted diagnostics; formatter and LSP binaries remain
scaffolds for later milestones.
Analysis is local and performs no network lookup. Consequently, it cannot check
citations, current facts, provenance, licensing, or similarity to external
sources.

View File

@ -14,9 +14,9 @@ archive reading stream one result at a time, so callers do not need to retain an
entire input tree in memory.
The current human-readable output contains the source identity, bounded evidence
confidence, and finding count. Findings currently cover configured non-ASCII
hyphen-like characters. More heuristic families and source-highlighted `miette`
diagnostics are planned.
confidence, and finding count. Findings cover the embedded or overridden
heuristic catalog and retain exact UTF-8 byte spans for later presentation.
Source-highlighted `miette` diagnostics are planned for the next milestone.
Exit statuses are stable:
@ -25,7 +25,9 @@ Exit statuses are stable:
- `2`: arguments are invalid or an operational/input error prevents completion.
Confidence is a summary of observable configured evidence. It is not an
AI-authorship probability.
AI-authorship probability. See [Heuristics and confidence](heuristics.md) for
the formula and [Limitations](limitations.md) before using thresholds in review
workflows.
Configuration layers can be selected with `--config-name NAME` and `--config
FILE`; `--max-document-bytes BYTES` is the current highest-precedence input

View File

@ -129,7 +129,7 @@ configuration is explainable, and all precedence/validation tests pass.
- [x] Build a reviewed evaluation corpus that is legally distributable and labels
observable low-care signals, not presumed author identity. Report per-rule
precision/recall and aggregate false-positive rates before tuning defaults.
- [ ] Document limitations, adversarial evasion, domain shift, and why the score
- [x] Document limitations, adversarial evasion, domain shift, and why the score
is not an AI-authorship probability.
Acceptance: every score decomposes into visible findings; no high-confidence