docs: define detector limitations
This commit is contained in:
parent
f1fe9f5d8d
commit
283a0542b9
11
README.md
11
README.md
|
|
@ -10,9 +10,11 @@ heuristics for human review; it does not prove AI authorship or identify a
|
|||
model.
|
||||
|
||||
Today, the detector reads UTF-8 files, directories, ZIP archives, and common
|
||||
ZIP-based office documents. Its first conservative rule family identifies
|
||||
configured non-ASCII hyphen-like characters and exposes safe ASCII replacement
|
||||
edits. Broader data-driven heuristics, terminal diagnostics, a formatter, and an
|
||||
ZIP-based office documents. Eleven conservative, configurable rules cover
|
||||
copied assistant scaffolding, generic style residue, source-code placeholders,
|
||||
tutorial narration, and configured non-ASCII characters. Rules retain exact
|
||||
source spans, apply path/content scopes, and feed a bounded, explainable
|
||||
confidence score. Source-highlighted terminal diagnostics, a formatter, and an
|
||||
LSP are planned in [tasks.md](tasks.md).
|
||||
|
||||
## Quick start
|
||||
|
|
@ -33,6 +35,9 @@ signals, not the probability that AI wrote the content.
|
|||
- [Command-line usage and current capabilities](docs/usage.md)
|
||||
- [Architecture and public library API](docs/architecture.md)
|
||||
- [Layered configuration](docs/configuration.md)
|
||||
- [Heuristics, confidence, and applicability](docs/heuristics.md)
|
||||
- [Evaluation corpus and current metrics](docs/evaluation.md)
|
||||
- [Limitations and responsible interpretation](docs/limitations.md)
|
||||
- [Development, tests, features, and coverage](docs/development.md)
|
||||
- [Design decision records](docs/decisions/)
|
||||
- [Background reading](docs/background.md)
|
||||
|
|
|
|||
|
|
@ -13,6 +13,8 @@ the details needed to use, embed, and develop John Slop:
|
|||
applicability, evidence damping, normalization, and score interpretation.
|
||||
- [Evaluation corpus](evaluation.md) records fixture provenance, per-rule
|
||||
precision/recall, aggregate false positives, and benchmark caveats.
|
||||
- [Limitations](limitations.md) covers false positives, evasion, domain shift,
|
||||
score semantics, evaluation limits, and operational boundaries.
|
||||
- [Development](development.md) covers the supported Rust version, feature
|
||||
matrix, snapshots, coverage, badges, and CI quality gates.
|
||||
- [Background](background.md) collects external context on observable signs of
|
||||
|
|
|
|||
|
|
@ -0,0 +1,76 @@
|
|||
# Limitations
|
||||
|
||||
John Slop is a review aid for observable low-care signals. It is not an
|
||||
AI-authorship detector, plagiarism detector, fact checker, security scanner, or
|
||||
quality oracle. A finding says only that configured text matched at a particular
|
||||
source span. Human context remains decisive.
|
||||
|
||||
## False positives
|
||||
|
||||
Legitimate material can contain every shipped signal. Documentation may quote a
|
||||
chat transcript, an article may discuss knowledge cutoffs, a tutorial may use
|
||||
step-by-step comments deliberately, and a tracked source-code TODO may be valid.
|
||||
Unicode punctuation can be correct typography even when a downstream format
|
||||
requires ASCII. Generic transitions are weak style evidence, not defects by
|
||||
themselves.
|
||||
|
||||
Use path scopes, rule disabling, and project-specific weights to reflect local
|
||||
policy. Do not automatically reject, shame, or attribute a document based on a
|
||||
score. Review the visible findings and original context.
|
||||
|
||||
## False negatives and evasion
|
||||
|
||||
Exact and regular-expression rules are inherently easy to evade through
|
||||
paraphrasing, case changes, spelling variation, inserted characters, or removal
|
||||
of obvious scaffolding. Careless work need not contain any known signal, while
|
||||
well-reviewed AI-assisted work may contain none. John Slop deliberately avoids
|
||||
opaque statistical authorship guesses, so a clean result is not evidence of
|
||||
human authorship or high quality.
|
||||
|
||||
Adversarial inputs can also target applicability assumptions: misleading file
|
||||
extensions, unusual generated-directory names, or stdin without a filename may
|
||||
select a different scope. Archive analysis uses the inner member path but does
|
||||
not infer semantic language from content.
|
||||
|
||||
## Domain and language shift
|
||||
|
||||
The starter prose rules primarily encode English phrases. Other languages,
|
||||
specialized editorial styles, historical corpora, transcripts, generated API
|
||||
documentation, notebooks, and uncommon programming languages are
|
||||
underrepresented. File-extension inference covers a conservative common set and
|
||||
defaults unknown paths to prose. Teams should validate rules against their own
|
||||
reviewed material before changing enforcement thresholds.
|
||||
|
||||
## Confidence is not probability
|
||||
|
||||
The score is a deterministic compression of configured weights, occurrence
|
||||
caps, and category damping. Those values are policy choices and have not been
|
||||
calibrated against population-level authorship data. A score of 75 does not mean
|
||||
there is a 75% probability of AI authorship, low care, or an error. Thresholds
|
||||
name review bands only, and changing the catalog changes their meaning.
|
||||
|
||||
Every score is traceable to findings, but that explainability does not make the
|
||||
underlying heuristic universally valid. Related signals can remain correlated
|
||||
even after category damping.
|
||||
|
||||
## Evaluation limits
|
||||
|
||||
The checked-in evaluation corpus is small, purpose-built, and intended to catch
|
||||
behavioral regressions. Its current perfect precision and recall reflect fixture
|
||||
construction, not real-world accuracy. It contains only one clean document and
|
||||
must not justify deployment claims or default-weight tuning. Representative,
|
||||
licensed, domain-specific samples and independent human review are required for
|
||||
such decisions.
|
||||
|
||||
## Input and operational boundaries
|
||||
|
||||
Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8,
|
||||
encrypted archives, unsafe archive members, and configured resource-limit
|
||||
violations are errors rather than clean results. Generated, vendor, dependency,
|
||||
and minified paths are skipped by default. The current command-line summary does
|
||||
not yet render source-highlighted diagnostics; formatter and LSP binaries remain
|
||||
scaffolds for later milestones.
|
||||
|
||||
Analysis is local and performs no network lookup. Consequently, it cannot check
|
||||
citations, current facts, provenance, licensing, or similarity to external
|
||||
sources.
|
||||
|
|
@ -14,9 +14,9 @@ archive reading stream one result at a time, so callers do not need to retain an
|
|||
entire input tree in memory.
|
||||
|
||||
The current human-readable output contains the source identity, bounded evidence
|
||||
confidence, and finding count. Findings currently cover configured non-ASCII
|
||||
hyphen-like characters. More heuristic families and source-highlighted `miette`
|
||||
diagnostics are planned.
|
||||
confidence, and finding count. Findings cover the embedded or overridden
|
||||
heuristic catalog and retain exact UTF-8 byte spans for later presentation.
|
||||
Source-highlighted `miette` diagnostics are planned for the next milestone.
|
||||
|
||||
Exit statuses are stable:
|
||||
|
||||
|
|
@ -25,7 +25,9 @@ Exit statuses are stable:
|
|||
- `2`: arguments are invalid or an operational/input error prevents completion.
|
||||
|
||||
Confidence is a summary of observable configured evidence. It is not an
|
||||
AI-authorship probability.
|
||||
AI-authorship probability. See [Heuristics and confidence](heuristics.md) for
|
||||
the formula and [Limitations](limitations.md) before using thresholds in review
|
||||
workflows.
|
||||
|
||||
Configuration layers can be selected with `--config-name NAME` and `--config
|
||||
FILE`; `--max-document-bytes BYTES` is the current highest-precedence input
|
||||
|
|
|
|||
2
tasks.md
2
tasks.md
|
|
@ -129,7 +129,7 @@ configuration is explainable, and all precedence/validation tests pass.
|
|||
- [x] Build a reviewed evaluation corpus that is legally distributable and labels
|
||||
observable low-care signals, not presumed author identity. Report per-rule
|
||||
precision/recall and aggregate false-positive rates before tuning defaults.
|
||||
- [ ] Document limitations, adversarial evasion, domain shift, and why the score
|
||||
- [x] Document limitations, adversarial evasion, domain shift, and why the score
|
||||
is not an AI-authorship probability.
|
||||
|
||||
Acceptance: every score decomposes into visible findings; no high-confidence
|
||||
|
|
|
|||
Loading…
Reference in New Issue