57 lines
3.1 KiB
Markdown
57 lines
3.1 KiB
Markdown
# Heuristics and evidence scoring
|
||
|
||
John Slop reports observable low-care signals. It does not classify authorship.
|
||
Every contribution to the score is represented by a finding with an exact
|
||
source span, stable rule ID, category, message, help, and configured weight.
|
||
|
||
The detector compiles the resolved rule catalog once. Exact phrases, regular
|
||
expressions, repeated-pattern rules, configured Unicode replacements, UTS #39
|
||
ASCII lookalikes, and invisible/control characters share the same finding
|
||
model. Content kind, extension, path globs, and conventional generated/vendor/
|
||
minified paths decide whether each rule applies. Findings are deduplicated by
|
||
rule and byte span, then sorted by start offset, end offset, and rule ID so
|
||
every adapter observes the same order.
|
||
|
||
Unicode review rules are deliberately distinct from replacement rules. The
|
||
detector can flag a lookalike, invisible scalar, or direction-changing control
|
||
at its exact span, but only the explicit ten-entry safe replacement allowlist
|
||
can produce a formatter edit or LSP quick fix. A safe replacement finding is
|
||
not duplicated as a confusable finding.
|
||
|
||
## Evidence-score formula
|
||
|
||
For each rule in deterministic catalog order, only the configured
|
||
`occurrence_cap` contributes to the evidence score. Its first occurrence receives the
|
||
full weight, then later occurrences receive one half, one quarter, and so on.
|
||
Findings beyond the cap remain visible but do not increase the score.
|
||
|
||
Rules are grouped by category as correlated evidence families. The first
|
||
matching rule in a category receives its full occurrence-adjusted contribution;
|
||
later distinct prompt-leakage rules receive successive factors of `0.90`, while
|
||
style, typography, and source-code rules receive successive factors of `0.75`.
|
||
Prompt/response markers and copied assistant disclaimers are stronger concrete
|
||
artifacts than generic stylistic similarities, but the remaining damping still
|
||
prevents a cluster of related phrases from being treated as fully independent
|
||
proof.
|
||
|
||
If the resulting evidence sum is `E`, the displayed score is
|
||
`round(100 × E / (E + 50))`, bounded to `0..=100`. The configured `low`,
|
||
`medium`, and `high` thresholds name the resulting band. Evidence below `low`
|
||
is explicitly reported as below threshold. Empty input and text with no
|
||
findings produce score zero and no evidence-band verdict. Binary, invalid UTF-8,
|
||
oversized, and unsupported inputs remain typed input errors; they are never
|
||
reported as clean analyses.
|
||
|
||
Every `Analysis` retains its complete finding list and can produce a compact
|
||
summary containing the numeric score, optional named band, and separate advice,
|
||
warning, and error counts. Summary generation never filters the evidence used
|
||
to explain the score.
|
||
|
||
Weights and thresholds are review policy, not calibrated probabilities. A
|
||
score of 75 does not mean a 75% chance that AI wrote the input.
|
||
|
||
The diagnostic showcase exercises five distinct prompt-leakage rules, including
|
||
two prompt/response markers. Together with its style and typography findings it
|
||
produces a low-care evidence score of 71 (medium). This is a deterministic policy score, not
|
||
an estimated probability.
|