john-slop/docs/heuristics.md

1.7 KiB
Raw Blame History

Heuristics and confidence

John Slop reports observable low-care signals. It does not classify authorship. Every contribution to the score is represented by a finding with an exact source span, stable rule ID, category, message, help, and configured weight.

The detector compiles the resolved rule catalog once. Exact phrases, regular expressions, repeated-pattern rules, and configured Unicode replacements share the same finding model. Content kind, extension, path globs, and conventional generated/vendor/minified paths decide whether each rule applies.

Confidence formula

For each rule in deterministic catalog order, only the configured occurrence_cap contributes to confidence. Its first occurrence receives the full weight, then later occurrences receive one half, one quarter, and so on. Findings beyond the cap remain visible but do not increase the score.

Rules are grouped by category as correlated evidence families. The first matching rule in a category receives its full occurrence-adjusted contribution; later distinct rules receive successive factors of 0.75. This prevents a cluster of closely related phrases from being treated as independent proof.

If the resulting evidence sum is E, the displayed score is round(100 × E / (E + 50)), bounded to 0..=100. The configured low, medium, and high thresholds name the resulting band. Evidence below low is explicitly reported as below threshold. Empty input and text with no findings produce score zero and no confidence verdict. Binary, invalid UTF-8, oversized, and unsupported inputs remain typed input errors; they are never reported as clean analyses.

Weights and thresholds are review policy, not calibrated probabilities. A score of 75 does not mean a 75% chance that AI wrote the input.