john-slop/AGENTS.md

13 KiB

John Slop contributor guide

Mission and product boundaries

John Slop is a transparent, local-first heuristic linter for low-effort or insufficiently reviewed text and source code. It reports concrete signals that are often associated with careless AI-assisted work. It must not claim to prove that a person used AI, identify a model, or infer authorship from writing style.

Treat the low-care evidence score as the bounded strength of configured signals, not as the probability that content is AI-generated. User-facing language must say "suspicious", "low-care signal", or "needs review" rather than making an accusation. A result must always be explainable from the findings that produced it.

The formatter scope is deliberately narrow: configured Unicode characters with ASCII replacements. The generated UTS #39 profile is broad and therefore must retain replace, non-mutating review, and off policies. The LSP publishes every applicable detector finding but exposes edits only under replacement policy, reusing the same library implementations as the CLI and formatter. Do not add prose rewriting, generative inference, or destructive archive editing without a separate design decision.

Required architecture

Keep domain logic in a library and binaries as thin adapters. The intended layout is:

src/
  lib.rs                     compatibility facade over workspace libraries
  bin/
    john-slop.rs             command-line detector
    john-slop-fmt.rs         replacement formatter
    john-slop-lsp.rs         stdio language server
crates/
  john-slop-core/
    assets/                  embedded replacement and heuristic catalogs
    src/                     config, input, detection, diagnostics, and edits
  john-slop-lsp/src/         protocol state, UTF-16 mapping, and code actions
tests/
  fixtures/                  small, reviewed input/output fixtures

Names can evolve when Rust conventions demand it, but preserve these boundaries:

  • input acquires text and never scores or changes it.
  • detection is pure over text plus validated rules.
  • replacement returns non-overlapping edits and does no I/O.
  • diagnostic renders findings but does not discover them.
  • the CLI, formatter, and LSP call the same config, detector, and replacement APIs.
  • filesystem writes are confined to explicit adapter methods.

Do not retain the current all-files-in-memory iterator design as the core API. Prefer a streaming iterator of per-file results so one unreadable or binary file does not erase findings from unrelated files. Preserve the outer archive path and inner member name as separate fields. Reject unsafe archive paths and apply size, entry-count, and nesting limits before extraction or decompression.

Core data contracts

Use typed, presentation-neutral domain values. The exact names may change, but the model must express at least:

struct Finding {
    rule_id: RuleId,
    category: Category,
    message: String,
    help: Option<String>,
    byte_span: Range<usize>,
    matched_text: String,
    weight: f64,
    fix: Option<TextEdit>,
}

struct Analysis {
    findings: Vec<Finding>,
    evidence_score: EvidenceScore,
}

struct TextEdit {
    byte_span: Range<usize>,
    replacement: String,
}

All internal spans are zero-based UTF-8 byte ranges into the exact source snapshot that was analyzed. Validate boundaries before slicing. Convert spans at the edge: miette::SourceSpan for terminal output and UTF-16 line/column ranges for LSP. Keep an indexed line map per document so LSP conversions are tested and not repeatedly O(n).

Rule IDs are stable, unique, lowercase kebab-case identifiers. Findings are sorted by source identity, span start, then rule ID so output is deterministic. Never use a hash-map iteration order as output order.

Heuristics and evidence scoring

Store data-driven heuristic definitions in crates/john-slop-core/assets/heuristics.toml, separately from replacement configuration. Compile and validate rules once at startup. Invalid regular expressions, duplicate IDs, negative/non-finite weights, invalid thresholds, and unsafe empty matches are configuration errors—not ignored rules.

Each rule needs an ID, category, rationale/message, help text, matcher type, weight, and optional applicability filters such as prose/source-code extensions. Start with high-precision, independently testable signals:

  • copied assistant disclaimers or knowledge-cutoff phrases;
  • prompt/response scaffolding accidentally left in a document;
  • templated filler and repeated conclusion/transition phrases;
  • suspicious Unicode/confusable punctuation when the configured corpus expects ASCII;
  • source-code placeholders, tutorial residue, fabricated references, or comments that merely narrate obvious code, using conservative language-specific rules.

Avoid weak demographic, dialect, spelling, grammar, vocabulary, or English-fluency proxies. Repetition alone must not trigger a high evidence-band verdict. Give users category and per-rule enablement controls, per-rule weight overrides, and path/language scopes. A rule is active only when both its category gate and individual enabled field are active.

Scoring must be documented and deterministic. Use bounded weighted evidence, apply a per-rule occurrence cap, and damp correlated findings from the same rule or family. Map the aggregate onto 0..=100 and named bands such as low, medium, and high; keep thresholds configurable. Tests must pin the formula, caps, threshold edges, and empty-input result. Never label the numeric result an accuracy or probability unless it has been independently calibrated against a published, representative corpus.

Every terminal finding must become a miette::Diagnostic with named source, precise label, stable diagnostic code (the rule ID), severity, message, and help. A summary score supplements findings; it never replaces their evidence. Machine output must offer a versioned JSON schema and must not include ANSI escapes.

Configuration contract

Use figment with its TOML support and Serde for typed layered configuration. Its providers allow embedded serialized defaults and later TOML providers to override individual values while retaining source metadata for useful errors. Use directories (or an equivalent platform-aware crate) to resolve the user configuration directory; do not hand-build it from HOME.

The precedence from lowest to highest is:

  1. embedded crates/john-slop-core/assets/config.toml and crates/john-slop-core/assets/heuristics.toml;
  2. runtime diagnostic catalogs: the optional user-wide diagnostics.toml, then explicitly selected catalogs in command-line order;
  3. user base config at $XDG_CONFIG_HOME/john-slop/config.toml, conventionally ~/.config/john-slop/config.toml on Linux;
  4. named user config at $XDG_CONFIG_HOME/john-slop/<config_name>.toml when a validated config name is selected;
  5. an explicit --config <path> file;
  6. explicit CLI flags for the current invocation.

Diagnostic catalog files may contain only schema_version and [[rules]]. They can add or patch data-driven rules but cannot define executable matcher algorithms, categories, replacements, thresholds, or input policy. Normal configuration layers intentionally load afterward so they can disable or tune catalog rules by stable ID.

Missing optional files are allowed; malformed files are fatal and identify their origin. A config name accepts only ASCII letters, digits, _, and -; it cannot contain separators, . segments, or an extension. Decide array behavior per field: replacement/rule tables merge by stable ID, while ignore globs and file extensions replace as complete arrays unless the schema explicitly says otherwise. Do not rely on a library's default array-merge semantics implicitly.

crates/john-slop-core/assets/unicode.toml is the canonical generated Unicode replacement and diagnostic catalog; config.toml owns other defaults. Each replacement entry has an ID, Unicode source string/code point, ASCII target, rationale, and enabled flag. Reject duplicate sources, empty sources, non-ASCII targets, and conflicting edits. Longer sources win matching precedence. Provide a command that prints the resolved configuration with secrets/redacted fields if secrets are ever introduced.

Configuration schema changes require a schema_version, migration notes, and tests for precedence. Embed assets with include_str! so installed binaries do not depend on the source tree or current working directory.

CLI, formatter, and LSP behavior

Use explicit Cargo binaries named john-slop, john-slop-fmt, and john-slop-lsp.

  • The detector reads files/directories/stdin, reports findings, and exits 0 below the selected threshold, 1 when findings cross it, and 2 for usage or operational/configuration errors.
  • The formatter supports stdin/stdout, --check, and an explicit --write. Default behavior must not rewrite a file. Writes must be atomic where the platform permits, preserve original bytes when no edit applies, preserve line endings, and be idempotent.
  • The LSP communicates only over stdin/stdout; logs go to stderr. It publishes applicable configured heuristic diagnostics and exposes quick-fix/code-action edits only for safe replacements. It must honor document versions, clear diagnostics on close, handle invalid URIs without panicking, and never write editor-owned files.

Replacement edits are generated from the original snapshot, sorted, checked for overlap, then applied from the end of the buffer toward the beginning. The formatter and LSP must produce byte-for-byte equivalent results for the same source and resolved config.

Errors, safety, and observability

  • Libraries return typed errors with thiserror; application boundaries attach path/rule context and render with miette.
  • Do not use unwrap, expect, panic!, todo!, or unreachable! on user-controlled input. An expect is acceptable only for an invariant proved during construction and explained in a comment.
  • Treat invalid UTF-8 and binary content as explicit skipped-input diagnostics according to policy; do not silently truncate or lossily rewrite it.
  • Do not follow symlinks by default. Respect ignore configuration and establish file-size, archive-size, archive-entry, and recursion limits.
  • Do not make network calls during analysis. Results should be reproducible and private by default.
  • Structured logs go to stderr and are disabled or quiet by default. Never mix logs with JSON, formatter stdout, or the LSP protocol stream.

Testing and quality gates

Each behavior change requires tests at the lowest useful layer plus an integration test when it crosses a binary boundary. Include positive, near-miss/negative, Unicode, empty, large-input, malformed-config, and ordering cases. Regression fixtures must state why they should or should not match.

Before considering a task complete, run:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets --all-features
cargo test --all-targets --no-default-features
cargo doc --no-deps

Run cargo tarpaulin --config tarpaulin.toml when tests or executable Rust code change. It enforces the 90% line-coverage floor. Jenkins renders the version, test, and coverage badges from package metadata and CI results; generated badge files and status snapshots do not belong in the repository.

Also run focused CLI golden tests and LSP protocol tests once those binaries exist. Favor property tests for replacement idempotence/non-overlap and fuzz targets for config parsing, span conversion, archives, and replacement planning. Benchmarks should cover a representative large document and directory; optimize only from measured results.

Change discipline

Keep commits and patches small enough to review. Do not combine a structural move with new detection behavior unless the new behavior requires it. Preserve public behavior during refactors or document the deliberate break. Update sample configuration, schema/version notes, CLI help, and tests together.

When implementing tasks.md, work in dependency order, mark only verified items complete, and add newly discovered work under the relevant milestone. Record a short decision note in the task file when changing configuration precedence, the score formula, a public data contract, or safety limits.

After completing and verifying each individual task, create a dedicated commit before starting the next task. Stage only the files that belong to that task; never include unrelated or pre-existing user changes. Use a concise conventional commit message that describes the completed behavior. If a task cannot be verified or committed, leave it unchecked and report the blocker instead of combining it with later work.

After every milestone's acceptance criteria and quality gates pass, create an annotated Git tag named v0.1.N, where N is the milestone number, on its completion commit. Before that completion commit, set the workspace package version and internal registry dependency requirements to 0.1.N, refresh Cargo.lock and cut the matching changelog entry. Jenkins derives the version badge from that metadata. Verify the manifests, lockfile, changelog, and binary --version output agree before tagging. Never move an existing milestone tag; add a corrective commit and a documented successor tag if a published milestone needs amendment.