docs: split detailed guides from readme

This commit is contained in:
Dandaedre 2026-09-29 12:55:53 +02:00
parent 239b5198e4
commit 25560fe5d2
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
7 changed files with 205 additions and 76 deletions

View File

@ -4,90 +4,39 @@
![tests](badges/test.svg)
![test coverage](badges/coverage.svg)
John Slop is an early-stage, local-first heuristic linter for concrete signals
of low-effort or insufficiently reviewed text and source code. It is intended to
highlight evidence for human review, not to prove that content was written by AI
or assign an authorship probability.
John Slop is an early-stage, local-first linter for concrete signs of low-effort
or insufficiently reviewed text and source code. It reports explainable
heuristics for human review; it does not prove AI authorship or identify a
model.
## Current prototype
Today, the detector reads UTF-8 files, directories, ZIP archives, and common
ZIP-based office documents. Its first conservative rule family identifies
configured non-ASCII hyphen-like characters and exposes safe ASCII replacement
edits. Broader data-driven heuristics, terminal diagnostics, a formatter, and an
LSP are planned in [tasks.md](tasks.md).
The repository currently contains proof-of-concept code that:
## Quick start
- streams UTF-8 text from a file or recursively from a directory;
- streams text entries from ZIP-based files, including `.zip`, `.docx`, `.xlsx`,
and `.pptx`;
- rejects missing paths and invalid UTF-8 with a diagnostic error;
- detects a configured-in-code set of hyphen-like Unicode characters; and
- returns typed findings, edits, and bounded evidence confidence for those
characters.
The temporary command-line entry point accepts one file, directory, or supported
ZIP-based path and prints a summary per document. Its interface will be replaced
by the planned Clap-based detector binary during Milestone 1.
Cargo exposes explicit `john-slop`, `john-slop-fmt`, and `john-slop-lsp` binary
targets. The formatter and LSP currently exit with an explanatory unsupported
message until their dedicated implementation milestones.
Run the detector over one or more paths with:
Rust 1.98.1 or newer is required.
```sh
cargo run --bin john-slop -- [--fail-confidence 0..100] <PATH>...
cargo run --bin john-slop -- path/to/document path/to/source-directory
```
The detector exits `0` below the selected threshold, `1` when any document meets
it, and `2` for invalid arguments or operational failures. Confidence describes
configured evidence, not an AI-authorship probability.
Use `--fail-confidence 0..100` to make the detector exit with status `1` when a
document reaches a chosen evidence threshold. Confidence measures configured
signals, not the probability that AI wrote the content.
Reusable consumers should call the library facade: `load_config`,
`analyze_source`, `plan_replacements`, and `read_inputs`. Replacement planning is
pure and shares the exact resolved configuration used by analysis.
Input iteration yields a separate result per document, so a bad directory entry
does not discard successfully read siblings.
Reads enforce the resource limits in
[ADR 0002](docs/decisions/0002-input-policy-and-limits.md), including bounded
file/member sizes, archive expansion and compression ratios, entry counts,
directory depth, safe member paths, strict UTF-8, and NUL-byte rejection.
Library input failures are typed `InputError` variants. Executable boundaries
attach operation context and render them with `miette`; library consumers do not
need to parse presentation strings.
## Documentation
The implementation is being restructured into a reusable library plus detector,
formatter, and LSP binaries. See [tasks.md](tasks.md) for the ordered backlog and
[AGENTS.md](AGENTS.md) for the design and contribution rules.
Architecture and product choices are recorded under
[`docs/decisions`](docs/decisions/), beginning with the
[core analysis contracts](docs/decisions/0001-core-analysis-contracts.md).
The default traversal, text-recognition, and archive safety policy is specified
in [ADR 0002](docs/decisions/0002-input-policy-and-limits.md).
- [Documentation index](docs/README.md)
- [Command-line usage and current capabilities](docs/usage.md)
- [Architecture and public library API](docs/architecture.md)
- [Development, tests, features, and coverage](docs/development.md)
- [Design decision records](docs/decisions/)
- [Background reading](docs/background.md)
## Development
Run the current test and quality suite with:
```sh
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets --all-features
cargo doc --no-deps
```
Behavior is intentionally captured with regression tests before the refactor so
that later changes are deliberate and reviewable.
Detector spans use zero-based UTF-8 byte offsets. Presentation layers will
convert those canonical offsets to terminal or LSP coordinates at their
boundaries.
The minimum supported Rust version (MSRV) is 1.88. CI checks both the MSRV and
the current stable toolchain against the committed dependency lockfile. Linux is
the currently exercised platform; support for other platforms will be declared
only after CI coverage is added.
The ordered implementation plan lives in [tasks.md](tasks.md), and contribution
rules live in [AGENTS.md](AGENTS.md).
John Slop is distributed under the [MIT license](LICENSE).
## Background reading
- [Wikipedia: Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia%3ASigns_of_AI_writing)
- [Wikipedia: WikiProject AI Cleanup](https://en.wikipedia.org/wiki/Wikipedia:WikiProject_AI_Cleanup)
- [The Verge: Wikipedia editors are battling AI slop](https://www.theverge.com/report/756810/wikipedia-ai-slop-policies-community-speedy-deletion)

19
docs/README.md Normal file
View File

@ -0,0 +1,19 @@
# John Slop documentation
The root [README](../README.md) is the short project overview. These guides hold
the details needed to use, embed, and develop John Slop:
- [Usage](usage.md) documents accepted inputs, detector output, exit statuses,
and the current formatter/LSP status.
- [Architecture](architecture.md) describes module boundaries, shared domain
values, input safety, and the reusable library facade.
- [Development](development.md) covers the supported Rust version, feature
matrix, snapshots, coverage, badges, and CI quality gates.
- [Background](background.md) collects external context on observable signs of
low-care AI-assisted writing.
- [Decision records](decisions/) preserve durable product and engineering
choices, including confidence semantics and input limits.
The future configuration, heuristic engine, formatter, and LSP contracts remain
specified in [tasks.md](../tasks.md) and [AGENTS.md](../AGENTS.md) until their
implementation milestones land.

44
docs/architecture.md Normal file
View File

@ -0,0 +1,44 @@
# Architecture
John Slop keeps detection and replacement behavior in a library. Executables are
thin adapters responsible for argument parsing, presentation, and process exit
statuses.
The current flow is:
```text
path -> input stream -> Document -> analysis -> Analysis + Finding
-> replacement planner -> TextEdit
```
The main modules have deliberately narrow responsibilities:
- `config` owns the validated embedded replacement catalog and input limits;
- `input` streams files, directory entries, and archive members as documents;
- `detector` produces explainable findings and bounded confidence;
- `replacement` produces non-overlapping edits without performing I/O;
- `model` contains presentation-neutral domain values;
- `error` exposes typed input failures for library consumers;
- `src/bin` contains the detector and the formatter/LSP adapters.
All source spans are zero-based UTF-8 byte ranges into the exact analyzed source
snapshot. Presentation layers will convert those canonical spans to terminal or
UTF-16 LSP coordinates at their boundaries. Archive sources retain both their
outer path and inner member identity.
## Public facade
The crate root exposes four entry points:
- `load_config` resolves the configuration embedded in the current build;
- `read_inputs` returns a streaming iterator of per-document results;
- `analyze_source` returns findings and evidence confidence for UTF-8 text;
- `plan_replacements` returns safe edits against the same source snapshot.
Configuration files and precedence layers are a later milestone. Keeping these
operations behind the facade lets those layers evolve without duplicating logic
across the three binaries.
See [ADR 0001](decisions/0001-core-analysis-contracts.md) for the foundational
contracts and [ADR 0002](decisions/0002-input-policy-and-limits.md) for traversal
and archive safety decisions.

13
docs/background.md Normal file
View File

@ -0,0 +1,13 @@
# Background reading
John Slop focuses on observable low-care signals and avoids making unsupported
authorship claims. These resources provide context for that product boundary and
the kinds of review problems the project aims to help surface:
- [Wikipedia: Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia%3ASigns_of_AI_writing)
- [Wikipedia: WikiProject AI Cleanup](https://en.wikipedia.org/wiki/Wikipedia:WikiProject_AI_Cleanup)
- [The Verge: Wikipedia editors are battling AI slop](https://www.theverge.com/report/756810/wikipedia-ai-slop-policies-community-speedy-deletion)
Project-specific decisions take precedence over external guidance. See
[ADR 0001](decisions/0001-core-analysis-contracts.md) for confidence and
authorship semantics.

60
docs/development.md Normal file
View File

@ -0,0 +1,60 @@
# Development
The project uses Rust edition 2024 and requires Rust 1.98.1 or newer. The
committed lockfile makes application builds reproducible. Linux is the currently
exercised platform; other platforms are not declared supported until CI covers
them.
## Quality gates
Run the same core checks required by the contributor guide:
```sh
cargo fmt --all -- --check
cargo clippy --locked --all-targets --all-features -- -D warnings
cargo test --locked --all-targets --all-features
cargo test --locked --all-targets --no-default-features
cargo doc --locked --no-deps --all-features
```
The default `fancy-diagnostics` feature enables `miette`'s rich terminal
renderer. The no-default-features test run ensures the shared library and all
binaries remain valid without that optional renderer. Focused unit and
integration tests cover domain invariants, input failures, archives, CLI exit
behavior, and feature configurations.
Insta snapshots are reserved for stable user-facing text such as CLI help and
usage errors. Structured domain values use direct assertions so failures remain
precise and easy to review. Accept snapshot changes only after inspecting the
generated diff.
## Coverage and badges
Run line coverage with Cargo Tarpaulin:
```sh
cargo tarpaulin --config tarpaulin.toml
```
The configuration enforces a 30% floor. The current suite measured 72.32%
(196/271 lines) on 2026-09-29. The low floor is intentional while CLI
subprocesses and future protocol adapters are not attributed consistently by
every backend; the tracked badge still reports the measured result.
After a clean test and coverage run, refresh the repository-local badges with:
```sh
rust_autobadges --high-coverage-percent 0.7 --mid-coverage-percent 0.3
```
Commit `badges/*.svg` and `.run_status.toml`; do not commit the generated
`tarpaulin-report.html`. Jenkins runs both feature configurations and enforces
the Tarpaulin floor before building release binaries.
## Test scope
Add behavior tests at the lowest useful layer and an integration test when a
change crosses a binary boundary. Important edge cases include malformed input,
invalid UTF-8, empty text, Unicode byte spans, deterministic ordering, archive
limits, and near misses for detection rules. The detailed expectations and
task-level commit discipline are in [AGENTS.md](../AGENTS.md).

44
docs/usage.md Normal file
View File

@ -0,0 +1,44 @@
# Usage
## Detector
Run the detector over one or more paths:
```sh
cargo run --bin john-slop -- [--fail-confidence 0..100] <PATH>...
```
Each path may be a UTF-8 text file, a directory, a ZIP archive, or a supported
ZIP-based office document (`.docx`, `.xlsx`, or `.pptx`). Directory traversal and
archive reading stream one result at a time, so callers do not need to retain an
entire input tree in memory.
The current human-readable output contains the source identity, bounded evidence
confidence, and finding count. Findings currently cover configured non-ASCII
hyphen-like characters. More heuristic families and source-highlighted `miette`
diagnostics are planned.
Exit statuses are stable:
- `0`: every analyzed document is below the requested confidence threshold;
- `1`: at least one document reaches the threshold;
- `2`: arguments are invalid or an operational/input error prevents completion.
Confidence is a summary of observable configured evidence. It is not an
AI-authorship probability.
## Supported input policy
Input reads use strict UTF-8 and reject NUL-containing text. They do not follow
symlinks by default. Limits on files, walk depth, archive entries, expanded
bytes, compression ratio, and member paths protect analysis from unbounded or
unsafe input. The exact policy is recorded in
[ADR 0002](decisions/0002-input-policy-and-limits.md).
## Formatter and LSP
Cargo already exposes `john-slop-fmt` and `john-slop-lsp` so packaging and CI
exercise their target names. Both are explicit scaffolds today and return an
unsupported diagnostic. Replacement planning is available through the library;
file-writing formatter behavior and the stdio language server arrive in later
milestones.

View File

@ -74,7 +74,7 @@ entry do not discard successful documents.
coverage, without snapshotting structured domain values unnecessarily.
- [x] Configure Tarpaulin with an enforceable coverage floor around 30%, run
`rust_autobadges`, and publish local version/test/coverage badges.
- [ ] Reduce the root README to a generic project landing page and move detailed
- [x] Reduce the root README to a generic project landing page and move detailed
usage, architecture, testing, coverage, and background links under `docs/`.
## Milestone 2 — Typed layered configuration