docs: publish formatter contract

This commit is contained in:
Dandaedre 2026-09-29 16:49:38 +02:00
parent 39c6eacdf3
commit 709ed232f4
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
9 changed files with 102 additions and 38 deletions

View File

@ -15,8 +15,9 @@ copied assistant scaffolding, generic style residue, source-code placeholders,
tutorial narration, and configured non-ASCII characters. Rules retain exact
source spans, apply path/content scopes, and feed a bounded, explainable
confidence score. The detector renders source-highlighted human diagnostics,
concise records, or versioned JSON Lines. A formatter and LSP are planned in
[tasks.md](tasks.md).
concise records, or versioned JSON Lines. The formatter applies only configured
Unicode-to-ASCII replacements, supports check/diff modes, and requires explicit
opt-in for atomic file writes. The LSP is planned in [tasks.md](tasks.md).
## Quick start
@ -24,12 +25,18 @@ Rust 1.98.1 or newer is required.
```sh
cargo run --bin john-slop -- path/to/document path/to/source-directory
cargo run --bin john-slop-fmt -- path/to/document
cargo run --bin john-slop-fmt -- --check path/to/document
```
Use `--fail-confidence 0..100` to make the detector exit with status `1` when a
document reaches a chosen evidence threshold. Confidence measures configured
signals, not the probability that AI wrote the content.
Formatter file mode prints transformed text without modifying the source. Use
`--write` explicitly for atomic in-place replacement, or pipe editor buffers
through `john-slop-fmt` over stdin/stdout.
## Documentation
- [Runnable Rust examples](examples/README.md)

View File

@ -8,7 +8,7 @@ the details needed to use, embed, and develop John Slop:
- [Usage](usage.md) documents accepted inputs, detector output, exit statuses,
and the current formatter/LSP status.
- [Editor setup](editors.md) provides Helix (`hx`) and Vim configuration for
the formatter and LSP adapter contracts, with current scaffold caveats.
the formatter and future LSP adapter, with current status caveats.
- [Output contracts](output.md) specifies human and concise presentation plus
the versioned JSON Lines schema and partial-error records.
- [Architecture](architecture.md) describes module boundaries, shared domain
@ -28,6 +28,6 @@ the details needed to use, embed, and develop John Slop:
- [Decision records](decisions/) preserve durable product and engineering
choices, including confidence semantics and input limits.
The future configuration, heuristic engine, formatter, and LSP contracts remain
specified in [tasks.md](../tasks.md) and [AGENTS.md](../AGENTS.md) until their
implementation milestones land.
Future LSP and release-hardening contracts remain specified in
[tasks.md](../tasks.md) and [AGENTS.md](../AGENTS.md) until their implementation
milestones land.

View File

@ -9,7 +9,8 @@ The current flow is:
```text
path -> input stream -> Document -> analysis -> Analysis + Finding
-> diagnostic/output adapters
-> replacement planner -> TextEdit
-> replacement planner -> TextEdit -> applier
-> formatter I/O
```
The main modules have deliberately narrow responsibilities:
@ -20,7 +21,8 @@ The main modules have deliberately narrow responsibilities:
explainable findings and bounded confidence;
- `diagnostic` binds findings to exact named snapshots as typed `miette`
diagnostics;
- `replacement` produces non-overlapping edits without performing I/O;
- `replacement` produces non-overlapping edits and validates/applies arbitrary
edit sets without performing I/O;
- `model` contains presentation-neutral domain values;
- `error` exposes typed input failures for library consumers;
- `src/bin` contains the detector presentation/exit policy and formatter/LSP
@ -40,7 +42,9 @@ The crate root exposes core entry points and presentation-neutral types:
- `read_inputs` returns a streaming iterator of per-document results;
- `read_stdin` returns one bounded strict-UTF-8 standard-input document;
- `analyze_source` returns findings and evidence confidence for UTF-8 text;
- `plan_replacements` returns safe edits against the same source snapshot.
- `plan_replacements` returns safe edits against the same source snapshot;
- `apply_replacements` validates boundaries and overlap, then applies edits in
descending byte order while borrowing no-op input unchanged.
`FindingDiagnostic` is the library bridge to `miette`; CLI-specific concise and
JSON rendering stays in the detector adapter so it cannot alter analysis.
@ -50,6 +54,11 @@ layers in order. Stable-ID merging and semantic validation happen before a
configuration reaches analysis. Keeping these operations behind the facade
avoids duplicating policy across the three binaries.
The formatter adapter owns filesystem mutation. It resolves configuration and
preflights every requested document before writing, then uses same-directory
temporary files and preserves permissions. Archive mutation remains outside the
library and adapter contract.
See [ADR 0001](decisions/0001-core-analysis-contracts.md) for the foundational
contracts and [ADR 0002](decisions/0002-input-policy-and-limits.md) for traversal
and archive safety decisions.

View File

@ -56,6 +56,12 @@ layer, invalid regular expressions or globs, unsafe matchers, non-ASCII
replacement targets, and invalid thresholds are fatal configuration
diagnostics.
Replacement `source` values are non-empty Unicode strings containing at least
one non-ASCII byte; `replacement` values are non-empty ASCII strings. Sources
must be unique. When enabled sources share a prefix, the longest matching source
wins at each byte offset, with stable rule ID ordering as the deterministic
tie-break. Edits always refer to the original UTF-8 snapshot.
Each heuristic also configures `severity` as `advice`, `warning`, or `error`.
Severity controls diagnostic presentation and summary counts; confidence still
comes from the independently configured weight and evidence formula.

View File

@ -1,10 +1,8 @@
# Editor setup
> **Current status:** these configurations are ready for the adapter contracts,
> but `john-slop-fmt` and `john-slop-lsp` are still explicit scaffolds. The
> formatter becomes operational in Milestone 5 and the language server in
> Milestone 6. Until then, both return an unsupported diagnostic when started
> without `--help`.
> **Current status:** `john-slop-fmt` is operational. `john-slop-lsp` remains an
> explicit Milestone 6 scaffold and returns an unsupported diagnostic when
> started without `--help`.
Install all three binaries somewhere on `PATH`, then confirm the editor-facing
help is available:
@ -37,7 +35,7 @@ auto-format = false
This complete custom language entry makes the repository's `example.txt`
fixtures useful without depending on a built-in plain-text grammar. Use
`:format` to run the formatter. Set `auto-format = true` only if save-time
replacement is wanted after Milestone 5.
replacement is wanted.
For an existing language, merge John Slop into that language's entry rather
than replacing its useful defaults. For example, Rust should retain
@ -96,10 +94,12 @@ current upstream documentation.
Open `examples/fixtures/typography/example.txt` in either editor. Its expected
formatter result is the adjacent `formatted.txt`; the shared manifest records
three replacements and one typography rule. The CLI is usable now for a
baseline:
three replacements and one typography rule. Compare detector and formatter
output with:
```sh
cargo run --bin john-slop -- --output concise \
examples/fixtures/typography/example.txt
cargo run --bin john-slop-fmt -- \
examples/fixtures/typography/example.txt
```

View File

@ -67,8 +67,11 @@ such decisions.
Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8,
encrypted archives, unsafe archive members, and configured resource-limit
violations are errors rather than clean results. Generated, vendor, dependency,
and minified paths are skipped by default. The formatter and LSP binaries remain
scaffolds for later milestones; only the detector currently presents findings.
and minified paths are skipped by default. The formatter handles strict UTF-8
files and configured ASCII replacements only; it does not rewrite ZIP-based
containers, prose, or source structure. Recursive writes require explicit
`--write --recursive` and preflight the complete input set. The LSP remains a
scaffold for Milestone 6.
Analysis is local and performs no network lookup. Consequently, it cannot check
citations, current facts, provenance, licensing, or similarity to external

View File

@ -72,8 +72,8 @@ planning. Run all example targets as part of the normal test matrix with
Its [shared fixture manifest](../examples/fixtures/manifest.toml) pairs several
`example.txt` inputs with expected detector rules, replacement counts, and
formatted output. The executable fixture test validates both the library and
detector CLI today; formatter and LSP integration tests will consume the same
corpus when those adapters are implemented.
the detector and formatter CLIs today; LSP integration tests will consume the
same corpus when that adapter is implemented.
## Supported input policy
@ -83,12 +83,46 @@ bytes, compression ratio, and member paths protect analysis from unbounded or
unsafe input. The exact policy is recorded in
[ADR 0002](decisions/0002-input-policy-and-limits.md).
## Formatter and LSP
## Formatter
Cargo already exposes `john-slop-fmt` and `john-slop-lsp` so packaging and CI
exercise their target names. Both are explicit scaffolds today and return an
unsupported diagnostic. Replacement planning is available through the library;
file-writing formatter behavior and the stdio language server arrive in later
milestones. Both scaffolds accept `--help` and show concise Helix and Vim setup;
the complete snippets and validation workflow live in
[Editor setup](editors.md).
With no path, `john-slop-fmt` reads one bounded strict-UTF-8 document from stdin
and writes only formatted bytes to stdout. With a file path, the default remains
read-only and prints the transformed file without changing it:
```sh
john-slop-fmt < article.txt > article.ascii.txt
john-slop-fmt article.txt
john-slop-fmt --check article.txt
john-slop-fmt --diff article.txt
john-slop-fmt --write article.txt
```
The modes are mutually exclusive:
- default mode emits formatted text and exits `0`;
- `--check` emits nothing and exits `1` when any replacement is needed;
- `--diff` emits a unified diff and exits `1` when any replacement is needed;
- `--write` emits nothing and atomically replaces changed files, exiting `0`;
- argument, configuration, UTF-8, input, or write failures exit `2`.
All modes use the same embedded/user/named/explicit configuration precedence as
the detector. `--config-name NAME` and `--config FILE` select those layers.
Replacement sources use deterministic longest-match semantics, and formatting
twice is byte-for-byte identical to formatting once.
Directories require `--recursive`, and recursive operation is deliberately
limited to `--check --recursive` or `--write --recursive`. Before a recursive
write, every input is read, decoded, planned, and edit-validated; a failure
prevents all writes. Changed files use a temporary file in the same directory,
copy original permissions, sync content, and replace by rename where the
platform supports it. Files needing no edit are never rewritten.
ZIP, DOCX, XLSX, and PPTX mutation is not supported. Extract and format an
individual UTF-8 file instead.
## LSP
Cargo exposes `john-slop-lsp` so packaging and CI exercise its target name, but
it remains an explicit Milestone 6 scaffold. Both adapter `--help` pages show
concise Helix and Vim setup; the complete snippets and validation workflow live
in [Editor setup](editors.md).

View File

@ -1,6 +1,6 @@
//! Shared editor-integration help for the formatter and LSP adapters.
pub const EDITOR_SETUP: &str = r#"EDITOR SETUP (fmt: Milestone 5; LSP: Milestone 6):
pub const EDITOR_SETUP: &str = r#"EDITOR SETUP (fmt available; LSP planned for Milestone 6):
Helix ~/.config/helix/languages.toml:
[language-server.john-slop]
command = "john-slop-lsp"

View File

@ -159,27 +159,32 @@ and remediation help; CI can consume stable machine output and exit statuses.
## Milestone 5 — Shared ASCII replacement engine and formatter
- [ ] Parse replacement definitions into a trie or deterministic longest-match
- [x] Parse replacement definitions into a trie or deterministic longest-match
planner and emit validated, non-overlapping `TextEdit`s against the original
source.
- [ ] Apply edits from highest to lowest byte offset. Preserve line endings and
- [x] Apply edits from highest to lowest byte offset. Preserve line endings and
return the original buffer unchanged when there are no edits.
- [ ] Reject non-character-boundary and overlapping edits as typed invariant
- [x] Reject non-character-boundary and overlapping edits as typed invariant
errors. Define deterministic behavior when sources share prefixes.
- [ ] Implement `john-slop-fmt` stdin/stdout mode, `--check`, `--diff`, and
- [x] Implement `john-slop-fmt` stdin/stdout mode, `--check`, `--diff`, and
explicit `--write`. Require an explicit policy for recursive directory writes.
- [ ] Make writes atomic with a same-directory temporary file plus rename when
- [x] Make writes atomic with a same-directory temporary file plus rename when
supported; retain permissions and never rewrite on analysis/config failure.
- [ ] Do not rewrite ZIP/DOCX/XLSX/PPTX members in this milestone; emit a clear
- [x] Do not rewrite ZIP/DOCX/XLSX/PPTX members in this milestone; emit a clear
unsupported-operation diagnostic.
- [ ] Add unit/property tests for idempotence, Unicode boundaries, longest match,
- [x] Add unit/property tests for idempotence, Unicode boundaries, longest match,
non-overlap, stable output, line endings, empty input, and no-op byte identity.
- [ ] Add integration tests proving `--check` exit status, stdout purity, atomic
- [x] Add integration tests proving `--check` exit status, stdout purity, atomic
failure behavior, config overrides, and formatter/detector agreement.
Acceptance: formatting twice equals formatting once; default invocation does not
modify files; every replacement comes from the resolved typed config.
Decision: `ReplacementRule::source` is a non-empty Unicode `String` rather than
a single `char`, enabling deterministic longest-match behavior for shared
prefixes. This is a deliberate public data-contract change made before the
first formatter milestone tag.
## Milestone 6 — LSP binary
- [ ] Select an actively maintained LSP framework after a short dependency/API