docs: publish formatter contract
This commit is contained in:
parent
39c6eacdf3
commit
709ed232f4
11
README.md
11
README.md
|
|
@ -15,8 +15,9 @@ copied assistant scaffolding, generic style residue, source-code placeholders,
|
|||
tutorial narration, and configured non-ASCII characters. Rules retain exact
|
||||
source spans, apply path/content scopes, and feed a bounded, explainable
|
||||
confidence score. The detector renders source-highlighted human diagnostics,
|
||||
concise records, or versioned JSON Lines. A formatter and LSP are planned in
|
||||
[tasks.md](tasks.md).
|
||||
concise records, or versioned JSON Lines. The formatter applies only configured
|
||||
Unicode-to-ASCII replacements, supports check/diff modes, and requires explicit
|
||||
opt-in for atomic file writes. The LSP is planned in [tasks.md](tasks.md).
|
||||
|
||||
## Quick start
|
||||
|
||||
|
|
@ -24,12 +25,18 @@ Rust 1.98.1 or newer is required.
|
|||
|
||||
```sh
|
||||
cargo run --bin john-slop -- path/to/document path/to/source-directory
|
||||
cargo run --bin john-slop-fmt -- path/to/document
|
||||
cargo run --bin john-slop-fmt -- --check path/to/document
|
||||
```
|
||||
|
||||
Use `--fail-confidence 0..100` to make the detector exit with status `1` when a
|
||||
document reaches a chosen evidence threshold. Confidence measures configured
|
||||
signals, not the probability that AI wrote the content.
|
||||
|
||||
Formatter file mode prints transformed text without modifying the source. Use
|
||||
`--write` explicitly for atomic in-place replacement, or pipe editor buffers
|
||||
through `john-slop-fmt` over stdin/stdout.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Runnable Rust examples](examples/README.md)
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ the details needed to use, embed, and develop John Slop:
|
|||
- [Usage](usage.md) documents accepted inputs, detector output, exit statuses,
|
||||
and the current formatter/LSP status.
|
||||
- [Editor setup](editors.md) provides Helix (`hx`) and Vim configuration for
|
||||
the formatter and LSP adapter contracts, with current scaffold caveats.
|
||||
the formatter and future LSP adapter, with current status caveats.
|
||||
- [Output contracts](output.md) specifies human and concise presentation plus
|
||||
the versioned JSON Lines schema and partial-error records.
|
||||
- [Architecture](architecture.md) describes module boundaries, shared domain
|
||||
|
|
@ -28,6 +28,6 @@ the details needed to use, embed, and develop John Slop:
|
|||
- [Decision records](decisions/) preserve durable product and engineering
|
||||
choices, including confidence semantics and input limits.
|
||||
|
||||
The future configuration, heuristic engine, formatter, and LSP contracts remain
|
||||
specified in [tasks.md](../tasks.md) and [AGENTS.md](../AGENTS.md) until their
|
||||
implementation milestones land.
|
||||
Future LSP and release-hardening contracts remain specified in
|
||||
[tasks.md](../tasks.md) and [AGENTS.md](../AGENTS.md) until their implementation
|
||||
milestones land.
|
||||
|
|
|
|||
|
|
@ -9,7 +9,8 @@ The current flow is:
|
|||
```text
|
||||
path -> input stream -> Document -> analysis -> Analysis + Finding
|
||||
-> diagnostic/output adapters
|
||||
-> replacement planner -> TextEdit
|
||||
-> replacement planner -> TextEdit -> applier
|
||||
-> formatter I/O
|
||||
```
|
||||
|
||||
The main modules have deliberately narrow responsibilities:
|
||||
|
|
@ -20,7 +21,8 @@ The main modules have deliberately narrow responsibilities:
|
|||
explainable findings and bounded confidence;
|
||||
- `diagnostic` binds findings to exact named snapshots as typed `miette`
|
||||
diagnostics;
|
||||
- `replacement` produces non-overlapping edits without performing I/O;
|
||||
- `replacement` produces non-overlapping edits and validates/applies arbitrary
|
||||
edit sets without performing I/O;
|
||||
- `model` contains presentation-neutral domain values;
|
||||
- `error` exposes typed input failures for library consumers;
|
||||
- `src/bin` contains the detector presentation/exit policy and formatter/LSP
|
||||
|
|
@ -40,7 +42,9 @@ The crate root exposes core entry points and presentation-neutral types:
|
|||
- `read_inputs` returns a streaming iterator of per-document results;
|
||||
- `read_stdin` returns one bounded strict-UTF-8 standard-input document;
|
||||
- `analyze_source` returns findings and evidence confidence for UTF-8 text;
|
||||
- `plan_replacements` returns safe edits against the same source snapshot.
|
||||
- `plan_replacements` returns safe edits against the same source snapshot;
|
||||
- `apply_replacements` validates boundaries and overlap, then applies edits in
|
||||
descending byte order while borrowing no-op input unchanged.
|
||||
|
||||
`FindingDiagnostic` is the library bridge to `miette`; CLI-specific concise and
|
||||
JSON rendering stays in the detector adapter so it cannot alter analysis.
|
||||
|
|
@ -50,6 +54,11 @@ layers in order. Stable-ID merging and semantic validation happen before a
|
|||
configuration reaches analysis. Keeping these operations behind the facade
|
||||
avoids duplicating policy across the three binaries.
|
||||
|
||||
The formatter adapter owns filesystem mutation. It resolves configuration and
|
||||
preflights every requested document before writing, then uses same-directory
|
||||
temporary files and preserves permissions. Archive mutation remains outside the
|
||||
library and adapter contract.
|
||||
|
||||
See [ADR 0001](decisions/0001-core-analysis-contracts.md) for the foundational
|
||||
contracts and [ADR 0002](decisions/0002-input-policy-and-limits.md) for traversal
|
||||
and archive safety decisions.
|
||||
|
|
|
|||
|
|
@ -56,6 +56,12 @@ layer, invalid regular expressions or globs, unsafe matchers, non-ASCII
|
|||
replacement targets, and invalid thresholds are fatal configuration
|
||||
diagnostics.
|
||||
|
||||
Replacement `source` values are non-empty Unicode strings containing at least
|
||||
one non-ASCII byte; `replacement` values are non-empty ASCII strings. Sources
|
||||
must be unique. When enabled sources share a prefix, the longest matching source
|
||||
wins at each byte offset, with stable rule ID ordering as the deterministic
|
||||
tie-break. Edits always refer to the original UTF-8 snapshot.
|
||||
|
||||
Each heuristic also configures `severity` as `advice`, `warning`, or `error`.
|
||||
Severity controls diagnostic presentation and summary counts; confidence still
|
||||
comes from the independently configured weight and evidence formula.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
# Editor setup
|
||||
|
||||
> **Current status:** these configurations are ready for the adapter contracts,
|
||||
> but `john-slop-fmt` and `john-slop-lsp` are still explicit scaffolds. The
|
||||
> formatter becomes operational in Milestone 5 and the language server in
|
||||
> Milestone 6. Until then, both return an unsupported diagnostic when started
|
||||
> without `--help`.
|
||||
> **Current status:** `john-slop-fmt` is operational. `john-slop-lsp` remains an
|
||||
> explicit Milestone 6 scaffold and returns an unsupported diagnostic when
|
||||
> started without `--help`.
|
||||
|
||||
Install all three binaries somewhere on `PATH`, then confirm the editor-facing
|
||||
help is available:
|
||||
|
|
@ -37,7 +35,7 @@ auto-format = false
|
|||
This complete custom language entry makes the repository's `example.txt`
|
||||
fixtures useful without depending on a built-in plain-text grammar. Use
|
||||
`:format` to run the formatter. Set `auto-format = true` only if save-time
|
||||
replacement is wanted after Milestone 5.
|
||||
replacement is wanted.
|
||||
|
||||
For an existing language, merge John Slop into that language's entry rather
|
||||
than replacing its useful defaults. For example, Rust should retain
|
||||
|
|
@ -96,10 +94,12 @@ current upstream documentation.
|
|||
|
||||
Open `examples/fixtures/typography/example.txt` in either editor. Its expected
|
||||
formatter result is the adjacent `formatted.txt`; the shared manifest records
|
||||
three replacements and one typography rule. The CLI is usable now for a
|
||||
baseline:
|
||||
three replacements and one typography rule. Compare detector and formatter
|
||||
output with:
|
||||
|
||||
```sh
|
||||
cargo run --bin john-slop -- --output concise \
|
||||
examples/fixtures/typography/example.txt
|
||||
cargo run --bin john-slop-fmt -- \
|
||||
examples/fixtures/typography/example.txt
|
||||
```
|
||||
|
|
|
|||
|
|
@ -67,8 +67,11 @@ such decisions.
|
|||
Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8,
|
||||
encrypted archives, unsafe archive members, and configured resource-limit
|
||||
violations are errors rather than clean results. Generated, vendor, dependency,
|
||||
and minified paths are skipped by default. The formatter and LSP binaries remain
|
||||
scaffolds for later milestones; only the detector currently presents findings.
|
||||
and minified paths are skipped by default. The formatter handles strict UTF-8
|
||||
files and configured ASCII replacements only; it does not rewrite ZIP-based
|
||||
containers, prose, or source structure. Recursive writes require explicit
|
||||
`--write --recursive` and preflight the complete input set. The LSP remains a
|
||||
scaffold for Milestone 6.
|
||||
|
||||
Analysis is local and performs no network lookup. Consequently, it cannot check
|
||||
citations, current facts, provenance, licensing, or similarity to external
|
||||
|
|
|
|||
|
|
@ -72,8 +72,8 @@ planning. Run all example targets as part of the normal test matrix with
|
|||
Its [shared fixture manifest](../examples/fixtures/manifest.toml) pairs several
|
||||
`example.txt` inputs with expected detector rules, replacement counts, and
|
||||
formatted output. The executable fixture test validates both the library and
|
||||
detector CLI today; formatter and LSP integration tests will consume the same
|
||||
corpus when those adapters are implemented.
|
||||
the detector and formatter CLIs today; LSP integration tests will consume the
|
||||
same corpus when that adapter is implemented.
|
||||
|
||||
## Supported input policy
|
||||
|
||||
|
|
@ -83,12 +83,46 @@ bytes, compression ratio, and member paths protect analysis from unbounded or
|
|||
unsafe input. The exact policy is recorded in
|
||||
[ADR 0002](decisions/0002-input-policy-and-limits.md).
|
||||
|
||||
## Formatter and LSP
|
||||
## Formatter
|
||||
|
||||
Cargo already exposes `john-slop-fmt` and `john-slop-lsp` so packaging and CI
|
||||
exercise their target names. Both are explicit scaffolds today and return an
|
||||
unsupported diagnostic. Replacement planning is available through the library;
|
||||
file-writing formatter behavior and the stdio language server arrive in later
|
||||
milestones. Both scaffolds accept `--help` and show concise Helix and Vim setup;
|
||||
the complete snippets and validation workflow live in
|
||||
[Editor setup](editors.md).
|
||||
With no path, `john-slop-fmt` reads one bounded strict-UTF-8 document from stdin
|
||||
and writes only formatted bytes to stdout. With a file path, the default remains
|
||||
read-only and prints the transformed file without changing it:
|
||||
|
||||
```sh
|
||||
john-slop-fmt < article.txt > article.ascii.txt
|
||||
john-slop-fmt article.txt
|
||||
john-slop-fmt --check article.txt
|
||||
john-slop-fmt --diff article.txt
|
||||
john-slop-fmt --write article.txt
|
||||
```
|
||||
|
||||
The modes are mutually exclusive:
|
||||
|
||||
- default mode emits formatted text and exits `0`;
|
||||
- `--check` emits nothing and exits `1` when any replacement is needed;
|
||||
- `--diff` emits a unified diff and exits `1` when any replacement is needed;
|
||||
- `--write` emits nothing and atomically replaces changed files, exiting `0`;
|
||||
- argument, configuration, UTF-8, input, or write failures exit `2`.
|
||||
|
||||
All modes use the same embedded/user/named/explicit configuration precedence as
|
||||
the detector. `--config-name NAME` and `--config FILE` select those layers.
|
||||
Replacement sources use deterministic longest-match semantics, and formatting
|
||||
twice is byte-for-byte identical to formatting once.
|
||||
|
||||
Directories require `--recursive`, and recursive operation is deliberately
|
||||
limited to `--check --recursive` or `--write --recursive`. Before a recursive
|
||||
write, every input is read, decoded, planned, and edit-validated; a failure
|
||||
prevents all writes. Changed files use a temporary file in the same directory,
|
||||
copy original permissions, sync content, and replace by rename where the
|
||||
platform supports it. Files needing no edit are never rewritten.
|
||||
|
||||
ZIP, DOCX, XLSX, and PPTX mutation is not supported. Extract and format an
|
||||
individual UTF-8 file instead.
|
||||
|
||||
## LSP
|
||||
|
||||
Cargo exposes `john-slop-lsp` so packaging and CI exercise its target name, but
|
||||
it remains an explicit Milestone 6 scaffold. Both adapter `--help` pages show
|
||||
concise Helix and Vim setup; the complete snippets and validation workflow live
|
||||
in [Editor setup](editors.md).
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
//! Shared editor-integration help for the formatter and LSP adapters.
|
||||
|
||||
pub const EDITOR_SETUP: &str = r#"EDITOR SETUP (fmt: Milestone 5; LSP: Milestone 6):
|
||||
pub const EDITOR_SETUP: &str = r#"EDITOR SETUP (fmt available; LSP planned for Milestone 6):
|
||||
Helix ~/.config/helix/languages.toml:
|
||||
[language-server.john-slop]
|
||||
command = "john-slop-lsp"
|
||||
|
|
|
|||
21
tasks.md
21
tasks.md
|
|
@ -159,27 +159,32 @@ and remediation help; CI can consume stable machine output and exit statuses.
|
|||
|
||||
## Milestone 5 — Shared ASCII replacement engine and formatter
|
||||
|
||||
- [ ] Parse replacement definitions into a trie or deterministic longest-match
|
||||
- [x] Parse replacement definitions into a trie or deterministic longest-match
|
||||
planner and emit validated, non-overlapping `TextEdit`s against the original
|
||||
source.
|
||||
- [ ] Apply edits from highest to lowest byte offset. Preserve line endings and
|
||||
- [x] Apply edits from highest to lowest byte offset. Preserve line endings and
|
||||
return the original buffer unchanged when there are no edits.
|
||||
- [ ] Reject non-character-boundary and overlapping edits as typed invariant
|
||||
- [x] Reject non-character-boundary and overlapping edits as typed invariant
|
||||
errors. Define deterministic behavior when sources share prefixes.
|
||||
- [ ] Implement `john-slop-fmt` stdin/stdout mode, `--check`, `--diff`, and
|
||||
- [x] Implement `john-slop-fmt` stdin/stdout mode, `--check`, `--diff`, and
|
||||
explicit `--write`. Require an explicit policy for recursive directory writes.
|
||||
- [ ] Make writes atomic with a same-directory temporary file plus rename when
|
||||
- [x] Make writes atomic with a same-directory temporary file plus rename when
|
||||
supported; retain permissions and never rewrite on analysis/config failure.
|
||||
- [ ] Do not rewrite ZIP/DOCX/XLSX/PPTX members in this milestone; emit a clear
|
||||
- [x] Do not rewrite ZIP/DOCX/XLSX/PPTX members in this milestone; emit a clear
|
||||
unsupported-operation diagnostic.
|
||||
- [ ] Add unit/property tests for idempotence, Unicode boundaries, longest match,
|
||||
- [x] Add unit/property tests for idempotence, Unicode boundaries, longest match,
|
||||
non-overlap, stable output, line endings, empty input, and no-op byte identity.
|
||||
- [ ] Add integration tests proving `--check` exit status, stdout purity, atomic
|
||||
- [x] Add integration tests proving `--check` exit status, stdout purity, atomic
|
||||
failure behavior, config overrides, and formatter/detector agreement.
|
||||
|
||||
Acceptance: formatting twice equals formatting once; default invocation does not
|
||||
modify files; every replacement comes from the resolved typed config.
|
||||
|
||||
Decision: `ReplacementRule::source` is a non-empty Unicode `String` rather than
|
||||
a single `char`, enabling deterministic longest-match behavior for shared
|
||||
prefixes. This is a deliberate public data-contract change made before the
|
||||
first formatter milestone tag.
|
||||
|
||||
## Milestone 6 — LSP binary
|
||||
|
||||
- [ ] Select an actively maintained LSP framework after a short dependency/API
|
||||
|
|
|
|||
Loading…
Reference in New Issue