diff --git a/README.md b/README.md index f9502f6..cd60ab8 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,9 @@ ZIP-based office documents. Eleven conservative, configurable rules cover copied assistant scaffolding, generic style residue, source-code placeholders, tutorial narration, and configured non-ASCII characters. Rules retain exact source spans, apply path/content scopes, and feed a bounded, explainable -confidence score. Source-highlighted terminal diagnostics, a formatter, and an -LSP are planned in [tasks.md](tasks.md). +confidence score. The detector renders source-highlighted human diagnostics, +concise records, or versioned JSON Lines. A formatter and LSP are planned in +[tasks.md](tasks.md). ## Quick start @@ -33,6 +34,7 @@ signals, not the probability that AI wrote the content. - [Documentation index](docs/README.md) - [Command-line usage and current capabilities](docs/usage.md) +- [Human, concise, and JSON output contracts](docs/output.md) - [Architecture and public library API](docs/architecture.md) - [Layered configuration](docs/configuration.md) - [Heuristics, confidence, and applicability](docs/heuristics.md) diff --git a/docs/README.md b/docs/README.md index f02d8ce..84e65c1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,6 +5,8 @@ the details needed to use, embed, and develop John Slop: - [Usage](usage.md) documents accepted inputs, detector output, exit statuses, and the current formatter/LSP status. +- [Output contracts](output.md) specifies human and concise presentation plus + the versioned JSON Lines schema and partial-error records. - [Architecture](architecture.md) describes module boundaries, shared domain values, input safety, and the reusable library facade. - [Configuration](configuration.md) documents schema version 1, source diff --git a/docs/architecture.md b/docs/architecture.md index c8816a2..e7f24e4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,6 +8,7 @@ The current flow is: ```text path -> input stream -> Document -> analysis -> Analysis + Finding + -> diagnostic/output adapters -> replacement planner -> TextEdit ``` @@ -17,10 +18,13 @@ The main modules have deliberately narrow responsibilities: - `input` streams files, directory entries, and archive members as documents; - `detector` compiles configured matchers and path scopes once, then produces explainable findings and bounded confidence; +- `diagnostic` binds findings to exact named snapshots as typed `miette` + diagnostics; - `replacement` produces non-overlapping edits without performing I/O; - `model` contains presentation-neutral domain values; - `error` exposes typed input failures for library consumers; -- `src/bin` contains the detector and the formatter/LSP adapters. +- `src/bin` contains the detector presentation/exit policy and formatter/LSP + adapters. All source spans are zero-based UTF-8 byte ranges into the exact analyzed source snapshot. Presentation layers will convert those canonical spans to terminal or @@ -30,13 +34,17 @@ so language and path filters behave the same inside and outside archives. ## Public facade -The crate root exposes four entry points: +The crate root exposes core entry points and presentation-neutral types: - `load_config` resolves the configuration embedded in the current build; - `read_inputs` returns a streaming iterator of per-document results; +- `read_stdin` returns one bounded strict-UTF-8 standard-input document; - `analyze_source` returns findings and evidence confidence for UTF-8 text; - `plan_replacements` returns safe edits against the same source snapshot. +`FindingDiagnostic` is the library bridge to `miette`; CLI-specific concise and +JSON rendering stays in the detector adapter so it cannot alter analysis. + `ConfigLoader` applies embedded, user base, named user, explicit file, and CLI layers in order. Stable-ID merging and semantic validation happen before a configuration reaches analysis. Keeping these operations behind the facade diff --git a/docs/limitations.md b/docs/limitations.md index 0afd64e..d4878cf 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -67,9 +67,8 @@ such decisions. Only strict UTF-8 text is analyzed. NUL-containing binary data, invalid UTF-8, encrypted archives, unsafe archive members, and configured resource-limit violations are errors rather than clean results. Generated, vendor, dependency, -and minified paths are skipped by default. The current command-line summary does -not yet render source-highlighted diagnostics; formatter and LSP binaries remain -scaffolds for later milestones. +and minified paths are skipped by default. The formatter and LSP binaries remain +scaffolds for later milestones; only the detector currently presents findings. Analysis is local and performs no network lookup. Consequently, it cannot check citations, current facts, provenance, licensing, or similarity to external diff --git a/docs/output.md b/docs/output.md new file mode 100644 index 0000000..9053bb4 --- /dev/null +++ b/docs/output.md @@ -0,0 +1,79 @@ +# Detector output contracts + +The detector exposes three formats through `--output`. All formats preserve the +same deterministic document and finding order. Requested paths retain command +line order, directory entries are sorted by filename, archive order follows the +container, and findings are sorted by byte span and rule ID. + +## Human + +`--output human` is the default. Every finding is rendered as a plain +`miette` diagnostic with the source name, exact excerpt and primary span, +severity, stable rule code, message, and remediation help. A final line per +document reports the confidence score/band and advice, warning, and error +counts. The narratable renderer deliberately avoids ANSI escapes. + +## Concise + +`--output concise` emits one line per finding followed by one summary line. A +finding line contains the source identity, half-open UTF-8 byte range, severity, +rule ID, and message. Recoverable input errors go to stderr in human and concise +modes. + +## JSON Lines schema version 1 + +`--output json` emits one compact JSON object per line. An analyzed document has +`kind: "analysis"` and this shape: + +```json +{ + "schema_version": 1, + "kind": "analysis", + "source": "article.md", + "summary": { + "confidence_score": 17, + "confidence_band": "below-threshold", + "advice": 1, + "warnings": 0, + "errors": 0, + "total": 1 + }, + "findings": [ + { + "rule_id": "typography-non-ascii-hyphen", + "category": "typography", + "severity": "advice", + "message": "a configured non-ASCII character has an ASCII replacement", + "help": "Review the punctuation or apply the configured safe replacement.", + "byte_span": { "start": 8, "end": 11 }, + "matched_text": "—", + "weight": 10.0, + "fix": { + "byte_span": { "start": 8, "end": 11 }, + "replacement": "-" + } + } + ] +} +``` + +`confidence_band` is `null` when there is no evidence. Spans are zero-based, +half-open UTF-8 byte offsets into the exact source snapshot, not character or +UTF-16 positions. `fix` is `null` for findings without a safe configured edit. + +A recoverable input failure has `kind: "input-error"`: + +```json +{ + "schema_version": 1, + "kind": "input-error", + "source": "broken.txt", + "message": "input broken.txt is not valid UTF-8" +} +``` + +JSON records are the only stdout content in JSON mode and never contain ANSI +escapes or logs. Fatal argument/configuration/setup failures can occur before +the JSON stream starts; they return status 2, leave stdout empty, and report on +stderr. Additive fields may appear within schema version 1; incompatible changes +require a new schema version. diff --git a/docs/usage.md b/docs/usage.md index ecec8cc..74a0e91 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -29,10 +29,9 @@ mode emits one object per analyzed document with `schema_version: 1`, summary fields, exact byte spans, and every finding. Machine modes never emit ANSI escapes or logs on stdout. -The current human-readable output contains the source identity, bounded evidence -confidence, and finding count. Findings cover the embedded or overridden -heuristic catalog and retain exact UTF-8 byte spans for later presentation. -Source-highlighted `miette` diagnostics are planned for the next milestone. +Findings cover the embedded or overridden heuristic catalog and retain exact +UTF-8 byte spans through every output mode. See [Output contracts](output.md) +for the human/concise formats and complete JSON Lines schema. Exit statuses are stable: @@ -58,10 +57,10 @@ the formula and [Limitations](limitations.md) before using thresholds in review workflows. Configuration layers can be selected with `--config-name NAME` and `--config -FILE`; `--max-document-bytes BYTES` is the current highest-precedence input -override. Use `john-slop config show` to print the resolved TOML and `john-slop -config paths` to inspect source precedence. See [Configuration](configuration.md) -for the schema and merge rules. +FILE`; command-line resource limits are the highest-precedence input overrides. +Use `john-slop config show` to print the resolved TOML and `john-slop config +paths` to inspect source precedence. See [Configuration](configuration.md) for +the schema and merge rules. ## Supported input policy