diff --git a/README.md b/README.md index 62071fc..5211c69 100644 --- a/README.md +++ b/README.md @@ -48,6 +48,7 @@ and Vim setup. ## Documentation - [Runnable Rust examples](examples/README.md) +- [Expected-failure Miette diagnostic showcase](examples/README.md#expected-failure-diagnostic-showcase) - [Documentation index](docs/README.md) - [Command-line usage and current capabilities](docs/usage.md) - [Helix and Vim formatter/LSP setup](docs/editors.md) diff --git a/docs/usage.md b/docs/usage.md index 1a97900..fd07930 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -75,6 +75,9 @@ Its [shared fixture manifest](../examples/fixtures/manifest.toml) pairs several `example.txt` inputs with expected detector rules, replacement counts, and formatted output. The executable fixture and protocol tests validate the library, detector, formatter, and LSP against the shared replacement contract. +The separate [expected-failure diagnostic showcase](../examples/README.md#expected-failure-diagnostic-showcase) +prints every current prose finding in graphical Miette form and intentionally +exits `1` when its low confidence threshold is reached. ## Supported input policy diff --git a/examples/README.md b/examples/README.md index b1c98de..182304a 100644 --- a/examples/README.md +++ b/examples/README.md @@ -43,3 +43,27 @@ The formatter integration test executes every case and compares stdout with its expected ASCII file. The manifest also names `lsp` as a consumer, and the in-process protocol suite proves that applying LSP edits produces the same bytes as the formatter. + +## Expected-failure diagnostic showcase + +[`diagnostic-showcase.txt`](diagnostic-showcase.txt) deliberately contains every +kind of currently shipped prose signal. Its adjacent example configuration +promotes assistant self-identification from warning to error, making one run +show Miette advice, warning, and error diagnostics together: + +```sh +cargo run --bin john-slop -- \ + --config examples/diagnostic-showcase.toml \ + --output miette --fail-level low \ + examples/diagnostic-showcase.txt +``` + +That command is expected to exit with status `1`: reaching the configured +threshold is the behavior under demonstration. The repository test asserts the +nonzero result rather than making the suite red. Run it with capture disabled to +print the complete report: + +```sh +cargo test --test example_fixtures \ + diagnostic_showcase_is_an_expected_threshold_failure -- --nocapture +``` diff --git a/examples/diagnostic-showcase.toml b/examples/diagnostic-showcase.toml new file mode 100644 index 0000000..80f3d7f --- /dev/null +++ b/examples/diagnostic-showcase.toml @@ -0,0 +1,5 @@ +# This example-only override demonstrates Miette's error presentation. The +# embedded catalog otherwise classifies this finding as a warning. +[[rules]] +id = "assistant-self-identification" +severity = "error" diff --git a/examples/diagnostic-showcase.txt b/examples/diagnostic-showcase.txt new file mode 100644 index 0000000..34e5137 --- /dev/null +++ b/examples/diagnostic-showcase.txt @@ -0,0 +1,12 @@ +User: Write a polished summary without checking any sources. +Assistant: Certainly! Here's a comprehensive answer. + +As an AI language model, I cannot browse the internet. My knowledge cutoff may +therefore make this inaccurate. It is important to note that the result is +seamless—production-ready, even. + +Moreover, the design is robust. +Furthermore, the implementation is scalable. +Additionally, the solution is intuitive. + +In conclusion, it is important to use this result as-is. diff --git a/tests/example_fixtures.rs b/tests/example_fixtures.rs index cd8bf27..7231ae9 100644 --- a/tests/example_fixtures.rs +++ b/tests/example_fixtures.rs @@ -135,3 +135,30 @@ fn formatter_cli_matches_every_shared_expected_file() { ); } } + +#[test] +fn diagnostic_showcase_is_an_expected_threshold_failure() { + let root = Path::new(env!("CARGO_MANIFEST_DIR")); + let config_home = tempfile::tempdir().expect("temp config home"); + let output = Command::new(env!("CARGO_BIN_EXE_john-slop")) + .env("XDG_CONFIG_HOME", config_home.path()) + .args(["--output", "miette", "--fail-level", "low", "--config"]) + .arg(root.join("examples/diagnostic-showcase.toml")) + .arg(root.join("examples/diagnostic-showcase.txt")) + .output() + .expect("run expected-failure diagnostic showcase"); + let stdout = String::from_utf8(output.stdout).expect("UTF-8 diagnostic output"); + + // `--nocapture` makes this a convenient manual tour of every diagnostic. + print!("{stdout}"); + assert_eq!(output.status.code(), Some(1)); + assert!(output.stderr.is_empty()); + assert!(stdout.contains("assistant-self-identification")); + assert!(stdout.contains("prompt-response-scaffolding")); + assert!(stdout.contains("typography-non-ascii-hyphen")); + assert!(stdout.contains("advice")); + assert!(stdout.contains("warning")); + assert!(stdout.contains("error")); + #[cfg(feature = "fancy-diagnostics")] + assert!(stdout.contains("╭─[")); +}