test: add expected-failure diagnostic showcase
This commit is contained in:
parent
b40112a724
commit
eeabed9010
|
|
@ -48,6 +48,7 @@ and Vim setup.
|
|||
## Documentation
|
||||
|
||||
- [Runnable Rust examples](examples/README.md)
|
||||
- [Expected-failure Miette diagnostic showcase](examples/README.md#expected-failure-diagnostic-showcase)
|
||||
- [Documentation index](docs/README.md)
|
||||
- [Command-line usage and current capabilities](docs/usage.md)
|
||||
- [Helix and Vim formatter/LSP setup](docs/editors.md)
|
||||
|
|
|
|||
|
|
@ -75,6 +75,9 @@ Its [shared fixture manifest](../examples/fixtures/manifest.toml) pairs several
|
|||
`example.txt` inputs with expected detector rules, replacement counts, and
|
||||
formatted output. The executable fixture and protocol tests validate the
|
||||
library, detector, formatter, and LSP against the shared replacement contract.
|
||||
The separate [expected-failure diagnostic showcase](../examples/README.md#expected-failure-diagnostic-showcase)
|
||||
prints every current prose finding in graphical Miette form and intentionally
|
||||
exits `1` when its low confidence threshold is reached.
|
||||
|
||||
## Supported input policy
|
||||
|
||||
|
|
|
|||
|
|
@ -43,3 +43,27 @@ The formatter integration test executes every case and compares stdout with its
|
|||
expected ASCII file. The manifest also names `lsp` as a consumer, and the
|
||||
in-process protocol suite proves that applying LSP edits produces the same bytes
|
||||
as the formatter.
|
||||
|
||||
## Expected-failure diagnostic showcase
|
||||
|
||||
[`diagnostic-showcase.txt`](diagnostic-showcase.txt) deliberately contains every
|
||||
kind of currently shipped prose signal. Its adjacent example configuration
|
||||
promotes assistant self-identification from warning to error, making one run
|
||||
show Miette advice, warning, and error diagnostics together:
|
||||
|
||||
```sh
|
||||
cargo run --bin john-slop -- \
|
||||
--config examples/diagnostic-showcase.toml \
|
||||
--output miette --fail-level low \
|
||||
examples/diagnostic-showcase.txt
|
||||
```
|
||||
|
||||
That command is expected to exit with status `1`: reaching the configured
|
||||
threshold is the behavior under demonstration. The repository test asserts the
|
||||
nonzero result rather than making the suite red. Run it with capture disabled to
|
||||
print the complete report:
|
||||
|
||||
```sh
|
||||
cargo test --test example_fixtures \
|
||||
diagnostic_showcase_is_an_expected_threshold_failure -- --nocapture
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
# This example-only override demonstrates Miette's error presentation. The
|
||||
# embedded catalog otherwise classifies this finding as a warning.
|
||||
[[rules]]
|
||||
id = "assistant-self-identification"
|
||||
severity = "error"
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
User: Write a polished summary without checking any sources.
|
||||
Assistant: Certainly! Here's a comprehensive answer.
|
||||
|
||||
As an AI language model, I cannot browse the internet. My knowledge cutoff may
|
||||
therefore make this inaccurate. It is important to note that the result is
|
||||
seamless—production-ready, even.
|
||||
|
||||
Moreover, the design is robust.
|
||||
Furthermore, the implementation is scalable.
|
||||
Additionally, the solution is intuitive.
|
||||
|
||||
In conclusion, it is important to use this result as-is.
|
||||
|
|
@ -135,3 +135,30 @@ fn formatter_cli_matches_every_shared_expected_file() {
|
|||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diagnostic_showcase_is_an_expected_threshold_failure() {
|
||||
let root = Path::new(env!("CARGO_MANIFEST_DIR"));
|
||||
let config_home = tempfile::tempdir().expect("temp config home");
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_john-slop"))
|
||||
.env("XDG_CONFIG_HOME", config_home.path())
|
||||
.args(["--output", "miette", "--fail-level", "low", "--config"])
|
||||
.arg(root.join("examples/diagnostic-showcase.toml"))
|
||||
.arg(root.join("examples/diagnostic-showcase.txt"))
|
||||
.output()
|
||||
.expect("run expected-failure diagnostic showcase");
|
||||
let stdout = String::from_utf8(output.stdout).expect("UTF-8 diagnostic output");
|
||||
|
||||
// `--nocapture` makes this a convenient manual tour of every diagnostic.
|
||||
print!("{stdout}");
|
||||
assert_eq!(output.status.code(), Some(1));
|
||||
assert!(output.stderr.is_empty());
|
||||
assert!(stdout.contains("assistant-self-identification"));
|
||||
assert!(stdout.contains("prompt-response-scaffolding"));
|
||||
assert!(stdout.contains("typography-non-ascii-hyphen"));
|
||||
assert!(stdout.contains("advice"));
|
||||
assert!(stdout.contains("warning"));
|
||||
assert!(stdout.contains("error"));
|
||||
#[cfg(feature = "fancy-diagnostics")]
|
||||
assert!(stdout.contains("╭─["));
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue