john-slop/docs/benchmarks.md

1.8 KiB
Raw Permalink Blame History

Performance benchmarks

Criterion benchmarks cover the scaling boundaries most likely to change architectural decisions:

  • analysis of an approximately 1 MiB clean prose document;
  • streaming and decoding 250 small files in deterministic path order;
  • compiling and analyzing with 110 rules (10× the embedded catalog); and
  • opening, validating, and reading a 100-member stored ZIP archive.

Run them on an otherwise idle machine:

cargo bench --locked --bench core

Use -- --quick for a smoke measurement. Criterion output under target/ is not a portable performance artifact: record the CPU, OS, Rust version, commit, and exact command when comparing results.

The Milestone 7 smoke baseline was measured on 2026-09-29 with cargo bench --locked --bench core -- --quick, Rust 1.100.0-nightly, Linux 7.2.7, and an AMD Ryzen 9 9950X3D:

Workload Criterion estimate
approximately 1 MiB document 15.686 ms / 65.36 MiB/s
250 small files 286.54 µs / 872.48 Kfiles/s
build and analyze 110 rules 9.474 ms
100-member stored archive 116.37 µs / 859.34 Kmembers/s

These quick-mode figures are orientation data, not a cross-machine performance guarantee. A release decision should use Criterion's normal sampling mode.

Milestone 7 retains the ordered single-threaded implementation. The baseline workloads complete in milliseconds on the development machine, and directory and archive results already stream rather than accumulating the entire input tree before analysis. Parallel traversal would add scheduling and output-order coordination without an established user-visible bottleneck. Reconsider it only with a representative workload, a reproducible before/after benchmark, and tests proving byte-for-byte output order remains unchanged.