From c88c5b8de28267a47bfb389802411a3d0cd163dd Mon Sep 17 00:00:00 2001 From: Dandaedre Date: Tue, 29 Sep 2026 12:11:33 +0200 Subject: [PATCH] docs: define input safety limits --- README.md | 2 + .../decisions/0002-input-policy-and-limits.md | 65 +++++++++++++++++++ tasks.md | 2 +- 3 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 docs/decisions/0002-input-policy-and-limits.md diff --git a/README.md b/README.md index 4abf303..4b55a22 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,8 @@ formatter, and LSP binaries. See [tasks.md](tasks.md) for the ordered backlog an Architecture and product choices are recorded under [`docs/decisions`](docs/decisions/), beginning with the [core analysis contracts](docs/decisions/0001-core-analysis-contracts.md). +The default traversal, text-recognition, and archive safety policy is specified +in [ADR 0002](docs/decisions/0002-input-policy-and-limits.md). ## Development diff --git a/docs/decisions/0002-input-policy-and-limits.md b/docs/decisions/0002-input-policy-and-limits.md new file mode 100644 index 0000000..71f325c --- /dev/null +++ b/docs/decisions/0002-input-policy-and-limits.md @@ -0,0 +1,65 @@ +# ADR 0002: Input policy and resource limits + +- Status: accepted +- Date: 2026-09-29 + +## Context + +The detector accepts individual files, directory trees, and ZIP-based +containers. Unbounded traversal or decompression can exhaust memory, CPU, file +descriptors, or disk. Extension-only detection can also misclassify binary files +as text. The input layer needs deterministic defaults before it is redesigned as +a streaming API. + +## Decision + +Limits apply per invocation and can be lowered by configuration or CLI flags. +Raising them requires an explicit option; there is no unlimited sentinel. + +| Resource | Default limit | +| --- | ---: | +| Plain file or archive member | 8 MiB | +| Input archive file | 64 MiB | +| Total expanded archive bytes | 256 MiB | +| Archive entries inspected | 10,000 | +| Expanded/compressed ratio per member | 100:1 | +| Directory traversal depth | 64 levels | +| Nested archive traversal | disabled (depth 0) | + +An exact file passed by the user is considered regardless of extension. During +directory traversal, configurable extension allowlists select likely prose, +markup, configuration, and source-code files. Content still has to decode as +strict UTF-8 and must not contain a NUL byte. Unsupported/binary content produces +a skipped-input result rather than a clean analysis or a lossy conversion. + +ZIP-based containers are recognized from both a supported extension (`zip`, +`docx`, `xlsx`, or `pptx`) and ZIP magic bytes. An extension/magic mismatch is a +typed input error. Members are streamed and counted before allocation. Directory +members and encrypted members are skipped with a reason. Absolute paths, +platform prefixes, and any path containing a parent (`..`) component are +rejected. Archive member names are identities only; members are never extracted +to the filesystem by the analysis path. + +Directory walking does not follow symbolic links. Hidden paths are skipped by +default. The default ignore set includes `.git`, `.hg`, `.svn`, `target`, +`node_modules`, `vendor`, common generated directories, source maps, and minified +assets. Repository ignore files are respected when the eventual walker supports +them. An explicitly named file can bypass path/extension ignores, but cannot +bypass content or resource-safety limits. + +One unreadable, invalid, or oversized path yields a per-document input error and +does not discard successful documents from the same directory or archive. +Results remain ordered by normalized source identity even if reading is later +parallelized. + +## Consequences + +- The input API must stream per-document results and track cumulative archive + counters. +- `Document` needs distinct outer-path and optional archive-member identities. +- Size checks must use checked arithmetic and occur before allocating from + metadata supplied by an archive. +- Tests need boundary cases immediately below, at, and above each configurable + limit, plus malicious archive names and misleading extensions. +- The README and CLI help must distinguish skipped inputs, operational errors, + and clean analyzed documents. diff --git a/tasks.md b/tasks.md index 45903b7..fedfb70 100644 --- a/tasks.md +++ b/tasks.md @@ -31,7 +31,7 @@ audit in CI. - [x] Record initial decisions in `docs/decisions/`: evidence confidence is not authorship probability; UTF-8 byte spans are canonical; formatter writes are opt-in; no archive mutation in the first formatter release. -- [ ] Define supported inputs and limits: extensions/MIME policy, maximum file +- [x] Define supported inputs and limits: extensions/MIME policy, maximum file bytes, archive bytes, expanded bytes, entry count, nesting, hidden files, symlinks, and ignore rules. - [ ] Establish an MSRV in `Cargo.toml` and CI, add license/repository metadata,