docs: define input safety limits

This commit is contained in:
Dandaedre 2026-09-29 12:11:33 +02:00
parent 7b5e6b330f
commit c88c5b8de2
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
3 changed files with 68 additions and 1 deletions

View File

@ -22,6 +22,8 @@ formatter, and LSP binaries. See [tasks.md](tasks.md) for the ordered backlog an
Architecture and product choices are recorded under
[`docs/decisions`](docs/decisions/), beginning with the
[core analysis contracts](docs/decisions/0001-core-analysis-contracts.md).
The default traversal, text-recognition, and archive safety policy is specified
in [ADR 0002](docs/decisions/0002-input-policy-and-limits.md).
## Development

View File

@ -0,0 +1,65 @@
# ADR 0002: Input policy and resource limits
- Status: accepted
- Date: 2026-09-29
## Context
The detector accepts individual files, directory trees, and ZIP-based
containers. Unbounded traversal or decompression can exhaust memory, CPU, file
descriptors, or disk. Extension-only detection can also misclassify binary files
as text. The input layer needs deterministic defaults before it is redesigned as
a streaming API.
## Decision
Limits apply per invocation and can be lowered by configuration or CLI flags.
Raising them requires an explicit option; there is no unlimited sentinel.
| Resource | Default limit |
| --- | ---: |
| Plain file or archive member | 8 MiB |
| Input archive file | 64 MiB |
| Total expanded archive bytes | 256 MiB |
| Archive entries inspected | 10,000 |
| Expanded/compressed ratio per member | 100:1 |
| Directory traversal depth | 64 levels |
| Nested archive traversal | disabled (depth 0) |
An exact file passed by the user is considered regardless of extension. During
directory traversal, configurable extension allowlists select likely prose,
markup, configuration, and source-code files. Content still has to decode as
strict UTF-8 and must not contain a NUL byte. Unsupported/binary content produces
a skipped-input result rather than a clean analysis or a lossy conversion.
ZIP-based containers are recognized from both a supported extension (`zip`,
`docx`, `xlsx`, or `pptx`) and ZIP magic bytes. An extension/magic mismatch is a
typed input error. Members are streamed and counted before allocation. Directory
members and encrypted members are skipped with a reason. Absolute paths,
platform prefixes, and any path containing a parent (`..`) component are
rejected. Archive member names are identities only; members are never extracted
to the filesystem by the analysis path.
Directory walking does not follow symbolic links. Hidden paths are skipped by
default. The default ignore set includes `.git`, `.hg`, `.svn`, `target`,
`node_modules`, `vendor`, common generated directories, source maps, and minified
assets. Repository ignore files are respected when the eventual walker supports
them. An explicitly named file can bypass path/extension ignores, but cannot
bypass content or resource-safety limits.
One unreadable, invalid, or oversized path yields a per-document input error and
does not discard successful documents from the same directory or archive.
Results remain ordered by normalized source identity even if reading is later
parallelized.
## Consequences
- The input API must stream per-document results and track cumulative archive
counters.
- `Document` needs distinct outer-path and optional archive-member identities.
- Size checks must use checked arithmetic and occur before allocating from
metadata supplied by an archive.
- Tests need boundary cases immediately below, at, and above each configurable
limit, plus malicious archive names and misleading extensions.
- The README and CLI help must distinguish skipped inputs, operational errors,
and clean analyzed documents.

View File

@ -31,7 +31,7 @@ audit in CI.
- [x] Record initial decisions in `docs/decisions/`: evidence confidence is not
authorship probability; UTF-8 byte spans are canonical; formatter writes are
opt-in; no archive mutation in the first formatter release.
- [ ] Define supported inputs and limits: extensions/MIME policy, maximum file
- [x] Define supported inputs and limits: extensions/MIME policy, maximum file
bytes, archive bytes, expanded bytes, entry count, nesting, hidden files,
symlinks, and ignore rules.
- [ ] Establish an MSRV in `Cargo.toml` and CI, add license/repository metadata,