3.0 KiB
ADR 0002: Input policy and resource limits
- Status: accepted
- Date: 2026-09-29
Context
The detector accepts individual files, directory trees, and ZIP-based containers. Unbounded traversal or decompression can exhaust memory, CPU, file descriptors, or disk. Extension-only detection can also misclassify binary files as text. The input layer needs deterministic defaults before it is redesigned as a streaming API.
Decision
Limits apply per invocation and can be lowered by configuration or CLI flags. Raising them requires an explicit option; there is no unlimited sentinel.
| Resource | Default limit |
|---|---|
| Plain file or archive member | 8 MiB |
| Input archive file | 64 MiB |
| Total expanded archive bytes | 256 MiB |
| Archive entries inspected | 10,000 |
| Expanded/compressed ratio per member | 100:1 |
| Directory traversal depth | 64 levels |
| Nested archive traversal | disabled (depth 0) |
An exact file passed by the user is considered regardless of extension. During directory traversal, configurable extension allowlists select likely prose, markup, configuration, and source-code files. Content still has to decode as strict UTF-8 and must not contain a NUL byte. Unsupported/binary content produces a skipped-input result rather than a clean analysis or a lossy conversion.
ZIP-based containers are recognized from both a supported extension (zip,
docx, xlsx, or pptx) and ZIP magic bytes. An extension/magic mismatch is a
typed input error. Members are streamed and counted before allocation. Directory
members and encrypted members are skipped with a reason. Absolute paths,
platform prefixes, and any path containing a parent (..) component are
rejected. Archive member names are identities only; members are never extracted
to the filesystem by the analysis path.
Directory walking does not follow symbolic links. Hidden paths are skipped by
default. The default ignore set includes .git, .hg, .svn, target,
node_modules, vendor, common generated directories, source maps, and minified
assets. Repository ignore files are respected when the eventual walker supports
them. An explicitly named file can bypass path/extension ignores, but cannot
bypass content or resource-safety limits.
One unreadable, invalid, or oversized path yields a per-document input error and does not discard successful documents from the same directory or archive. Results remain ordered by normalized source identity even if reading is later parallelized.
Consequences
- The input API must stream per-document results and track cumulative archive counters.
Documentneeds distinct outer-path and optional archive-member identities.- Size checks must use checked arithmetic and occur before allocating from metadata supplied by an archive.
- Tests need boundary cases immediately below, at, and above each configurable limit, plus malicious archive names and misleading extensions.
- The README and CLI help must distinguish skipped inputs, operational errors, and clean analyzed documents.