66 lines
3.0 KiB
Markdown
66 lines
3.0 KiB
Markdown
# ADR 0002: Input policy and resource limits
|
|
|
|
- Status: accepted
|
|
- Date: 2026-09-29
|
|
|
|
## Context
|
|
|
|
The detector accepts individual files, directory trees, and ZIP-based
|
|
containers. Unbounded traversal or decompression can exhaust memory, CPU, file
|
|
descriptors, or disk. Extension-only detection can also misclassify binary files
|
|
as text. The input layer needs deterministic defaults before it is redesigned as
|
|
a streaming API.
|
|
|
|
## Decision
|
|
|
|
Limits apply per invocation and can be lowered by configuration or CLI flags.
|
|
Raising them requires an explicit option; there is no unlimited sentinel.
|
|
|
|
| Resource | Default limit |
|
|
| --- | ---: |
|
|
| Plain file or archive member | 8 MiB |
|
|
| Input archive file | 64 MiB |
|
|
| Total expanded archive bytes | 256 MiB |
|
|
| Archive entries inspected | 10,000 |
|
|
| Expanded/compressed ratio per member | 100:1 |
|
|
| Directory traversal depth | 64 levels |
|
|
| Nested archive traversal | disabled (depth 0) |
|
|
|
|
An exact file passed by the user is considered regardless of extension. During
|
|
directory traversal, configurable extension allowlists select likely prose,
|
|
markup, configuration, and source-code files. Content still has to decode as
|
|
strict UTF-8 and must not contain a NUL byte. Unsupported/binary content produces
|
|
a skipped-input result rather than a clean analysis or a lossy conversion.
|
|
|
|
ZIP-based containers are recognized from both a supported extension (`zip`,
|
|
`docx`, `xlsx`, or `pptx`) and ZIP magic bytes. An extension/magic mismatch is a
|
|
typed input error. Members are streamed and counted before allocation. Directory
|
|
members and encrypted members are skipped with a reason. Absolute paths,
|
|
platform prefixes, and any path containing a parent (`..`) component are
|
|
rejected. Archive member names are identities only; members are never extracted
|
|
to the filesystem by the analysis path.
|
|
|
|
Directory walking does not follow symbolic links. Hidden paths are skipped by
|
|
default. The default ignore set includes `.git`, `.hg`, `.svn`, `target`,
|
|
`node_modules`, `vendor`, common generated directories, source maps, and minified
|
|
assets. Repository ignore files are respected when the eventual walker supports
|
|
them. An explicitly named file can bypass path/extension ignores, but cannot
|
|
bypass content or resource-safety limits.
|
|
|
|
One unreadable, invalid, or oversized path yields a per-document input error and
|
|
does not discard successful documents from the same directory or archive.
|
|
Results remain ordered by normalized source identity even if reading is later
|
|
parallelized.
|
|
|
|
## Consequences
|
|
|
|
- The input API must stream per-document results and track cumulative archive
|
|
counters.
|
|
- `Document` needs distinct outer-path and optional archive-member identities.
|
|
- Size checks must use checked arithmetic and occur before allocating from
|
|
metadata supplied by an archive.
|
|
- Tests need boundary cases immediately below, at, and above each configurable
|
|
limit, plus malicious archive names and misleading extensions.
|
|
- The README and CLI help must distinguish skipped inputs, operational errors,
|
|
and clean analyzed documents.
|