feat: define generic report v1 contract and YAML JSON validator
Reportist CI Reportist passed all currently implemented CI checks.
Beffroi/reportist/pipeline/head This commit looks good Details

This commit is contained in:
Dandaedre 2026-09-15 16:32:36 +02:00
parent 3b73d1a03c
commit e2ada627d0
14 changed files with 1164 additions and 18 deletions

3
.gitignore vendored
View File

@ -102,3 +102,6 @@ core.*
#
# uv.lock / Cargo.lock should normally be committed when applicable.
# =============================================================================
# Isolated Jenkins validation environment
.ci-venv/

View File

@ -37,10 +37,10 @@ visual baseline without importing private reports into generic fixtures.
Depends on Pass 0 for the selected slice.
- [ ] RT1.1 Define `schema/report.schema.json` with explicit schema version, required/optional fields, metadata, branding, and generic report content; preserve useful legacy compatibility deliberately.
- [ ] RT1.2 Map existing report fields and diagnostics presentation needs to generic content structures; keep evidence interpretation outside the renderer.
- [ ] RT1.3 Define external asset resolution, theme overrides, missing branding behavior, and neutral defaults without organisation-specific names, authors, logos, or incident terminology.
- [ ] RT1.4 Specify YAML/JSON validation errors, unsupported schema handling, and schema evolution/migration policy.
- [x] RT1.1 Define `schema/report.schema.json` with explicit schema version, required/optional fields, metadata, branding, and generic report content; preserve useful legacy compatibility deliberately.
- [x] RT1.2 Map existing report fields and diagnostics presentation needs to generic content structures; keep evidence interpretation outside the renderer.
- [x] RT1.3 Define external asset resolution, theme overrides, missing branding behavior, and neutral defaults without organisation-specific names, authors, logos, or incident terminology.
- [x] RT1.4 Specify YAML/JSON validation errors, unsupported schema handling, and schema evolution/migration policy.
- [ ] RT1.5 Define a simple CLI with explicit input and output paths, documented exit codes, and installation requirements; publish synthetic valid/invalid examples.
Exit: unrelated callers can author valid reports without knowing internal Typst
@ -101,3 +101,11 @@ contract is sufficient for any report-producing application.
- Validation: macro baseline command rendered both report PDFs and rejected missing author/date and unknown theme. Reviewed the technical report first page; later-page/long-content/alternate-branding coverage remains open.
- Existing Beffroi logo coupling remains visible in legacy output. No generic renderer implementation or public schema is claimed yet.
- Next: RT1 schema/validation fixtures, then RT2 generic rendering extraction.
### 2026-09-15 — Versioned contract pass
- Completed RT1.1–RT1.4: generic report v1 schema, YAML/JSON validator and paired synthetic examples, optional branding/path contract, generic blocks, and version/legacy mapping documentation.
- Evidence: [report contract](docs/report-contract.md), `schema/report.schema.json`, `tools/validate.py`, `examples/`, `tests/test_contract.py`.
- Validation: eight unittest cases with many invalid-input subcases passed on Python 3.11 and 3.14, including an isolated standalone copy. Covers YAML/JSON parity, minimal reports, invalid dates/versions, duplicate keys, generic branding shape, nested blocks and parser errors.
- Jenkins now runs the contract tests. Rendering, actual asset loading and visual branding are not implemented or validated by these schema tests.
- Next: finish RT1.5 exact rendering command/output behavior, then RT2 extraction. No render command is claimed yet.

23
Jenkinsfile vendored
View File

@ -46,15 +46,18 @@ pipeline {
* - print `typst --version`.
*/
/*
* TODO: Validate Report Schema
*
* Once schema/report.schema.json exists:
*
* - validate the schema itself;
* - validate examples/*.yaml;
* - reject malformed report definitions.
*/
stage('Validate Contract') {
steps {
sh '''
set -eu
command -v uv >/dev/null 2>&1 || { echo "uv is required for the isolated validation environment" >&2; exit 1; }
uv venv --python 3.11 --allow-existing .ci-venv
uv pip sync --python .ci-venv/bin/python requirements.txt
.ci-venv/bin/python --version
.ci-venv/bin/python -m unittest discover -s tests -p 'test_*.py'
'''
}
}
/*
* TODO: Render Example
@ -87,7 +90,7 @@ pipeline {
conclusion: 'SUCCESS',
name: 'Reportist CI',
summary: 'Reportist passed all currently implemented CI checks.',
text: 'Repository validation completed successfully. Renderer validation will be enabled as functionality is migrated.',
text: 'Repository and report contract tests passed. PDF renderer implementation and rendering validation remain pending.',
title: 'Reportist CI passed'
)

View File

@ -26,6 +26,12 @@ See [AGENT.md](AGENT.md) for repository ownership and development rules.
[AGENTS-TASKS.md](AGENTS-TASKS.md) tracks migration passes, completed work,
acceptance criteria, and the next-pass handoff.
## Contract validation
The version-1 schema, standalone validator, and synthetic examples are available.
See [contract documentation](docs/report-contract.md) for setup and validation commands.
Collection/analysis and PDF rendering are still separate migration tasks.
## License
Software source code is licensed under the **MIT License**.

View File

@ -42,7 +42,7 @@ Defaults use A4, DejaVu Sans and DejaVu Sans Mono. No external Typst packages ar
imported. The new CLI must accept explicit input/output paths independently of
legacy incident directory conventions.
## Field mapping for version 1 (draft, not implemented)
## Field mapping for version 1
| Existing fields | Proposed treatment |
| --- | --- |
@ -58,8 +58,9 @@ legacy incident directory conventions.
| `actions_title`, `actions`: title, status, text, commands | Generic action display; commands are inert text. |
| `diagnostics.hypotheses` and lists | Map to generic sections/cards or a documented generic assessment block. |
Schema validation, YAML/JSON parity, optional-field behavior, branding, and the
standalone CLI remain RT1/RT2 tasks; this is not a released interface.
Version-1 schema validation, YAML/JSON parity, optional-field and branding shape
checks are now implemented; see [report contract](report-contract.md). Rendering,
asset loading, and actual branding/layout behavior remain unimplemented.
## Baseline
@ -77,4 +78,5 @@ this is a known coupling to remove. Other pages, long-content pagination,
alternate branding, all optional constructs, and font-version reproducibility
still require tests.
Next: RT1 schema and validation examples, then the generic single-report slice.
Next: define the rendering command precisely and implement the generic
single-report slice against the validated contract.

108
docs/report-contract.md Normal file
View File

@ -0,0 +1,108 @@
# Report contract version 1
[report.schema.json](../schema/report.schema.json) is the implemented JSON Schema
Draft 2020-12 interface for generic YAML/JSON report definitions. Validation is
available now; PDF rendering is the next implementation slice.
## Standalone validation
Python 3.11 or newer is required. From this component checkout:
```bash
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python tools/validate.py examples/example-report.yaml
.venv/bin/python tools/validate.py examples/example-report.json
.venv/bin/python -m unittest discover -s tests -p 'test_*.py'
```
On Windows substitute `.venv\Scripts\python.exe`. With uv, use `uv venv` and
`uv pip sync --python .venv/bin/python requirements.txt`. The dependency lock pins
transitive versions. Validation needs no Typst or sibling repositories and works
from any working directory using explicit paths. It writes no files. Exit codes:
0 valid, 1 invalid/unreadable input or missing dependencies, 2 CLI usage error.
Errors identify the offending JSON path; unsupported file extensions are rejected.
## Required and optional content
Only `schema_version: 1`, nonblank `title`, and `metadata` containing nonblank
`Author` and a calendar-valid `Date` string (`YYYY-MM-DD`) are required.
`examples/minimal-report.json` demonstrates omitted optional fields. The larger
YAML/JSON examples have identical parsed content and contain no organisation assets
or actual incident data.
Optional fields retain useful legacy structure:
- `id`, `subtitle`, `report_type` (`technical` or `briefing`), `status`, `severity`.
- `summary`, `closing`: supplied prose, never generated from technical observations.
- `timeline`: rows with `time`, `event`, optional `notes`.
- `findings`: title, optional ID/severity/status, separate `evidence` and `assessment`,
and optional `evidence_refs` containing artifact, JSON pointer, and optional label.
- `sections`, `appendices`: titled sections containing generic content blocks.
- `actions_title`, `actions`: action titles, optional status/text/command strings.
- `provenance`: generic source records with artifact and optional SHA-256/schema
version/producer, and optional `rendered_by` tool name/version. Do not fabricate
missing historical provenance or claim a render before it happens.
Finding IDs, when supplied, must be unique. Evidence pointers are syntax-checked;
this validator does not open referenced evidence or prove report assertions.
Additional metadata labels may contain JSON scalar values. Unknown fields elsewhere
are rejected, including legacy `diagnostics`, top-level `organization` and named
`theme` fields. Status is a supplied nonblank display label; severity is one of
`info`, `low`, `medium`, `high`, `critical`, `warning`, `ok`, `success`, `unanalysed`.
## Blocks
| `type` | Required content | Optional content |
| --- | --- | --- |
| `text` | `text` | — |
| `bullets` | nonempty `items` of strings | — |
| `steps` | nonempty `items` of strings | — |
| `callout` | `text` | `title`, `kind` (severity token) |
| `code` | `text` | — |
| `subsection` | `title`, nonempty nested `content` | — |
Unknown or misspelled blocks fail validation. Text and commands are data, not code
for the validator to execute. The renderer must preserve that boundary.
## Branding and path resolution
`branding` is optional and contains optional `organization`, `logo`, and `theme`.
A theme is an object with optional `font`, `mono_font`, and six-digit hex colors:
`ink`, `muted`, `line`, `paper_soft`, `accent`, `info`, `warning`, `critical`,
`success`. No organisation, author, logo, or incident name is hard-coded.
Logo and artifact paths use relative forward-slash paths without parent/dot
segments, backslashes, drive letters, or control characters. For the future
renderer, resolve assets relative to the input report directory; missing branding
means neutral defaults with no logo, and a missing specified asset is an error.
This pass validates the contract shape, not asset availability or actual layout.
The intended rendering CLI will accept one report and an explicit output path;
there is no implemented render command yet.
## YAML/JSON behavior
Input must fit the JSON data model: string mapping keys, strings, finite numbers,
booleans, null, arrays, and objects. Quote dates/timestamps and identifier-like
values so YAML cannot reinterpret them. Duplicate keys, non-finite numbers, unsafe
tags, recursive aliases, and YAML merge keys are rejected. Acyclic aliases are
allowed when their expanded values satisfy the same schema. Multiple YAML documents
are not accepted. YAML and JSON validation use the same schema and semantic checks.
## Legacy migration and compatibility
Add an explicit version and title to legacy reports, retain `metadata.Date` and
`metadata.Author`, and move `organization` into `branding.organization`. Resolve
named legacy palettes into explicit `branding.theme` values. Supply a logo only
when the report's owner provides it. Map diagnostic hypotheses and fixed domain
sections into generic sections/cards; they are not a required renderer feature.
Do not relabel Windows raw captures as report inputs.
Unsupported versions fail validation. Breaking structural or semantic changes
require a new schema version and migration notes. The pinned legacy examples remain
unchanged for visual comparison; the new examples show contract-v1 authoring.
See [JSON Schema validation](https://json-schema.org/draft/2020-12/json-schema-validation)
and [jsonschema format checking](https://python-jsonschema.readthedocs.io/en/stable/validate/)
for the underlying validation rules. Local semantic checks additionally reject
repeated finding IDs.

View File

@ -0,0 +1,130 @@
{
"id": "INC-2026-001",
"title": "Service Interruption Investigation",
"subtitle": "Incident analysis and shift handoff",
"report_type": "technical",
"status": "open",
"severity": "high",
"metadata": {
"Date": "2026-09-05",
"Author": "Your Name",
"Service": "Example API",
"Environment": "Production",
"Owner": "Operations"
},
"summary": "The API is available but intermittently slow. Database connection exhaustion is the strongest current lead. No permanent data loss has been observed.\n",
"timeline": [
{
"time": "09:15",
"event": "Alert received",
"notes": "Latency exceeded the five-second threshold."
},
{
"time": "09:22",
"event": "Initial mitigation",
"notes": "One unhealthy application instance was removed from rotation."
}
],
"findings": [
{
"title": "Connection pool saturation",
"severity": "high",
"status": "confirmed",
"evidence": "Pool utilization remained at 100 percent during each latency spike.\n",
"assessment": "Requests are waiting for database connections. Query duration and pool sizing both need review.\n",
"id": "pool-saturation",
"evidence_refs": [
{
"artifact": "observations.json",
"pointer": "/metrics/pool_utilization",
"label": "Synthetic metric"
}
]
},
{
"title": "No evidence of data loss",
"severity": "info",
"status": "confirmed",
"evidence": "Record counts and replication checks are consistent across nodes.\n",
"assessment": "Continue monitoring, but recovery work can prioritize availability.\n"
}
],
"sections": [
{
"title": "Operational precautions",
"content": [
{
"type": "bullets",
"items": [
"Avoid schema changes during diagnosis.",
"Record exact timestamps for new latency spikes."
]
},
{
"type": "callout",
"title": "Escalation condition",
"kind": "critical",
"text": "Escalate immediately if replication lag exceeds five minutes."
}
]
},
{
"title": "Diagnostic procedure",
"content": [
{
"type": "steps",
"items": [
"Capture application and database metrics.",
"Correlate slow requests with query duration.",
"Test a pool-size adjustment in staging."
]
},
{
"type": "subsection",
"title": "Useful command",
"content": [
{
"type": "code",
"text": "systemctl status example-api"
}
]
}
]
}
],
"actions_title": "Next-shift plan",
"actions": [
{
"title": "Capture a diagnostic bundle",
"status": "pending",
"text": "Collect logs and metrics covering the next occurrence.",
"commands": [
"journalctl -u example-api --since '1 hour ago'"
]
},
{
"title": "Validate mitigation in staging",
"status": "pending",
"text": "Load-test the proposed connection-pool configuration."
}
],
"closing": "Service is operational with intermittent degradation. The next priority is to capture a complete occurrence and validate the pool adjustment in staging.\n",
"schema_version": 1,
"branding": {
"organization": "Example Organization",
"theme": {
"accent": "#086f83"
}
},
"appendices": [
{
"title": "Example provenance",
"content": [
{
"type": "text",
"text": "All content is synthetic."
}
]
}
]
}

View File

@ -0,0 +1,95 @@
id: INC-2026-001
title: Service Interruption Investigation
subtitle: Incident analysis and shift handoff
report_type: technical
status: open
severity: high
metadata:
Date: '2026-09-05'
Author: Your Name
Service: Example API
Environment: Production
Owner: Operations
summary: 'The API is available but intermittently slow. Database connection exhaustion
is the strongest current lead. No permanent data loss has been observed.
'
timeline:
- time: 09:15
event: Alert received
notes: Latency exceeded the five-second threshold.
- time: 09:22
event: Initial mitigation
notes: One unhealthy application instance was removed from rotation.
findings:
- title: Connection pool saturation
severity: high
status: confirmed
evidence: 'Pool utilization remained at 100 percent during each latency spike.
'
assessment: 'Requests are waiting for database connections. Query duration and pool
sizing both need review.
'
id: pool-saturation
evidence_refs:
- artifact: observations.json
pointer: /metrics/pool_utilization
label: Synthetic metric
- title: No evidence of data loss
severity: info
status: confirmed
evidence: 'Record counts and replication checks are consistent across nodes.
'
assessment: 'Continue monitoring, but recovery work can prioritize availability.
'
sections:
- title: Operational precautions
content:
- type: bullets
items:
- Avoid schema changes during diagnosis.
- Record exact timestamps for new latency spikes.
- type: callout
title: Escalation condition
kind: critical
text: Escalate immediately if replication lag exceeds five minutes.
- title: Diagnostic procedure
content:
- type: steps
items:
- Capture application and database metrics.
- Correlate slow requests with query duration.
- Test a pool-size adjustment in staging.
- type: subsection
title: Useful command
content:
- type: code
text: systemctl status example-api
actions_title: Next-shift plan
actions:
- title: Capture a diagnostic bundle
status: pending
text: Collect logs and metrics covering the next occurrence.
commands:
- journalctl -u example-api --since '1 hour ago'
- title: Validate mitigation in staging
status: pending
text: Load-test the proposed connection-pool configuration.
closing: 'Service is operational with intermittent degradation. The next priority
is to capture a complete occurrence and validate the pool adjustment in staging.
'
schema_version: 1
branding:
organization: Example Organization
theme:
accent: '#086f83'
appendices:
- title: Example provenance
content:
- type: text
text: All content is synthetic.

View File

@ -0,0 +1,8 @@
{
"schema_version": 1,
"title": "Synthetic minimal report",
"metadata": {
"Date": "2026-09-15",
"Author": "Example Author"
}
}

3
requirements.in Normal file
View File

@ -0,0 +1,3 @@
jsonschema==4.26.0
rfc3339-validator==0.1.4
PyYAML==6.0.3

26
requirements.txt Normal file
View File

@ -0,0 +1,26 @@
# This file was autogenerated by uv via the following command:
# uv pip compile --python-version 3.11 --universal requirements.in -o requirements.txt
attrs==26.1.0
# via
# jsonschema
# referencing
jsonschema==4.26.0
# via -r requirements.in
jsonschema-specifications==2025.9.1
# via jsonschema
pyyaml==6.0.3
# via -r requirements.in
referencing==0.37.0
# via
# jsonschema
# jsonschema-specifications
rfc3339-validator==0.1.4
# via -r requirements.in
rpds-py==2026.6.3
# via
# jsonschema
# referencing
six==1.17.0
# via rfc3339-validator
typing-extensions==4.16.0 ; python_full_version < '3.13'
# via referencing

523
schema/report.schema.json Normal file
View File

@ -0,0 +1,523 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Reportist structured report version 1",
"type": "object",
"properties": {
"schema_version": {
"type": "integer",
"const": 1
},
"title": {
"type": "string",
"pattern": "\\S"
},
"subtitle": {
"type": "string",
"pattern": "\\S"
},
"id": {
"type": "string",
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$(?![\\s\\S])"
},
"report_type": {
"enum": [
"technical",
"briefing"
]
},
"status": {
"type": "string",
"pattern": "\\S"
},
"severity": {
"enum": [
"info",
"low",
"medium",
"high",
"critical",
"warning",
"ok",
"success",
"unanalysed"
]
},
"metadata": {
"type": "object",
"properties": {
"Date": {
"type": "string",
"format": "date"
},
"Author": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"Date",
"Author"
],
"additionalProperties": {
"type": [
"string",
"number",
"boolean",
"null"
]
}
},
"branding": {
"type": "object",
"properties": {
"organization": {
"type": "string",
"pattern": "\\S"
},
"logo": {
"type": "string",
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.?(?:/|$))[^\\\\:\\x00-\\x1f]+$(?![\\s\\S])"
},
"theme": {
"type": "object",
"properties": {
"font": {
"type": "string",
"pattern": "\\S"
},
"mono_font": {
"type": "string",
"pattern": "\\S"
},
"ink": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"muted": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"line": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"paper_soft": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"accent": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"info": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"warning": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"critical": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
},
"success": {
"type": "string",
"pattern": "^#[0-9a-fA-F]{6}$(?![\\s\\S])"
}
},
"required": [],
"additionalProperties": false
}
},
"required": [],
"additionalProperties": false
},
"summary": {
"type": "string",
"pattern": "\\S"
},
"closing": {
"type": "string",
"pattern": "\\S"
},
"timeline": {
"type": "array",
"items": {
"type": "object",
"properties": {
"time": {
"type": "string",
"pattern": "\\S"
},
"event": {
"type": "string",
"pattern": "\\S"
},
"notes": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"time",
"event"
],
"additionalProperties": false
}
},
"findings": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string",
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$(?![\\s\\S])"
},
"title": {
"type": "string",
"pattern": "\\S"
},
"severity": {
"enum": [
"info",
"low",
"medium",
"high",
"critical",
"warning",
"ok",
"success",
"unanalysed"
]
},
"status": {
"type": "string",
"pattern": "\\S"
},
"evidence": {
"type": "string",
"pattern": "\\S"
},
"assessment": {
"type": "string",
"pattern": "\\S"
},
"evidence_refs": {
"type": "array",
"items": {
"type": "object",
"properties": {
"artifact": {
"type": "string",
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.?(?:/|$))[^\\\\:\\x00-\\x1f]+$(?![\\s\\S])"
},
"pointer": {
"type": "string",
"pattern": "^(?:/(?:[^~/]|~[01])*)*$(?![\\s\\S])"
},
"label": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"artifact",
"pointer"
],
"additionalProperties": false
},
"uniqueItems": true
}
},
"required": [
"title"
],
"additionalProperties": false
}
},
"sections": {
"type": "array",
"items": {
"$ref": "#/$defs/section"
}
},
"appendices": {
"type": "array",
"items": {
"$ref": "#/$defs/section"
}
},
"actions_title": {
"type": "string",
"pattern": "\\S"
},
"actions": {
"type": "array",
"items": {
"type": "object",
"properties": {
"title": {
"type": "string",
"pattern": "\\S"
},
"status": {
"type": "string",
"pattern": "\\S"
},
"text": {
"type": "string",
"pattern": "\\S"
},
"commands": {
"type": "array",
"items": {
"type": "string",
"pattern": "\\S"
}
}
},
"required": [
"title"
],
"additionalProperties": false
}
},
"provenance": {
"type": "object",
"properties": {
"sources": {
"type": "array",
"items": {
"type": "object",
"properties": {
"artifact": {
"type": "string",
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.?(?:/|$))[^\\\\:\\x00-\\x1f]+$(?![\\s\\S])"
},
"sha256": {
"type": "string",
"pattern": "^[a-f0-9]{64}$(?![\\s\\S])"
},
"schema_version": {
"type": "integer",
"minimum": 1
},
"producer": {
"type": "object",
"properties": {
"name": {
"type": "string",
"pattern": "\\S"
},
"version": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"name",
"version"
],
"additionalProperties": false
}
},
"required": [
"artifact"
],
"additionalProperties": false
}
},
"rendered_by": {
"type": "object",
"properties": {
"name": {
"type": "string",
"pattern": "\\S"
},
"version": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"name",
"version"
],
"additionalProperties": false
}
},
"required": [],
"additionalProperties": false
}
},
"required": [
"schema_version",
"title",
"metadata"
],
"additionalProperties": false,
"$defs": {
"section": {
"type": "object",
"properties": {
"title": {
"type": "string",
"pattern": "\\S"
},
"content": {
"type": "array",
"items": {
"$ref": "#/$defs/block"
},
"minItems": 1
}
},
"required": [
"title",
"content"
],
"additionalProperties": false
},
"block": {
"oneOf": [
{
"type": "object",
"properties": {
"type": {
"const": "text"
},
"text": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"type",
"text"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "bullets"
},
"items": {
"type": "array",
"items": {
"type": "string",
"pattern": "\\S"
},
"minItems": 1
}
},
"required": [
"type",
"items"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "steps"
},
"items": {
"type": "array",
"items": {
"type": "string",
"pattern": "\\S"
},
"minItems": 1
}
},
"required": [
"type",
"items"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "callout"
},
"title": {
"type": "string",
"pattern": "\\S"
},
"kind": {
"enum": [
"info",
"low",
"medium",
"high",
"critical",
"warning",
"ok",
"success",
"unanalysed"
]
},
"text": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"type",
"text"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "code"
},
"text": {
"type": "string",
"pattern": "\\S"
}
},
"required": [
"type",
"text"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "subsection"
},
"title": {
"type": "string",
"pattern": "\\S"
},
"content": {
"type": "array",
"items": {
"$ref": "#/$defs/block"
},
"minItems": 1
}
},
"required": [
"type",
"title",
"content"
],
"additionalProperties": false
}
]
}
}
}

114
tests/test_contract.py Normal file
View File

@ -0,0 +1,114 @@
import copy
import importlib.util
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location("report_validation", ROOT / "tools/validate.py")
validation = importlib.util.module_from_spec(spec)
spec.loader.exec_module(validation)
class ReportContractTests(unittest.TestCase):
def setUp(self):
self.document = validation.load_report(ROOT / "examples/example-report.json")
def test_yaml_json_parity_and_minimal_report(self):
self.assertEqual(self.document, validation.load_report(ROOT / "examples/example-report.yaml"))
self.assertEqual(validation.validate_document(self.document), [])
self.assertEqual(validation.validate_document(validation.load_report(ROOT / "examples/minimal-report.json")), [])
def test_optional_branding_replacement(self):
self.document["branding"] = {"organization": "Independent Example", "logo": "assets/example.svg",
"theme": {"accent": "#112233", "font": "Example Font"}}
self.assertEqual(validation.validate_document(self.document), [])
del self.document["branding"]
self.assertEqual(validation.validate_document(self.document), [])
def test_required_fields_and_versions(self):
for path, value in [(("schema_version",), 2), (("schema_version",), True),
(("title",), " "), (("metadata", "Author"), ""),
(("metadata", "Date"), "2026-02-30"),
(("metadata", "Date"), "2026-09-15T00:00:00Z")]:
with self.subTest(path=path):
doc = copy.deepcopy(self.document)
target = doc
for key in path[:-1]:
target = target[key]
target[path[-1]] = value
self.assertTrue(validation.validate_document(doc))
for key in ("Date", "Author"):
doc = copy.deepcopy(self.document)
del doc["metadata"][key]
self.assertTrue(validation.validate_document(doc))
def test_legacy_domain_fields_rejected(self):
for key in ("diagnostics", "organization", "theme", "collect"):
with self.subTest(key=key):
doc = copy.deepcopy(self.document)
doc[key] = {}
self.assertTrue(validation.validate_document(doc))
def test_all_blocks_and_unknown_nested_blocks(self):
blocks = [
{"type": "text", "text": "Text"}, {"type": "bullets", "items": ["Item"]},
{"type": "steps", "items": ["Step"]}, {"type": "callout", "text": "Note", "kind": "warning"},
{"type": "code", "text": "echo example"},
]
self.document["sections"] = [{"title": "Blocks", "content": [
{"type": "subsection", "title": "Nested", "content": blocks}]}]
self.assertEqual(validation.validate_document(self.document), [])
blocks.append({"type": "execute", "command": "anything"})
self.assertTrue(validation.validate_document(self.document))
blocks.pop()
blocks[0]["extra"] = "typo"
self.assertTrue(validation.validate_document(self.document))
def test_bad_branding_and_references(self):
for path in ("../secret.svg", "/logo.svg", "C:\\logo.svg", "assets/../logo.svg", "logo.svg\n"):
with self.subTest(path=path):
self.document["branding"]["logo"] = path
self.assertTrue(validation.validate_document(self.document))
self.setUp()
self.document["branding"]["theme"]["accent"] = "not-a-color"
self.assertTrue(validation.validate_document(self.document))
self.setUp()
self.document["findings"][0]["evidence_refs"][0]["pointer"] = "/bad~3escape"
self.assertTrue(validation.validate_document(self.document))
self.setUp()
self.document["findings"].append(copy.deepcopy(self.document["findings"][0]))
self.assertIn("duplicate finding", " ".join(validation.validate_document(self.document)))
def test_cli_parsing_errors_and_non_json_yaml(self):
with tempfile.TemporaryDirectory() as temp:
for suffix, content in [
("yaml", "title: first\ntitle: second\n"),
("yaml", "metadata: {Date: 2026-09-15}\n"),
("yaml", "value: .nan\n"), ("yaml", "value: &cycle [*cycle]\n"),
("yaml", "1: bad key\n"), ("yaml", "value: !!python/object:invalid {}\n"),
("yaml", "base: &base {a: 1}\nvalue: {<<: *base}\n"),
("json", '{"title":"first","title":"second"}'),
("json", '{"value":1e999}'), ("json", '{"value":NaN}'),
("txt", "{}"), ("yaml", "[unclosed"),
]:
with self.subTest(content=content):
path = Path(temp) / ("invalid." + suffix)
path.write_text(content)
result = subprocess.run([sys.executable, str(ROOT / "tools/validate.py"), str(path)],
cwd=temp, text=True, capture_output=True)
self.assertEqual(result.returncode, 1, result.stdout)
self.assertNotIn("Traceback", result.stderr)
def test_cli_from_another_directory(self):
with tempfile.TemporaryDirectory() as temp:
result = subprocess.run([sys.executable, str(ROOT / "tools/validate.py"),
str(ROOT / "examples/example-report.yaml")], cwd=temp, text=True, capture_output=True)
self.assertEqual(result.returncode, 0, result.stderr)
if __name__ == "__main__":
unittest.main()

117
tools/validate.py Normal file
View File

@ -0,0 +1,117 @@
#!/usr/bin/env python3
"""Validate Reportist contract v1 from a YAML or JSON report."""
import argparse
import json
import math
from pathlib import Path
import sys
try:
from jsonschema import Draft202012Validator, FormatChecker
import yaml
except ImportError:
raise SystemExit("Install validation dependencies with: python -m pip install -r requirements.txt")
SCHEMA = json.loads((Path(__file__).resolve().parents[1] / "schema/report.schema.json").read_text())
Draft202012Validator.check_schema(SCHEMA)
VALIDATOR = Draft202012Validator(SCHEMA, format_checker=FormatChecker())
def unique_object(pairs):
result = {}
for key, value in pairs:
if not isinstance(key, str):
raise ValueError("mapping keys must be strings")
if key in result:
raise ValueError(f"duplicate mapping key: {key}")
result[key] = value
return result
class ReportLoader(yaml.SafeLoader):
"""Safe YAML restricted to the JSON data model, without duplicate keys."""
def construct_mapping(loader, node):
return unique_object((loader.construct_object(k), loader.construct_object(v))
for k, v in node.value)
ReportLoader.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, construct_mapping)
def check_json_value(value, ancestors=None):
ancestors = set() if ancestors is None else ancestors
if value is None or isinstance(value, (str, bool, int)):
return
if isinstance(value, float):
if not math.isfinite(value):
raise ValueError("non-finite numbers are not supported")
return
if isinstance(value, (dict, list)):
if id(value) in ancestors:
raise ValueError("cyclic YAML aliases are not supported")
ancestors.add(id(value))
if isinstance(value, dict):
if any(not isinstance(key, str) for key in value):
raise ValueError("mapping keys must be strings")
children = value.values()
else:
children = value
for child in children:
check_json_value(child, ancestors)
ancestors.remove(id(value))
return
raise ValueError("report values must use JSON types; quote YAML dates and timestamps")
def load_report(path):
path = Path(path)
text = path.read_text(encoding="utf-8")
if path.suffix.lower() == ".json":
document = json.loads(text, object_pairs_hook=unique_object)
elif path.suffix.lower() in (".yaml", ".yml"):
document = yaml.load(text, Loader=ReportLoader)
else:
raise ValueError("expected a .yaml, .yml, or .json report")
check_json_value(document)
return document
def validate_document(document):
check_json_value(document)
errors = []
for error in VALIDATOR.iter_errors(document):
path = "/" + "/".join(str(p).replace("~", "~0").replace("/", "~1") for p in error.absolute_path)
# oneOf's headline is intentionally concise; its path locates the offending block.
errors.append(f"{path}: {error.message}")
if errors:
return sorted(errors)
ids = set()
for index, finding in enumerate(document.get("findings", [])):
if "id" in finding:
if finding["id"] in ids:
errors.append(f"/findings/{index}/id: duplicate finding ID")
ids.add(finding["id"])
return errors
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("input", type=Path)
args = parser.parse_args(argv)
try:
errors = validate_document(load_report(args.input))
except (OSError, ValueError, yaml.YAMLError, RecursionError) as error:
print(f"error: {error}", file=sys.stderr)
return 1
if errors:
print("\n".join(errors), file=sys.stderr)
return 1
print(f"Valid report v1: {args.input}")
return 0
if __name__ == "__main__":
sys.exit(main())