ci: run pinned component contract checks

This commit is contained in:
Dandaedre 2026-09-15 22:08:58 +02:00
parent 7ed97a36cf
commit 53cdfee3c5
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
6 changed files with 88 additions and 15 deletions

View File

@ -69,7 +69,7 @@ Depends on P0.5–P0.6 for the selected slice.
- [x] P1.1 **win-diagnostics:** define `schemas/diagnostic.schema.json`, schema versioning, metadata, collection status, observations, findings, warnings, and evidence references. Distinguish missing/failed checks from healthy results.
- [x] P1.2 **win-diagnostics:** document how legacy capture schemas 1 and 2 map to the new diagnostic artifact, including unsupported versions and retained raw evidence.
- [x] P1.3 **reportist:** define `schema/report.schema.json` and versioned YAML/JSON input, required/optional fields, generic content blocks, external branding/assets, and actionable validation errors. Map useful legacy report fields explicitly.
- [ ] P1.4 **beffroi-reports:** define the adapter mapping and incident structure; distinguish machine findings, human analysis, and recipient-facing conclusions. Specify how regeneration preserves human edits.
- [x] P1.4 **beffroi-reports:** define the adapter mapping and incident structure; distinguish machine findings, human analysis, and recipient-facing conclusions. Specify how regeneration preserves human edits.
- [ ] P1.5 **components:** document independent CLI input/output paths, exit codes, dependencies, and schema compatibility. Add synthetic valid/invalid contract examples and validation checks in their owning repositories.
Exit: producer and consumer agree on explicit artifacts without requiring access
@ -126,6 +126,8 @@ Depends on a working end-to-end slice from Passes 2–4. Work in beffroi-reporti
- [ ] P5.3 Add high-level workflow commands based on the component CLIs for collection, offline analysis, draft report generation, rendering, and a complete incident run. Keep implementation in components.
- [ ] P5.4 Add an integration test: sanitized capture → analysis → diagnostic artifact → report adapter → report input → Reportist → PDF. Validate schemas, expected evidence/findings, and PDF output; use isolated temporary outputs.
- [ ] P5.5 Invoke component checks and the integration test from Jenkins; report actual validation coverage, tool versions, and artifacts accurately.
- Component contract suites are wired through `make check` and Jenkins; the
end-to-end integration test remains pending.
- [ ] P5.6 Pin the tested component commits intentionally and document external dependencies plus clean-checkout reproduction commands; verify `git clone --recursive` followed by bootstrap and integration testing.
Exit: a pinned integration revision runs the sanitized pipeline reproducibly,
@ -147,6 +149,34 @@ reports remain understandable/recreatable and individual tools need no macro rep
## Pass handoff log
### 2026-09-15 — Adapter regeneration policy
- Completed P1.4 / BR1.4 as a specification in the reports component. Generated
drafts use immutable directories with exact evidence snapshots and ownership
hashes; linked regeneration detects edits and never replaces reviewed sources.
- Defined planned CLI inputs, conflict recovery, publication failure behavior and
synthetic acceptance cases. Implementation and preservation tests remain open.
- Validation: checked the contract against existing diagnostic evidence-root
handling and Reportist schema boundaries; whitespace checks passed.
- The reports component revision and integration gitlink are committed together
after validation so the regeneration policy remains reproducibly pinned.
- Next: implement the adapter and preservation tests in beffroi-reports, and
finish collection/analysis/render CLI contracts under P1.5.
### 2026-09-15 — Root component-check orchestration
- Advanced the component-check portion of P5.5 with `scripts/check-components.sh`,
`make check`, and a Jenkins stage using the same runner. P5.5 remains open until
the complete sanitized pipeline exists and runs in CI.
- Each component uses its own Python 3.11 environment and pinned requirements;
the runner prints uv/Python versions and checked-out revisions. No component
implementation or gitlink changed.
- Validation: `make check` passed all 10 diagnostic and eight report contract
tests with Python 3.11.14 and uv 0.12.13. Jenkins itself was not executed.
- Documented setup, dependency downloads, isolated environments, and coverage.
- Next: finish the CLI contracts and draft regeneration policy, then implement
diagnostic, rendering, and adapter slices before the full integration test.
### 2026-09-15 — Initial planning pass
- Completed: P0.1–P0.4. Added this tracker and corrected root README navigation/layout descriptions.

19
Jenkinsfile vendored
View File

@ -76,18 +76,11 @@ pipeline {
* We therefore intentionally do not require them here forever.
*/
/*
* TODO: Component CI
*
* Once component-level validation commands exist, invoke them
* here rather than duplicating their implementation.
*
* For example:
*
* diagnostics/... validate
* reporter/... validate
* reports/... validate
*/
stage('Component Contract Checks') {
steps {
sh 'bash scripts/check-components.sh'
}
}
/*
* TODO: Integration Test
@ -159,7 +152,7 @@ pipeline {
conclusion: 'SUCCESS',
name: 'Beffroi Reporting CI',
summary: 'Beffroi Reporting workspace passed all currently implemented CI checks.',
text: 'The macro repository and its component submodules were checked out and validated successfully.',
text: 'Workspace and diagnostic/report contract checks passed. Analysis, adapter, and PDF integration checks remain pending.',
title: 'Reporting workspace healthy'
)

4
Makefile Normal file
View File

@ -0,0 +1,4 @@
.PHONY: check
check:
bash scripts/check-components.sh

View File

@ -49,6 +49,24 @@ For an existing checkout:
git submodule update --init --recursive
```
## Run component checks
On Linux/macOS with Bash, Git, Make, and `uv` available, run:
```bash
make check
```
The same runner is used by Jenkins (`bash scripts/check-components.sh`). It
creates separate Python 3.11 environments under `.venv/component-checks/` and
synchronizes each component's pinned requirements. On first use, uv may download
Python and dependencies, requiring network access. No system packages are changed.
The runner prints tool versions and component revisions and stops on any failure.
Current coverage is the diagnostic and report contract suites. The reports adapter
has no executable suite yet; this command does not verify analysis or PDF output.
Submodules must already be initialized with the command above.
## Push component commits and the workspace together
After committing changes inside each component and committing their updated

@ -1 +1 @@
Subproject commit 79f93e43d5917e5b2d22c647c6e73dc98b6c3108
Subproject commit 12970575d33a6d166828ccabdba63ddecc1abf9e

View File

@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Run the checks owned by the pinned components in separate environments.
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/.."
command -v uv >/dev/null 2>&1 || {
echo 'uv is required to prepare isolated component test environments.' >&2
exit 1
}
for component in win-diagnostics reportist; do
if [[ ! -f "$component/requirements.txt" || ! -f "$component/tests/test_contract.py" ]]; then
echo "Missing $component checks. Run: git submodule update --init --recursive" >&2
exit 1
fi
done
uv --version
for component in win-diagnostics reportist; do
echo "Checking $component ($(git -C "$component" rev-parse HEAD))"
environment=".venv/component-checks/$component"
uv venv --python 3.11 --allow-existing "$environment"
uv pip sync --python "$environment/bin/python" "$component/requirements.txt"
"$environment/bin/python" --version
"$environment/bin/python" -m unittest discover -s "$component/tests" -p 'test_*.py' -v
done
echo 'Component contract checks passed. Full pipeline and PDF checks remain pending.'