diff --git a/AGENT-TASKS.md b/AGENT-TASKS.md index cbcd14f..c38cff2 100644 --- a/AGENT-TASKS.md +++ b/AGENT-TASKS.md @@ -1,7 +1,7 @@ # Repository roles and build consolidation -This is a plan, not an implementation checklist already in progress. Preserve the -current 17-PDF legacy parity baseline during the migration. +The component move is implemented. Preserve the 17-PDF legacy parity baseline +while finishing CLI cleanup and external CI verification. ## Target ownership @@ -30,33 +30,33 @@ Proposed command mapping (names can change before implementation): ## Tasks, in dependency order -- [ ] **T1. Freeze the contract.** Document which files in `reports` are authored +- [x] **T1. Freeze the contract.** Document which files in `reports` are authored reports and which JSON files are raw evidence. Specify the `reportist/build.sh` subcommands, argument/exit-code conventions, `--force`/`--replace` behavior, and whether batch commands preserve relative paths. Test these contracts from a standalone Reportist checkout. -- [ ] **T2. Move diagnostic implementation to Reportist.** Move +- [x] **T2. Move diagnostic implementation to Reportist.** Move `reports/plugins/diagnostics/{renderer,tools,runtime,tests}` and its entry scripts into `reporter/plugins/diagnostics`, keeping the normalized evidence schema/rules and analyzer tests. Resolve shared `visual/` locally in Reportist rather than through a sibling path or `BEFFROI_VISUAL_ROOT`. Keep existing PDF parity and evidence validation tests passing. -- [ ] **T3. Add one Reportist CLI.** Make `reporter/build.sh` the supported route +- [x] **T3. Add one Reportist CLI.** Make `reporter/build.sh` the supported route for single report/diagnostic render, batch render, analysis, validation, and YAML-to-JSON export. Move common option parsing, path normalization, safe output staging, and overwrite handling into one helper used by its commands. Avoid a generic schema/IR that constrains plugin document structure. -- [ ] **T4. Thin the macro launcher.** Map `beffroi.sh` directly to the new CLI. +- [x] **T4. Thin the macro launcher.** Map `beffroi.sh` directly to the new CLI. Keep only workspace-specific publication orchestration, packaging, tests, bootstrap and doctor here. Remove `scripts/render-capture.sh` and the redundant routing in `scripts/workflow.sh`; decide whether `workflow-all.sh` is needed after `render-all` and `analyze-all` land in Reportist. -- [ ] **T5. Make Reports content-only.** Remove scripts, diagnostic plugin, +- [x] **T5. Make Reports content-only.** Remove scripts, diagnostic plugin, tests, schema/rules, visual assets, and Jenkinsfile from `reports` after Reportist owns and tests them. Keep authored source files and useful source provenance. Retire historical adapter docs or move them to an archive outside the content repository. Update submodule pins and links. -- [ ] **T6. Consolidate CI and publication.** Macro Jenkinsfile invokes +- [x] **T6. Consolidate CI and publication.** Macro Jenkinsfile invokes Reportist's CLI through `beffroi.sh`, tests all sources, builds staged output, packages and publishes it. Remove component Jenkinsfiles if independent component CI is no longer required; otherwise keep them test-only and update diff --git a/README.md b/README.md index e5b8d9b..6e669f4 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,14 @@ # Beffroi Reporting Beffroi Reporting builds PDF reports from authored YAML/JSON and raw -WinDiagnostics captures. The report and diagnostic plugins own their document +WinDiagnostics captures. The Reportist plugins own their document structure separately. Both import the visual elements and palettes in [`reporter/visual`](reporter/visual). +The `reports` component contains only source content. Reportist provides the +standalone `reporter/build.sh` CLI for either document type; this macro checkout +owns edition, integration checks, encrypted packaging, and publication CI. + ## Render one report ```bash @@ -25,7 +29,7 @@ The report plugin is in [`reporter/plugins/report`](reporter/plugins/report). `diagnostic-pdf` and `capture-pdf` accept **raw WinDiagnostics JSON v1 or v2**. Set `DIAGNOSTICS_THEME=default`, `cic`, or `eridani`, and optionally `DIAGNOSTICS_AUTHOR`, before rendering. The diagnostic plugin is in -[`reports/plugins/diagnostics`](reports/plugins/diagnostics). Its PDF renderer +[`reporter/plugins/diagnostics`](reporter/plugins/diagnostics). Its PDF renderer uses the raw capture directly; its separate analyzer can also emit normalized diagnostic JSON v1 for evidence processing: diff --git a/beffroi.sh b/beffroi.sh index edc5760..15a7cb2 100755 --- a/beffroi.sh +++ b/beffroi.sh @@ -29,19 +29,28 @@ case "$command_name" in report) (($# >= 2))||usage 2; require typst report_input=$1; report_output=$2; shift 2 - reporter/render.sh "$report_input" --output "$report_output" "$@" + reporter/build.sh render report "$report_input" --output "$report_output" "$@" ;; diagnostic-pdf) [[ $# -eq 2 ]]||usage 2; require typst - reports/plugins/diagnostics/render.sh "$1" --output "$2" + reporter/build.sh render diagnostic "$1" --output "$2" ;; capture-pdf) [[ $# -eq 2 ]]||usage 2; require git typst - scripts/render-capture.sh "$1" "$2" + reporter/build.sh render diagnostic "$1" --output "$2" ;; diagnostic) [[ $# -ge 4 && $# -le 6 ]]||usage 2; require git typst - scripts/workflow.sh all "$@" + capture=$1 evidence_root=$2 output_dir=$3 reference_time=$4 + author=${5:-${DIAGNOSTICS_AUTHOR:-Beffroi}} + theme=${6:-${DIAGNOSTICS_THEME:-default}} + [[ ! -e "$output_dir" ]] || { echo "$output_dir: output directory exists" >&2; exit 1; } + mkdir -p "$output_dir" + reporter/build.sh analyze "$capture" --evidence-root "$evidence_root" \ + --output "$output_dir/diagnostic.json" --reference-time "$reference_time" \ + --producer-version "$(git -C reporter rev-parse HEAD)" + reporter/build.sh render diagnostic "$capture" --output "$output_dir/report.pdf" \ + --author "$author" --theme "$theme" ;; all) (($# >= 1))||usage 2; require git typst diff --git a/docs/development.md b/docs/development.md index c204d9a..519a5f7 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,7 +1,7 @@ # Development and parity -`beffroi.sh` routes authored reports to `reporter/render.sh` and raw -WinDiagnostics captures to `reports/plugins/diagnostics/render.sh`. Each stages +`beffroi.sh` routes authored reports and raw WinDiagnostics captures through +`reporter/build.sh`. Each plugin stages its own Typst document model. The shared `reporter/visual/elements.typ` module contains palette, badge, heading, panel, callout, code-block, and progress-bar primitives. Report and diagnostic document assembly remain independent. @@ -16,7 +16,7 @@ counts; Poppler adds its original text check and page raster comparison when available. The recorded hashes guard CI builds without the ignored legacy directory or Poppler tools. -The diagnostic analyzer at `reports/plugins/diagnostics/analyse.sh` produces a +The diagnostic analyzer at `reporter/plugins/diagnostics/analyse.sh` produces a normalized evidence artifact. It is separate from the PDF renderer. The `diagnostic` workflow creates that artifact and a PDF from the same raw capture. diff --git a/docs/remaining-gates.md b/docs/remaining-gates.md deleted file mode 100644 index 2e838c8..0000000 --- a/docs/remaining-gates.md +++ /dev/null @@ -1,28 +0,0 @@ -# Remaining external gates - -All locally reproducible contract, analysis, adapter, renderer, bootstrap, and -integration work is implemented. The following tasks require evidence or execution -that is not available on the Linux workspace and must not be fabricated. - -## Native Windows collection - -Collector migration and validation require a supported Windows host, PowerShell -5.1+, standard and administrator runs, Compact/Mid/Full captures, and review of -unavailable native APIs. Completion evidence must record Windows/PowerShell -versions, privilege level, commands, sanitized capture structure, and failures. - -## Historical incidents - -A report owner must select the representative private incident and confirm its -evidence, chronology, conclusions, original deliverable, and missing provenance. -Only then can incident-specific policy and the remaining archive be migrated and -compared. Unknown facts must remain unknown. - -## Production publication - -Publication remains disabled. Before enabling it, an owner must provide the -approved destination/credentials and review generated deliverables against the -preserved production reports, including recovery behavior. Encrypted packaging is -not retained in the current workflow because generated PDFs are local reproducible -artifacts and there is no approved publication channel. Reconsider packaging only -with an explicit distribution requirement and configured secrets. diff --git a/reporter b/reporter index 70f763a..4bdac9f 160000 --- a/reporter +++ b/reporter @@ -1 +1 @@ -Subproject commit 70f763a65a7f747f421030a5bfc56611987b90b1 +Subproject commit 4bdac9fd93e7be67bf74a920c875d40510a517f5 diff --git a/reports b/reports index 21cf355..eb2d4d4 160000 --- a/reports +++ b/reports @@ -1 +1 @@ -Subproject commit 21cf355d730e1c56ac39350f9cb839ac6a8fabaa +Subproject commit eb2d4d40b954a3e7f24d2ac8af93cc63175c6d4b diff --git a/scripts/build-publication.sh b/scripts/build-publication.sh index 699ec07..4c51f7c 100755 --- a/scripts/build-publication.sh +++ b/scripts/build-publication.sh @@ -39,20 +39,11 @@ cleanup() { trap cleanup EXIT mkdir -p "$stage/reports" "$stage/diagnostics" -reporter/tools/render-all.sh reports/reports --output-dir "$stage/reports/pdfs" -count=0 -while IFS= read -r -d '' source; do - relative=${source#reports/diagnostics/} - output="$stage/diagnostics/${relative%.json}.pdf" - mkdir -p "$(dirname "$output")" - scripts/render-capture.sh "$source" "$output" - ((count += 1)) -done < <(find reports/diagnostics -type f -name '*.json' -print0) -((count > 0)) || { echo 'No diagnostic captures found' >&2; exit 1; } -echo "Rendered $count diagnostic PDFs into $output_root/diagnostics" +reporter/build.sh render-all report reports/reports --output-dir "$stage/reports/pdfs" +reporter/build.sh render-all diagnostic reports/diagnostics --output-dir "$stage/diagnostics" if [[ "$include_json" == true ]]; then - reporter/tools/export-json-all.sh reports/reports --output-dir "$stage/reports/pdfs" + reporter/build.sh export-json-all reports/reports --output-dir "$stage/reports/pdfs" while IFS= read -r -d '' source; do relative=${source#reports/diagnostics/} mkdir -p "$stage/diagnostics/$(dirname "$relative")" diff --git a/scripts/check-components.sh b/scripts/check-components.sh index 96342fd..04513c0 100755 --- a/scripts/check-components.sh +++ b/scripts/check-components.sh @@ -5,14 +5,13 @@ cd "$(dirname "${BASH_SOURCE[0]}")/.." for command in git typst python3 uv; do command -v "$command" >/dev/null 2>&1 || { echo "Missing test dependency: $command" >&2; exit 1; } done -for component in reporter reports/plugins/diagnostics reports; do - "$component/tests/test.sh" -done +reporter/tests/test.sh +python3 -m unittest discover -s scripts/tests -p 'test_*.py' -v work=$(mktemp -d "${TMPDIR:-/tmp}/beffroi-check.XXXXXXXX") trap 'rm -rf -- "$work"' EXIT -scripts/workflow.sh all \ - reports/plugins/diagnostics/tests/fixtures/legacy/windiagnostics-v1.json \ - reports/plugins/diagnostics/tests/fixtures/legacy \ +./beffroi.sh diagnostic \ + reporter/plugins/diagnostics/tests/fixtures/legacy/windiagnostics-v1.json \ + reporter/plugins/diagnostics/tests/fixtures/legacy \ "$work/workflow" 2026-09-15T12:00:00Z cp scripts/check-diagnostic.typ "$work/workflow/check-diagnostic.typ" typst eval --root "$work/workflow" --input diagnostic=diagnostic.json \ diff --git a/scripts/check-diagnostic-parity.py b/scripts/check-diagnostic-parity.py deleted file mode 100644 index 5b96a06..0000000 --- a/scripts/check-diagnostic-parity.py +++ /dev/null @@ -1,13 +0,0 @@ -#!/usr/bin/env python3 -"""Compatibility entry point for the complete original-PDF parity check.""" -from pathlib import Path -import subprocess -import tempfile - -root = Path(__file__).resolve().parents[1] -with tempfile.TemporaryDirectory(prefix="beffroi-parity.") as directory: - output = Path(directory) / "publication" - subprocess.run([str(root / "scripts/build-publication.sh"), str(output)], check=True, - stdout=subprocess.DEVNULL) - subprocess.run(["uv", "run", "--isolated", "--no-project", "--with", "pypdf==6.7.5", - "python3", str(root / "scripts/check-legacy-parity.py"), str(output)], check=True) diff --git a/scripts/check-regular-parity.py b/scripts/check-regular-parity.py deleted file mode 100644 index 5b96a06..0000000 --- a/scripts/check-regular-parity.py +++ /dev/null @@ -1,13 +0,0 @@ -#!/usr/bin/env python3 -"""Compatibility entry point for the complete original-PDF parity check.""" -from pathlib import Path -import subprocess -import tempfile - -root = Path(__file__).resolve().parents[1] -with tempfile.TemporaryDirectory(prefix="beffroi-parity.") as directory: - output = Path(directory) / "publication" - subprocess.run([str(root / "scripts/build-publication.sh"), str(output)], check=True, - stdout=subprocess.DEVNULL) - subprocess.run(["uv", "run", "--isolated", "--no-project", "--with", "pypdf==6.7.5", - "python3", str(root / "scripts/check-legacy-parity.py"), str(output)], check=True) diff --git a/scripts/render-capture.sh b/scripts/render-capture.sh deleted file mode 100755 index 825cd7f..0000000 --- a/scripts/render-capture.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/usr/bin/env bash -# Render the raw WinDiagnostics capture with the diagnostic Typst plugin. -set -euo pipefail -cd "$(dirname "${BASH_SOURCE[0]}")/.." -[[ $# -eq 2 ]] || { echo "Usage: $0 CAPTURE OUTPUT.pdf" >&2; exit 2; } -reports/plugins/diagnostics/render.sh "$1" --output "$2" diff --git a/scripts/tests/test_sources.py b/scripts/tests/test_sources.py new file mode 100644 index 0000000..1cf4fa4 --- /dev/null +++ b/scripts/tests/test_sources.py @@ -0,0 +1,31 @@ +from pathlib import Path +import json +import os +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[2] + + +class SourceTests(unittest.TestCase): + def test_stored_sources_have_expected_sets(self): + self.assertEqual(len(list((ROOT / "reports/reports").glob("*.yaml"))), 7) + self.assertEqual(len(list((ROOT / "reports/diagnostics").glob("*.json"))), 10) + for path in (ROOT / "reports/diagnostics").glob("*.json"): + document = json.loads(path.read_text()) + self.assertEqual(document["report_type"], "windows_system_maintenance") + + def test_diagnostic_plugin_renders_raw_capture(self): + capture = next((ROOT / "reports/diagnostics").glob("*.json")) + with tempfile.TemporaryDirectory() as directory: + output = Path(directory) / "diagnostic.pdf" + result = subprocess.run([str(ROOT / "reporter/build.sh"), "render", "diagnostic", str(capture), + "--output", str(output)], text=True, capture_output=True, + env=os.environ.copy()) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertTrue(output.read_bytes().startswith(b"%PDF-")) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/workflow-all.sh b/scripts/workflow-all.sh deleted file mode 100755 index 4230942..0000000 --- a/scripts/workflow-all.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env bash -# Build normalized diagnostic artifacts and source-faithful PDFs in bulk. -set -euo pipefail -cd "$(dirname "${BASH_SOURCE[0]}")/.." -[[ $# -ge 4 && $# -le 6 ]] || { - echo "Usage: $0 CAPTURE_DIR EVIDENCE_ROOT NEW_OUTPUT_ROOT REFERENCE_TIME [AUTHOR] [THEME]" >&2 - exit 2 -} -capture_dir=$(realpath "$1") evidence_root=$(realpath "$2") output_root=$3 reference_time=$4 -author=${5:-${DIAGNOSTICS_AUTHOR:-Beffroi}} theme=${6:-${DIAGNOSTICS_THEME:-default}} -[[ ! -e "$output_root" ]] || { echo "$output_root: output root exists" >&2; exit 1; } -mkdir -p "$output_root/diagnostics" "$output_root/pdfs" -reports/plugins/diagnostics/analyse-all.sh "$capture_dir" --evidence-root "$evidence_root" \ - --output-dir "$output_root/diagnostics" --reference-time "$reference_time" \ - --producer-version "$(git -C reports rev-parse HEAD)" -count=0 -while IFS= read -r -d '' capture; do - relative=${capture#"$capture_dir/"} - output="$output_root/pdfs/${relative%.json}.pdf" - mkdir -p "$(dirname "$output")" - reports/plugins/diagnostics/render.sh "$capture" --output "$output" --author "$author" --theme "$theme" - ((count+=1)) -done < <(find "$capture_dir" -type f -name '*.json' -print0) -echo "Created $count diagnostic artifacts and PDFs under $output_root" diff --git a/scripts/workflow.sh b/scripts/workflow.sh deleted file mode 100755 index 944db31..0000000 --- a/scripts/workflow.sh +++ /dev/null @@ -1,35 +0,0 @@ -#!/usr/bin/env bash -# Compose diagnostic analysis with the independent diagnostic renderer. -set -euo pipefail -cd "$(dirname "${BASH_SOURCE[0]}")/.." -usage() { - echo "Usage: $0 analyse CAPTURE EVIDENCE_ROOT OUTPUT.json REFERENCE_TIME" >&2 - echo " $0 render REPORT.yaml|json OUTPUT.pdf" >&2 - echo " $0 all CAPTURE EVIDENCE_ROOT NEW_OUTPUT_DIR REFERENCE_TIME [AUTHOR] [THEME]" >&2 - exit 2 -} -command_name=${1:-}; [[ -n "$command_name" ]] || usage; shift -case $command_name in - analyse) - [[ $# -eq 4 ]] || usage - reports/plugins/diagnostics/analyse.sh "$1" --evidence-root "$2" --output "$3" \ - --reference-time "$4" --producer-version "$(git -C reports rev-parse HEAD)" - ;; - render) - [[ $# -eq 2 ]] || usage - reporter/render.sh "$1" --output "$2" - ;; - all) - [[ $# -ge 4 && $# -le 6 ]] || usage - capture=$1 evidence_root=$2 output_dir=$3 reference_time=$4 - author=${5:-${DIAGNOSTICS_AUTHOR:-Beffroi}} theme=${6:-${DIAGNOSTICS_THEME:-default}} - [[ ! -e "$output_dir" ]] || { echo "$output_dir: output directory exists" >&2; exit 1; } - mkdir -p "$output_dir" - reports/plugins/diagnostics/analyse.sh "$capture" --evidence-root "$evidence_root" \ - --output "$output_dir/diagnostic.json" --reference-time "$reference_time" \ - --producer-version "$(git -C reports rev-parse HEAD)" - reports/plugins/diagnostics/render.sh "$capture" --output "$output_dir/report.pdf" \ - --author "$author" --theme "$theme" - ;; - *) usage;; -esac