From 24778d4652c2e2794d1b8e8c6f5592dbbe7655f7 Mon Sep 17 00:00:00 2001 From: Dandaedre Date: Thu, 17 Sep 2026 00:10:54 +0200 Subject: [PATCH] Consolidate Beffroi vision architecture and operations docs --- README.md | 92 +++++++++++++------------------------------- docs/architecture.md | 65 +++++++++++++++++++++++++++++++ docs/development.md | 25 ------------ docs/operations.md | 59 ++++++++++++++++++++++++++++ docs/vision.md | 42 ++++++++++++++++++++ reporter | 2 +- reports | 2 +- 7 files changed, 194 insertions(+), 93 deletions(-) create mode 100644 docs/architecture.md delete mode 100644 docs/development.md create mode 100644 docs/operations.md create mode 100644 docs/vision.md diff --git a/README.md b/README.md index 6e669f4..605f9a6 100644 --- a/README.md +++ b/README.md @@ -1,73 +1,33 @@ # Beffroi Reporting -Beffroi Reporting builds PDF reports from authored YAML/JSON and raw -WinDiagnostics captures. The Reportist plugins own their document -structure separately. Both import the visual elements and palettes in -[`reporter/visual`](reporter/visual). - -The `reports` component contains only source content. Reportist provides the -standalone `reporter/build.sh` CLI for either document type; this macro checkout -owns edition, integration checks, encrypted packaging, and publication CI. - -## Render one report +This repository is the editing and publishing workspace for Beffroi reports. +It pins two components: **Reports** holds authored YAML and raw diagnostic JSON +sources; **Reportist** turns those sources into PDFs with independent Typst +plugins and shared graphical elements. The macro pipeline checks, packages, +and publishes the results. ```bash -./beffroi.sh report reports/reports/2026-09-04-clevo-handoff.yaml report.pdf -``` - -Authored reports use `organization`, `theme`, `metadata`, `summary`, `sections`, -`findings`, `actions`, and any report-specific fields. Available named palettes -are `default`, `cic`, and `eridani`; a custom theme dictionary is also accepted. -The report plugin is in [`reporter/plugins/report`](reporter/plugins/report). - -## Render one diagnostic - -```bash -./beffroi.sh diagnostic-pdf reports/diagnostics/windows_system_report_DESKTOP-SAU019H_20260915_121057.json diagnostic.pdf -``` - -`diagnostic-pdf` and `capture-pdf` accept **raw WinDiagnostics JSON v1 or v2**. -Set `DIAGNOSTICS_THEME=default`, `cic`, or `eridani`, and optionally -`DIAGNOSTICS_AUTHOR`, before rendering. The diagnostic plugin is in -[`reporter/plugins/diagnostics`](reporter/plugins/diagnostics). Its PDF renderer -uses the raw capture directly; its separate analyzer can also emit normalized -diagnostic JSON v1 for evidence processing: - -```bash -./beffroi.sh diagnostic CAPTURE.json EVIDENCE_ROOT NEW_OUTPUT_DIR REFERENCE_TIME [AUTHOR] [THEME] -``` - -That command writes `diagnostic.json` and `report.pdf`. - -## Build the publication - -```bash -./beffroi.sh all build/publication -./beffroi.sh all build/publication --replace --include-json -``` - -The build creates seven authored report PDFs and ten diagnostic PDFs from the -stored sources. `--include-json` exports authored report JSON and copies raw -diagnostic JSON alongside the PDFs. `--replace` stages a complete build before -replacing an existing output directory. - -Run `./beffroi.sh test` to test the plugins and compare every published PDF with -the original Typst renderer. When `legacy/typst_reporter/build` is present, the -comparison checks extracted text, page counts, and rasterized pages. Without -that local reference, it checks recorded original text hashes and page counts. - -## Commands - -```text -./beffroi.sh report INPUT.yaml|json OUTPUT.pdf [--force] -./beffroi.sh diagnostic-pdf CAPTURE.json OUTPUT.pdf -./beffroi.sh capture-pdf CAPTURE.json OUTPUT.pdf -./beffroi.sh diagnostic CAPTURE EVIDENCE_ROOT OUTPUT_DIR REFERENCE_TIME [AUTHOR] [THEME] -./beffroi.sh all OUTPUT_DIR [--include-json] [--replace] -./beffroi.sh package BUILD_DIR ARTIFACT_DIR -./beffroi.sh test ./beffroi.sh bootstrap -./beffroi.sh doctor +./beffroi.sh report reports/reports/2026-09-04-clevo-handoff.yaml /tmp/report.pdf +./beffroi.sh diagnostic-pdf reports/diagnostics/windows_system_report_DESKTOP-SAU019H_20260915_121057.json /tmp/diagnostic.pdf +./beffroi.sh test +./beffroi.sh all build/publication --replace ``` -See [`docs/development.md`](docs/development.md) for implementation details. +`beffroi.sh diagnostic CAPTURE EVIDENCE_ROOT NEW_OUTPUT_DIR REFERENCE_TIME +[AUTHOR] [THEME]` also creates normalized evidence JSON. `capture-pdf` is an +alias for `diagnostic-pdf`. `--include-json` on `all` includes source JSON in +the publication. `beffroi.sh package BUILD_DIR ARTIFACT_DIR` creates encrypted +archives when `REPORT_ZIP_PASSWORD` is set. + +## Documentation + +- [Vision](docs/vision.md): goals, repository roles, and design rules. +- [Architecture](docs/architecture.md): components, data formats, and flow. +- [Operations](docs/operations.md): editing, verification, CI, and delivery. +- [Reportist CLI](reporter/docs/build-cli.md) and [plugin guide](reporter/docs/plugins.md): render commands and how to add a plugin. +- [Report source contract](reporter/docs/report-contract.md) and [source provenance](reports/docs/preserved-source.md): input expectations and origin. + +Run `./beffroi.sh doctor` to check local tools. `./beffroi.sh --help` lists all +workspace commands. The original Typst project under `legacy/typst_reporter` +is retained locally as the visual parity reference for the 17 stored PDFs. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..331c953 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,65 @@ +# Architecture + +```mermaid +flowchart LR + S[Reports: YAML and raw JSON sources] --> M[Macro: beffroi.sh and publication build] + M --> C[Reportist: build.sh] + C --> R[Report plugin] + C --> D[Diagnostic plugin] + V[Shared visual elements and themes] --> R + V --> D + D --> A[Normalized evidence JSON, when requested] + R --> P[PDFs] + D --> P + P --> Z[Macro: encrypted archives and Jenkins publication] +``` + +## Component contracts + +**Reports** stores authored YAML reports in `reports/reports/` and raw +WinDiagnostics JSON captures in `reports/diagnostics/`. The raw captures are +source evidence for the diagnostic renderer. This repository contains no +plugin, analyzer, visual asset, test runner, or Jenkinsfile. Source provenance +is documented in `reports/docs/preserved-source.md`. + +**Reportist** is the rendering engine. Its public entry point is +`reporter/build.sh`; see its [CLI contract](../reporter/docs/build-cli.md). +`reporter/plugins/report/` interprets authored reports. +`reporter/plugins/diagnostics/` interprets raw captures and, separately, +produces normalized evidence JSON. These plugins have independent Typst +models. Both import `reporter/visual/elements.typ` and use the palettes in +`reporter/visual/themes.yaml`. `reporter/tools/render-common.sh` shares shell +option, path, and temporary staging rules. Each renderer stages its own Typst +model and resources so it can compile without the macro repository. + +**The macro repository** pins both components as submodules. `beffroi.sh` +provides convenient commands for editing and checking one source. +`scripts/build-publication.sh` builds all stored sources into a new staged +publication and replaces an existing build only after rendering succeeds. +`scripts/package-reports.sh` creates AES-encrypted `Reports.zip` and +`Diagnostics.zip`. The root `Jenkinsfile` tests, builds, packages, archives, +and publishes those archives from `main`. + +## Data and output boundaries + +| Input or output | Format | Owner | +| --- | --- | --- | +| Authored report | YAML or JSON with fields defined by the report plugin | Reports content; Reportist interpretation | +| Raw capture | WinDiagnostics JSON v1/v2 | Reports content; diagnostic plugin interpretation | +| Normalized diagnostic | Diagnostic evidence JSON v1 | Diagnostic analyzer and validator | +| PDF | Plugin-specific layout | Reportist renderer | +| Publication directory and encrypted ZIPs | Batch artifacts | Macro repository | + +A normalized diagnostic is an evidence artifact. The diagnostic PDF is rendered +from the raw capture, not from a report-shaped intermediate representation. +Neither plugin is required to conform to the other's schema. New plugin kinds +are routed explicitly in `reporter/build.sh`; there is no automatic registry. + +## Verification + +`./beffroi.sh test` runs the component suites, a diagnostic analysis workflow, +and a full publication build. The parity checker verifies the set of 17 PDFs, +recorded text hashes, and page counts. When original PDFs and Poppler are +available locally, it also compares rasterized pages. New plugins should add +focused input/render tests; an approved existing output needs a regression +oracle before its renderer is changed. diff --git a/docs/development.md b/docs/development.md deleted file mode 100644 index 519a5f7..0000000 --- a/docs/development.md +++ /dev/null @@ -1,25 +0,0 @@ -# Development and parity - -`beffroi.sh` routes authored reports and raw WinDiagnostics captures through -`reporter/build.sh`. Each plugin stages -its own Typst document model. The shared `reporter/visual/elements.typ` module -contains palette, badge, heading, panel, callout, code-block, and progress-bar -primitives. Report and diagnostic document assembly remain independent. - -The original renderer is retained locally under `legacy/typst_reporter`. Run its -`render.sh --all` and `render-diagnostics.sh --all Beffroi` scripts to refresh -the reference PDFs. `./beffroi.sh test` builds the stored publication and runs -`scripts/check-legacy-parity.py` against all 17 PDFs. With the local reference -available, the check compares every page raster as well as extracted text and -pagination. The portable CI check uses pinned `pypdf` via `uv` for text and page -counts; Poppler adds its original text check and page raster comparison when -available. The recorded hashes guard CI builds without the ignored legacy -directory or Poppler tools. - -The diagnostic analyzer at `reporter/plugins/diagnostics/analyse.sh` produces a -normalized evidence artifact. It is separate from the PDF renderer. The -`diagnostic` workflow creates that artifact and a PDF from the same raw capture. - -`./beffroi.sh all OUTPUT_DIR --replace` stages a new publication next to the -output directory and replaces it only after all PDFs are ready. `--include-json` -adds JSON source exports. diff --git a/docs/operations.md b/docs/operations.md new file mode 100644 index 0000000..4a23ab8 --- /dev/null +++ b/docs/operations.md @@ -0,0 +1,59 @@ +# Editing, building, and publishing + +## Setup and editing + +Run `./beffroi.sh bootstrap` after cloning to initialize the pinned Reports +and Reportist submodules. `./beffroi.sh doctor` checks the core tools. Edit +source files in `reports/reports/` or add a raw capture to +`reports/diagnostics/`; implementation changes belong in Reportist. Consult +[the report input contract](../reporter/docs/report-contract.md) for authored +YAML/JSON and [the diagnostic plugin](../reporter/plugins/diagnostics/README.md) +for raw capture behavior. + +Render one source while editing: + +```bash +./beffroi.sh report reports/reports/2026-09-04-clevo-handoff.yaml /tmp/report.pdf +./beffroi.sh diagnostic-pdf reports/diagnostics/windows_system_report_DESKTOP-SAU019H_20260915_121057.json /tmp/diagnostic.pdf +``` + +`capture-pdf` is a compatibility alias for `diagnostic-pdf`. For evidence +processing, `./beffroi.sh diagnostic CAPTURE EVIDENCE_ROOT NEW_OUTPUT_DIR +REFERENCE_TIME [AUTHOR] [THEME]` writes both `diagnostic.json` and +`report.pdf`. The PDF renderer reads the raw capture directly. + +## Local verification + +Run `./beffroi.sh test` before publishing. It executes Reportist's report and +diagnostic suites, checks the stored source set, builds all PDFs, and compares +all seven reports and ten diagnostics to the recorded legacy baseline. The +original renderer in `legacy/typst_reporter` is retained as a read-only local +reference. Its ignored `build/` PDFs enable raster comparison when available; +CI can still check recorded page counts and text hashes without them. + +Build the complete publication with: + +```bash +./beffroi.sh all build/publication --replace +./beffroi.sh all build/publication --replace --include-json +``` + +`--replace` builds beside the existing directory, then swaps it after success. +`--include-json` adds authored report JSON exports and copies raw capture JSON +beside the PDFs. Single-document rendering refuses an existing PDF unless +`--force` is supplied. + +## CI and delivery + +On `main`, the macro Jenkinsfile checks out pinned submodules, runs tests, +builds the publication, creates password-protected AES ZIP archives, archives +them as CI artifacts, and copies `Reports.zip` and `Diagnostics.zip` to its +configured deployment directory. Jenkins supplies `REPORT_ZIP_PASSWORD` from +credentials. The password must not be committed or printed. The standalone +Reportist Jenkinsfile runs plugin tests; Reports has no Jenkinsfile. + +`./beffroi.sh package BUILD_DIR ARTIFACT_DIR` runs the same packaging locally +when `REPORT_ZIP_PASSWORD` is set. It requires `uv` and pinned `pyzipper`. +Package output is checked by reopening the archives. Publication and deployment +are external actions; review generated sources and artifacts before pushing a +change that triggers the main-branch pipeline. diff --git a/docs/vision.md b/docs/vision.md new file mode 100644 index 0000000..0b157c5 --- /dev/null +++ b/docs/vision.md @@ -0,0 +1,42 @@ +# Project vision + +Beffroi Reporting turns reviewed source material into reproducible PDF reports. +An authored report and a machine capture are different kinds of source; each +gets a renderer that understands its own meaning and layout. They can share a +visual language without sharing a forced document schema. + +## The three repositories + +| Repository | Owns | Does not own | +| --- | --- | --- | +| Beffroi Reporting (macro) | Editing workspace, pinned components, integration checks, publication staging, encrypted archives, deployment CI | Report layouts or diagnostic rules | +| Beffroi Reports | Authored YAML/JSON and raw diagnostic captures, with editorial provenance | Executable rendering or analysis code | +| Reportist (`reporter/` here) | Standalone renderers, diagnostic analysis, validation, shared graphical elements, themes, plugin tests, and `build.sh` | The production source collection or publication destination | + +A Reportist plugin owns its input format, interpretation, validation, and +Typst document model. The report plugin and diagnostic plugin are peers. A +future plugin may produce a document with a different structure; it should use +shared visual elements where useful, but should not be translated into the +report plugin's data shape simply to render. + +The common surface is graphical: palettes, badges, pills, headings, panels, +callouts, progress bars, and similar reusable elements in `reporter/visual/`. +Plugin-specific composition remains in `reporter/plugins//`. + +## Working rules + +- Source content stays reviewable in text files. Generated PDFs and encrypted + archives are build artifacts. +- Typst handles structured data, interpretation, and page composition. Shell + or Python handles directory traversal, input staging, hashes, atomic output, + encryption, Git, and CI deployment. +- A plugin must work from a standalone Reportist checkout. The macro checkout + may call it, but must not be required by it. +- Changes to existing renderers must preserve approved output. The current + regression baseline covers seven authored reports and ten diagnostics. +- The macro Jenkins pipeline is the publishing authority. Reportist's CI tests + its own code; the Reports repository contains content and has no renderer CI. + +See [architecture](architecture.md) for the component boundaries and +[operations](operations.md) for the editing and publishing workflow. New +renderers follow the [Reportist plugin guide](../reporter/docs/plugins.md). diff --git a/reporter b/reporter index b268af7..6f29f59 160000 --- a/reporter +++ b/reporter @@ -1 +1 @@ -Subproject commit b268af7cb7a1c82bf86edce760afe92a66b0d7ad +Subproject commit 6f29f59d54773e7f305ebfe6e249ad5c68d8efcc diff --git a/reports b/reports index eb2d4d4..39de436 160000 --- a/reports +++ b/reports @@ -1 +1 @@ -Subproject commit eb2d4d40b954a3e7f24d2ac8af93cc63175c6d4b +Subproject commit 39de436eac70b3a76148f70085faae668ab55c38