diff --git a/AGENTS-TASKS.md b/AGENTS-TASKS.md index 33fde17..e120320 100644 --- a/AGENTS-TASKS.md +++ b/AGENTS-TASKS.md @@ -56,10 +56,11 @@ Keep cross-repository milestones here consistent with those component checklists - [x] P0.3 Record initial ownership boundaries, coupling, and phased acceptance criteria here. - [x] P0.4 Link this tracker and the actual root instruction file from the README; distinguish current and target paths. - [ ] P0.5 Complete a file/function-level inventory before extraction, including rule/watchlist classification, report fields, CLI behavior, licenses, fonts, and external dependencies. Record source revisions and destination decisions without copying private evidence into this tracker. -- [ ] P0.6 Establish reproducible legacy baselines using reviewed synthetic/sanitized fixtures: expected findings, collection completeness, report structure, and representative PDFs. Record exact commands, tool versions, and failures. +- [x] P0.6 Establish reproducible legacy baselines using reviewed synthetic/sanitized fixtures: expected findings, collection completeness, report structure, and representative PDFs. Record exact commands, tool versions, and failures. Exit: every behavior in the first extraction slice has an owner and an observable -baseline. Inventory and baseline execution have not yet been completed. +baseline. The first synthetic baseline is complete; full inventory and private +incident evidence mapping remain open under P0.5 and component tasks. ## Pass 1 — Define contracts and compatibility @@ -161,3 +162,12 @@ reports remain understandable/recreatable and individual tools need no macro rep - Validation: checked new documentation links, unique task IDs, checkbox syntax, and whitespace. No runtime implementation changed or pipeline tests ran. - Commit handoff: component documentation commits are pinned by the enclosing integration commit; use its gitlinks to identify the exact revisions. - Next: complete the source inventories and sanitized baselines in each component, coordinated by P0.5–P0.6. Implementation milestones remain unchecked. + +### 2026-09-15 — Inventory and executable baseline pass + +- Completed P0.6 for five synthetic diagnostic captures and two synthetic report examples; added `scripts/check-legacy-baseline.py` and [reproduction instructions](docs/migration-baseline.md). +- Component inventories classify collector/normalizer functions, 53 event rules, seven watchlist entries, generic layout/fields, and private source ownership. First contract decisions are drafts, not implemented schemas. +- Validation: five expected snapshots, seven rendered PDFs with extracted titles, and three invalid-input checks passed with Typst 0.15.1. First-page visual review covered technical and Compact reports. Component CI file-presence checks passed locally; Jenkins itself was not run. +- Preserved concurrent component checkout updates that renamed instructions to `AGENT.md` and corrected reports CI license paths. References remain unchanged. +- Limits: no native Windows collection (PowerShell unavailable), migrated engine/renderer/adapter, private historical rendering, full pagination review, or advisory revalidation. P0.5 remains open for the remaining detailed historical/packaging inventory. +- Next: implement WD1/RT1 versioned schemas and synthetic validation examples; finish BR0.5 claim/evidence mapping before moving private incidents. diff --git a/README.md b/README.md index e37064a..e85f0a6 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,9 @@ git submodule update --init --recursive acceptance criteria, and the next-pass handoff. [AGENTS.md](AGENTS.md) defines repository ownership and rules for agents working across the workspace. +The first executable migration check is the [synthetic legacy baseline](docs/migration-baseline.md). +It records diagnostic behavior and renders example PDFs before extraction. + ## License Root-level software and orchestration code is licensed under the **MIT License**. diff --git a/beffroi-reports b/beffroi-reports index 35498b3..10c59dd 160000 --- a/beffroi-reports +++ b/beffroi-reports @@ -1 +1 @@ -Subproject commit 35498b3d8e4e09ef1e4f993491b0837878d663db +Subproject commit 10c59ddaa3d22730244416a7dcd1842d1e00e6e3 diff --git a/docs/migration-baseline.md b/docs/migration-baseline.md new file mode 100644 index 0000000..06441b0 --- /dev/null +++ b/docs/migration-baseline.md @@ -0,0 +1,65 @@ +# First migration baseline + +## Run + +Requirements: Python 3, Git, Typst, Poppler's `pdftotext`, DejaVu Sans and DejaVu +Sans Mono fonts. Dependencies are checked, not installed automatically. +Choose a new output directory: + +```bash +python3 scripts/check-legacy-baseline.py --output /tmp/beffroi-baseline-review +``` + +The runner reads an explicit source allowlist from legacy renderer commit +`1d381904fff3d378a7c0e04588e75310a9b7ed60`, stages it in a temporary directory, +and executes synthetic fixtures owned by the destination components. It never +materializes legacy real captures/reports or modifies reference working trees. +Temporary sources are removed after execution. + +Outputs: five diagnostic PDFs, two example report PDFs, observed diagnostic JSON, +and a manifest of source revision, fixture hashes, Typst version, available +DejaVu fonts, and negative checks. Existing output directories are rejected. +PDFs are review artifacts, not committed content. + +## Verification + +- Five snapshots match reviewed expected diagnostic output: collection coverage, + findings, grouping, severity/confidence, and system facts. +- Independent assertions cover schema-1 service failure, schema-2 event counts and + grouping, incomplete-collection confidence, Compact unavailable checks, + unexpected collection status, and six watchlist candidate matches. +- Seven PDFs compile, have PDF signatures, and contain expected extracted titles. +- Missing author/date and an unknown theme fail with expected errors. + +This is a legacy comparison harness, **not the new integration pipeline**. +The new diagnostic schema, renderer, adapter and full pipeline remain +unimplemented. The old regression entry point mixes synthetic tests with private +captures and is deliberately not executed by this harness. + +## Observations and limits + +Initial run: 2026-09-15, Linux, Typst `0.15.1 (586e1bd4)`. PowerShell is absent; +no native collection ran. Font versions and metadata affect PDF bytes, so byte +identity is not asserted. The fixture dates explicitly fix analysis reference time. + +Schema-1 produces one service failure. Full/partial schema-2 preserves unknown +events and caps incomplete-collection confidence at 50. Compact unavailable checks +with zero counts remain unavailable. Its legacy display still shows a high numeric +score despite partial collection: this is observed behavior, not an accepted new +health contract. Generic synthetic reports still acquire the Beffroi logo. + +First-page visual review covered the technical report and Compact diagnostic: +both were readable without visible clipping and the diagnostic showed its partial +collection warning. Other pages, long-content pagination and alternate branding +remain open. Synthetic checks do not verify historical content or advisory +accuracy. Linked vendor guidance has not been revalidated. + +## Next pass + +- [Diagnostic inventory and draft contract](../win-diagnostics/docs/migration-inventory.md) +- [Renderer inventory and draft field mapping](../reportist/docs/migration-inventory.md) +- [Private preservation and adapter decisions](../beffroi-reports/docs/migration-inventory.md) + +Implement versioned schemas with valid/invalid synthetic examples, then extract +one offline diagnostic and generic rendering slice against this baseline. Finish +private incident evidence mapping before claiming historical migration is verified. diff --git a/reportist b/reportist index 2c1c085..3b73d1a 160000 --- a/reportist +++ b/reportist @@ -1 +1 @@ -Subproject commit 2c1c085ca2940e3908c16035398d739a8773644c +Subproject commit 3b73d1a03c25c1d9ac56959768b894c64b607980 diff --git a/scripts/check-legacy-baseline.py b/scripts/check-legacy-baseline.py new file mode 100644 index 0000000..816396a --- /dev/null +++ b/scripts/check-legacy-baseline.py @@ -0,0 +1,139 @@ +#!/usr/bin/env python3 +"""Reproduce the migration baseline without modifying the legacy repositories.""" + +import argparse +import hashlib +import json +from pathlib import Path +import shutil +import subprocess +import tempfile + + +ROOT = Path(__file__).resolve().parents[1] +SOURCE = ROOT / "references/typist_reporter" +REVISION = "1d381904fff3d378a7c0e04588e75310a9b7ed60" +FIXTURES = ROOT / "win-diagnostics/tests/fixtures/legacy" +REPORTS = ROOT / "reportist/tests/fixtures/legacy" +EXPECTED = ROOT / "win-diagnostics/tests/legacy-baseline.json" +# Explicit allowlist: never materialize real reports or diagnostic captures. +SOURCE_FILES = ( + "LICENSE", "report.typ", "diagnostics.typ", "modules/model.typ", + "modules/diagnostics-model.typ", "modules/diagnostics-normalize.typ", + "config/themes.yaml", "config/diagnostics-rules.yaml", + "config/diagnostics-watchlist.yaml", "assets/beffroi.png", +) +BRIDGE = '''#import "modules/diagnostics-normalize.typ": normalize +#let data = normalize(json(sys.inputs.at("fixture")), + rules: yaml("config/diagnostics-rules.yaml"), source-name: "synthetic fixture") +#metadata(( + schema_version: data.schema_version, + mode: data.mode, + collection_incomplete: data.collection_incomplete, + analysis_confidence: data.analysis_confidence, + health_score: data.health_score, + event_counts: data.event_counts, + issue_counts: data.issue_counts, + system: data.system, + findings: data.findings, + event_groups: data.event_groups, + coverage: data.coverage, +)) +''' + + +def run(*args, cwd=None): + return subprocess.run(args, cwd=cwd, check=True, capture_output=True).stdout + + +def verify_semantics(results): + v1 = results["windiagnostics-v1"] + assert v1["system"]["ram_total_gb"] == 31.82 + assert any(f["title"] == "Service Control Manager · Event 7000" + for f in v1["findings"]) + full = results["windiagnostics-v2-full"] + assert full["event_counts"] == dict(critical=0, error=1, warning=2, information=1, unknown=0) + assert full["collection_incomplete"] and full["analysis_confidence"] <= 50 + assert any(e["level"] == 3 and e["count"] == 2 for e in full["event_groups"]) + compact = results["windiagnostics-v2-compact"] + assert not any(f["title"] == "Devices report problems" for f in compact["findings"]) + partial = results["windiagnostics-v2-partial"] + assert any(c["section"] == "services" and c["severity"] == "warning" + for c in partial["coverage"]) + watch = results["windiagnostics-watchlist"] + assert [f["watch_id"] for f in watch["findings"] if "watch_id" in f] == [ + "winring0", "dell-dbutil", "gigabyte-oc-guru", "obs-overlay", "obs-audio-hooks", "smb1"] + assert all(f["weight_factor"] == 0 for f in watch["findings"] if "watch_id" in f) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, required=True, help="New directory for PDFs and observed JSON") + args = parser.parse_args() + for tool in ("git", "typst", "pdftotext"): + if not shutil.which(tool): + parser.error(f"{tool} is required; install it explicitly before running this check") + if args.output.exists(): + parser.error("output directory already exists; choose a new directory") + output = args.output.resolve() + output.mkdir(parents=True) + results = {} + manifest = {"source_revision": REVISION, "typst": run("typst", "--version").decode().strip(), + "fixture_sha256": {}, "pdfs": []} + with tempfile.TemporaryDirectory(prefix="beffroi-legacy-baseline-") as directory: + work = Path(directory) + for name in SOURCE_FILES: + target = work / name + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(run("git", "-C", str(SOURCE), "show", f"{REVISION}:{name}")) + (work / "baseline.typ").write_text(BRIDGE) + for fixture in sorted(FIXTURES.glob("*.json")): + shutil.copyfile(fixture, work / fixture.name) + manifest["fixture_sha256"][fixture.name] = hashlib.sha256(fixture.read_bytes()).hexdigest() + raw = run("typst", "query", "--root", directory, "--input", f"fixture={fixture.name}", + "baseline.typ", "", "--field", "value", "--one", cwd=work) + results[fixture.stem] = json.loads(raw) + pdf = output / f"{fixture.stem}.pdf" + run("typst", "compile", "--root", directory, "--input", f"diagnostics={fixture.name}", + "--input", "author=Synthetic baseline", "diagnostics.typ", str(pdf), cwd=work) + manifest["pdfs"].append(pdf.name) + verify_semantics(results) + for fixture in sorted(REPORTS.glob("*.yaml")): + shutil.copyfile(fixture, work / fixture.name) + manifest["fixture_sha256"][fixture.name] = hashlib.sha256(fixture.read_bytes()).hexdigest() + pdf = output / f"report-{fixture.stem}.pdf" + run("typst", "compile", "--root", directory, "--input", f"report={fixture.name}", + "report.typ", str(pdf), cwd=work) + manifest["pdfs"].append(pdf.name) + for name, content, message in ( + ("missing-author", 'metadata:\n Date: "2026-09-15"\n', "metadata.Author"), + ("missing-date", 'metadata:\n Author: "Synthetic"\n', "metadata.Date"), + ("unknown-theme", 'theme: unknown\nmetadata:\n Author: "Synthetic"\n Date: "2026-09-15"\n', "Unknown report theme"), + ): + (work / "invalid.yaml").write_text(content) + result = subprocess.run(["typst", "compile", "--input", "report=invalid.yaml", + "report.typ", str(work / "invalid.pdf")], cwd=work, capture_output=True) + assert result.returncode != 0 and message in result.stderr.decode(), name + manifest["negative_checks"] = ["missing-author", "missing-date", "unknown-theme"] + for name in manifest["pdfs"]: + assert (output / name).read_bytes().startswith(b"%PDF-"), name + text = run("pdftotext", str(output / name), "-").decode() + expected_title = {"report-minimal.pdf": "Short report title", + "report-technical.pdf": "Service Interruption Investigation"}.get( + name, "Windows diagnostic assessment") + assert expected_title in " ".join(text.split()), f"Missing rendered title: {name}" + manifest["fonts"] = [font for font in run("typst", "fonts").decode().splitlines() + if "DejaVu" in font] + (output / "diagnostics.json").write_text(json.dumps(results, indent=2, ensure_ascii=False) + "\n") + (output / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n") + if not EXPECTED.exists(): + parser.exit(1, f"Observed baseline written to {output}; reviewed expected snapshot is missing\n") + assert results == json.loads(EXPECTED.read_text()), "Diagnostic baseline differs; inspect diagnostics.json" + print(f"PASS: 5 diagnostic snapshots, 7 PDFs, 3 invalid inputs; artifacts: {output}") + + +if __name__ == "__main__": + try: + main() + except subprocess.CalledProcessError as error: + raise SystemExit(error.stderr.decode(errors="replace")) from error diff --git a/win-diagnostics b/win-diagnostics index 861757f..c0d7f42 160000 --- a/win-diagnostics +++ b/win-diagnostics @@ -1 +1 @@ -Subproject commit 861757f0d165429e789e042cf4d015ab5b72f85c +Subproject commit c0d7f42e8ca31e7f8de4f3475211a52b8db9e3d4