diff --git a/AGENTS-TASKS.md b/AGENTS-TASKS.md index e266049..c7d1f73 100644 --- a/AGENTS-TASKS.md +++ b/AGENTS-TASKS.md @@ -64,9 +64,7 @@ the public interface, with collection limitations documented. Depends on normalized evidence and the diagnostic contract. - [x] WD3.1 Port reusable rules and watchlist matching with traceable evidence; keep investigation-specific conclusions and organisational recommendations outside this engine. -- [ ] WD3.2 Preserve and test event grouping, deduplication, correlation, unknown-event behavior, collection completeness, severity, and confidence. - - Primary event grouping, unknown events, watchlist gates, completeness, severity, - and confidence are covered; reliability-history deduplication remains pending. +- [x] WD3.2 Preserve and test event grouping, deduplication, correlation, unknown-event behavior, collection completeness, severity, and confidence. - [x] WD3.3 Make time-dependent analysis repeatable using a defined reference time; document intentional differences from legacy behavior. - [x] WD3.4 Emit schema-valid `diagnostic.json`; verify every finding's evidence reference resolves and raw observations remain available. - [x] WD3.5 Run sanitized regression fixtures for known diagnostic situations without the original machine; record exact commands and results. @@ -80,15 +78,26 @@ Depends on the working diagnostic slice. - [x] WD4.1 Correct Jenkins' instruction-file check; enable schema, fixture, and analysis checks as they become available, with accurate coverage reporting. - [ ] WD4.2 Validate native collection on supported Windows/PowerShell environments; clearly document anything not tested by the non-Windows Jenkins agent. -- [ ] WD4.3 Document installation, collection, offline analysis, output schema, privacy, dependencies, and troubleshooting from a fresh standalone clone. -- [ ] WD4.4 Verify the standalone clone needs neither Reportist, beffroi-reports, the macro repository, nor legacy sources; audit fixtures for identifying data. -- [ ] WD4.5 Record a verified component revision and public commands for the integration repository's sanitized pipeline test; keep integration orchestration there. +- [x] WD4.3 Document installation, collection, offline analysis, output schema, privacy, dependencies, and troubleshooting from a fresh standalone clone. +- [x] WD4.4 Verify the standalone clone needs neither Reportist, beffroi-reports, the macro repository, nor legacy sources; audit fixtures for identifying data. +- [x] WD4.5 Record a verified component revision and public commands for the integration repository's sanitized pipeline test; keep integration orchestration there. Exit: a fresh clone collects/analyses supported Windows evidence and produces a valid diagnostic result independently of reporting tools. ## Handoff log +### 2026-09-16 — Reliability deduplication and standalone handoff + +- Completed WD3.2 and WD4.3–WD4.5. Reviewed warning/critical reliability records + supplement bounded event logs while same-second/provider/ID/message duplicates + are omitted. Every retained record keeps its raw JSON pointer. +- Documented standalone installation, analysis, validation, privacy, + troubleshooting, and native-collection limitations. Runtime requires no sibling, + macro, renderer, reports, or legacy checkout. +- Native collection remains WD2.1–WD2.2/WD4.2 and requires a supported Windows + PowerShell worker; it is not claimed from Linux fixture validation. + ### 2026-09-15 — Complete event and watchlist catalog - Completed WD3.1 and advanced WD3.2 for the selected event-analysis scope: all 53 event rules diff --git a/README.md b/README.md index edf9871..66e9261 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,9 @@ required for reproducibility. Existing output requires `--force`. Exit codes are CLI usage. See the [rule migration status](docs/rules.md). Native Windows collection and broader non-event evidence correlation are still migration work. +See [standalone use](docs/standalone.md) for installation, exact commands, +privacy guidance, troubleshooting, and native-collection limitations. + ## License Software source code is licensed under the **MIT License**. diff --git a/docs/diagnostic-contract.md b/docs/diagnostic-contract.md index abf75ef..f8f39a1 100644 --- a/docs/diagnostic-contract.md +++ b/docs/diagnostic-contract.md @@ -99,8 +99,11 @@ python tools/analyse.py capture.json \ Input must be inside the explicit evidence root so output retains a portable source path. Existing output requires `--force`. Exit codes are 0 for a validated artifact, 1 for input, validation, or I/O failure, and 2 for invalid invocation. -The current rule catalog contains the first 15 reviewed event classifications; -other captured events remain traceable `unanalysed` findings. See [rules.md](rules.md). +The current catalog contains all 53 reviewed event classifications and seven +advisory watchlist entries. Actionable reliability-history records supplement +bounded event logs and are deduplicated by provider, event ID, normalized message, +and timestamp second. Other events remain traceable `unanalysed` findings. See +[rules.md](rules.md). ## Mapping legacy captures @@ -126,8 +129,8 @@ remain migration comparisons, not new-schema examples. Consumers select by explicit artifact version and reject unsupported versions. Breaking field or semantic changes require a new version and documented migration. Do not relabel raw captures or silently reshape historical artifacts. Contract v1 -is the initial implementation. The offline analyser is a bounded migration slice, -not yet the complete legacy rule/watchlist engine or collector. +is the initial implementation. Event/watchlist analysis is migrated; native +collection and broader non-event health analysis remain pending. Schema and format behavior follow [JSON Schema Draft 2020-12](https://json-schema.org/draft/2020-12/json-schema-validation) and the [jsonschema validator documentation](https://python-jsonschema.readthedocs.io/en/stable/validate/). diff --git a/docs/rules.md b/docs/rules.md index 86c8df5..4cb5c6f 100644 --- a/docs/rules.md +++ b/docs/rules.md @@ -41,3 +41,7 @@ OBS audio hooks, and SMBv1. Matches are exact and case-insensitive; driver paths match by basename, versions and enabled states are enforced where specified, and OBS/Intel prerequisite gates must pass. Results remain warning-level candidates whose descriptions explicitly avoid claims of exploitation or causation. + +Actionable warning/critical reliability-history records supplement bounded event +logs. Records duplicating an event by provider, ID, whitespace-normalized message, +and timestamp second are omitted; distinct records retain their raw pointer. diff --git a/docs/standalone.md b/docs/standalone.md new file mode 100644 index 0000000..076da16 --- /dev/null +++ b/docs/standalone.md @@ -0,0 +1,47 @@ +# Standalone use + +## Install and verify + +Python 3.11 or newer is required for offline validation and analysis. Reportist, +beffroi-reports, the integration workspace, and legacy sources are not runtime +dependencies. + +```bash +python3 -m venv .venv +.venv/bin/python -m pip install -r requirements.txt +.venv/bin/python -m unittest discover -s tests -p 'test_*.py' +``` + +```bash +.venv/bin/python tools/analyse.py capture.json \ + --evidence-root . --output diagnostic.json \ + --reference-time 2026-09-15T12:00:00Z \ + --producer-version "$(git rev-parse HEAD)" +.venv/bin/python tools/validate.py diagnostic.json --evidence-root . +``` + +Both commands return 0 on success, 1 for input/validation/I/O failures, and 2 for +invalid CLI usage. Existing analysis output requires `--force`. + +## Collection status + +Native collection has not yet been migrated into this standalone repository or +validated on a supported Windows worker. Offline analysis is supported and tested +on sanitized schema-1/2 captures. Missing, skipped, unavailable, or truncated +sections are limitations and never evidence of system health. + +## Privacy and troubleshooting + +Captures can contain host names, user paths, software inventory, device IDs, +events, and security configuration. Store real captures in the ignored +`collection/`, `captures/`, or `local-data/` directories and review every fixture +before committing it. + +- “input must be inside evidence root”: choose the capture's containing directory. +- Unsupported schema: retain the source unchanged and use schema 1 or 2. +- Invalid timestamp: use timezone-qualified ISO 8601 times. +- Existing output: choose a new path or pass `--force` deliberately. +- Evidence mismatch: validate against the unchanged capture used for analysis. + +The fixtures use synthetic values. The integration repository pins the verified +revision and owns cross-component orchestration. diff --git a/tests/test_analyse.py b/tests/test_analyse.py index b5413b5..6df4b51 100644 --- a/tests/test_analyse.py +++ b/tests/test_analyse.py @@ -3,6 +3,7 @@ import hashlib import importlib.util import json from pathlib import Path +import shutil import subprocess import sys import tempfile @@ -104,6 +105,36 @@ class AnalyseTests(unittest.TestCase): self.assertEqual(result["findings"][0]["confidence"], 0) self.assertEqual(len(result["findings"][0]["evidence"]), 3) + def test_reliability_history_adds_actionable_events_without_duplicates(self): + capture = analyser.validation.load_json(FIXTURES / "windiagnostics-v2-full.json") + capture = copy.deepcopy(capture) + event = {"TimeCreated": "2026-09-14T12:00:00.100Z", + "ProviderName": "Service Control Manager", "Id": 7000, + "Message": "Synthetic service failure"} + capture["sections"]["recent_system_events"]["data"] = [event] + capture["sections"]["recent_system_events"]["count"] = 1 + capture["sections"]["recent_application_events"]["data"] = [] + capture["sections"]["recent_application_events"]["count"] = 0 + capture["sections"]["reliability_records"]["data"] = [ + {"TimeGenerated": "2026-09-14T12:00:00.900Z", + "SourceName": "Service Control Manager", "EventIdentifier": "7000", + "Message": "Synthetic service failure"}, + {"TimeGenerated": "2026-09-13T12:00:00Z", + "SourceName": "Service Control Manager", "EventIdentifier": 7000, + "Message": "Separate reliability failure"}, + {"TimeGenerated": "2026-09-13T12:00:00Z", "SourceName": "EventLog", + "EventIdentifier": 1, "Message": "Unreviewed reliability noise"}, + ] + capture["sections"]["reliability_records"]["count"] = 3 + result = analyser.analyse(capture, "capture.json", "0" * 64, + self.reference_time, "test-revision") + finding = next(item for item in result["findings"] + if item["id"] == "event-service-control-manager-7000") + self.assertEqual(finding["evidence"], ["event-recent_system_events-1", + "event-reliability-records-2"]) + self.assertNotIn("event-reliability-records-1", + {item["id"] for item in result["observations"]}) + def test_watchlist_fixture_emits_six_traceable_candidates(self): result = self.run_fixture("windiagnostics-watchlist.json") findings = {finding["id"]: finding for finding in result["findings"] @@ -220,6 +251,21 @@ class AnalyseTests(unittest.TestCase): "--reference-time", self.reference_time, "--producer-version", "test-revision", ], cwd=self.work, text=True, capture_output=True) self.assertEqual(result.returncode, 0, result.stderr) + + def test_standalone_component_copy_analyses_fixture(self): + standalone = self.work / "standalone" + shutil.copytree(ROOT, standalone, ignore=shutil.ignore_patterns( + ".git", ".ci-venv", "__pycache__", "*.pyc")) + fixture = standalone / "tests" / "fixtures" / "legacy" / "windiagnostics-v1.json" + output = self.work / "standalone-diagnostic.json" + result = subprocess.run([ + sys.executable, str(standalone / "tools" / "analyse.py"), str(fixture), + "--evidence-root", str(fixture.parent), "--output", str(output), + "--reference-time", self.reference_time, + "--producer-version", "standalone-test", + ], cwd=self.work, text=True, capture_output=True) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(output.read_text())["schema_version"], 1) self.assertTrue(output.is_file()) diff --git a/tools/analyse.py b/tools/analyse.py index af5ab75..6de9485 100644 --- a/tools/analyse.py +++ b/tools/analyse.py @@ -363,6 +363,20 @@ def selected_event_sections(capture): return [(name, sections[name]) for name in names if name in sections] +def event_identity(event): + provider = str(event.get("ProviderName", "Unknown provider")) + raw_id = event.get("Id", "unknown") + event_id = raw_id if type(raw_id) in (int, str) else "unknown" + return provider, event_id + + +def event_signature(provider, event_id, timestamp, message): + normalized_message = " ".join(str(message or "").split()) + parsed = valid_timestamp(timestamp) + second = datetime.fromisoformat(parsed.replace("Z", "+00:00")).replace(microsecond=0) if parsed else timestamp + return provider, str(event_id), second, normalized_message + + def analyse(capture, artifact, artifact_hash, reference_time, producer_version): version = capture.get("schema_version") if type(version) is not int or version not in (1, 2): @@ -383,6 +397,7 @@ def analyse(capture, artifact, artifact_hash, reference_time, producer_version): collection_status = "complete" if complete else "partial" observations = [] groups = OrderedDict() + event_signatures = set() for section_name, section in selected_event_sections(capture): data = section.get("data", []) if isinstance(section, dict) else [] if not isinstance(data, list): @@ -398,9 +413,36 @@ def analyse(capture, artifact, artifact_hash, reference_time, producer_version): if observed_at: observation["observed_at"] = observed_at observations.append(observation) - provider = str(event.get("ProviderName", "Unknown provider")) - raw_event_id = event.get("Id", "unknown") - event_id = raw_event_id if type(raw_event_id) in (int, str) else "unknown" + provider, event_id = event_identity(event) + groups.setdefault((provider, event_id), []).append(observation_id) + event_signatures.add(event_signature(provider, event_id, event.get("TimeCreated"), + event.get("Message"))) + + reliability = capture["sections"].get("reliability_records") + reliability_data = reliability.get("data", []) if isinstance(reliability, dict) else [] + if isinstance(reliability_data, list): + for index, record in enumerate(reliability_data): + if not isinstance(record, dict): + continue + provider = str(record.get("SourceName", "Unknown provider")) + raw_id = record.get("EventIdentifier", "unknown") + event_id = int(raw_id) if isinstance(raw_id, str) and raw_id.isdigit() else raw_id + rule = EVENT_RULES.get((provider, event_id)) + if not rule or rule["severity"] not in ("warning", "critical"): + continue + signature = event_signature(provider, event_id, record.get("TimeGenerated"), + record.get("Message")) + if signature in event_signatures: + continue + event_signatures.add(signature) + observation_id = f"event-reliability-records-{index + 1}" + observation = {"id": observation_id, "section": "reliability_records", + "source_pointer": pointer("sections", "reliability_records", "data", index), + "value": record} + observed_at = valid_timestamp(record.get("TimeGenerated")) + if observed_at: + observation["observed_at"] = observed_at + observations.append(observation) groups.setdefault((provider, event_id), []).append(observation_id) findings = []