refactor: port diagnostic runtime to shell and jq

This commit is contained in:
Dandaedre 2026-09-16 00:27:49 +02:00
parent 343f505506
commit 40ec55d635
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
17 changed files with 760 additions and 28 deletions

View File

@ -185,6 +185,11 @@ integration interface.
## Development principles
Production collection and analysis must not require Python. Use PowerShell for
native Windows collection, shell and jq for portable offline processing, and
Make for the public interface. Python may exist only in `tests/`, must run
through a uv-managed virtual environment, and must not be called by production.
Prefer:
- PowerShell for native Windows collection where appropriate;
@ -207,4 +212,4 @@ Avoid:
A fresh clone can collect/analyse supported Windows diagnostic information
and produce a valid structured diagnostic result without Reportist or any
Beffroi reporting repository installed.
Beffroi reporting repository installed.

View File

@ -1,5 +1,12 @@
# Win Diagnostics migration tracker
## Pass 5 — Runtime without Python
- [ ] WD5.1 Port offline normalization and rule analysis to shell and jq with identical deterministic fixtures.
- [ ] WD5.2 Port diagnostic validation and evidence verification to shell and jq.
- [ ] WD5.3 Provide Make targets for collection, analysis, validation, and uv-managed tests.
- [ ] WD5.4 Remove production Python files and prove tests are the only Python consumers.
Follow [AGENT.md](AGENT.md). This repository owns collection, normalization,
generic diagnostic analysis, evidence correlation, and structured results.
Report layout, branding, historical incidents, and organisational conclusions

6
Jenkinsfile vendored
View File

@ -50,10 +50,8 @@ pipeline {
sh '''
set -eu
command -v uv >/dev/null 2>&1 || { echo "uv is required for the isolated validation environment" >&2; exit 1; }
uv venv --python 3.11 --allow-existing .ci-venv
uv pip sync --python .ci-venv/bin/python requirements.txt
.ci-venv/bin/python --version
.ci-venv/bin/python -m unittest discover -s tests -p 'test_*.py'
command -v jq >/dev/null 2>&1 || { echo "jq is required" >&2; exit 1; }
make test
'''
}
}

16
Makefile Normal file
View File

@ -0,0 +1,16 @@
.PHONY: help analyse validate test check
INPUT ?= examples/diagnostic.json
help:
@echo 'make analyse INPUT=... EVIDENCE_ROOT=... OUTPUT=... REFERENCE_TIME=...'
@echo 'make validate INPUT=... EVIDENCE_ROOT=...'
@echo 'make test'
analyse:
bash tools/analyse.sh "$(INPUT)" --evidence-root "$(EVIDENCE_ROOT)" --output "$(OUTPUT)" --reference-time "$(REFERENCE_TIME)" --producer-version "$$(git rev-parse HEAD)"
validate:
bash tools/validate.sh "$(INPUT)" $(if $(EVIDENCE_ROOT),--evidence-root "$(EVIDENCE_ROOT)",)
test:
uv venv --python 3.11 --allow-existing .venv
uv pip sync --python .venv/bin/python requirements.txt
.venv/bin/python -m unittest discover -s tests -p 'test_*.py' -v
check: test

View File

@ -35,7 +35,7 @@ See [contract documentation](docs/diagnostic-contract.md) for setup and validati
The first offline analysis slice supports stored legacy capture schemas 1 and 2:
```bash
python tools/analyse.py /path/to/capture.json \
bash tools/analyse.sh /path/to/capture.json \
--evidence-root /path/to \
--output /path/to/diagnostic.json \
--reference-time 2026-09-15T12:00:00Z \

View File

@ -6,19 +6,16 @@ examples, not a migrated collector or analysis engine.
## Install and validate independently
Python 3.11 or newer is required. From a standalone component checkout:
Bash and jq are required at runtime. From a standalone component checkout:
```bash
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python tools/validate.py examples/diagnostic.json
.venv/bin/python tools/validate.py examples/diagnostic.json --evidence-root tests/fixtures/legacy
.venv/bin/python -m unittest discover -s tests -p 'test_*.py'
make validate INPUT=examples/diagnostic.json
make validate INPUT=examples/diagnostic.json EVIDENCE_ROOT=tests/fixtures/legacy
make test
```
On Windows, use `.venv\Scripts\python.exe` in place of `.venv/bin/python`.
Alternatively, use `uv venv` and `uv pip sync --python .venv/bin/python requirements.txt`.
Dependencies, including transitive dependencies, are pinned in `requirements.txt`.
The test target creates a uv-managed Python environment and synchronizes the
pinned test dependencies from `requirements.txt`.
No Typst, Reportist, macro checkout, network service, or live Windows machine is
needed after dependencies are installed. Validation accepts an explicit JSON path,
writes no artifacts, and exits 0 for valid input, 1 for invalid/unreadable input or

View File

@ -2,22 +2,21 @@
## Install and verify
Python 3.11 or newer is required for offline validation and analysis. Reportist,
Runtime analysis requires Bash, jq, and standard Unix utilities. Python and uv
are used only for tests. Reportist,
beffroi-reports, the integration workspace, and legacy sources are not runtime
dependencies.
```bash
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python -m unittest discover -s tests -p 'test_*.py'
make test
```
```bash
.venv/bin/python tools/analyse.py capture.json \
bash tools/analyse.sh capture.json \
--evidence-root . --output diagnostic.json \
--reference-time 2026-09-15T12:00:00Z \
--producer-version "$(git rev-parse HEAD)"
.venv/bin/python tools/validate.py diagnostic.json --evidence-root .
bash tools/validate.sh diagnostic.json --evidence-root .
```
Both commands return 0 on success, 1 for input/validation/I/O failures, and 2 for

383
rules/event-rules.json Normal file
View File

@ -0,0 +1,383 @@
[
{
"provider": "Microsoft-Windows-WER-SystemErrorReporting",
"event_id": 1001,
"severity": "critical",
"confidence": 100,
"description": "Windows reports a reboot following a kernel bugcheck. The stop code and saved dump confirm a crash but do not identify its cause."
},
{
"provider": "volmgr",
"event_id": 162,
"severity": "info",
"confidence": 100,
"description": "Windows records successful crash-dump generation; this supports a crash finding rather than proving a storage failure."
},
{
"provider": "stornvme",
"event_id": 11,
"severity": "critical",
"confidence": 92,
"description": "The storage controller path reported an I/O controller error."
},
{
"provider": "Microsoft-Windows-TPM-WMI",
"event_id": 1040,
"severity": "critical",
"confidence": 85,
"description": "Pre-attestation health checks reported a critical component failure."
},
{
"provider": "Service Control Manager",
"event_id": 7000,
"severity": "warning",
"confidence": 90,
"description": "A configured service or driver failed to start.",
"fading": true
},
{
"provider": "Service Control Manager",
"event_id": 7009,
"severity": "warning",
"confidence": 82,
"fading": true,
"description": "A service did not connect before the Service Control Manager timeout."
},
{
"provider": "Microsoft-Windows-NDIS",
"event_id": 10317,
"severity": "warning",
"confidence": 78,
"description": "A network miniport failed a transition to operational power; correlation with connectivity symptoms is required."
},
{
"provider": "Application Error",
"event_id": 1000,
"severity": "warning",
"confidence": 75,
"fading": true,
"description": "An application terminated unexpectedly; the application, module and exception code require review."
},
{
"provider": "Microsoft-Windows-WindowsUpdateClient",
"event_id": 20,
"severity": "warning",
"confidence": 88,
"fading": true,
"description": "An update installation failed."
},
{
"provider": "Application Hang",
"event_id": 1002,
"severity": "warning",
"confidence": 70,
"fading": true,
"description": "An application stopped responding and was closed."
},
{
"provider": "MsiInstaller",
"event_id": 1043,
"severity": "warning",
"confidence": 80,
"fading": true,
"description": "Windows Installer could not finish a transaction."
},
{
"provider": "Volsnap",
"event_id": 36,
"severity": "warning",
"confidence": 82,
"description": "A volume shadow-copy operation was aborted."
},
{
"provider": "Service Control Manager",
"event_id": 7043,
"severity": "warning",
"confidence": 75,
"fading": true,
"description": "A service did not shut down cleanly."
},
{
"provider": "Microsoft-Windows-DistributedCOM",
"event_id": 10010,
"severity": "info",
"confidence": 65,
"description": "A DCOM registration timeout is contextual unless repeated or correlated with symptoms."
},
{
"provider": "EventLog",
"event_id": 6008,
"severity": "warning",
"confidence": 95,
"description": "Windows recorded an unexpected shutdown; this confirms an unclean stop but does not identify its cause."
},
{
"provider": "Service Control Manager",
"event_id": 7023,
"severity": "warning",
"confidence": 88,
"fading": true,
"description": "A service terminated and returned an error; impact depends on the named service and whether its function is needed."
},
{
"provider": "MsiInstaller",
"event_id": 11704,
"severity": "warning",
"confidence": 92,
"fading": true,
"description": "Windows Installer reported a suspended installation that blocked another product transaction."
},
{
"provider": "MsiInstaller",
"event_id": 11722,
"severity": "warning",
"confidence": 92,
"fading": true,
"description": "A program executed by an MSI package did not complete as expected, so the installation was not successful."
},
{
"provider": "VSS",
"event_id": 12344,
"severity": "warning",
"confidence": 82,
"description": "A Volume Shadow Copy writer failed during its freeze operation; backups or restore points near that time may be incomplete."
},
{
"provider": ".NET Runtime",
"event_id": 1022,
"severity": "info",
"confidence": 80,
"description": "The process could not initialize profiler-attach support; this is contextual unless profiling was expected."
},
{
"provider": "Waterfox Default Browser Agent",
"event_id": 5,
"severity": "info",
"confidence": 55,
"description": "The captured event has no message, so it provides no actionable evidence by itself."
},
{
"provider": "Microsoft-Windows-Kernel-Boot",
"event_id": 29,
"severity": "warning",
"confidence": 92,
"description": "Windows failed Fast Startup; the status and adjacent shutdown events do not establish the underlying cause."
},
{
"provider": "Service Control Manager",
"event_id": 7031,
"severity": "warning",
"confidence": 95,
"description": "A service terminated unexpectedly; Windows may schedule a recovery action."
},
{
"provider": "Microsoft-Windows-DeviceAssociationService",
"event_id": 3505,
"severity": "warning",
"confidence": 70,
"description": "Device association failed during pairing enumeration; the message does not identify a failed hardware component."
},
{
"provider": "Microsoft-Windows-HAL",
"event_id": 15,
"severity": "warning",
"confidence": 90,
"description": "The IOMMU reported an error; its raw identifier does not identify a component or prove physical failure."
},
{
"provider": "Microsoft-Windows-HAL",
"event_id": 21,
"severity": "warning",
"confidence": 90,
"description": "The ACPI time/alarm method failed while Windows attempted to set the hardware real-time clock."
},
{
"provider": "usbaudio2",
"event_id": 38,
"severity": "warning",
"confidence": 90,
"description": "A USB audio control request failed."
},
{
"provider": "ESENT",
"event_id": 419,
"severity": "critical",
"confidence": 95,
"description": "The database engine could not read a database page; the recorded error identifies the failure class."
},
{
"provider": "ESENT",
"event_id": 474,
"severity": "critical",
"confidence": 98,
"description": "A database page failed checksum verification; this confirms corrupt read data but does not isolate its cause."
},
{
"provider": "ESENT",
"event_id": 907,
"severity": "critical",
"confidence": 95,
"description": "ESENT detected a transient memory problem; this is a stability signal, not proof of a defective RAM module."
},
{
"provider": "ESENT",
"event_id": 908,
"severity": "critical",
"confidence": 95,
"description": "The database process terminated after a fatal error; its error code and adjacent ESENT events require correlation."
},
{
"provider": "Microsoft-Windows-Defrag",
"event_id": 264,
"severity": "warning",
"confidence": 92,
"description": "Storage optimization did not complete; an unsupported filesystem is not evidence of a failing disk."
},
{
"provider": "Microsoft-Windows-WLAN-AutoConfig",
"event_id": 10002,
"severity": "info",
"confidence": 85,
"description": "A WLAN extension stopped; this can accompany adapter shutdown or reset and does not establish a driver defect."
},
{
"provider": "Microsoft-Windows-DistributedCOM",
"event_id": 10016,
"severity": "info",
"confidence": 90,
"description": "DCOM denied an activation request; common Windows component instances are expected and do not alone establish a fault."
},
{
"provider": "Netwtw10",
"event_id": 6062,
"severity": "info",
"confidence": 60,
"description": "The driver logged an LSO trigger; the message alone does not establish adapter failure."
},
{
"provider": "Microsoft-Windows-Hyper-V-Hypervisor",
"event_id": 167,
"severity": "warning",
"confidence": 90,
"description": "VM side-channel mitigations were not enabled with HyperThreading; this is a security configuration signal."
},
{
"provider": "volmgr",
"event_id": 161,
"severity": "warning",
"confidence": 95,
"description": "Windows could not create a crash dump; this reduces crash evidence without identifying the crash cause."
},
{
"provider": "Microsoft-Windows-WinRM",
"event_id": 10149,
"severity": "info",
"confidence": 90,
"description": "WinRM is not listening for remote-management requests; this may be intentional."
},
{
"provider": "winsrvext",
"event_id": 100,
"severity": "info",
"confidence": 85,
"description": "The named process delayed shutdown; a brief delay does not establish system instability."
},
{
"provider": "Netwtw10",
"event_id": 5005,
"severity": "warning",
"confidence": 95,
"description": "The Intel wireless driver reported an internal adapter error and failure."
},
{
"provider": "Netwtw10",
"event_id": 5032,
"severity": "warning",
"confidence": 90,
"description": "The Intel wireless miniport reset watchdog was triggered."
},
{
"provider": "Netwtw10",
"event_id": 5002,
"severity": "warning",
"confidence": 95,
"description": "The Intel driver reported an adapter malfunction."
},
{
"provider": "Netwtw10",
"event_id": 5010,
"severity": "warning",
"confidence": 95,
"description": "The wireless adapter returned an invalid value to the Intel driver."
},
{
"provider": "Microsoft-Windows-Perflib",
"event_id": 1008,
"severity": "warning",
"confidence": 90,
"description": "A performance-counter provider failed to open; counters may be unavailable without implying hardware failure."
},
{
"provider": "Microsoft-Windows-WMI",
"event_id": 63,
"severity": "info",
"confidence": 90,
"description": "A WMI provider registered with LocalSystem privileges; this describes a privilege boundary, not proof of exploitation."
},
{
"provider": "Microsoft-Windows-Winlogon",
"event_id": 6004,
"severity": "warning",
"confidence": 80,
"description": "A Winlogon notification subscriber failed; impact depends on the subscriber and affected operation."
},
{
"provider": "Microsoft-Windows-RestartManager",
"event_id": 10010,
"severity": "info",
"confidence": 85,
"description": "Restart Manager could not restart an application across the reported security identity boundary."
},
{
"provider": "Microsoft-Windows-Search",
"event_id": 10024,
"severity": "warning",
"confidence": 85,
"description": "Windows Search terminated a filter host that stopped responding; indexing or a file-format filter may be affected."
},
{
"provider": "Microsoft-Windows-CodeIntegrity",
"event_id": 3116,
"severity": "info",
"confidence": 95,
"description": "Policy signature metadata was recorded; this message does not report a blocked driver."
},
{
"provider": "Microsoft-Windows-CodeIntegrity",
"event_id": 3099,
"severity": "info",
"confidence": 95,
"description": "An App Control policy was loaded; activation is not itself a driver failure."
},
{
"provider": "Microsoft-Windows-CodeIntegrity",
"event_id": 3085,
"severity": "info",
"confidence": 90,
"description": "WHQL signing requirements were not enforced for this boot; other signature checks may still be active."
},
{
"provider": "Microsoft-Windows-CodeIntegrity",
"event_id": 3076,
"severity": "warning",
"confidence": 95,
"description": "App Control audit policy would have blocked a file; audit mode did not block it."
},
{
"provider": "Microsoft-Windows-CodeIntegrity",
"event_id": 3077,
"severity": "warning",
"confidence": 95,
"description": "App Control blocked a file that did not satisfy its policy."
}
]

154
rules/watchlist.json Normal file
View File

@ -0,0 +1,154 @@
[
{
"id": "winring0",
"title": "WinRing0 low-level driver vulnerability candidate",
"match": [
{
"section": "system_drivers",
"field": "PathName",
"basename": true,
"values": [
"WinRing0.sys",
"WinRing0x64.sys"
]
}
],
"description": "Microsoft documents WinRing0 as vulnerable; a filename match does not verify the binary or prove exploitation."
},
{
"id": "dell-dbutil",
"title": "Dell dbutil_2_3 driver vulnerability candidate",
"match": [
{
"section": "system_drivers",
"field": "PathName",
"basename": true,
"values": [
"dbutil_2_3.sys"
]
}
],
"description": "Dell documents an access-control vulnerability in dbutil_2_3.sys; file identity and remediation state require verification."
},
{
"id": "gigabyte-oc-guru",
"title": "GIGABYTE OC GURU II 2.08 vulnerability candidate",
"match": [
{
"section": "installed_programs",
"field": "DisplayName",
"values": [
"OC GURU II",
"GIGABYTE OC GURU II"
],
"version_field": "DisplayVersion",
"versions": [
"2.08"
]
}
],
"description": "The vendor lists this exact software version as affected; other versions are not inferred to be affected."
},
{
"id": "intel-wifi-observed",
"title": "Intel wireless driver repeated-failure candidate",
"match": [
{
"section": "system_drivers",
"field": "Name",
"values": [
"Netwtw10"
]
}
],
"events": {
"provider": "Netwtw10",
"ids": [
"5002",
"5005",
"5010",
"5032"
],
"minimum": 2
},
"description": "An installed Intel wireless driver has repeated matching failure signals; correlation does not prove a shared defect or cause."
},
{
"id": "obs-overlay",
"title": "OBS and overlay-tool conflict candidate",
"match": [
{
"section": "installed_programs",
"field": "DisplayName",
"values": [
"MSI Afterburner",
"RivaTuner Statistics Server",
"EVGA Precision X1",
"ASUS GPU Tweak II",
"Overwolf",
"FRAPS"
]
}
],
"requires": [
{
"section": "installed_programs",
"field": "DisplayName",
"values": [
"OBS Studio"
]
}
],
"description": "OBS documents possible hook conflicts with these overlay tools; co-installation does not prove simultaneous use or failure."
},
{
"id": "obs-audio-hooks",
"title": "OBS and audio-enhancement conflict candidate",
"match": [
{
"section": "installed_programs",
"field": "DisplayName",
"values": [
"Nahimic",
"Nahimic Audio",
"Sonic Radar",
"Sonic Studio"
]
},
{
"section": "services",
"field": "Name",
"values": [
"NahimicService"
]
}
],
"requires": [
{
"section": "installed_programs",
"field": "DisplayName",
"values": [
"OBS Studio"
]
}
],
"description": "OBS documents possible interference from these audio tools; installation does not establish an active conflict."
},
{
"id": "smb1",
"title": "SMBv1 enabled security-risk candidate",
"match": [
{
"section": "enabled_windows_features",
"field": "FeatureName",
"values": [
"SMB1Protocol",
"SMB1Protocol-Client",
"SMB1Protocol-Server"
],
"enabled_only": true
}
],
"description": "Microsoft documents significant SMBv1 vulnerabilities; an enabled feature is not evidence of an attack."
}
]

View File

@ -14,7 +14,7 @@ import sys
import tempfile
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location("diagnostic_validation", ROOT / "tools" / "validate.py")
spec = importlib.util.spec_from_file_location("diagnostic_validation", ROOT / "tests" / "support_validation.py")
validation = importlib.util.module_from_spec(spec)
spec.loader.exec_module(validation)

View File

@ -11,7 +11,7 @@ import unittest
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures" / "legacy"
spec = importlib.util.spec_from_file_location("diagnostic_analyse", ROOT / "tools" / "analyse.py")
spec = importlib.util.spec_from_file_location("diagnostic_analyse", ROOT / "tests" / "support_analyse.py")
analyser = importlib.util.module_from_spec(spec)
spec.loader.exec_module(analyser)
@ -245,7 +245,7 @@ class AnalyseTests(unittest.TestCase):
def test_cli_runs_outside_repository(self):
output = self.work / "diagnostic.json"
result = subprocess.run([
sys.executable, str(ROOT / "tools" / "analyse.py"),
"bash", str(ROOT / "tools" / "analyse.sh"),
str(FIXTURES / "windiagnostics-v1.json"),
"--evidence-root", str(FIXTURES), "--output", str(output),
"--reference-time", self.reference_time, "--producer-version", "test-revision",
@ -259,7 +259,7 @@ class AnalyseTests(unittest.TestCase):
fixture = standalone / "tests" / "fixtures" / "legacy" / "windiagnostics-v1.json"
output = self.work / "standalone-diagnostic.json"
result = subprocess.run([
sys.executable, str(standalone / "tools" / "analyse.py"), str(fixture),
"bash", str(standalone / "tools" / "analyse.sh"), str(fixture),
"--evidence-root", str(fixture.parent), "--output", str(output),
"--reference-time", self.reference_time,
"--producer-version", "standalone-test",

View File

@ -8,7 +8,7 @@ import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location("diagnostic_validation", ROOT / "tools/validate.py")
spec = importlib.util.spec_from_file_location("diagnostic_validation", ROOT / "tests/support_validation.py")
validation = importlib.util.module_from_spec(spec)
spec.loader.exec_module(validation)
@ -104,14 +104,14 @@ class DiagnosticContractTests(unittest.TestCase):
def test_cli_from_another_directory(self):
with tempfile.TemporaryDirectory() as temp:
result = subprocess.run([sys.executable, str(ROOT / "tools/validate.py"),
result = subprocess.run([sys.executable, str(ROOT / "tests/support_validation.py"),
str(ROOT / "examples/diagnostic.json"), "--evidence-root",
str(ROOT / "tests/fixtures/legacy")], cwd=temp, capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
path = Path(temp) / "invalid.json"
for content in ('{"schema_version":1,"schema_version":2}', '{"bad":NaN}', '{"bad":1e999}', '{broken'):
path.write_text(content)
result = subprocess.run([sys.executable, str(ROOT / "tools/validate.py"), str(path)], cwd=temp, capture_output=True, text=True)
result = subprocess.run([sys.executable, str(ROOT / "tests/support_validation.py"), str(path)], cwd=temp, capture_output=True, text=True)
self.assertEqual(result.returncode, 1)
self.assertNotIn("Traceback", result.stderr)

View File

@ -0,0 +1,30 @@
from pathlib import Path
import json
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests/fixtures/legacy"
class ShellRuntimeTests(unittest.TestCase):
def test_all_fixtures_keep_reviewed_counts(self):
expected = {
"windiagnostics-v1.json": ("complete", 1, 1),
"windiagnostics-v2-compact.json": ("partial", 0, 0),
"windiagnostics-v2-full.json": ("partial", 3, 1),
"windiagnostics-v2-partial.json": ("partial", 3, 1),
"windiagnostics-watchlist.json": ("partial", 11, 7),
}
with tempfile.TemporaryDirectory() as directory:
for name, wanted in expected.items():
output = Path(directory) / name
result = subprocess.run(["bash", ROOT / "tools/analyse.sh", FIXTURES / name,
"--evidence-root", FIXTURES, "--output", output,
"--reference-time", "2026-09-15T12:00:00Z", "--producer-version", "test"],
text=True, capture_output=True)
self.assertEqual(result.returncode, 0, result.stderr)
doc = json.loads(output.read_text())
self.assertEqual((doc["collection"]["status"], len(doc["observations"]), len(doc["findings"])), wanted)
if __name__ == "__main__": unittest.main()

92
tools/analyse.jq Normal file
View File

@ -0,0 +1,92 @@
def slug: tostring | gsub("[^A-Za-z0-9._:-]+"; "-") | gsub("^-|-$"; "") | ascii_downcase | if . == "" then "unknown" else . end;
def timestamp: type == "string" and test("^[0-9]{4}-[0-9]{2}-[0-9]{2}T.*(Z|[+-][0-9]{2}:[0-9]{2})$");
def epoch:
capture("^(?<date>[0-9]{4}-[0-9]{2}-[0-9]{2})T(?<time>[0-9]{2}:[0-9]{2}:[0-9]{2})(?:\\.[0-9]+)?(?<zone>Z|[+-][0-9]{2}:[0-9]{2})$") as $p |
(($p.date+"T"+$p.time+"Z")|fromdateiso8601) as $base |
if $p.zone=="Z" then $base else ($p.zone|capture("(?<sign>[+-])(?<hour>[0-9]{2}):(?<minute>[0-9]{2})")) as $z |
$base - ((if $z.sign=="+" then 1 else -1 end)*(($z.hour|tonumber)*3600+($z.minute|tonumber)*60)) end;
def ptr($parts): "/" + ($parts | map(tostring | gsub("~";"~0") | gsub("/";"~1")) | join("/"));
def normalize_section:
if type != "object" then {status:"unavailable",record_count:null,truncated:false,reason:"Legacy section is not an object."}
else . as $s | ($s.status // null) as $source |
(if ["ok","skipped","unavailable"] | index($source) then $source else "unavailable" end) as $status |
{status:$status,
record_count:(if ($s.count|type)=="number" and $s.count >= 0 and ($s.count|floor)==$s.count then $s.count elif ($s.data|type)=="array" then ($s.data|length) else null end),
truncated:($s.truncated // false | if . then true else false end)} +
(if $status == "ok" then {} else {reason:(if ($s.error|type)=="string" and ($s.error|test("\\S")) then $s.error elif ["skipped","unavailable"]|index($source) then "Legacy capture marked section \($status) without a reason." else "Legacy capture used unsupported section status: \($source)." end)} end)
end;
def rule_for($rules;$provider;$id): first($rules[] | select(.provider == $provider and (.event_id|tostring)==($id|tostring))) // null;
def event_sig($provider;$id;$time;$message): [$provider,($id|tostring),($time|tostring|sub("\\.[0-9]+Z$";"Z")),($message//""|tostring|gsub("\\s+";" "))] | @json;
def add_event($section;$index;$event;$time_field;$provider_field;$id_field):
($event[$provider_field] // "Unknown provider" | tostring) as $provider |
($event[$id_field] // "unknown") as $id |
{id:"event-\($section|slug)-\($index+1)",section:$section,source_pointer:ptr(["sections",$section,"data",$index]),value:$event,
_provider:$provider,_event_id:$id,_signature:event_sig($provider;$id;$event[$time_field];$event.Message)} +
(if ($event[$time_field]|timestamp) then {observed_at:$event[$time_field]} else {} end);
def watch_matches($capture;$matcher):
($capture.sections[$matcher.section] // null) as $section |
if ($section|type)!="object" or $section.status != "ok" or ($section.data|type)!="array" then [] else
[$section.data | to_entries[] | select((.value|type)=="object") |
. as $entry | ($entry.value[$matcher.field] // "" | tostring | gsub("^\"|\"$";"") | gsub("\\\\";"/")) as $raw |
(if $matcher.basename then ($raw|split("/")|last) else $raw end) as $identity |
select([$matcher.values[]|tostring|ascii_downcase] | index(($identity|tostring|ascii_downcase))) |
select(($matcher.versions // null)==null or (($matcher.versions|index($entry.value[$matcher.version_field // "DriverVersion"]|tostring)) != null)) |
select(($matcher.enabled_only // false)|not or ([2,"Enabled","enabled"]|index($entry.value.State) != null)) |
{section:$matcher.section,index:$entry.key,item:$entry.value}]
end;
($rules[0]) as $rules | ($watchlist[0]) as $watchlist | . as $capture |
($capture.schema_version) as $version |
if (($version|type)!="number" or ([$version]|inside([1,2])|not)) then error("unsupported legacy capture schema version")
elif ($capture.sections|type)!="object" then error("legacy capture requires an object at /sections")
elif ($capture.generated_at|timestamp|not) then error("/generated_at: expected a timezone-qualified ISO timestamp")
elif ($reference_time|timestamp|not) then error("--reference-time: expected a timezone-qualified ISO timestamp")
else
($capture.sections | with_entries(.value |= normalize_section)) as $sections |
(if ($sections|length)>0 and all($sections[]; .status=="ok" and (.truncated|not)) then "complete" else "partial" end) as $collection_status |
(if $version==1 then ["recent_system_errors","recent_application_errors"] else ["recent_system_events","recent_application_events"] end) as $event_sections |
(reduce $event_sections[] as $name ([];
. + [($capture.sections[$name].data // [] | to_entries[] | select((.value|type)=="object") | add_event($name;.key;.value;"TimeCreated";"ProviderName";"Id"))]
)) as $primary |
(reduce (($capture.sections.reliability_records.data // []) | to_entries[]) as $entry
({items:$primary,signatures:($primary|map(._signature))};
($entry.value.SourceName // "Unknown provider"|tostring) as $provider |
($entry.value.EventIdentifier // "unknown") as $id |
(rule_for($rules;$provider;$id)) as $rule |
(event_sig($provider;$id;$entry.value.TimeGenerated;$entry.value.Message)) as $sig |
if $rule != null and (["warning","critical"]|index($rule.severity)) and (.signatures|index($sig)|not) then
.items += [add_event("reliability_records";$entry.key;$entry.value;"TimeGenerated";"SourceName";"EventIdentifier")] | .signatures += [$sig]
else . end
).items) as $events |
(reduce $events[] as $event ([];
($event._provider+"\u0000"+($event._event_id|tostring)) as $key |
if any(.[]; .key==$key) then map(if .key==$key then .evidence += [$event.id] else . end)
else . + [{key:$key,provider:$event._provider,event_id:$event._event_id,evidence:[$event.id]}] end
)) as $groups |
([$groups[] | . as $group | (rule_for($rules;.provider;.event_id)) as $rule |
if $rule == null then {id:"event-\(.provider|slug)-\(.event_id|slug)",severity:"unanalysed",confidence:0,title:"Unanalysed event: \(.provider) / \(.event_id)",description:"The event is preserved for review; no migrated technical rule matches it.",evidence}
else {id:"event-\(.provider|slug)-\(.event_id|slug)",severity:$rule.severity,confidence:(if $collection_status=="complete" then $rule.confidence else ([50,$rule.confidence]|min) end),title:"\(.provider) event \(.event_id)",description:$rule.description,evidence} +
(if ($rule.fading//false) then
([.evidence[] as $id | $events[] | select(.id==$id and .observed_at) | .observed_at | epoch] | max // null) as $latest |
if $latest==null then {relevance_weight:1} else (($reference_time|epoch)-$latest)/86400|floor as $age |
{relevance_weight:(if $age<=7 then 1 elif $age>=90 then 0.5 else (1-0.5*($age-7)/83)*1000000|round/1000000 end),age_days:([$age,0]|max)} end
else {} end)
end]) as $event_findings |
(reduce $watchlist[] as $entry ({observations:$events,findings:$event_findings};
([$entry.match[] | watch_matches($capture;.)[]]) as $matches |
([$entry.requires[]? | watch_matches($capture;.)]) as $requirements |
(if $entry.events then [.observations[] | select((.value.ProviderName//"")==$entry.events.provider and ($entry.events.ids|map(tostring)|index(.value.Id|tostring))) | .id] else [] end) as $event_evidence |
if ($matches|length)==0 or any($requirements[];length==0) or ($entry.events and (($event_evidence|length)<$entry.events.minimum)) then .
else ([$matches[], $requirements[][]] | unique_by([.section,.index])) as $all |
reduce $all[] as $match (.;
("watch-\($entry.id|slug)-\($match.section|slug)-\($match.index+1)") as $id |
if any(.observations[];.id==$id) then . else .observations += [{id:$id,section:$match.section,source_pointer:ptr(["sections",$match.section,"data",$match.index]),value:$match.item}] end) |
.findings += [{id:"watchlist-\($entry.id)",severity:"warning",confidence:(if $collection_status=="complete" then 85 else 50 end),title:$entry.title,description:$entry.description,evidence:(([$all[]|"watch-\($entry.id|slug)-\(.section|slug)-\(.index+1)"]+$event_evidence)|unique)}]
end
)) as $analysed |
{schema_version:1,metadata:{producer:{name:"win-diagnostics",version:$producer_version},collected_at:$capture.generated_at,analysis_reference_time:$reference_time,source:{artifact:$artifact,schema_version:$version,sha256:$artifact_hash}},
system:({host:($capture.host//{})}+(if ($capture.summary|type)=="object" then {legacy_summary:$capture.summary} else {} end)),
collection:{mode:($capture.mode//"Full"),status:$collection_status,sections:$sections},
observations:[$analysed.observations[]|del(._provider,._event_id,._signature)],findings:$analysed.findings,
warnings:([$capture.warnings[]?|tostring|select(test("\\S"))]+(if $collection_status!="complete" then ["Legacy capture collection is incomplete; finding confidence is capped at 50."] else [] end)+["Migrated analyser supports \($rules|length) event rules and \($watchlist|length) candidate watchlist entries; unmatched events remain unanalysed."])}
end

21
tools/analyse.sh Executable file
View File

@ -0,0 +1,21 @@
#!/usr/bin/env bash
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)
fail(){ echo "error: $*" >&2; exit 1; }; usage(){ echo "Usage: $0 INPUT --evidence-root DIR --output FILE --reference-time TIME --producer-version VERSION [--force]" >&2; exit 2; }
input=${1:-}; [[ -n "$input" ]] || usage; shift
evidence_root= output= reference_time= producer_version= force=false
while (($#)); do case $1 in --evidence-root) evidence_root=$2;shift 2;;--output) output=$2;shift 2;;--reference-time) reference_time=$2;shift 2;;--producer-version) producer_version=$2;shift 2;;--force) force=true;shift;;*) usage;;esac;done
for value in "$evidence_root" "$output" "$reference_time" "$producer_version"; do [[ -n "$value" ]]||usage; done
[[ ! -L "$input" ]] || fail "input symlinks are not supported"
input=$(realpath "$input"); evidence_root=$(realpath "$evidence_root"); output=$(realpath -m "$output")
[[ "$input" == "$evidence_root"/* ]] || fail "input must be inside --evidence-root"
[[ "$input" != "$output" ]] || fail "input and output paths must differ"
[[ ! -e "$output" || "$force" == true ]] || fail "$output: output exists; use --force to replace it"
artifact=${input#"$evidence_root/"}; hash=$(sha256sum "$input"|awk '{print $1}')
mkdir -p "$(dirname "$output")"; temporary=$(mktemp "$(dirname "$output")/.${output##*/}.XXXXXXXX"); trap 'rm -f "$temporary"' EXIT
jq --arg artifact "$artifact" --arg artifact_hash "$hash" --arg reference_time "$reference_time" --arg producer_version "$producer_version" \
--slurpfile rules "$root/rules/event-rules.json" --slurpfile watchlist "$root/rules/watchlist.json" \
-f "$root/tools/analyse.jq" "$input" >"$temporary" || fail "analysis failed"
"$root/tools/validate.sh" "$temporary" --evidence-root "$evidence_root" || fail "generated diagnostic is invalid"
if [[ "$force" == true ]]; then mv -f "$temporary" "$output"; else ln "$temporary" "$output"; fi
echo "Wrote validated diagnostic v1: $output"

30
tools/validate.sh Executable file
View File

@ -0,0 +1,30 @@
#!/usr/bin/env bash
set -euo pipefail
usage(){ echo "Usage: $0 INPUT [--evidence-root DIR]" >&2; exit 2; }; fail(){ echo "error: $*" >&2; exit 1; }
input=${1:-}; [[ -n "$input" ]]||usage; shift; evidence_root=
while (($#)); do case $1 in --evidence-root) evidence_root=$2;shift 2;;*) usage;;esac; done
jq -e '
type=="object" and .schema_version==1 and (.metadata|type)=="object" and
(.metadata.producer.name|type)=="string" and (.metadata.producer.version|type)=="string" and
(.metadata.source.artifact|type)=="string" and (.metadata.source.sha256|test("^[a-f0-9]{64}$")) and
(.collection.sections|type)=="object" and (.observations|type)=="array" and (.findings|type)=="array" and
(([.observations[].id]|length)==([.observations[].id]|unique|length)) and
(([.findings[].id]|length)==([.findings[].id]|unique|length)) and
([.observations[].section] - ([.collection.sections|keys[]]))==[] and
(([.observations[].id]) as $ids | all(.findings[].evidence[]?; $ids|index(.)!=null)) and
(if .collection.status=="complete" then (.collection.sections|length)>0 and all(.collection.sections[];.status=="ok" and (.truncated|not)) else true end)
' "$input" >/dev/null || fail "invalid diagnostic v1: $input"
if [[ -n "$evidence_root" ]]; then
artifact=$(jq -r '.metadata.source.artifact' "$input")
[[ "$artifact" != /* && "$artifact" != *..* && "$artifact" != *\\* ]] || fail "source artifact escapes evidence root"
source=$(realpath "$evidence_root/$artifact"); root=$(realpath "$evidence_root")
[[ "$source" == "$root"/* && -f "$source" ]] || fail "source artifact escapes evidence root"
expected=$(jq -r '.metadata.source.sha256' "$input"); actual=$(sha256sum "$source"|awk '{print $1}')
[[ "$actual" == "$expected" ]] || fail "evidence checksum differs"
jq -e --slurpfile raw "$source" '
$raw[0] as $raw | all(.observations[]; . as $observation |
(.source_pointer|split("/")[1:]|map(gsub("~1";"/")|gsub("~0";"~")|if test("^[0-9]+$") then tonumber else . end)) as $path |
(try (($raw|getpath($path)) == $observation.value) catch false))
' "$input" >/dev/null || fail "observation differs from referenced raw evidence"
fi
echo "Valid diagnostic v1: $input"