diff --git a/docs/fix-rust-realease.task.md b/docs/fix-rust-realease.task.md new file mode 100644 index 0000000..0d491b9 --- /dev/null +++ b/docs/fix-rust-realease.task.md @@ -0,0 +1,331 @@ +# Task: support complete portable application bundles in rust_realease + +## Objective + +Extend [rust_realease](https://proxy.dandaedre.com/gitea/idle-org/rust_realease) +so a single release invocation can build a Rust application, prepare its +target-specific runtime dependencies, assemble complete portable archives, +and checksum, publish and verify those archives through the normal workflow. + +This is generic release packaging functionality. Do not hard-code FFmpeg, +BtbN, this application, or its target list into the release tool. +Project-specific dependency acquisition may remain a small script; archive +creation, build-output discovery and publication must belong to rust_realease. + +This document is an implementation specification for the upstream repository, +not a request to publish a release or change this project's CI into a publisher. +Inspect upstream code and instructions before implementing. The configuration +and hook protocol below are **proposed APIs**, not existing supported options. +Equivalent naming is acceptable if all behavioral requirements are met. + +## Observed baseline and concrete problem + +The consumer is `pptx-reporter` in +[Beffroi/beffroi-tools](https://proxy.dandaedre.com/gitea/Beffroi/beffroi-tools), +on `feature/rust-reporter`. The locally tested publisher is +`rust_realease 0.1.6-rc.2`. The current +[advanced usage documentation](https://proxy.dandaedre.com/gitea/idle-org/rust_realease/src/branch/main/docs/advanced-usage.md) +was checked on 2026-10-20. Distinguish that installed version from upstream main; +reuse any capabilities that have landed since, rather than implementing them twice. + +The reporter has two Cargo binaries, `pptx-reporter` and `pptx-reporter-gui`. +It embeds Typst and fonts, but invokes an external `ffprobe` executable. +Windows users should extract an archive and double-click the GUI with no Python, +Typst, PowerPoint or separate FFmpeg installation. Python may be a build-host +dependency for a configured preparation script. + +Current gaps: + +- The documented release archives contain the package's declared Cargo binaries. + They do not provide a documented mechanism for runtime executables, notices, + templates, README files or other application resources. +- The documented formats are `tgz`, `binary` and `both`. Windows desktop users + need an ordinary ZIP as well as the existing cargo-binstall distribution. +- Cross `pre-build` prepares a container image/build environment. It is not a + packaging hook with access to completed binaries and target bundle staging. +- A dirty checkout may build in a fresh, publisher-owned target directory. + A project packager cannot safely assume `target//release/`. +- A successful cargo-binstall check of the declared Rust binaries does not by + itself establish that adjacent runtime executables/resources are installed. + +The workaround lives in this consumer repository: + +- `scripts/fetch_ffprobe.py`: download a target build, check its SHA-256 and + PE/ELF architecture, retain notices/provenance, cache outside Git, and perform + a native version check when possible. +- `scripts/package-portable.py`: independently locate Cargo outputs, copy the + dependency and notices, produce Windows ZIP/Linux tar.gz and checksum files. +- `rust-release.toml`: currently documents calling that packager separately. + +Replace the second script's responsibilities with upstream support. Keep +provider-specific download logic reusable through a thin preparation adapter. + +## Required application layout + +Produce these portable distributions for the currently supported targets: + +| Target | Format | Runtime dependency | +| --- | --- | --- | +| `x86_64-pc-windows-gnu` | ZIP | `ffprobe.exe`, built for Windows x86-64 | +| `x86_64-unknown-linux-gnu` | tar.gz/tgz | `ffprobe`, built for Linux x86-64 | + +Each archive has one predictable root directory: + +```text +pptx-reporter-portable--v/ + pptx-reporter[.exe] + pptx-reporter-gui[.exe] + ffprobe[.exe] + README.txt + LICENSES/ + reporter-MIT.txt + DejaVu.txt + ffmpeg/ + + provenance.json +``` + +Executable permissions must survive Linux extraction. Windows executable names +must have `.exe`. Do not accidentally bundle the host's FFprobe, unrelated +FFmpeg programs, build caches or source checkout. Allow explicitly declared DLLs, +shared libraries and resource directories for applications that need them; +this project's selected FFprobe distributions do not need separate FFmpeg DLLs. + +## Required capabilities + +### 1. Declarative bundles alongside existing artifacts + +Support named bundles with target filters, per-target archive formats, a root +directory/name template, Cargo binary selection, and explicit extra files or +directory trees with destination paths inside the bundle. + +Existing configurations must retain their current behavior. Existing +`package.format`, `--artifact-format`, archive names and cargo-binstall metadata +must keep their existing meanings; named bundles are additional artifacts. +Reject unknown or inconsistent configuration rather than silently ignoring it. +If a selected bundle lacks a format for a selected target, fail clearly. +`--target` must also filter hooks, dependency fetches and bundle creation. + +Illustrative configuration for this consumer: + +```toml +[package] +format = "tgz" # Keep the existing cargo-binstall artifacts. + +[[package.bundles]] +name = "portable" +targets = ["x86_64-pc-windows-gnu", "x86_64-unknown-linux-gnu"] +formats = { x86_64-pc-windows-gnu = "zip", x86_64-unknown-linux-gnu = "tgz" } +archive-name = "{name}-portable-{target}-v{version}" +root = "{name}-portable-{target}-v{version}" +cargo-binaries = "all" +files = [ + { source = "LICENSE", destination = "LICENSES/reporter-MIT.txt" }, + { source = "assets/fonts/LICENSE", destination = "LICENSES/DejaVu.txt" }, + { source = "docs/windows-quickstart.txt", destination = "README.txt" }, +] + +[package.bundles.prepare] +command = ["python3", "scripts/prepare-release.py", "--context", "{context-file}", "--manifest", "{manifest-file}"] +inputs = ["scripts/prepare-release.py", "scripts/fetch_ffprobe.py"] +timeout-seconds = 600 +required-destinations = ["ffprobe{binary-ext}", "LICENSES/ffmpeg/LICENSE.txt", "LICENSES/ffmpeg/provenance.json"] +``` + +`prepare-release.py` above is a proposed thin adapter, not an existing file. +It would call this project's downloader and return the prepared file list. +Define whether static source paths are package- or workspace-relative; default +to package-relative and provide an explicit workspace-relative form. +This must work for selected members of Cargo workspaces. +In templates, `{name}` is the Cargo package name, `{bundle}` is the bundle name, +and `{binary-ext}` is the target suffix. Append the selected archive format's +extension to `archive-name`; do not reinterpret it as a filesystem source path. + +### 2. A host-side preparation hook with a stable contract + +Run preparation once per selected bundle/target, after successful compilation +and before archive creation, checksums, release creation or uploads. This is +packaging work on the host, not a command inside a Cross build container. + +Use an argument vector, not implicit shell interpolation. Explicit shell use +can be configured as `["sh", "..."]`. Define the working directory and supply a +versioned JSON context file containing: + +- Package name/version, target triple, profile and target executable suffix. +- Workspace and selected package roots, and the source commit/dirty state. +- An exact mapping of declared binary names to **actual built executable paths**. +- Fresh hook output/staging directory, persistent cache directory, and result + manifest path, all absolute. +- Bundle name and protocol version. + +Resolve binary paths from actual build artifacts or the validated build cache. +Do not reconstruct them from a conventional target directory. Support +`CARGO_TARGET_DIR`, publisher-owned fresh build directories, non-default profiles +and workspace members. + +The hook writes only its outputs into its supplied output directory and emits +a manifest. For example: + +```json +{ + "schema_version": 1, + "target": "x86_64-pc-windows-gnu", + "files": [ + { + "source": "ffprobe.exe", + "destination": "ffprobe.exe", + "kind": "executable", + "sha256": "" + }, + { + "source": "notices/provenance.json", + "destination": "LICENSES/ffmpeg/provenance.json", + "kind": "file", + "sha256": "" + }, + { + "source": "notices/LICENSE.txt", + "destination": "LICENSES/ffmpeg/LICENSE.txt", + "kind": "file", + "sha256": "" + } + ] +} +``` + +Paths in that manifest are relative to the hook output directory and bundle +root respectively. The publisher validates them, copies only declared files, +checks the recorded hashes, and owns final archive construction. Static inputs +and Cargo binaries are also hashed and included in an artifact manifest. + +Stream hook output with target/bundle labels. Identify the failing command and +exit status. Stop on nonzero exit, timeout, cancellation, missing required +outputs, invalid manifest or hash mismatch. Clean temporary staging and do not +create/upload a partial release. Terminate hook subprocesses on cancellation. +Do not pass Gitea tokens or publication credentials into hooks; preserve the +existing child-process credential isolation. + +### 3. Runtime dependency integrity and caching + +A hook can download dependencies into the supplied persistent cache; it must +not need tracked binaries or tracked generated outputs. Allow explicit digest +pins and immutable release identifiers through declared configuration/inputs. + +For this consumer, continue using verified target-specific non-shared LGPL +FFprobe builds. Preserve their upstream notices and provenance, including +provider URL, release/asset IDs, archive SHA-256 and extracted file hashes. +Do not make a provider-specific built-in downloader a prerequisite for this fix. + +Keep build-cache reuse separate from bundle preparation. Reusing valid Rust +executables must not accidentally reuse stale auxiliary files. A straightforward +first implementation can rerun preparation for each packaging invocation while +the hook reuses its own checksum-verified download cache. + +If prepared bundles themselves are cached, fingerprint the effective bundle +configuration, hook command and declared inputs, static file contents, actual +binary hashes, and resolved dependency identities/hashes. Changes in a moving +`latest` dependency must be resolved before accepting a bundle cache hit. +Do not treat an unchanged Git commit as proof of identical runtime dependencies. + +### 4. Validate and package all files consistently + +Reject absolute destinations, `..`, destination collisions, duplicate entries, +symlinks/hardlinks escaping approved roots, device files and missing required +inputs. For Windows bundles also reject case-insensitive destination collisions +and invalid/reserved Windows destination names. Hooks cannot replace a declared +Cargo executable by writing another file to its destination. + +Check target architecture/format for declared runtime executables. A foreign +executable must not be run on the build host merely because packaging succeeded. +Allow an explicit native smoke-test command; record passed, failed and skipped +checks distinctly. Cross-compilation is not Windows runtime validation. + +Produce archives atomically with deterministic ordering, normalized ownership, +timestamps and permissions, preserving the intended executable bits. Identical +inputs should produce identical archives. Avoid leaking absolute host paths or +environment secrets into generated public manifests. + +### 5. Integrate bundles into the complete release lifecycle + +- `--dry-run`: validate configuration and show bundle formats, target selections, + static inclusions, hook commands and declared required outputs. Do not execute + hooks, fetch dependencies or produce archives. Mark dynamic results unresolved. +- `--compile-only` and `--stop-after package`: build/reuse binaries, run preparation + and produce complete local bundles and checksums. No Gitea token, release, + tag creation or push is required. +- `--skip-tests`: skip the existing test phase only; do not bypass hook success, + file/hash checks or archive validation. +- Normal publication: finish and validate **all** requested artifacts before + release creation/upload. Include named bundles in the existing `SHA256SUMS`, + collision handling, `--replace-assets` and post-upload download verification. + Keep existing draft/prerelease behavior and do not delete unrelated assets. +- Print every resulting artifact path and its checks performed/skipped. Provide + a machine-readable artifact manifest so consumers do not scrape console output. +- Fail before remote mutations if any bundle fails. Do not promise transactional + rollback of remote uploads where the existing backend cannot provide it. + +### 6. Preserve the cargo-binstall contract + +Keep the existing canonical `{name}-{target}-v{version}.tgz` archives and their +metadata/install checks working. Portable archives use distinct names, for +example `{name}-portable-{target}-v{version}.zip`, avoiding ambiguous asset matches. + +Do not assume extra files in an archive become installed beside Cargo binaries +by cargo-binstall. Verify its actual behavior before promising that. For this +consumer the supported baseline is: + +- The portable bundle contains everything and is validated as a complete tree. +- The cargo-binstall installation still installs the declared Rust binaries. + Its usage documentation explicitly states the remaining external FFprobe + requirement unless runtime-dependency installation is actually implemented + and tested end to end. + +Archive verification must understand the selected artifact's declared contents, +not reject a valid portable bundle for containing more than Cargo binaries. + +## Tests and acceptance criteria + +Use local fixture hooks/downloads and small fixture binaries in automated tests; +do not depend on mutable GitHub releases or download hundreds of MB per test. + +- [ ] Legacy configurations produce unchanged artifact names, layout and + cargo-binstall behavior, including multiple binaries. +- [ ] A fixture hook receives correct target/version/profile, source roots and + real executable paths for clean, dirty, cached and custom-target-dir builds. +- [ ] The Windows ZIP and Linux tar.gz contain both application binaries, a + target-matching FFprobe fixture, README and license/provenance files. +- [ ] Target filtering does not run hooks or fetch dependencies for other targets. +- [ ] Nonzero exit, timeout/cancellation, missing executable/license, bad hashes, + malformed manifests and architecture mismatches prevent publication. +- [ ] Traversal, escaping links, duplicate/case-colliding destinations, reserved + Windows names and attempts to replace Cargo binaries are rejected. +- [ ] Paths with spaces and Unicode are handled without shell reinterpretation. +- [ ] Dry-run leaves no hook marker, download or archive. Compile-only runs + preparation and packaging with no token, tag, release or push. +- [ ] Changing a hook/input/license/dependency digest invalidates affected bundle + reuse while permitting unchanged Rust binaries to remain cached. +- [ ] Repeated packaging of identical inputs is byte-identical; stale staging + files never appear in a later archive; executable modes are correct. +- [ ] Mocked publication includes all bundle checksums, collision checks and + post-upload verification; one failed target causes no release creation. +- [ ] Existing cargo-binstall checks still pass and are reported separately from + portable-tree and native-runtime checks. +- [ ] Update upstream docs/schema/examples and implement a consumer migration + recipe using the actual downloader, not an FFmpeg-specific publisher fork. + +Final consumer smoke test, once the implementation is available: + +1. Configure the portable bundle and thin downloader adapter in this project. +2. Run `rust_realease . --package pptx-reporter --compile-only`. +3. Verify both portable archives and checksums, and run Linux extraction/probing + and report generation using the bundled FFprobe without relying on PATH. +4. On Windows, extract the ZIP and launch the GUI without Python, Typst or system + FFmpeg. Generate reports using its adjacent FFprobe. Record this as unverified + if no Windows validation machine is available. +5. Confirm no dependency binary, archive or generated staging file entered Git. + Remove the separate project archive packager once upstream owns its function. + +Do not create an actual public release merely to test this task. This project's +Jenkins remains checks/coverage/badges only; release publication stays explicit. +Signing, installers, automatic updates and rewriting dependency downloaders in +Rust are outside this task. diff --git a/docs/rust-reporter.md b/docs/rust-reporter.md index 43eec10..142ae1d 100644 --- a/docs/rust-reporter.md +++ b/docs/rust-reporter.md @@ -94,6 +94,10 @@ python3 scripts/package-portable.py --target x86_64-unknown-linux-gnu \ --fetch-ffprobe ``` +The [upstream packaging task](fix-rust-realease.task.md) specifies the hooks, +runtime-file inclusion and portable artifact support needed to replace this +separate packaging step with the normal `rust_realease` workflow. + This creates a ZIP under `dist/`, with both reporter binaries, FFprobe, licences, download provenance, quick-start instructions and a SHA-256 checksum (Linux uses `.tar.gz`). These commands expect the reporter binaries under