beffroi-tools/docs/fix-rust-realease.task.md

332 lines
17 KiB
Markdown

# Task: support complete portable application bundles in rust_realease
## Objective
Extend [rust_realease](https://proxy.dandaedre.com/gitea/idle-org/rust_realease)
so a single release invocation can build a Rust application, prepare its
target-specific runtime dependencies, assemble complete portable archives,
and checksum, publish and verify those archives through the normal workflow.
This is generic release packaging functionality. Do not hard-code FFmpeg,
BtbN, this application, or its target list into the release tool.
Project-specific dependency acquisition may remain a small script; archive
creation, build-output discovery and publication must belong to rust_realease.
This document is an implementation specification for the upstream repository,
not a request to publish a release or change this project's CI into a publisher.
Inspect upstream code and instructions before implementing. The configuration
and hook protocol below are **proposed APIs**, not existing supported options.
Equivalent naming is acceptable if all behavioral requirements are met.
## Observed baseline and concrete problem
The consumer is `pptx-reporter` in
[Beffroi/beffroi-tools](https://proxy.dandaedre.com/gitea/Beffroi/beffroi-tools),
on `feature/rust-reporter`. The locally tested publisher is
`rust_realease 0.1.6-rc.2`. The current
[advanced usage documentation](https://proxy.dandaedre.com/gitea/idle-org/rust_realease/src/branch/main/docs/advanced-usage.md)
was checked on 2026-10-20. Distinguish that installed version from upstream main;
reuse any capabilities that have landed since, rather than implementing them twice.
The reporter has two Cargo binaries, `pptx-reporter` and `pptx-reporter-gui`.
It embeds Typst and fonts, but invokes an external `ffprobe` executable.
Windows users should extract an archive and double-click the GUI with no Python,
Typst, PowerPoint or separate FFmpeg installation. Python may be a build-host
dependency for a configured preparation script.
Current gaps:
- The documented release archives contain the package's declared Cargo binaries.
They do not provide a documented mechanism for runtime executables, notices,
templates, README files or other application resources.
- The documented formats are `tgz`, `binary` and `both`. Windows desktop users
need an ordinary ZIP as well as the existing cargo-binstall distribution.
- Cross `pre-build` prepares a container image/build environment. It is not a
packaging hook with access to completed binaries and target bundle staging.
- A dirty checkout may build in a fresh, publisher-owned target directory.
A project packager cannot safely assume `target/<triple>/release/<binary>`.
- A successful cargo-binstall check of the declared Rust binaries does not by
itself establish that adjacent runtime executables/resources are installed.
The workaround lives in this consumer repository:
- `scripts/fetch_ffprobe.py`: download a target build, check its SHA-256 and
PE/ELF architecture, retain notices/provenance, cache outside Git, and perform
a native version check when possible.
- `scripts/package-portable.py`: independently locate Cargo outputs, copy the
dependency and notices, produce Windows ZIP/Linux tar.gz and checksum files.
- `rust-release.toml`: currently documents calling that packager separately.
Replace the second script's responsibilities with upstream support. Keep
provider-specific download logic reusable through a thin preparation adapter.
## Required application layout
Produce these portable distributions for the currently supported targets:
| Target | Format | Runtime dependency |
| --- | --- | --- |
| `x86_64-pc-windows-gnu` | ZIP | `ffprobe.exe`, built for Windows x86-64 |
| `x86_64-unknown-linux-gnu` | tar.gz/tgz | `ffprobe`, built for Linux x86-64 |
Each archive has one predictable root directory:
```text
pptx-reporter-portable-<target>-v<version>/
pptx-reporter[.exe]
pptx-reporter-gui[.exe]
ffprobe[.exe]
README.txt
LICENSES/
reporter-MIT.txt
DejaVu.txt
ffmpeg/
<upstream notices>
provenance.json
```
Executable permissions must survive Linux extraction. Windows executable names
must have `.exe`. Do not accidentally bundle the host's FFprobe, unrelated
FFmpeg programs, build caches or source checkout. Allow explicitly declared DLLs,
shared libraries and resource directories for applications that need them;
this project's selected FFprobe distributions do not need separate FFmpeg DLLs.
## Required capabilities
### 1. Declarative bundles alongside existing artifacts
Support named bundles with target filters, per-target archive formats, a root
directory/name template, Cargo binary selection, and explicit extra files or
directory trees with destination paths inside the bundle.
Existing configurations must retain their current behavior. Existing
`package.format`, `--artifact-format`, archive names and cargo-binstall metadata
must keep their existing meanings; named bundles are additional artifacts.
Reject unknown or inconsistent configuration rather than silently ignoring it.
If a selected bundle lacks a format for a selected target, fail clearly.
`--target` must also filter hooks, dependency fetches and bundle creation.
Illustrative configuration for this consumer:
```toml
[package]
format = "tgz" # Keep the existing cargo-binstall artifacts.
[[package.bundles]]
name = "portable"
targets = ["x86_64-pc-windows-gnu", "x86_64-unknown-linux-gnu"]
formats = { x86_64-pc-windows-gnu = "zip", x86_64-unknown-linux-gnu = "tgz" }
archive-name = "{name}-portable-{target}-v{version}"
root = "{name}-portable-{target}-v{version}"
cargo-binaries = "all"
files = [
{ source = "LICENSE", destination = "LICENSES/reporter-MIT.txt" },
{ source = "assets/fonts/LICENSE", destination = "LICENSES/DejaVu.txt" },
{ source = "docs/windows-quickstart.txt", destination = "README.txt" },
]
[package.bundles.prepare]
command = ["python3", "scripts/prepare-release.py", "--context", "{context-file}", "--manifest", "{manifest-file}"]
inputs = ["scripts/prepare-release.py", "scripts/fetch_ffprobe.py"]
timeout-seconds = 600
required-destinations = ["ffprobe{binary-ext}", "LICENSES/ffmpeg/LICENSE.txt", "LICENSES/ffmpeg/provenance.json"]
```
`prepare-release.py` above is a proposed thin adapter, not an existing file.
It would call this project's downloader and return the prepared file list.
Define whether static source paths are package- or workspace-relative; default
to package-relative and provide an explicit workspace-relative form.
This must work for selected members of Cargo workspaces.
In templates, `{name}` is the Cargo package name, `{bundle}` is the bundle name,
and `{binary-ext}` is the target suffix. Append the selected archive format's
extension to `archive-name`; do not reinterpret it as a filesystem source path.
### 2. A host-side preparation hook with a stable contract
Run preparation once per selected bundle/target, after successful compilation
and before archive creation, checksums, release creation or uploads. This is
packaging work on the host, not a command inside a Cross build container.
Use an argument vector, not implicit shell interpolation. Explicit shell use
can be configured as `["sh", "..."]`. Define the working directory and supply a
versioned JSON context file containing:
- Package name/version, target triple, profile and target executable suffix.
- Workspace and selected package roots, and the source commit/dirty state.
- An exact mapping of declared binary names to **actual built executable paths**.
- Fresh hook output/staging directory, persistent cache directory, and result
manifest path, all absolute.
- Bundle name and protocol version.
Resolve binary paths from actual build artifacts or the validated build cache.
Do not reconstruct them from a conventional target directory. Support
`CARGO_TARGET_DIR`, publisher-owned fresh build directories, non-default profiles
and workspace members.
The hook writes only its outputs into its supplied output directory and emits
a manifest. For example:
```json
{
"schema_version": 1,
"target": "x86_64-pc-windows-gnu",
"files": [
{
"source": "ffprobe.exe",
"destination": "ffprobe.exe",
"kind": "executable",
"sha256": "<actual 64-character SHA-256>"
},
{
"source": "notices/provenance.json",
"destination": "LICENSES/ffmpeg/provenance.json",
"kind": "file",
"sha256": "<actual 64-character SHA-256>"
},
{
"source": "notices/LICENSE.txt",
"destination": "LICENSES/ffmpeg/LICENSE.txt",
"kind": "file",
"sha256": "<actual 64-character SHA-256>"
}
]
}
```
Paths in that manifest are relative to the hook output directory and bundle
root respectively. The publisher validates them, copies only declared files,
checks the recorded hashes, and owns final archive construction. Static inputs
and Cargo binaries are also hashed and included in an artifact manifest.
Stream hook output with target/bundle labels. Identify the failing command and
exit status. Stop on nonzero exit, timeout, cancellation, missing required
outputs, invalid manifest or hash mismatch. Clean temporary staging and do not
create/upload a partial release. Terminate hook subprocesses on cancellation.
Do not pass Gitea tokens or publication credentials into hooks; preserve the
existing child-process credential isolation.
### 3. Runtime dependency integrity and caching
A hook can download dependencies into the supplied persistent cache; it must
not need tracked binaries or tracked generated outputs. Allow explicit digest
pins and immutable release identifiers through declared configuration/inputs.
For this consumer, continue using verified target-specific non-shared LGPL
FFprobe builds. Preserve their upstream notices and provenance, including
provider URL, release/asset IDs, archive SHA-256 and extracted file hashes.
Do not make a provider-specific built-in downloader a prerequisite for this fix.
Keep build-cache reuse separate from bundle preparation. Reusing valid Rust
executables must not accidentally reuse stale auxiliary files. A straightforward
first implementation can rerun preparation for each packaging invocation while
the hook reuses its own checksum-verified download cache.
If prepared bundles themselves are cached, fingerprint the effective bundle
configuration, hook command and declared inputs, static file contents, actual
binary hashes, and resolved dependency identities/hashes. Changes in a moving
`latest` dependency must be resolved before accepting a bundle cache hit.
Do not treat an unchanged Git commit as proof of identical runtime dependencies.
### 4. Validate and package all files consistently
Reject absolute destinations, `..`, destination collisions, duplicate entries,
symlinks/hardlinks escaping approved roots, device files and missing required
inputs. For Windows bundles also reject case-insensitive destination collisions
and invalid/reserved Windows destination names. Hooks cannot replace a declared
Cargo executable by writing another file to its destination.
Check target architecture/format for declared runtime executables. A foreign
executable must not be run on the build host merely because packaging succeeded.
Allow an explicit native smoke-test command; record passed, failed and skipped
checks distinctly. Cross-compilation is not Windows runtime validation.
Produce archives atomically with deterministic ordering, normalized ownership,
timestamps and permissions, preserving the intended executable bits. Identical
inputs should produce identical archives. Avoid leaking absolute host paths or
environment secrets into generated public manifests.
### 5. Integrate bundles into the complete release lifecycle
- `--dry-run`: validate configuration and show bundle formats, target selections,
static inclusions, hook commands and declared required outputs. Do not execute
hooks, fetch dependencies or produce archives. Mark dynamic results unresolved.
- `--compile-only` and `--stop-after package`: build/reuse binaries, run preparation
and produce complete local bundles and checksums. No Gitea token, release,
tag creation or push is required.
- `--skip-tests`: skip the existing test phase only; do not bypass hook success,
file/hash checks or archive validation.
- Normal publication: finish and validate **all** requested artifacts before
release creation/upload. Include named bundles in the existing `SHA256SUMS`,
collision handling, `--replace-assets` and post-upload download verification.
Keep existing draft/prerelease behavior and do not delete unrelated assets.
- Print every resulting artifact path and its checks performed/skipped. Provide
a machine-readable artifact manifest so consumers do not scrape console output.
- Fail before remote mutations if any bundle fails. Do not promise transactional
rollback of remote uploads where the existing backend cannot provide it.
### 6. Preserve the cargo-binstall contract
Keep the existing canonical `{name}-{target}-v{version}.tgz` archives and their
metadata/install checks working. Portable archives use distinct names, for
example `{name}-portable-{target}-v{version}.zip`, avoiding ambiguous asset matches.
Do not assume extra files in an archive become installed beside Cargo binaries
by cargo-binstall. Verify its actual behavior before promising that. For this
consumer the supported baseline is:
- The portable bundle contains everything and is validated as a complete tree.
- The cargo-binstall installation still installs the declared Rust binaries.
Its usage documentation explicitly states the remaining external FFprobe
requirement unless runtime-dependency installation is actually implemented
and tested end to end.
Archive verification must understand the selected artifact's declared contents,
not reject a valid portable bundle for containing more than Cargo binaries.
## Tests and acceptance criteria
Use local fixture hooks/downloads and small fixture binaries in automated tests;
do not depend on mutable GitHub releases or download hundreds of MB per test.
- [ ] Legacy configurations produce unchanged artifact names, layout and
cargo-binstall behavior, including multiple binaries.
- [ ] A fixture hook receives correct target/version/profile, source roots and
real executable paths for clean, dirty, cached and custom-target-dir builds.
- [ ] The Windows ZIP and Linux tar.gz contain both application binaries, a
target-matching FFprobe fixture, README and license/provenance files.
- [ ] Target filtering does not run hooks or fetch dependencies for other targets.
- [ ] Nonzero exit, timeout/cancellation, missing executable/license, bad hashes,
malformed manifests and architecture mismatches prevent publication.
- [ ] Traversal, escaping links, duplicate/case-colliding destinations, reserved
Windows names and attempts to replace Cargo binaries are rejected.
- [ ] Paths with spaces and Unicode are handled without shell reinterpretation.
- [ ] Dry-run leaves no hook marker, download or archive. Compile-only runs
preparation and packaging with no token, tag, release or push.
- [ ] Changing a hook/input/license/dependency digest invalidates affected bundle
reuse while permitting unchanged Rust binaries to remain cached.
- [ ] Repeated packaging of identical inputs is byte-identical; stale staging
files never appear in a later archive; executable modes are correct.
- [ ] Mocked publication includes all bundle checksums, collision checks and
post-upload verification; one failed target causes no release creation.
- [ ] Existing cargo-binstall checks still pass and are reported separately from
portable-tree and native-runtime checks.
- [ ] Update upstream docs/schema/examples and implement a consumer migration
recipe using the actual downloader, not an FFmpeg-specific publisher fork.
Final consumer smoke test, once the implementation is available:
1. Configure the portable bundle and thin downloader adapter in this project.
2. Run `rust_realease . --package pptx-reporter --compile-only`.
3. Verify both portable archives and checksums, and run Linux extraction/probing
and report generation using the bundled FFprobe without relying on PATH.
4. On Windows, extract the ZIP and launch the GUI without Python, Typst or system
FFmpeg. Generate reports using its adjacent FFprobe. Record this as unverified
if no Windows validation machine is available.
5. Confirm no dependency binary, archive or generated staging file entered Git.
Remove the separate project archive packager once upstream owns its function.
Do not create an actual public release merely to test this task. This project's
Jenkins remains checks/coverage/badges only; release publication stays explicit.
Signing, installers, automatic updates and rewriting dependency downloaders in
Rust are outside this task.