332 lines
17 KiB
Markdown
332 lines
17 KiB
Markdown
# Task: support complete portable application bundles in rust_realease
|
|
|
|
## Objective
|
|
|
|
Extend [rust_realease](https://proxy.dandaedre.com/gitea/idle-org/rust_realease)
|
|
so a single release invocation can build a Rust application, prepare its
|
|
target-specific runtime dependencies, assemble complete portable archives,
|
|
and checksum, publish and verify those archives through the normal workflow.
|
|
|
|
This is generic release packaging functionality. Do not hard-code FFmpeg,
|
|
BtbN, this application, or its target list into the release tool.
|
|
Project-specific dependency acquisition may remain a small script; archive
|
|
creation, build-output discovery and publication must belong to rust_realease.
|
|
|
|
This document is an implementation specification for the upstream repository,
|
|
not a request to publish a release or change this project's CI into a publisher.
|
|
Inspect upstream code and instructions before implementing. The configuration
|
|
and hook protocol below are **proposed APIs**, not existing supported options.
|
|
Equivalent naming is acceptable if all behavioral requirements are met.
|
|
|
|
## Observed baseline and concrete problem
|
|
|
|
The consumer is `pptx-reporter` in
|
|
[Beffroi/beffroi-tools](https://proxy.dandaedre.com/gitea/Beffroi/beffroi-tools),
|
|
on `feature/rust-reporter`. The locally tested publisher is
|
|
`rust_realease 0.1.6-rc.2`. The current
|
|
[advanced usage documentation](https://proxy.dandaedre.com/gitea/idle-org/rust_realease/src/branch/main/docs/advanced-usage.md)
|
|
was checked on 2026-10-20. Distinguish that installed version from upstream main;
|
|
reuse any capabilities that have landed since, rather than implementing them twice.
|
|
|
|
The reporter has two Cargo binaries, `pptx-reporter` and `pptx-reporter-gui`.
|
|
It embeds Typst and fonts, but invokes an external `ffprobe` executable.
|
|
Windows users should extract an archive and double-click the GUI with no Python,
|
|
Typst, PowerPoint or separate FFmpeg installation. Python may be a build-host
|
|
dependency for a configured preparation script.
|
|
|
|
Current gaps:
|
|
|
|
- The documented release archives contain the package's declared Cargo binaries.
|
|
They do not provide a documented mechanism for runtime executables, notices,
|
|
templates, README files or other application resources.
|
|
- The documented formats are `tgz`, `binary` and `both`. Windows desktop users
|
|
need an ordinary ZIP as well as the existing cargo-binstall distribution.
|
|
- Cross `pre-build` prepares a container image/build environment. It is not a
|
|
packaging hook with access to completed binaries and target bundle staging.
|
|
- A dirty checkout may build in a fresh, publisher-owned target directory.
|
|
A project packager cannot safely assume `target/<triple>/release/<binary>`.
|
|
- A successful cargo-binstall check of the declared Rust binaries does not by
|
|
itself establish that adjacent runtime executables/resources are installed.
|
|
|
|
The workaround lives in this consumer repository:
|
|
|
|
- `scripts/fetch_ffprobe.py`: download a target build, check its SHA-256 and
|
|
PE/ELF architecture, retain notices/provenance, cache outside Git, and perform
|
|
a native version check when possible.
|
|
- `scripts/package-portable.py`: independently locate Cargo outputs, copy the
|
|
dependency and notices, produce Windows ZIP/Linux tar.gz and checksum files.
|
|
- `rust-release.toml`: currently documents calling that packager separately.
|
|
|
|
Replace the second script's responsibilities with upstream support. Keep
|
|
provider-specific download logic reusable through a thin preparation adapter.
|
|
|
|
## Required application layout
|
|
|
|
Produce these portable distributions for the currently supported targets:
|
|
|
|
| Target | Format | Runtime dependency |
|
|
| --- | --- | --- |
|
|
| `x86_64-pc-windows-gnu` | ZIP | `ffprobe.exe`, built for Windows x86-64 |
|
|
| `x86_64-unknown-linux-gnu` | tar.gz/tgz | `ffprobe`, built for Linux x86-64 |
|
|
|
|
Each archive has one predictable root directory:
|
|
|
|
```text
|
|
pptx-reporter-portable-<target>-v<version>/
|
|
pptx-reporter[.exe]
|
|
pptx-reporter-gui[.exe]
|
|
ffprobe[.exe]
|
|
README.txt
|
|
LICENSES/
|
|
reporter-MIT.txt
|
|
DejaVu.txt
|
|
ffmpeg/
|
|
<upstream notices>
|
|
provenance.json
|
|
```
|
|
|
|
Executable permissions must survive Linux extraction. Windows executable names
|
|
must have `.exe`. Do not accidentally bundle the host's FFprobe, unrelated
|
|
FFmpeg programs, build caches or source checkout. Allow explicitly declared DLLs,
|
|
shared libraries and resource directories for applications that need them;
|
|
this project's selected FFprobe distributions do not need separate FFmpeg DLLs.
|
|
|
|
## Required capabilities
|
|
|
|
### 1. Declarative bundles alongside existing artifacts
|
|
|
|
Support named bundles with target filters, per-target archive formats, a root
|
|
directory/name template, Cargo binary selection, and explicit extra files or
|
|
directory trees with destination paths inside the bundle.
|
|
|
|
Existing configurations must retain their current behavior. Existing
|
|
`package.format`, `--artifact-format`, archive names and cargo-binstall metadata
|
|
must keep their existing meanings; named bundles are additional artifacts.
|
|
Reject unknown or inconsistent configuration rather than silently ignoring it.
|
|
If a selected bundle lacks a format for a selected target, fail clearly.
|
|
`--target` must also filter hooks, dependency fetches and bundle creation.
|
|
|
|
Illustrative configuration for this consumer:
|
|
|
|
```toml
|
|
[package]
|
|
format = "tgz" # Keep the existing cargo-binstall artifacts.
|
|
|
|
[[package.bundles]]
|
|
name = "portable"
|
|
targets = ["x86_64-pc-windows-gnu", "x86_64-unknown-linux-gnu"]
|
|
formats = { x86_64-pc-windows-gnu = "zip", x86_64-unknown-linux-gnu = "tgz" }
|
|
archive-name = "{name}-portable-{target}-v{version}"
|
|
root = "{name}-portable-{target}-v{version}"
|
|
cargo-binaries = "all"
|
|
files = [
|
|
{ source = "LICENSE", destination = "LICENSES/reporter-MIT.txt" },
|
|
{ source = "assets/fonts/LICENSE", destination = "LICENSES/DejaVu.txt" },
|
|
{ source = "docs/windows-quickstart.txt", destination = "README.txt" },
|
|
]
|
|
|
|
[package.bundles.prepare]
|
|
command = ["python3", "scripts/prepare-release.py", "--context", "{context-file}", "--manifest", "{manifest-file}"]
|
|
inputs = ["scripts/prepare-release.py", "scripts/fetch_ffprobe.py"]
|
|
timeout-seconds = 600
|
|
required-destinations = ["ffprobe{binary-ext}", "LICENSES/ffmpeg/LICENSE.txt", "LICENSES/ffmpeg/provenance.json"]
|
|
```
|
|
|
|
`prepare-release.py` above is a proposed thin adapter, not an existing file.
|
|
It would call this project's downloader and return the prepared file list.
|
|
Define whether static source paths are package- or workspace-relative; default
|
|
to package-relative and provide an explicit workspace-relative form.
|
|
This must work for selected members of Cargo workspaces.
|
|
In templates, `{name}` is the Cargo package name, `{bundle}` is the bundle name,
|
|
and `{binary-ext}` is the target suffix. Append the selected archive format's
|
|
extension to `archive-name`; do not reinterpret it as a filesystem source path.
|
|
|
|
### 2. A host-side preparation hook with a stable contract
|
|
|
|
Run preparation once per selected bundle/target, after successful compilation
|
|
and before archive creation, checksums, release creation or uploads. This is
|
|
packaging work on the host, not a command inside a Cross build container.
|
|
|
|
Use an argument vector, not implicit shell interpolation. Explicit shell use
|
|
can be configured as `["sh", "..."]`. Define the working directory and supply a
|
|
versioned JSON context file containing:
|
|
|
|
- Package name/version, target triple, profile and target executable suffix.
|
|
- Workspace and selected package roots, and the source commit/dirty state.
|
|
- An exact mapping of declared binary names to **actual built executable paths**.
|
|
- Fresh hook output/staging directory, persistent cache directory, and result
|
|
manifest path, all absolute.
|
|
- Bundle name and protocol version.
|
|
|
|
Resolve binary paths from actual build artifacts or the validated build cache.
|
|
Do not reconstruct them from a conventional target directory. Support
|
|
`CARGO_TARGET_DIR`, publisher-owned fresh build directories, non-default profiles
|
|
and workspace members.
|
|
|
|
The hook writes only its outputs into its supplied output directory and emits
|
|
a manifest. For example:
|
|
|
|
```json
|
|
{
|
|
"schema_version": 1,
|
|
"target": "x86_64-pc-windows-gnu",
|
|
"files": [
|
|
{
|
|
"source": "ffprobe.exe",
|
|
"destination": "ffprobe.exe",
|
|
"kind": "executable",
|
|
"sha256": "<actual 64-character SHA-256>"
|
|
},
|
|
{
|
|
"source": "notices/provenance.json",
|
|
"destination": "LICENSES/ffmpeg/provenance.json",
|
|
"kind": "file",
|
|
"sha256": "<actual 64-character SHA-256>"
|
|
},
|
|
{
|
|
"source": "notices/LICENSE.txt",
|
|
"destination": "LICENSES/ffmpeg/LICENSE.txt",
|
|
"kind": "file",
|
|
"sha256": "<actual 64-character SHA-256>"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Paths in that manifest are relative to the hook output directory and bundle
|
|
root respectively. The publisher validates them, copies only declared files,
|
|
checks the recorded hashes, and owns final archive construction. Static inputs
|
|
and Cargo binaries are also hashed and included in an artifact manifest.
|
|
|
|
Stream hook output with target/bundle labels. Identify the failing command and
|
|
exit status. Stop on nonzero exit, timeout, cancellation, missing required
|
|
outputs, invalid manifest or hash mismatch. Clean temporary staging and do not
|
|
create/upload a partial release. Terminate hook subprocesses on cancellation.
|
|
Do not pass Gitea tokens or publication credentials into hooks; preserve the
|
|
existing child-process credential isolation.
|
|
|
|
### 3. Runtime dependency integrity and caching
|
|
|
|
A hook can download dependencies into the supplied persistent cache; it must
|
|
not need tracked binaries or tracked generated outputs. Allow explicit digest
|
|
pins and immutable release identifiers through declared configuration/inputs.
|
|
|
|
For this consumer, continue using verified target-specific non-shared LGPL
|
|
FFprobe builds. Preserve their upstream notices and provenance, including
|
|
provider URL, release/asset IDs, archive SHA-256 and extracted file hashes.
|
|
Do not make a provider-specific built-in downloader a prerequisite for this fix.
|
|
|
|
Keep build-cache reuse separate from bundle preparation. Reusing valid Rust
|
|
executables must not accidentally reuse stale auxiliary files. A straightforward
|
|
first implementation can rerun preparation for each packaging invocation while
|
|
the hook reuses its own checksum-verified download cache.
|
|
|
|
If prepared bundles themselves are cached, fingerprint the effective bundle
|
|
configuration, hook command and declared inputs, static file contents, actual
|
|
binary hashes, and resolved dependency identities/hashes. Changes in a moving
|
|
`latest` dependency must be resolved before accepting a bundle cache hit.
|
|
Do not treat an unchanged Git commit as proof of identical runtime dependencies.
|
|
|
|
### 4. Validate and package all files consistently
|
|
|
|
Reject absolute destinations, `..`, destination collisions, duplicate entries,
|
|
symlinks/hardlinks escaping approved roots, device files and missing required
|
|
inputs. For Windows bundles also reject case-insensitive destination collisions
|
|
and invalid/reserved Windows destination names. Hooks cannot replace a declared
|
|
Cargo executable by writing another file to its destination.
|
|
|
|
Check target architecture/format for declared runtime executables. A foreign
|
|
executable must not be run on the build host merely because packaging succeeded.
|
|
Allow an explicit native smoke-test command; record passed, failed and skipped
|
|
checks distinctly. Cross-compilation is not Windows runtime validation.
|
|
|
|
Produce archives atomically with deterministic ordering, normalized ownership,
|
|
timestamps and permissions, preserving the intended executable bits. Identical
|
|
inputs should produce identical archives. Avoid leaking absolute host paths or
|
|
environment secrets into generated public manifests.
|
|
|
|
### 5. Integrate bundles into the complete release lifecycle
|
|
|
|
- `--dry-run`: validate configuration and show bundle formats, target selections,
|
|
static inclusions, hook commands and declared required outputs. Do not execute
|
|
hooks, fetch dependencies or produce archives. Mark dynamic results unresolved.
|
|
- `--compile-only` and `--stop-after package`: build/reuse binaries, run preparation
|
|
and produce complete local bundles and checksums. No Gitea token, release,
|
|
tag creation or push is required.
|
|
- `--skip-tests`: skip the existing test phase only; do not bypass hook success,
|
|
file/hash checks or archive validation.
|
|
- Normal publication: finish and validate **all** requested artifacts before
|
|
release creation/upload. Include named bundles in the existing `SHA256SUMS`,
|
|
collision handling, `--replace-assets` and post-upload download verification.
|
|
Keep existing draft/prerelease behavior and do not delete unrelated assets.
|
|
- Print every resulting artifact path and its checks performed/skipped. Provide
|
|
a machine-readable artifact manifest so consumers do not scrape console output.
|
|
- Fail before remote mutations if any bundle fails. Do not promise transactional
|
|
rollback of remote uploads where the existing backend cannot provide it.
|
|
|
|
### 6. Preserve the cargo-binstall contract
|
|
|
|
Keep the existing canonical `{name}-{target}-v{version}.tgz` archives and their
|
|
metadata/install checks working. Portable archives use distinct names, for
|
|
example `{name}-portable-{target}-v{version}.zip`, avoiding ambiguous asset matches.
|
|
|
|
Do not assume extra files in an archive become installed beside Cargo binaries
|
|
by cargo-binstall. Verify its actual behavior before promising that. For this
|
|
consumer the supported baseline is:
|
|
|
|
- The portable bundle contains everything and is validated as a complete tree.
|
|
- The cargo-binstall installation still installs the declared Rust binaries.
|
|
Its usage documentation explicitly states the remaining external FFprobe
|
|
requirement unless runtime-dependency installation is actually implemented
|
|
and tested end to end.
|
|
|
|
Archive verification must understand the selected artifact's declared contents,
|
|
not reject a valid portable bundle for containing more than Cargo binaries.
|
|
|
|
## Tests and acceptance criteria
|
|
|
|
Use local fixture hooks/downloads and small fixture binaries in automated tests;
|
|
do not depend on mutable GitHub releases or download hundreds of MB per test.
|
|
|
|
- [ ] Legacy configurations produce unchanged artifact names, layout and
|
|
cargo-binstall behavior, including multiple binaries.
|
|
- [ ] A fixture hook receives correct target/version/profile, source roots and
|
|
real executable paths for clean, dirty, cached and custom-target-dir builds.
|
|
- [ ] The Windows ZIP and Linux tar.gz contain both application binaries, a
|
|
target-matching FFprobe fixture, README and license/provenance files.
|
|
- [ ] Target filtering does not run hooks or fetch dependencies for other targets.
|
|
- [ ] Nonzero exit, timeout/cancellation, missing executable/license, bad hashes,
|
|
malformed manifests and architecture mismatches prevent publication.
|
|
- [ ] Traversal, escaping links, duplicate/case-colliding destinations, reserved
|
|
Windows names and attempts to replace Cargo binaries are rejected.
|
|
- [ ] Paths with spaces and Unicode are handled without shell reinterpretation.
|
|
- [ ] Dry-run leaves no hook marker, download or archive. Compile-only runs
|
|
preparation and packaging with no token, tag, release or push.
|
|
- [ ] Changing a hook/input/license/dependency digest invalidates affected bundle
|
|
reuse while permitting unchanged Rust binaries to remain cached.
|
|
- [ ] Repeated packaging of identical inputs is byte-identical; stale staging
|
|
files never appear in a later archive; executable modes are correct.
|
|
- [ ] Mocked publication includes all bundle checksums, collision checks and
|
|
post-upload verification; one failed target causes no release creation.
|
|
- [ ] Existing cargo-binstall checks still pass and are reported separately from
|
|
portable-tree and native-runtime checks.
|
|
- [ ] Update upstream docs/schema/examples and implement a consumer migration
|
|
recipe using the actual downloader, not an FFmpeg-specific publisher fork.
|
|
|
|
Final consumer smoke test, once the implementation is available:
|
|
|
|
1. Configure the portable bundle and thin downloader adapter in this project.
|
|
2. Run `rust_realease . --package pptx-reporter --compile-only`.
|
|
3. Verify both portable archives and checksums, and run Linux extraction/probing
|
|
and report generation using the bundled FFprobe without relying on PATH.
|
|
4. On Windows, extract the ZIP and launch the GUI without Python, Typst or system
|
|
FFmpeg. Generate reports using its adjacent FFprobe. Record this as unverified
|
|
if no Windows validation machine is available.
|
|
5. Confirm no dependency binary, archive or generated staging file entered Git.
|
|
Remove the separate project archive packager once upstream owns its function.
|
|
|
|
Do not create an actual public release merely to test this task. This project's
|
|
Jenkins remains checks/coverage/badges only; release publication stays explicit.
|
|
Signing, installers, automatic updates and rewriting dependency downloaders in
|
|
Rust are outside this task.
|