Specify upstream portable packaging support for rust_realease
This commit is contained in:
parent
96ae48f9f2
commit
0558a92c2b
|
|
@ -0,0 +1,331 @@
|
|||
# Task: support complete portable application bundles in rust_realease
|
||||
|
||||
## Objective
|
||||
|
||||
Extend [rust_realease](https://proxy.dandaedre.com/gitea/idle-org/rust_realease)
|
||||
so a single release invocation can build a Rust application, prepare its
|
||||
target-specific runtime dependencies, assemble complete portable archives,
|
||||
and checksum, publish and verify those archives through the normal workflow.
|
||||
|
||||
This is generic release packaging functionality. Do not hard-code FFmpeg,
|
||||
BtbN, this application, or its target list into the release tool.
|
||||
Project-specific dependency acquisition may remain a small script; archive
|
||||
creation, build-output discovery and publication must belong to rust_realease.
|
||||
|
||||
This document is an implementation specification for the upstream repository,
|
||||
not a request to publish a release or change this project's CI into a publisher.
|
||||
Inspect upstream code and instructions before implementing. The configuration
|
||||
and hook protocol below are **proposed APIs**, not existing supported options.
|
||||
Equivalent naming is acceptable if all behavioral requirements are met.
|
||||
|
||||
## Observed baseline and concrete problem
|
||||
|
||||
The consumer is `pptx-reporter` in
|
||||
[Beffroi/beffroi-tools](https://proxy.dandaedre.com/gitea/Beffroi/beffroi-tools),
|
||||
on `feature/rust-reporter`. The locally tested publisher is
|
||||
`rust_realease 0.1.6-rc.2`. The current
|
||||
[advanced usage documentation](https://proxy.dandaedre.com/gitea/idle-org/rust_realease/src/branch/main/docs/advanced-usage.md)
|
||||
was checked on 2026-10-20. Distinguish that installed version from upstream main;
|
||||
reuse any capabilities that have landed since, rather than implementing them twice.
|
||||
|
||||
The reporter has two Cargo binaries, `pptx-reporter` and `pptx-reporter-gui`.
|
||||
It embeds Typst and fonts, but invokes an external `ffprobe` executable.
|
||||
Windows users should extract an archive and double-click the GUI with no Python,
|
||||
Typst, PowerPoint or separate FFmpeg installation. Python may be a build-host
|
||||
dependency for a configured preparation script.
|
||||
|
||||
Current gaps:
|
||||
|
||||
- The documented release archives contain the package's declared Cargo binaries.
|
||||
They do not provide a documented mechanism for runtime executables, notices,
|
||||
templates, README files or other application resources.
|
||||
- The documented formats are `tgz`, `binary` and `both`. Windows desktop users
|
||||
need an ordinary ZIP as well as the existing cargo-binstall distribution.
|
||||
- Cross `pre-build` prepares a container image/build environment. It is not a
|
||||
packaging hook with access to completed binaries and target bundle staging.
|
||||
- A dirty checkout may build in a fresh, publisher-owned target directory.
|
||||
A project packager cannot safely assume `target/<triple>/release/<binary>`.
|
||||
- A successful cargo-binstall check of the declared Rust binaries does not by
|
||||
itself establish that adjacent runtime executables/resources are installed.
|
||||
|
||||
The workaround lives in this consumer repository:
|
||||
|
||||
- `scripts/fetch_ffprobe.py`: download a target build, check its SHA-256 and
|
||||
PE/ELF architecture, retain notices/provenance, cache outside Git, and perform
|
||||
a native version check when possible.
|
||||
- `scripts/package-portable.py`: independently locate Cargo outputs, copy the
|
||||
dependency and notices, produce Windows ZIP/Linux tar.gz and checksum files.
|
||||
- `rust-release.toml`: currently documents calling that packager separately.
|
||||
|
||||
Replace the second script's responsibilities with upstream support. Keep
|
||||
provider-specific download logic reusable through a thin preparation adapter.
|
||||
|
||||
## Required application layout
|
||||
|
||||
Produce these portable distributions for the currently supported targets:
|
||||
|
||||
| Target | Format | Runtime dependency |
|
||||
| --- | --- | --- |
|
||||
| `x86_64-pc-windows-gnu` | ZIP | `ffprobe.exe`, built for Windows x86-64 |
|
||||
| `x86_64-unknown-linux-gnu` | tar.gz/tgz | `ffprobe`, built for Linux x86-64 |
|
||||
|
||||
Each archive has one predictable root directory:
|
||||
|
||||
```text
|
||||
pptx-reporter-portable-<target>-v<version>/
|
||||
pptx-reporter[.exe]
|
||||
pptx-reporter-gui[.exe]
|
||||
ffprobe[.exe]
|
||||
README.txt
|
||||
LICENSES/
|
||||
reporter-MIT.txt
|
||||
DejaVu.txt
|
||||
ffmpeg/
|
||||
<upstream notices>
|
||||
provenance.json
|
||||
```
|
||||
|
||||
Executable permissions must survive Linux extraction. Windows executable names
|
||||
must have `.exe`. Do not accidentally bundle the host's FFprobe, unrelated
|
||||
FFmpeg programs, build caches or source checkout. Allow explicitly declared DLLs,
|
||||
shared libraries and resource directories for applications that need them;
|
||||
this project's selected FFprobe distributions do not need separate FFmpeg DLLs.
|
||||
|
||||
## Required capabilities
|
||||
|
||||
### 1. Declarative bundles alongside existing artifacts
|
||||
|
||||
Support named bundles with target filters, per-target archive formats, a root
|
||||
directory/name template, Cargo binary selection, and explicit extra files or
|
||||
directory trees with destination paths inside the bundle.
|
||||
|
||||
Existing configurations must retain their current behavior. Existing
|
||||
`package.format`, `--artifact-format`, archive names and cargo-binstall metadata
|
||||
must keep their existing meanings; named bundles are additional artifacts.
|
||||
Reject unknown or inconsistent configuration rather than silently ignoring it.
|
||||
If a selected bundle lacks a format for a selected target, fail clearly.
|
||||
`--target` must also filter hooks, dependency fetches and bundle creation.
|
||||
|
||||
Illustrative configuration for this consumer:
|
||||
|
||||
```toml
|
||||
[package]
|
||||
format = "tgz" # Keep the existing cargo-binstall artifacts.
|
||||
|
||||
[[package.bundles]]
|
||||
name = "portable"
|
||||
targets = ["x86_64-pc-windows-gnu", "x86_64-unknown-linux-gnu"]
|
||||
formats = { x86_64-pc-windows-gnu = "zip", x86_64-unknown-linux-gnu = "tgz" }
|
||||
archive-name = "{name}-portable-{target}-v{version}"
|
||||
root = "{name}-portable-{target}-v{version}"
|
||||
cargo-binaries = "all"
|
||||
files = [
|
||||
{ source = "LICENSE", destination = "LICENSES/reporter-MIT.txt" },
|
||||
{ source = "assets/fonts/LICENSE", destination = "LICENSES/DejaVu.txt" },
|
||||
{ source = "docs/windows-quickstart.txt", destination = "README.txt" },
|
||||
]
|
||||
|
||||
[package.bundles.prepare]
|
||||
command = ["python3", "scripts/prepare-release.py", "--context", "{context-file}", "--manifest", "{manifest-file}"]
|
||||
inputs = ["scripts/prepare-release.py", "scripts/fetch_ffprobe.py"]
|
||||
timeout-seconds = 600
|
||||
required-destinations = ["ffprobe{binary-ext}", "LICENSES/ffmpeg/LICENSE.txt", "LICENSES/ffmpeg/provenance.json"]
|
||||
```
|
||||
|
||||
`prepare-release.py` above is a proposed thin adapter, not an existing file.
|
||||
It would call this project's downloader and return the prepared file list.
|
||||
Define whether static source paths are package- or workspace-relative; default
|
||||
to package-relative and provide an explicit workspace-relative form.
|
||||
This must work for selected members of Cargo workspaces.
|
||||
In templates, `{name}` is the Cargo package name, `{bundle}` is the bundle name,
|
||||
and `{binary-ext}` is the target suffix. Append the selected archive format's
|
||||
extension to `archive-name`; do not reinterpret it as a filesystem source path.
|
||||
|
||||
### 2. A host-side preparation hook with a stable contract
|
||||
|
||||
Run preparation once per selected bundle/target, after successful compilation
|
||||
and before archive creation, checksums, release creation or uploads. This is
|
||||
packaging work on the host, not a command inside a Cross build container.
|
||||
|
||||
Use an argument vector, not implicit shell interpolation. Explicit shell use
|
||||
can be configured as `["sh", "..."]`. Define the working directory and supply a
|
||||
versioned JSON context file containing:
|
||||
|
||||
- Package name/version, target triple, profile and target executable suffix.
|
||||
- Workspace and selected package roots, and the source commit/dirty state.
|
||||
- An exact mapping of declared binary names to **actual built executable paths**.
|
||||
- Fresh hook output/staging directory, persistent cache directory, and result
|
||||
manifest path, all absolute.
|
||||
- Bundle name and protocol version.
|
||||
|
||||
Resolve binary paths from actual build artifacts or the validated build cache.
|
||||
Do not reconstruct them from a conventional target directory. Support
|
||||
`CARGO_TARGET_DIR`, publisher-owned fresh build directories, non-default profiles
|
||||
and workspace members.
|
||||
|
||||
The hook writes only its outputs into its supplied output directory and emits
|
||||
a manifest. For example:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"target": "x86_64-pc-windows-gnu",
|
||||
"files": [
|
||||
{
|
||||
"source": "ffprobe.exe",
|
||||
"destination": "ffprobe.exe",
|
||||
"kind": "executable",
|
||||
"sha256": "<actual 64-character SHA-256>"
|
||||
},
|
||||
{
|
||||
"source": "notices/provenance.json",
|
||||
"destination": "LICENSES/ffmpeg/provenance.json",
|
||||
"kind": "file",
|
||||
"sha256": "<actual 64-character SHA-256>"
|
||||
},
|
||||
{
|
||||
"source": "notices/LICENSE.txt",
|
||||
"destination": "LICENSES/ffmpeg/LICENSE.txt",
|
||||
"kind": "file",
|
||||
"sha256": "<actual 64-character SHA-256>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Paths in that manifest are relative to the hook output directory and bundle
|
||||
root respectively. The publisher validates them, copies only declared files,
|
||||
checks the recorded hashes, and owns final archive construction. Static inputs
|
||||
and Cargo binaries are also hashed and included in an artifact manifest.
|
||||
|
||||
Stream hook output with target/bundle labels. Identify the failing command and
|
||||
exit status. Stop on nonzero exit, timeout, cancellation, missing required
|
||||
outputs, invalid manifest or hash mismatch. Clean temporary staging and do not
|
||||
create/upload a partial release. Terminate hook subprocesses on cancellation.
|
||||
Do not pass Gitea tokens or publication credentials into hooks; preserve the
|
||||
existing child-process credential isolation.
|
||||
|
||||
### 3. Runtime dependency integrity and caching
|
||||
|
||||
A hook can download dependencies into the supplied persistent cache; it must
|
||||
not need tracked binaries or tracked generated outputs. Allow explicit digest
|
||||
pins and immutable release identifiers through declared configuration/inputs.
|
||||
|
||||
For this consumer, continue using verified target-specific non-shared LGPL
|
||||
FFprobe builds. Preserve their upstream notices and provenance, including
|
||||
provider URL, release/asset IDs, archive SHA-256 and extracted file hashes.
|
||||
Do not make a provider-specific built-in downloader a prerequisite for this fix.
|
||||
|
||||
Keep build-cache reuse separate from bundle preparation. Reusing valid Rust
|
||||
executables must not accidentally reuse stale auxiliary files. A straightforward
|
||||
first implementation can rerun preparation for each packaging invocation while
|
||||
the hook reuses its own checksum-verified download cache.
|
||||
|
||||
If prepared bundles themselves are cached, fingerprint the effective bundle
|
||||
configuration, hook command and declared inputs, static file contents, actual
|
||||
binary hashes, and resolved dependency identities/hashes. Changes in a moving
|
||||
`latest` dependency must be resolved before accepting a bundle cache hit.
|
||||
Do not treat an unchanged Git commit as proof of identical runtime dependencies.
|
||||
|
||||
### 4. Validate and package all files consistently
|
||||
|
||||
Reject absolute destinations, `..`, destination collisions, duplicate entries,
|
||||
symlinks/hardlinks escaping approved roots, device files and missing required
|
||||
inputs. For Windows bundles also reject case-insensitive destination collisions
|
||||
and invalid/reserved Windows destination names. Hooks cannot replace a declared
|
||||
Cargo executable by writing another file to its destination.
|
||||
|
||||
Check target architecture/format for declared runtime executables. A foreign
|
||||
executable must not be run on the build host merely because packaging succeeded.
|
||||
Allow an explicit native smoke-test command; record passed, failed and skipped
|
||||
checks distinctly. Cross-compilation is not Windows runtime validation.
|
||||
|
||||
Produce archives atomically with deterministic ordering, normalized ownership,
|
||||
timestamps and permissions, preserving the intended executable bits. Identical
|
||||
inputs should produce identical archives. Avoid leaking absolute host paths or
|
||||
environment secrets into generated public manifests.
|
||||
|
||||
### 5. Integrate bundles into the complete release lifecycle
|
||||
|
||||
- `--dry-run`: validate configuration and show bundle formats, target selections,
|
||||
static inclusions, hook commands and declared required outputs. Do not execute
|
||||
hooks, fetch dependencies or produce archives. Mark dynamic results unresolved.
|
||||
- `--compile-only` and `--stop-after package`: build/reuse binaries, run preparation
|
||||
and produce complete local bundles and checksums. No Gitea token, release,
|
||||
tag creation or push is required.
|
||||
- `--skip-tests`: skip the existing test phase only; do not bypass hook success,
|
||||
file/hash checks or archive validation.
|
||||
- Normal publication: finish and validate **all** requested artifacts before
|
||||
release creation/upload. Include named bundles in the existing `SHA256SUMS`,
|
||||
collision handling, `--replace-assets` and post-upload download verification.
|
||||
Keep existing draft/prerelease behavior and do not delete unrelated assets.
|
||||
- Print every resulting artifact path and its checks performed/skipped. Provide
|
||||
a machine-readable artifact manifest so consumers do not scrape console output.
|
||||
- Fail before remote mutations if any bundle fails. Do not promise transactional
|
||||
rollback of remote uploads where the existing backend cannot provide it.
|
||||
|
||||
### 6. Preserve the cargo-binstall contract
|
||||
|
||||
Keep the existing canonical `{name}-{target}-v{version}.tgz` archives and their
|
||||
metadata/install checks working. Portable archives use distinct names, for
|
||||
example `{name}-portable-{target}-v{version}.zip`, avoiding ambiguous asset matches.
|
||||
|
||||
Do not assume extra files in an archive become installed beside Cargo binaries
|
||||
by cargo-binstall. Verify its actual behavior before promising that. For this
|
||||
consumer the supported baseline is:
|
||||
|
||||
- The portable bundle contains everything and is validated as a complete tree.
|
||||
- The cargo-binstall installation still installs the declared Rust binaries.
|
||||
Its usage documentation explicitly states the remaining external FFprobe
|
||||
requirement unless runtime-dependency installation is actually implemented
|
||||
and tested end to end.
|
||||
|
||||
Archive verification must understand the selected artifact's declared contents,
|
||||
not reject a valid portable bundle for containing more than Cargo binaries.
|
||||
|
||||
## Tests and acceptance criteria
|
||||
|
||||
Use local fixture hooks/downloads and small fixture binaries in automated tests;
|
||||
do not depend on mutable GitHub releases or download hundreds of MB per test.
|
||||
|
||||
- [ ] Legacy configurations produce unchanged artifact names, layout and
|
||||
cargo-binstall behavior, including multiple binaries.
|
||||
- [ ] A fixture hook receives correct target/version/profile, source roots and
|
||||
real executable paths for clean, dirty, cached and custom-target-dir builds.
|
||||
- [ ] The Windows ZIP and Linux tar.gz contain both application binaries, a
|
||||
target-matching FFprobe fixture, README and license/provenance files.
|
||||
- [ ] Target filtering does not run hooks or fetch dependencies for other targets.
|
||||
- [ ] Nonzero exit, timeout/cancellation, missing executable/license, bad hashes,
|
||||
malformed manifests and architecture mismatches prevent publication.
|
||||
- [ ] Traversal, escaping links, duplicate/case-colliding destinations, reserved
|
||||
Windows names and attempts to replace Cargo binaries are rejected.
|
||||
- [ ] Paths with spaces and Unicode are handled without shell reinterpretation.
|
||||
- [ ] Dry-run leaves no hook marker, download or archive. Compile-only runs
|
||||
preparation and packaging with no token, tag, release or push.
|
||||
- [ ] Changing a hook/input/license/dependency digest invalidates affected bundle
|
||||
reuse while permitting unchanged Rust binaries to remain cached.
|
||||
- [ ] Repeated packaging of identical inputs is byte-identical; stale staging
|
||||
files never appear in a later archive; executable modes are correct.
|
||||
- [ ] Mocked publication includes all bundle checksums, collision checks and
|
||||
post-upload verification; one failed target causes no release creation.
|
||||
- [ ] Existing cargo-binstall checks still pass and are reported separately from
|
||||
portable-tree and native-runtime checks.
|
||||
- [ ] Update upstream docs/schema/examples and implement a consumer migration
|
||||
recipe using the actual downloader, not an FFmpeg-specific publisher fork.
|
||||
|
||||
Final consumer smoke test, once the implementation is available:
|
||||
|
||||
1. Configure the portable bundle and thin downloader adapter in this project.
|
||||
2. Run `rust_realease . --package pptx-reporter --compile-only`.
|
||||
3. Verify both portable archives and checksums, and run Linux extraction/probing
|
||||
and report generation using the bundled FFprobe without relying on PATH.
|
||||
4. On Windows, extract the ZIP and launch the GUI without Python, Typst or system
|
||||
FFmpeg. Generate reports using its adjacent FFprobe. Record this as unverified
|
||||
if no Windows validation machine is available.
|
||||
5. Confirm no dependency binary, archive or generated staging file entered Git.
|
||||
Remove the separate project archive packager once upstream owns its function.
|
||||
|
||||
Do not create an actual public release merely to test this task. This project's
|
||||
Jenkins remains checks/coverage/badges only; release publication stays explicit.
|
||||
Signing, installers, automatic updates and rewriting dependency downloaders in
|
||||
Rust are outside this task.
|
||||
|
|
@ -94,6 +94,10 @@ python3 scripts/package-portable.py --target x86_64-unknown-linux-gnu \
|
|||
--fetch-ffprobe
|
||||
```
|
||||
|
||||
The [upstream packaging task](fix-rust-realease.task.md) specifies the hooks,
|
||||
runtime-file inclusion and portable artifact support needed to replace this
|
||||
separate packaging step with the normal `rust_realease` workflow.
|
||||
|
||||
This creates a ZIP under `dist/`, with both reporter binaries, FFprobe,
|
||||
licences, download provenance, quick-start instructions and a SHA-256 checksum
|
||||
(Linux uses `.tar.gz`). These commands expect the reporter binaries under
|
||||
|
|
|
|||
Loading…
Reference in New Issue