diff --git a/Cross.toml b/Cross.toml index bead784..49c60c6 100644 --- a/Cross.toml +++ b/Cross.toml @@ -1,4 +1,7 @@ [build] +pre-build = [ + "apt-get update && apt-get install --assume-yes --no-install-recommends libudev-dev pkg-config", +] [build.env] passthrough = ["RUSTUP_TOOLCHAIN"] diff --git a/README.md b/README.md index 813822f..ec1adb9 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,9 @@ tag, push, release-note, private-repository, and asset-replacement workflows. - [Advanced usage](docs/advanced-usage.md) covers sources, configuration, metadata preparation, pipeline stages, artifact formats, authentication, this repository's release procedure, and troubleshooting. +- [Cross configuration](docs/advanced-usage.md#cross-configuration) explains + target images, environment passthrough, and global or target-specific + `pre-build` commands for native build dependencies. - `rust_realease --help` is the complete option reference. ## Development diff --git a/docs/advanced-usage.md b/docs/advanced-usage.md index 9b6f82b..acf0111 100644 --- a/docs/advanced-usage.md +++ b/docs/advanced-usage.md @@ -75,6 +75,77 @@ fallback-url = "https://proxy.dandaedre.com/gitea/idle-org/john-slop" URLs are normalized, including removal of a trailing `.git`; SCP-style and SSH URLs normalize to HTTPS. The scheme remains significant for HTTP(S), so add separate rules if the same origin is used through both HTTP and SSH. Each rule matches one repository, not an entire host. The fallback must use HTTPS because it becomes the Gitea API and public download base. It is also written to `package.repository` during metadata repair. Git continues to clone and push through `origin`; the mapping tells the safety checks that the internal and public repositories are equivalent. +## Cross configuration + +`rust_realease` runs Cross from the acquired project root, once for every +target in `build.targets` (or supplied with `--target`). Put `Cross.toml` in +that root to configure the container builds. Cross also accepts equivalent +settings below `[workspace.metadata.cross]` in the root `Cargo.toml`; when both +are present, `Cross.toml` takes precedence. Cross environment variables take +precedence over both. See the +[Cross configuration reference](https://github.com/cross-rs/cross/blob/main/docs/config_file.md) +for every setting supported by Cross. + +This repository uses the following configuration: + +```toml +[build] +pre-build = [ + "apt-get update && apt-get install --assume-yes --no-install-recommends libudev-dev pkg-config", +] + +[build.env] +passthrough = ["RUSTUP_TOOLCHAIN"] + +[target.x86_64-unknown-linux-musl] +image = "ghcr.io/cross-rs/x86_64-unknown-linux-musl:main" + +[target.aarch64-unknown-linux-musl] +image = "ghcr.io/cross-rs/aarch64-unknown-linux-musl:main" + +[target.x86_64-pc-windows-gnu] +image = "ghcr.io/cross-rs/x86_64-pc-windows-gnu:main" +``` + +The tables have distinct jobs: + +- `[build]` applies defaults to every target. Its `pre-build` commands run + while Cross constructs a derived container image, before Cargo builds the + package. Commands run as container-image build steps and must be valid for + every configured base image. +- `[build.env]` controls the environment exposed to builds. The + `RUSTUP_TOOLCHAIN` passthrough keeps a caller-selected toolchain available + inside Cross. Add only variables the container build actually needs. +- `[target.TRIPLE]` selects settings for one target. Here, each `image` pins + the corresponding image repository to its `main` tag. + +For a dependency needed by only one target, put `pre-build` on that target +instead of under `[build]`: + +```toml +[target.aarch64-unknown-linux-musl] +image = "ghcr.io/cross-rs/aarch64-unknown-linux-musl:main" +pre-build = [ + "dpkg --add-architecture $CROSS_DEB_ARCH", + "apt-get update && apt-get install --assume-yes --no-install-recommends libudev-dev:$CROSS_DEB_ARCH pkg-config", +] +``` + +Cross supplies `CROSS_DEB_ARCH` when it builds a custom image, which avoids +hard-coding Debian architecture names. A target-specific `pre-build` may also +name an executable script relative to the workspace root when a list of shell +commands becomes unwieldy. Treat all `pre-build` content as trusted code: it +is executed by the container engine. `rust_realease` accepts `pre-build` in +both supported configuration locations, validates the TOML, and leaves the +commands for Cross to execute. Custom `dockerfile` and `runner` options remain +unsupported by the publisher. + +`--prepare-repo` creates `Cross.toml` when necessary and preserves existing +settings. It always ensures `RUSTUP_TOOLCHAIN` is in +`build.env.passthrough`; `--cross-image-tag TAG` additionally writes the image +for every configured release target. Add project-specific `pre-build` +commands directly to `Cross.toml` and commit them like other build inputs. + ## Metadata and preparation The publisher checks the selected member's repository and, when archives are enabled, its cargo-binstall metadata. `--fix-metadata` repairs supported entries in that member's `Cargo.toml` while preserving comments where practical. It does not commit, tag, or push. To preview the full preparation diff, use `--prepare-repo --dry-run`. @@ -90,11 +161,11 @@ bin-dir = "{ name }-{ target }-v{ version }/{ bin }{ binary-ext }" The generated `pkg-url` uses the configured tag prefix; the archive filename itself retains `-v{version}`. Unsupported binstall keys are reported as metadata conflicts. A preparation run may create one extra fix commit, as intended. `--commit-preparation` requires a clean worktree and index. `--create-tag` requires that commit option, and `--push-preparation` requires both. No option moves an existing tag or force pushes. -For repositories whose build scripts need a selected Rust toolchain inside Cross, preparation can add the supported `Cross.toml` environment setting. `--cross-image-tag TAG` explicitly selects `ghcr.io/cross-rs/TARGET:TAG` images. Existing unrelated Cross settings are preserved. Repository-provided Cross hooks such as `pre-build`, `dockerfile`, and `runner` are rejected by the release build. +For repositories whose build scripts need a selected Rust toolchain inside Cross, preparation can add the supported `Cross.toml` environment setting. `--cross-image-tag TAG` explicitly selects `ghcr.io/cross-rs/TARGET:TAG` images. Existing unrelated Cross settings, including `pre-build`, are preserved. ## Pipeline stages -`--compile-only` runs through the package stage and then stops: it performs host checks, cross-compiles, and writes local artifacts without requiring a Gitea token or creating or uploading a release. It is the explicit artifact-build mode and conflicts with `--dry-run`, `--prepare-repo`, and `--stop-after`. For this repository, `./release.sh --compile-only` provides the same token-free path; a later `./release.sh TOKEN_FILE` publication reuses the completed build when its fingerprint still matches. +`--compile-only` runs through the package stage and then stops: it performs host checks, cross-compiles, and writes local artifacts without requiring a Gitea token or creating or uploading a release. It is the explicit artifact-build mode and conflicts with `--dry-run`, `--prepare-repo`, and `--stop-after`. A later publication from the same clean commit reuses the completed build when its fingerprint still matches. `--stop-after` can stop at `inspect`, `metadata`, `verify`, `build`, `package`, `release`, `upload`, or `binstall`. `verify` runs host `cargo check` and `cargo test`; `--skip-tests` skips only tests. A successful Cross build from a clean checkout records its Git commit, package, targets, profile, lock mode, container engine, active Rust toolchain, and binary list. A later run with the same inputs reuses those regular binary files from Cargo's target directory; dirty checkouts, changed inputs, missing files, and invalid cache records build in a fresh target directory instead. `--recompile` explicitly bypasses a matching completed build. `release` creates or reuses the Gitea release; `upload` uploads archives and checksums. The final stage downloads assets anonymously for public repositories or with Gitea authentication for private repositories, checks their checksums and archive layout, and runs isolated `cargo-binstall` installs for public targets. `--skip-binstall-test` skips that install check, while retaining download verification. @@ -121,17 +192,23 @@ Normal publication does not create a commit, tag, or push. Existing asset names ## Releasing this repository -Install the publisher from the checkout, then use the repository wrapper. The -wrapper selects this package, publishes both archive and standalone binary -assets, and forwards its token file to `--token-file` when publishing: +Install the publisher from the checkout, then invoke it directly. This +repository publishes both archive and standalone binary assets. Its current +version is a release candidate, so publication also uses `--prerelease`: ```bash cargo install --path . --locked -./release.sh --compile-only -./release.sh ../gitea_token.txt --dry-run -./release.sh ../gitea_token.txt +rust_realease . --package rust_realease --artifact-format both --compile-only +rust_realease . --package rust_realease --artifact-format both \ + --token-file ../gitea_token.txt --prerelease --dry-run +rust_realease . --package rust_realease --artifact-format both \ + --token-file ../gitea_token.txt --prerelease ``` +Omit `--prerelease` for a stable version. The token-free compile command can +run first; a later publication reuses its completed build when the commit and +build inputs still match. + ## CI badge pipeline Jenkins runs the locked test suite and Tarpaulin configuration, enforces the diff --git a/release.sh b/release.sh deleted file mode 100755 index dfa7e14..0000000 --- a/release.sh +++ /dev/null @@ -1,45 +0,0 @@ -#!/bin/sh -set -eu - -usage() { - echo "usage: $0 [rust_realease options...]" >&2 - echo " $0 --compile-only [rust_realease options...]" >&2 - exit 2 -} - -[ "$#" -ge 1 ] || usage -case "$1" in - --compile-only) - token_file= - ;; - *) - token_file=$1 - shift - if [ ! -f "$token_file" ] || [ ! -r "$token_file" ]; then - echo "token file is not a readable regular file: $token_file" >&2 - exit 2 - fi - ;; -esac - -if ! command -v rust_realease >/dev/null 2>&1; then - echo "rust_realease is not installed or not available in PATH" >&2 - exit 2 -fi - -repository_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) -manifest_version=$(sed -n 's/^version = "\([^"]*\)"/\1/p' "$repository_dir/Cargo.toml" | head -n 1) - -set -- \ - "$repository_dir" \ - --package rust_realease \ - --artifact-format both \ - "$@" -if [ -n "$token_file" ]; then - set -- "$@" --token-file "$token_file" -fi -case "$manifest_version" in - *-*) set -- "$@" --prerelease ;; -esac - -exec rust_realease "$@" diff --git a/src/build.rs b/src/build.rs index 71b92a7..5ff6b7a 100644 --- a/src/build.rs +++ b/src/build.rs @@ -118,7 +118,7 @@ fn validate_cross_config(root: &Path) -> Result<(), Error> { .and_then(|value| value.get("metadata")) .and_then(|value| value.get("cross")) { - reject_hooks(cross)?; + reject_unsupported_options(cross)?; } let path = root.join("Cross.toml"); if path.exists() { @@ -126,24 +126,24 @@ fn validate_cross_config(root: &Path) -> Result<(), Error> { .map_err(|e| Error(format!("cannot read {}: {e}", path.display())))?; let cross: toml::Value = toml::from_str(&source) .map_err(|e| Error(format!("invalid {}: {e}", path.display())))?; - reject_hooks(&cross)?; + reject_unsupported_options(&cross)?; } Ok(()) } -fn reject_hooks(value: &toml::Value) -> Result<(), Error> { +fn reject_unsupported_options(value: &toml::Value) -> Result<(), Error> { match value { toml::Value::Table(table) => { for (key, child) in table { - if matches!(key.as_str(), "pre-build" | "dockerfile" | "runner") { + if matches!(key.as_str(), "dockerfile" | "runner") { return Err(Error(format!("Repository cross configuration contains unsupported executable option {key:?}."))); } - reject_hooks(child)?; + reject_unsupported_options(child)?; } } toml::Value::Array(array) => { for child in array { - reject_hooks(child)?; + reject_unsupported_options(child)?; } } _ => (), @@ -520,11 +520,17 @@ mod tests { ); } #[test] - fn cross_hooks_are_rejected() { + fn cross_pre_build_is_allowed_but_other_executable_options_are_rejected() { let doc: toml::Value = - toml::from_str("[target.x86_64-unknown-linux-musl]\npre-build=['echo unsafe']") + toml::from_str("[target.x86_64-unknown-linux-musl]\npre-build=['apt-get update']") .unwrap(); - assert!(reject_hooks(&doc).is_err()); + assert!(reject_unsupported_options(&doc).is_ok()); + + for option in ["dockerfile = 'Dockerfile'", "runner = 'custom-runner'"] { + let doc: toml::Value = + toml::from_str(&format!("[target.x86_64-unknown-linux-musl]\n{option}")).unwrap(); + assert!(reject_unsupported_options(&doc).is_err()); + } } #[test] @@ -543,7 +549,7 @@ mod tests { fs::write( temp.path().join("Cargo.toml"), - "[workspace.metadata.cross.target.demo]\nrunner = 'unsafe'\n", + "[workspace.metadata.cross.target.demo]\nrunner = 'custom-runner'\n", ) .unwrap(); assert!(validate_cross_config(temp.path()) @@ -552,6 +558,13 @@ mod tests { .contains("runner")); fs::write(temp.path().join("Cargo.toml"), "[workspace]\n").unwrap(); + fs::write( + temp.path().join("Cross.toml"), + "[build]\npre-build = ['apt-get update']\n", + ) + .unwrap(); + validate_cross_config(temp.path()).unwrap(); + fs::write(temp.path().join("Cross.toml"), "invalid = [").unwrap(); assert!(validate_cross_config(temp.path()) .unwrap_err() diff --git a/tests/release_wrapper.rs b/tests/release_wrapper.rs deleted file mode 100644 index 15c04e9..0000000 --- a/tests/release_wrapper.rs +++ /dev/null @@ -1,165 +0,0 @@ -#![cfg(unix)] - -use std::{ - env, fs, - os::unix::fs::PermissionsExt, - path::Path, - process::{Command, Output}, -}; - -use tempfile::TempDir; - -const DEVELOPMENT_ORIGIN: &str = "git@git.dandaedre.com:idle-org/rust_realease.git"; -const RELEASE_CONFIG: &str = "[[repository.fallbacks]]\n\ -origin = \"git@git.dandaedre.com:idle-org/rust_realease.git\"\n\ -fallback-url = \"https://proxy.dandaedre.com/gitea/idle-org/rust_realease\"\n"; - -fn run(command: &mut Command) -> Output { - command.output().expect("test command should start") -} - -fn git(repository: &Path, arguments: &[&str]) -> Output { - run(Command::new("git") - .arg("-C") - .arg(repository) - .args(arguments)) -} - -#[test] -fn release_wrapper_passes_the_token_file_and_preserves_the_ssh_origin() { - let temporary = TempDir::new().expect("temporary directory should be created"); - let repository = temporary.path().join("repository"); - let fake_bin = temporary.path().join("bin"); - fs::create_dir_all(&repository).expect("repository directory should be created"); - fs::create_dir_all(&fake_bin).expect("fake binary directory should be created"); - - for name in ["release.sh", "rust-release.toml"] { - fs::copy( - Path::new(env!("CARGO_MANIFEST_DIR")).join(name), - repository.join(name), - ) - .expect("release file should be copied"); - } - fs::write( - repository.join("Cargo.toml"), - "[package]\nname = \"rust_realease\"\nversion = \"0.1.3\"\n", - ) - .expect("manifest should be written"); - fs::write(repository.join("token.txt"), "test-token\n").expect("token file should be written"); - - let fake_release = fake_bin.join("rust_realease"); - fs::write( - &fake_release, - "#!/bin/sh\n\ - [ -z \"${GITEA_TOKEN:-}\" ] || exit 91\n\ - git -C \"$1\" remote get-url origin >\"$OBSERVED_ORIGIN\"\n\ - cat \"$1/rust-release.toml\" >\"$OBSERVED_CONFIG\"\n\ - printf '%s\\n' \"$@\" >\"$OBSERVED_ARGS\"\n", - ) - .expect("fake release executable should be written"); - for path in [&fake_release, &repository.join("release.sh")] { - let mut permissions = fs::metadata(path) - .expect("script metadata should be readable") - .permissions(); - permissions.set_mode(0o755); - fs::set_permissions(path, permissions).expect("script should be executable"); - } - - assert!(git(&repository, &["init", "--quiet"]).status.success()); - assert!(git( - &repository, - &["remote", "add", "origin", DEVELOPMENT_ORIGIN] - ) - .status - .success()); - - let observed_origin = temporary.path().join("observed-origin.txt"); - let observed_config = temporary.path().join("observed-config.toml"); - let observed_args = temporary.path().join("observed-args.txt"); - let path = format!( - "{}:{}", - fake_bin.display(), - env::var("PATH").expect("PATH should be available") - ); - let output = run(Command::new(repository.join("release.sh")) - .arg(repository.join("token.txt")) - .arg("--dry-run") - .env("PATH", path) - .env("OBSERVED_ORIGIN", &observed_origin) - .env("OBSERVED_CONFIG", &observed_config) - .env("OBSERVED_ARGS", &observed_args)); - - assert!(output.status.success()); - assert_eq!( - fs::read_to_string(&observed_origin) - .expect("fake release should record the origin") - .trim(), - DEVELOPMENT_ORIGIN - ); - assert_eq!( - fs::read_to_string(&observed_config).expect("fake release should record the configuration"), - RELEASE_CONFIG - ); - let arguments = - fs::read_to_string(&observed_args).expect("fake release should record publisher arguments"); - assert!(arguments.lines().any(|argument| argument == "both")); - assert!(arguments.lines().any(|argument| argument == "--token-file")); - assert!(arguments - .lines() - .any(|argument| argument == repository.join("token.txt").to_string_lossy())); - assert!(arguments.lines().any(|argument| argument == "--dry-run")); - assert!(!arguments.lines().any(|argument| argument == "--prerelease")); -} - -#[test] -fn release_wrapper_builds_without_a_token_file() { - let temporary = TempDir::new().expect("temporary directory should be created"); - let repository = temporary.path().join("repository"); - let fake_bin = temporary.path().join("bin"); - fs::create_dir_all(&repository).expect("repository directory should be created"); - fs::create_dir_all(&fake_bin).expect("fake binary directory should be created"); - fs::copy( - Path::new(env!("CARGO_MANIFEST_DIR")).join("release.sh"), - repository.join("release.sh"), - ) - .expect("release wrapper should be copied"); - fs::write( - repository.join("Cargo.toml"), - "[package]\nname = \"rust_realease\"\nversion = \"0.1.3\"\n", - ) - .expect("manifest should be written"); - - let fake_release = fake_bin.join("rust_realease"); - fs::write( - &fake_release, - "#!/bin/sh\nprintf '%s\\n' \"$@\" >\"$OBSERVED_ARGS\"\n", - ) - .expect("fake release executable should be written"); - for path in [&fake_release, &repository.join("release.sh")] { - let mut permissions = fs::metadata(path) - .expect("script metadata should be readable") - .permissions(); - permissions.set_mode(0o755); - fs::set_permissions(path, permissions).expect("script should be executable"); - } - - let observed_args = temporary.path().join("observed-args.txt"); - let path = format!( - "{}:{}", - fake_bin.display(), - env::var("PATH").expect("PATH should be available") - ); - let output = run(Command::new(repository.join("release.sh")) - .arg("--compile-only") - .env("PATH", path) - .env("OBSERVED_ARGS", &observed_args) - .env_remove("GITEA_TOKEN")); - - assert!(output.status.success()); - let arguments = - fs::read_to_string(&observed_args).expect("fake release should record publisher arguments"); - assert!(arguments - .lines() - .any(|argument| argument == "--compile-only")); - assert!(!arguments.lines().any(|argument| argument == "--token-file")); -}