Port mocked Gitea publication and artifact verification

This commit is contained in:
Dandaedre 2026-09-17 02:01:32 +02:00
parent 291faaee48
commit e73571a67f
Signed by: Dandaedre
GPG Key ID: 440C6592E8A26D99
11 changed files with 1727 additions and 54 deletions

View File

@ -14,9 +14,9 @@ The Python implementation remains the behavioral oracle until the Rust replaceme
- Defer live API release tests until a disposable token is provided.
- Use the small `target_test_project` submodule for fast build and cross-target experiments.
Current Rust milestone: `inspect`, `metadata`, repository preparation, host `verify`, Cross `build`, and local `package` stages are implemented. Preparation has disposable Git tests for dry-run, commit, tag safety, and destination refusal; atomic push still needs an isolated integration test. The tiny fixture built and packaged all three default targets locally. Gitea publication remains in the Python worker.
Current Rust milestone: inspection, metadata, preparation, host verification, Cross build, packaging, Gitea publication, anonymous downloads, and isolated cargo-binstall orchestration are implemented. Gitea paths pass local HTTP mock tests; live publication and real cargo-binstall installation still need a disposable token. Preparation has disposable Git tests for dry-run, commit, tag safety, and destination refusal; atomic push still needs an isolated integration test. The tiny fixture built and packaged all three default targets locally.
Progress tracking: check a task only when the Rust behavior is implemented and locally verified. Keep phase gates open until their full fixture/parity criteria pass. The Python baseline has 45 passing tests and two opt-in tests skipped. CI runs Cargo tests only; Cross and live Gitea tests are deferred. Next work: validate a real local target build, complete repository preparation, and port mocked Gitea publication and artifact verification.
Progress tracking: check a task only when the Rust behavior is implemented and locally verified. Keep phase gates open until their full fixture/parity criteria pass. The Python baseline has 45 passing tests and two opt-in tests skipped. CI runs Cargo tests only; Cross and live Gitea tests are deferred. Next work: isolated preparation-push test, broader parity fixtures, and offline publication edge cases.
Work in small, testable phases. Commit migration work in coherent increments.
@ -329,7 +329,7 @@ Test-only as useful:
- [x] Apply configured/CLI tag prefix.
- [x] Validate explicit `--tag`.
- [x] Validate local tag points to HEAD where required.
- [ ] Validate remote tag exists and points to expected commit before publication.
- [x] Validate remote tag exists and points to expected commit before publication.
- [ ] Preserve preparation-mode distinction: intended tag naming without checking out an existing tag.
- [ ] Never move an existing tag.
- [ ] Never rewrite Cargo package version.
@ -502,38 +502,38 @@ Test-only as useful:
### Client
- [ ] Parse base instance/repository coordinates.
- [ ] Load token only from configured environment variable.
- [ ] Use HTTPS for authenticated publication.
- [ ] Set useful user-agent.
- [ ] Implement contextual API errors.
- [x] Parse base instance/repository coordinates.
- [x] Load token only from configured environment variable.
- [x] Use HTTPS for authenticated publication.
- [x] Set useful user-agent.
- [x] Implement contextual API errors.
### Releases
- [ ] Get remote tag.
- [ ] Get release by tag.
- [ ] Create release.
- [ ] Reuse existing release.
- [ ] Preserve draft/prerelease fields.
- [x] Get remote tag.
- [x] Get release by tag.
- [x] Create release.
- [x] Reuse existing release.
- [x] Preserve draft/prerelease fields.
### Assets
- [ ] List release assets once according to current Gitea API behavior.
- [ ] Do not reintroduce the historical repeated pagination bug.
- [ ] Upload assets.
- [ ] Fail on duplicate filename by default.
- [ ] Delete/replace only under `--replace-assets`.
- [ ] Never touch unrelated assets.
- [x] List release assets once according to current Gitea API behavior.
- [x] Do not reintroduce the historical repeated pagination bug.
- [x] Upload assets.
- [x] Fail on duplicate filename by default.
- [x] Delete/replace only under `--replace-assets`.
- [x] Never touch unrelated assets.
### Tests
- [ ] Mock release absence -> create.
- [ ] Mock existing release -> reuse.
- [ ] Asset listing regression test.
- [ ] Collision refusal.
- [ ] Explicit replacement.
- [ ] HTTP failure context.
- [ ] Token redaction.
- [x] Mock release absence -> create.
- [x] Mock existing release -> reuse.
- [x] Asset listing regression test.
- [x] Collision refusal.
- [x] Explicit replacement.
- [x] HTTP failure context.
- [x] Token redaction.
### Gate
@ -545,25 +545,25 @@ Test-only as useful:
### Artifact verification
- [ ] Build expected asset URLs.
- [ ] Download published archives.
- [ ] Prefer anonymous downloads for public assets.
- [ ] Validate SHA256SUMS.
- [ ] Validate archive root.
- [ ] Validate binary filename.
- [ ] Validate Unix executable bit where meaningful.
- [x] Build expected asset URLs.
- [x] Download published archives.
- [x] Prefer anonymous downloads for public assets.
- [x] Validate SHA256SUMS.
- [x] Validate archive root.
- [x] Validate binary filename.
- [x] Validate Unix executable bit where meaningful.
### cargo-binstall
- [ ] Use selected package manifest.
- [ ] Create temporary Cargo home/install root.
- [ ] Install without touching user's normal Cargo bin.
- [x] Use selected package manifest.
- [x] Create temporary Cargo home/install root.
- [x] Install without touching user's normal Cargo bin.
- [ ] Verify all default target contracts where supported.
- [ ] Respect `--skip-binstall-test`.
- [x] Respect `--skip-binstall-test`.
### Tests
- [ ] Local/mock HTTP asset server integration.
- [x] Local/mock HTTP asset server integration.
- [ ] Real cargo-binstall integration fixture.
- [ ] Check all three default target artifact layouts.

864
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -16,3 +16,4 @@ regex = "1"
flate2 = "1.1"
sha2 = "0.10"
similar = "2"
reqwest = { version = "0.12", default-features = false, features = ["blocking", "json", "multipart", "rustls-tls"] }

2
Jenkinsfile vendored
View File

@ -17,7 +17,7 @@ pipeline {
stage('Cargo Tests') {
steps {
sh 'cargo test --locked --offline --all'
sh 'cargo test --locked --all'
}
}
}

View File

@ -1,8 +1,8 @@
# Rust Binstall Publisher
> Rust migration status: the Rust binary supports local and remote Git inspection, metadata repair, repository preparation, host check/test (`--stop-after verify`), Cross builds (`--stop-after build`), and local `.tgz`/SHA256SUMS packaging (`--stop-after package`). Preparation has disposable Git tests for dry-run, commit, and tag safety; its remote push path still needs an isolated integration test. The tiny fixture built and packaged all three default targets locally. The Python worker remains the publication implementation.
> Rust migration status: the Rust binary supports inspection, metadata repair, repository preparation, host check/test, Cross builds, local packaging, Gitea release/upload, anonymous artifact verification, and isolated cargo-binstall verification. The Gitea and download paths have local HTTP mock tests; a live disposable release and real cargo-binstall install still need a token. Preparation's remote push path still needs an isolated integration test. The Python worker remains the proven publication reference until those parity gates pass.
Inspect locally with `cargo run -- target_test_project --stop-after inspect`. Validate metadata with `cargo run -- target_test_project --stop-after metadata`; add `--dry-run` to preview proposed repairs or `--fix-metadata` to write them. Preview repository changes with `cargo run -- target_test_project --prepare-repo --dry-run`. Use `--commit-preparation`, `--create-tag`, and `--push-preparation` only when those Git writes are intended. Run host checks with `cargo run -- target_test_project --stop-after verify`. Build and package locally with `--stop-after package` once Cross and a container backend are available. Run the Rust checks with `cargo fmt --check`, `cargo clippy --offline --all-targets --all-features -- -D warnings`, and `cargo test --offline`. Live Gitea tests are deferred until a disposable token is available.
Inspect locally with `cargo run -- target_test_project --stop-after inspect`. Validate metadata with `cargo run -- target_test_project --stop-after metadata`; add `--dry-run` to preview proposed repairs or `--fix-metadata` to write them. Preview repository changes with `cargo run -- target_test_project --prepare-repo --dry-run`. Use `--commit-preparation`, `--create-tag`, and `--push-preparation` only when those Git writes are intended. Run host checks with `cargo run -- target_test_project --stop-after verify`. Build and package locally with `--stop-after package` once Cross and a container backend are available. Run the Rust checks with `cargo fmt --check`, `cargo clippy --all-targets --all-features -- -D warnings`, and `cargo test --all`. Live Gitea tests are deferred until a disposable token is available.
A small Python release tool for Rust CLI projects on Gitea, releasing one package with one binary at a time. It inspects Cargo metadata, checks version tags, cross-compiles binaries, creates reproducible `.tgz` archives and SHA-256 checksums, uploads a release, and verifies the downloads with cargo-binstall.

View File

@ -2,10 +2,12 @@ use crate::{
build,
config::{self, Overrides},
error::Error,
gitea::Gitea,
manifest,
model::Stage,
policy, prepare, project,
runner::Runner,
verify,
};
use clap::Parser;
use std::path::PathBuf;
@ -104,6 +106,11 @@ pub fn run(args: Cli) -> Result<(), Error> {
.to_str()
.ok_or_else(|| Error("Project path is not UTF-8.".into()))?;
let selected = project::inspect(root, args.package.as_deref(), &runner)?;
let publishing = !args.prepare_repo
&& !args.dry_run
&& args
.stop_after
.is_none_or(|stage| stage.order() >= Stage::Release.order());
let tag = policy::expected_tag(&selected.version, &config.tag_prefix);
if args.tag.as_ref().is_some_and(|provided| provided != &tag) {
return Err(Error(format!(
@ -122,6 +129,19 @@ pub fn run(args: Cli) -> Result<(), Error> {
{
return Ok(());
}
if publishing
&& !runner
.text(
"git",
&["status", "--porcelain", "--untracked-files=normal"],
&checkout.root,
)?
.is_empty()
{
return Err(Error(
"Publication requires a clean checkout before metadata repair.".into(),
));
}
let candidate = args
.repository_url
.as_deref()
@ -135,6 +155,15 @@ pub fn run(args: Cli) -> Result<(), Error> {
)
})?;
let repository = project::normalize_repository(&candidate)?;
if publishing {
if let Some(origin) = project::origin_url(&checkout.root, &runner) {
if project::normalize_repository(&origin)? != repository {
return Err(Error(
"Git origin differs from the release repository.".into(),
));
}
}
}
let issues = manifest::changes(&selected.manifest, &repository, &config.tag_prefix)?;
if issues.is_empty() {
println!("Binstall metadata: valid");
@ -216,6 +245,18 @@ pub fn run(args: Cli) -> Result<(), Error> {
"Resolve the prerequisite issues above before building.".into(),
));
}
if publishing {
project::validate_checkout_tag(&checkout.root, &tag, &runner)?;
if std::env::var(&config.token_env)
.ok()
.is_none_or(|value| value.is_empty())
{
return Err(Error(format!(
"Set {} before publishing.",
config.token_env
)));
}
}
build::verify_source(&checkout.root, &selected, &config, args.skip_tests, &runner)?;
if args.stop_after == Some(Stage::Verify) {
return Ok(());
@ -248,9 +289,28 @@ pub fn run(args: Cli) -> Result<(), Error> {
if args.stop_after == Some(Stage::Package) {
return Ok(());
}
Err(Error(
"Rust Gitea publication is still being ported; use release.sh for publication.".into(),
))
let commit = project::head_commit(&checkout.root, &runner)?;
let api = Gitea::new(&repository, &config.token_env)?;
let release = api.ensure_release(&tag, &commit, args.draft, args.prerelease)?;
println!("Release ready: {tag}");
if args.stop_after == Some(Stage::Release) {
return Ok(());
}
let mut paths: Vec<&std::path::Path> = artifacts.iter().map(|a| a.path.as_path()).collect();
paths.push(sums.as_path());
api.upload(release.id, &paths, args.replace_assets)?;
if args.stop_after == Some(Stage::Upload) {
return Ok(());
}
if release.draft {
return Err(Error("Draft uploaded. Public verification requires publishing the draft; use --stop-after upload for draft workflows.".into()));
}
verify::verify_published(&repository, &tag, &artifacts, &sums)?;
if !args.skip_binstall_test {
verify::verify_binstall(&checkout.root, &selected, &config.targets, &runner)?;
}
println!("Release completed and published artifacts verified.");
Ok(())
}
#[cfg(test)]

View File

@ -1 +1,426 @@
//! Placeholder for the future gitea implementation.
use crate::error::Error;
use reqwest::{
blocking::{
multipart::{Form, Part},
Client,
},
header::{HeaderMap, HeaderValue, AUTHORIZATION},
Method, StatusCode,
};
use serde::Deserialize;
use serde_json::{json, Value};
use std::{env, fs::File, path::Path, time::Duration};
#[derive(Debug, Deserialize)]
pub struct RemoteRelease {
pub id: u64,
#[serde(default)]
pub draft: bool,
}
#[derive(Debug, Deserialize)]
struct Asset {
id: u64,
name: String,
}
pub struct Gitea {
client: Client,
base: String,
}
impl Gitea {
pub fn new(repository: &str, token_env: &str) -> Result<Self, Error> {
let token = env::var(token_env)
.ok()
.filter(|s| !s.is_empty())
.ok_or_else(|| Error(format!("Set {token_env} before publishing.")))?;
if !repository.starts_with("https://") {
return Err(Error(
"Authenticated Gitea publishing requires HTTPS.".into(),
));
}
Self::with_token(repository, &token)
}
fn with_token(repository: &str, token: &str) -> Result<Self, Error> {
let parsed = url::Url::parse(repository)
.map_err(|_| Error("Invalid release repository URL.".into()))?;
let parts: Vec<_> = parsed
.path_segments()
.ok_or_else(|| Error("Repository URL must end in owner/repository.".into()))?
.filter(|part| !part.is_empty())
.collect();
if parts.len() < 2 {
return Err(Error("Repository URL must end in owner/repository.".into()));
}
let prefix = parts[..parts.len() - 2].join("/");
let owner = parts[parts.len() - 2];
let name = parts[parts.len() - 1];
let origin = parsed.origin().ascii_serialization();
let base = if prefix.is_empty() {
format!("{origin}/api/v1/repos/{owner}/{name}")
} else {
format!("{origin}/{prefix}/api/v1/repos/{owner}/{name}")
};
let mut headers = HeaderMap::new();
headers.insert(
AUTHORIZATION,
HeaderValue::from_str(&format!("token {token}"))
.map_err(|_| Error("Invalid Gitea token value.".into()))?,
);
let client = Client::builder()
.default_headers(headers)
.user_agent("rust_realease/0.1")
.timeout(Duration::from_secs(120))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|_| Error("Cannot initialize Gitea HTTP client.".into()))?;
Ok(Self { client, base })
}
fn request(
&self,
method: Method,
path: &str,
body: Option<Value>,
missing_ok: bool,
) -> Result<Option<Value>, Error> {
let url = format!("{}{path}", self.base);
let mut request = self.client.request(method.clone(), &url);
if let Some(body) = body {
request = request.json(&body);
}
let response = request
.send()
.map_err(|_| Error(format!("Gitea {method} {path}: connection failed.")))?;
if missing_ok && response.status() == StatusCode::NOT_FOUND {
return Ok(None);
}
if !response.status().is_success() {
return Err(Error(format!(
"Gitea {method} {path}: HTTP {}.",
response.status().as_u16()
)));
}
let bytes = response
.bytes()
.map_err(|_| Error(format!("Gitea {method} {path}: response failed.")))?;
if bytes.is_empty() {
return Ok(Some(Value::Null));
}
serde_json::from_slice(&bytes)
.map(Some)
.map_err(|_| Error(format!("Gitea {method} {path}: invalid JSON response.")))
}
pub fn ensure_release(
&self,
tag: &str,
commit: &str,
draft: bool,
prerelease: bool,
) -> Result<RemoteRelease, Error> {
let encoded = encode(tag);
let remote = self
.request(Method::GET, &format!("/tags/{encoded}"), None, true)?
.ok_or_else(|| {
Error(format!(
"Remote tag {tag:?} is missing. Push the tag explicitly before publishing."
))
})?;
if remote.pointer("/commit/sha").and_then(Value::as_str) != Some(commit) {
return Err(Error(
"Remote tag commit differs from the checkout; refusing publication.".into(),
));
}
let release = if let Some(existing) = self.request(
Method::GET,
&format!("/releases/tags/{encoded}"),
None,
true,
)? {
existing
} else {
self.request(Method::POST, "/releases", Some(json!({"tag_name": tag, "name": tag, "target_commitish": commit, "draft": draft, "prerelease": prerelease})), false)?
.ok_or_else(|| Error("Gitea release creation returned no response.".into()))?
};
serde_json::from_value(release)
.map_err(|_| Error("Gitea release response is missing id or draft state.".into()))
}
fn assets(&self, release_id: u64) -> Result<Vec<Asset>, Error> {
let response = self
.request(
Method::GET,
&format!("/releases/{release_id}/assets"),
None,
false,
)?
.ok_or_else(|| Error("Gitea assets response is empty.".into()))?;
serde_json::from_value(response)
.map_err(|_| Error("Gitea assets response is invalid.".into()))
}
pub fn upload(&self, release_id: u64, paths: &[&Path], replace: bool) -> Result<(), Error> {
let existing = self.assets(release_id)?;
let names: Vec<_> = paths
.iter()
.map(|p| {
p.file_name()
.and_then(|s| s.to_str())
.ok_or_else(|| Error("Invalid upload filename.".into()))
})
.collect::<Result<_, _>>()?;
let collisions: Vec<_> = existing
.iter()
.filter(|asset| names.contains(&asset.name.as_str()))
.collect();
if !collisions.is_empty() && !replace {
return Err(Error(format!("Release assets already exist: {}. Use --replace-assets to replace them explicitly.", collisions.iter().map(|a| a.name.as_str()).collect::<Vec<_>>().join(", "))));
}
for (path, name) in paths.iter().zip(names.iter()) {
for asset in collisions.iter().filter(|asset| asset.name == *name) {
self.request(
Method::DELETE,
&format!("/releases/{release_id}/assets/{}", asset.id),
None,
false,
)?;
}
let input = File::open(path)
.map_err(|e| Error(format!("cannot open asset {}: {e}", path.display())))?;
let size = input
.metadata()
.map_err(|e| Error(format!("cannot inspect asset {}: {e}", path.display())))?
.len();
let part = Part::reader_with_length(input, size)
.file_name((*name).to_owned())
.mime_str("application/octet-stream")
.map_err(|_| Error("Invalid upload MIME type.".into()))?;
let mut url = url::Url::parse(&format!("{}/releases/{release_id}/assets", self.base))
.map_err(|_| Error("Invalid Gitea upload URL.".into()))?;
url.query_pairs_mut().append_pair("name", name);
let response = self
.client
.post(url)
.multipart(Form::new().part("attachment", part))
.send()
.map_err(|_| {
Error(format!(
"Gitea POST /releases/{release_id}/assets: connection failed."
))
})?;
if !response.status().is_success() {
return Err(Error(format!(
"Gitea POST /releases/{release_id}/assets: HTTP {}.",
response.status().as_u16()
)));
}
println!("Uploaded {name}");
}
Ok(())
}
}
fn encode(segment: &str) -> String {
url::form_urlencoded::byte_serialize(segment.as_bytes()).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use std::{
io::{Read, Write},
net::TcpListener,
thread,
time::Duration,
};
struct Reply {
method: &'static str,
path: &'static str,
status: u16,
body: &'static str,
}
fn server(replies: Vec<Reply>) -> (String, thread::JoinHandle<Vec<String>>) {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let address = format!("http://{}", listener.local_addr().unwrap());
let handle = thread::spawn(move || {
let mut requests = Vec::new();
for reply in replies {
let (mut stream, _) = listener.accept().unwrap();
stream
.set_read_timeout(Some(Duration::from_secs(5)))
.unwrap();
let mut bytes = Vec::new();
let mut buffer = [0_u8; 8192];
let header_end = loop {
let n = stream.read(&mut buffer).unwrap();
assert!(n > 0, "client closed before headers");
bytes.extend_from_slice(&buffer[..n]);
if let Some(index) = bytes.windows(4).position(|w| w == b"\r\n\r\n") {
break index + 4;
}
assert!(bytes.len() < 64 * 1024);
};
let headers = String::from_utf8_lossy(&bytes[..header_end]).to_string();
let first = headers.lines().next().unwrap().to_owned();
assert!(
first.starts_with(&format!("{} {} ", reply.method, reply.path)),
"{first}"
);
let length = headers
.lines()
.find_map(|line| {
line.to_ascii_lowercase()
.strip_prefix("content-length: ")
.and_then(|v| v.trim().parse::<usize>().ok())
})
.unwrap_or(0);
while bytes.len() < header_end + length {
let n = stream.read(&mut buffer).unwrap();
assert!(n > 0, "client closed before body");
bytes.extend_from_slice(&buffer[..n]);
}
requests.push(first);
let reason = if reply.status == 200 {
"OK"
} else if reply.status == 201 {
"Created"
} else if reply.status == 204 {
"No Content"
} else if reply.status == 404 {
"Not Found"
} else {
"Unauthorized"
};
write!(stream, "HTTP/1.1 {} {}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}", reply.status, reason, reply.body.len(), reply.body).unwrap();
}
requests
});
(address, handle)
}
#[test]
fn creates_or_reuses_release_after_remote_tag_check() {
let base = "/gitea/api/v1/repos/team/demo";
let (host, handle) = server(vec![
Reply {
method: "GET",
path: "/gitea/api/v1/repos/team/demo/tags/v0.1.0",
status: 200,
body: r#"{"commit":{"sha":"abc"}}"#,
},
Reply {
method: "GET",
path: "/gitea/api/v1/repos/team/demo/releases/tags/v0.1.0",
status: 404,
body: "{}",
},
Reply {
method: "POST",
path: "/gitea/api/v1/repos/team/demo/releases",
status: 201,
body: r#"{"id":7,"draft":false}"#,
},
]);
let api = Gitea::with_token(&format!("{host}/gitea/team/demo"), "test-token").unwrap();
let release = api.ensure_release("v0.1.0", "abc", false, true).unwrap();
assert_eq!(release.id, 7);
assert_eq!(handle.join().unwrap().len(), 3);
let (host, handle) = server(vec![
Reply {
method: "GET",
path: "/gitea/api/v1/repos/team/demo/tags/v0.1.0",
status: 200,
body: r#"{"commit":{"sha":"abc"}}"#,
},
Reply {
method: "GET",
path: "/gitea/api/v1/repos/team/demo/releases/tags/v0.1.0",
status: 200,
body: r#"{"id":9,"draft":true}"#,
},
]);
let api = Gitea::with_token(&format!("{host}/gitea/team/demo"), "test-token").unwrap();
assert_eq!(
api.ensure_release("v0.1.0", "abc", false, false)
.unwrap()
.id,
9
);
assert_eq!(handle.join().unwrap().len(), 2);
let _ = base;
}
#[test]
fn collisions_refuse_or_replace_only_matching_asset() {
let temp = tempfile::tempdir().unwrap();
let path = temp.path().join("one.tgz");
std::fs::write(&path, b"archive").unwrap();
let assets = r#"[{"id":3,"name":"one.tgz"},{"id":4,"name":"unrelated.tgz"}]"#;
let (host, handle) = server(vec![Reply {
method: "GET",
path: "/api/v1/repos/team/demo/releases/7/assets",
status: 200,
body: assets,
}]);
let api = Gitea::with_token(&format!("{host}/team/demo"), "test-token").unwrap();
assert!(api
.upload(7, &[&path], false)
.unwrap_err()
.to_string()
.contains("--replace-assets"));
assert_eq!(handle.join().unwrap().len(), 1);
let (host, handle) = server(vec![
Reply {
method: "GET",
path: "/api/v1/repos/team/demo/releases/7/assets",
status: 200,
body: assets,
},
Reply {
method: "DELETE",
path: "/api/v1/repos/team/demo/releases/7/assets/3",
status: 204,
body: "",
},
Reply {
method: "POST",
path: "/api/v1/repos/team/demo/releases/7/assets?name=one.tgz",
status: 201,
body: r#"{"id":5}"#,
},
]);
let api = Gitea::with_token(&format!("{host}/team/demo"), "test-token").unwrap();
api.upload(7, &[&path], true).unwrap();
assert_eq!(handle.join().unwrap().len(), 3);
}
#[test]
fn api_errors_do_not_echo_server_body_or_token() {
let (host, handle) = server(vec![Reply {
method: "GET",
path: "/api/v1/repos/team/demo/tags/v0.1.0",
status: 401,
body: "test-token",
}]);
let api = Gitea::with_token(&format!("{host}/team/demo"), "test-token").unwrap();
let error = api
.ensure_release("v0.1.0", "abc", false, false)
.unwrap_err()
.to_string();
assert!(error.contains("HTTP 401"));
assert!(!error.contains("test-token"));
handle.join().unwrap();
}
#[test]
fn encodes_tag_and_requires_https() {
assert_eq!(encode("v1.0+build"), "v1.0%2Bbuild");
assert!(Gitea::new("http://example.com/team/demo", "GITEA_TOKEN").is_err());
}
}

View File

@ -4,6 +4,7 @@ mod build;
mod cli;
mod config;
mod error;
mod gitea;
mod manifest;
mod model;
mod package;
@ -11,6 +12,7 @@ mod policy;
mod prepare;
mod project;
mod runner;
mod verify;
use clap::Parser;

View File

@ -140,7 +140,7 @@ fn unix_mode(metadata: &fs::Metadata) -> u32 {
}
}
fn checksum(path: &Path) -> Result<String, Error> {
pub fn checksum(path: &Path) -> Result<String, Error> {
let mut input =
File::open(path).map_err(|e| Error(format!("cannot open {}: {e}", path.display())))?;
let mut hasher = Sha256::new();

View File

@ -241,6 +241,10 @@ pub fn origin_url(root: &Path, runner: &Runner) -> Option<String> {
.ok()
}
pub fn head_commit(root: &Path, runner: &Runner) -> Result<String, Error> {
runner.text("git", &["rev-parse", "HEAD"], root)
}
pub fn validate_checkout_tag(root: &Path, tag: &str, runner: &Runner) -> Result<(), Error> {
let tagged = runner
.text(

View File

@ -1 +1,332 @@
//! Placeholder for the future verify implementation.
use crate::{
error::Error,
package::{self, Artifact},
policy,
project::SelectedPackage,
runner::Runner,
};
use flate2::read::GzDecoder;
use reqwest::blocking::Client;
use std::{
fs::{self, File},
io::{Read, Write},
path::Path,
time::Duration,
};
pub fn asset_url(repository: &str, tag: &str, filename: &str) -> String {
let encode =
|s: &str| -> String { url::form_urlencoded::byte_serialize(s.as_bytes()).collect() };
format!(
"{repository}/releases/download/{}/{}",
encode(tag),
encode(filename)
)
}
pub fn verify_archive(path: &Path, artifact: &Artifact) -> Result<(), Error> {
let mut tar = Vec::new();
let file =
File::open(path).map_err(|e| Error(format!("cannot open {}: {e}", path.display())))?;
GzDecoder::new(file)
.read_to_end(&mut tar)
.map_err(|e| Error(format!("{}: invalid gzip archive: {e}", path.display())))?;
if tar.len() < 2048 {
return Err(Error(format!("{}: truncated tar archive.", path.display())));
}
let directory = &tar[..512];
let first = member_name(directory)?;
if first != artifact.root || directory[156] != b'5' {
return Err(Error(format!(
"{}: archive root does not match binstall metadata.",
path.display()
)));
}
let binary = &tar[512..1024];
let second = member_name(binary)?;
if second != format!("{}/{}", artifact.root, artifact.binary) || binary[156] != b'0' {
return Err(Error(format!(
"{}: archive binary does not match binstall metadata.",
path.display()
)));
}
let mode = octal(&binary[100..108])?;
if !artifact.target.split('-').any(|part| part == "windows") && mode & 0o111 == 0 {
return Err(Error(format!(
"{}: binary is not executable.",
path.display()
)));
}
let size = octal(&binary[124..136])?;
let rounded = size
.checked_add(511)
.ok_or_else(|| Error("Archive size overflow.".into()))?
/ 512
* 512;
let end = 1024_u64
.checked_add(rounded)
.and_then(|n| n.checked_add(1024))
.ok_or_else(|| Error("Archive size overflow.".into()))?;
if end > tar.len() as u64 {
return Err(Error(format!(
"{}: truncated binary or tar footer.",
path.display()
)));
}
if tar[(end - 1024) as usize..].iter().any(|b| *b != 0) {
return Err(Error(format!(
"{}: archive contains unexpected members.",
path.display()
)));
}
Ok(())
}
fn member_name(header: &[u8]) -> Result<String, Error> {
let name = text_field(&header[..100])?;
let prefix = text_field(&header[345..500])?;
if prefix.is_empty() {
Ok(name)
} else {
Ok(format!("{prefix}/{name}"))
}
}
fn text_field(field: &[u8]) -> Result<String, Error> {
let end = field.iter().position(|b| *b == 0).unwrap_or(field.len());
std::str::from_utf8(&field[..end])
.map(str::to_owned)
.map_err(|_| Error("Archive contains non-UTF-8 paths.".into()))
}
fn octal(field: &[u8]) -> Result<u64, Error> {
let bytes = field
.iter()
.take_while(|b| **b != 0)
.copied()
.collect::<Vec<_>>();
let string =
std::str::from_utf8(&bytes).map_err(|_| Error("Invalid tar numeric field.".into()))?;
u64::from_str_radix(string.trim(), 8).map_err(|_| Error("Invalid tar numeric field.".into()))
}
pub fn verify_published(
repository: &str,
tag: &str,
artifacts: &[Artifact],
sums: &Path,
) -> Result<(), Error> {
let downloads = tempfile::Builder::new()
.prefix("rust-realease-download-")
.tempdir()
.map_err(|e| Error(format!("cannot create download directory: {e}")))?;
let client = Client::builder()
.timeout(Duration::from_secs(120))
.redirect(reqwest::redirect::Policy::limited(10))
.build()
.map_err(|_| Error("Cannot initialize anonymous download client.".into()))?;
for local in artifacts
.iter()
.map(|a| a.path.as_path())
.chain(std::iter::once(sums))
{
let filename = local
.file_name()
.and_then(|s| s.to_str())
.ok_or_else(|| Error("Invalid asset filename.".into()))?;
let mut response = client
.get(asset_url(repository, tag, filename))
.send()
.map_err(|_| Error(format!("{filename}: public download failed.")))?;
if response.status() != reqwest::StatusCode::OK {
return Err(Error(format!(
"{filename}: public download returned HTTP {}.",
response.status().as_u16()
)));
}
let expected = fs::metadata(local)
.map_err(|e| {
Error(format!(
"cannot inspect local asset {}: {e}",
local.display()
))
})?
.len();
let path = downloads.path().join(filename);
let mut output = File::create(&path)
.map_err(|e| Error(format!("cannot create download {}: {e}", path.display())))?;
let mut count = 0_u64;
let mut buffer = [0_u8; 64 * 1024];
loop {
let n = response
.read(&mut buffer)
.map_err(|_| Error(format!("{filename}: public download failed.")))?;
if n == 0 {
break;
}
count = count
.checked_add(n as u64)
.ok_or_else(|| Error("Download size overflow.".into()))?;
if count > expected {
return Err(Error(format!(
"{filename}: download exceeds expected size."
)));
}
output
.write_all(&buffer[..n])
.map_err(|e| Error(format!("cannot write download: {e}")))?;
}
drop(output);
if package::checksum(&path)? != package::checksum(local)? {
return Err(Error(format!(
"{filename}: published SHA-256 differs from local artifact."
)));
}
if let Some(artifact) = artifacts.iter().find(|a| a.path == local) {
verify_archive(&path, artifact)?;
println!("Verified URL, checksum and archive: {}", artifact.target);
}
}
println!("Verified published SHA256SUMS");
Ok(())
}
pub fn binstall_arguments(project: &SelectedPackage, target: &str, install: &Path) -> Vec<String> {
vec![
"binstall".into(),
project.name.clone(),
"--manifest-path".into(),
project.manifest.display().to_string(),
"--version".into(),
format!("={}", project.version),
"--targets".into(),
target.into(),
"--strategies".into(),
"crate-meta-data".into(),
"--no-confirm".into(),
"--no-track".into(),
"--disable-telemetry".into(),
"--no-discover-github-token".into(),
"--install-path".into(),
install.display().to_string(),
]
}
pub fn verify_binstall(
root: &Path,
project: &SelectedPackage,
targets: &[String],
runner: &Runner,
) -> Result<(), Error> {
for target in targets {
let temp = tempfile::Builder::new()
.prefix("rust-realease-binstall-")
.tempdir()
.map_err(|e| Error(format!("cannot create isolated install directory: {e}")))?;
let cargo_home = temp.path().join("cargo-home");
let install = temp.path().join("install");
fs::create_dir_all(&cargo_home)
.map_err(|e| Error(format!("cannot create isolated Cargo home: {e}")))?;
fs::create_dir_all(&install)
.map_err(|e| Error(format!("cannot create isolated install root: {e}")))?;
let args = binstall_arguments(project, target, &install);
let borrowed: Vec<_> = args.iter().map(String::as_str).collect();
let cargo_home = cargo_home
.to_str()
.ok_or_else(|| Error("Cargo home path is not UTF-8.".into()))?;
runner.output_with("cargo", &borrowed, root, &[("CARGO_HOME", cargo_home)], &[])?;
let binary = install.join(policy::binary_filename(&project.binary, target));
if !binary.is_file() {
return Err(Error(format!(
"cargo-binstall did not install the expected binary for {target}."
)));
}
println!("cargo-binstall installation verified: {target}");
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn anonymous_downloads_match_local_archives_and_checksums() {
use std::{net::TcpListener, thread};
let temp = tempfile::tempdir().unwrap();
let binary = temp.path().join("demo");
fs::write(&binary, b"binary").unwrap();
let (artifacts, sums) = package::package(
temp.path(),
"demo",
"0.1.0",
"demo",
&[("x86_64-unknown-linux-musl".into(), binary)],
)
.unwrap();
let filename = artifacts[0]
.path
.file_name()
.unwrap()
.to_str()
.unwrap()
.to_owned();
let bodies = [
(filename, fs::read(&artifacts[0].path).unwrap()),
("SHA256SUMS".into(), fs::read(&sums).unwrap()),
];
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let repository = format!("http://{}/team/demo", listener.local_addr().unwrap());
let handle = thread::spawn(move || {
for (filename, body) in bodies {
let (mut stream, _) = listener.accept().unwrap();
let mut buffer = [0_u8; 4096];
let count = stream.read(&mut buffer).unwrap();
let request = String::from_utf8_lossy(&buffer[..count]);
assert!(request.starts_with(&format!(
"GET /team/demo/releases/download/v0.1.0/{filename} "
)));
assert!(!request.to_ascii_lowercase().contains("authorization:"));
write!(
stream,
"HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n",
body.len()
)
.unwrap();
stream.write_all(&body).unwrap();
}
});
verify_published(&repository, "v0.1.0", &artifacts, &sums).unwrap();
handle.join().unwrap();
}
#[test]
fn local_archive_validation_and_binstall_command() {
let temp = tempfile::tempdir().unwrap();
let binary = temp.path().join("demo");
fs::write(&binary, b"binary").unwrap();
let (artifacts, _) = package::package(
temp.path(),
"demo",
"0.1.0",
"demo",
&[("x86_64-unknown-linux-musl".into(), binary)],
)
.unwrap();
verify_archive(&artifacts[0].path, &artifacts[0]).unwrap();
let project = SelectedPackage {
name: "demo".into(),
version: "0.1.0".into(),
binary: "demo".into(),
manifest: temp.path().join("Cargo.toml"),
repository: None,
target_directory: temp.path().join("target"),
};
let args = binstall_arguments(&project, "x86_64-unknown-linux-musl", temp.path());
assert!(args.contains(&"--no-track".to_owned()));
assert!(args.contains(&"--no-discover-github-token".to_owned()));
assert_eq!(
asset_url("https://example.com/team/demo", "release+1", "one.tgz"),
"https://example.com/team/demo/releases/download/release%2B1/one.tgz"
);
}
}