Add local reference pictures for layer crop previews
Gitea/canvas-pilot/pipeline/head There was a failure building this commit Details

This commit is contained in:
Dandaedre 2026-11-18 16:21:36 +01:00
parent 709fdcbb59
commit 9f5a8d737d
20 changed files with 343 additions and 17 deletions

2
Cargo.lock generated
View File

@ -602,7 +602,7 @@ dependencies = [
[[package]]
name = "canvas-pilot"
version = "0.9.70"
version = "0.9.71"
dependencies = [
"base64 0.22.1",
"craft-dwg",

View File

@ -4,7 +4,7 @@ default-members = ["crates/canvas-pilot"]
resolver = "3"
[workspace.package]
version = "0.9.70"
version = "0.9.71"
edition = "2024"
license = "MIT"
repository = "https://proxy.dandaedre.com/gitea/idle-org/canvas-pilot"

View File

@ -496,5 +496,9 @@
"preset.quarter": "Quarter",
"preset.tiling": "Tiling",
"preset.portrait": "Portrait",
"preset.landscape": "Landscape"
"preset.landscape": "Landscape",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "Kvarono",
"preset.tiling": "Kahelado",
"preset.portrait": "Vertikale",
"preset.landscape": "Horizontale"
"preset.landscape": "Horizontale",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "Cuarto",
"preset.tiling": "Mosaico",
"preset.portrait": "Vertical",
"preset.landscape": "Horizontal"
"preset.landscape": "Horizontal",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "Quart",
"preset.tiling": "Mosaïque",
"preset.portrait": "Portrait",
"preset.landscape": "Paysage"
"preset.landscape": "Paysage",
"ui.reference-picture": "Image de référence",
"ui.load-reference-picture": "Charger une image de référence…",
"ui.input-reference-pictures": "Images de référence des entrées",
"ui.reference-picture-help": "Référence locale uniquement ; jamais envoyée à l’appareil. Utilise le format source, le crop, le zoom, le fit et le masque du calque. Choisissez une entrée à capturer pour la remplacer."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "Τέταρτον",
"preset.tiling": "Πλακόστρωσις",
"preset.portrait": "Ὄρθιον",
"preset.landscape": "Πλάγιον"
"preset.landscape": "Πλάγιον",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "चौथाई",
"preset.tiling": "टाइल व्यवस्था",
"preset.portrait": "लंबवत",
"preset.landscape": "क्षैतिज"
"preset.landscape": "क्षैतिज",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "4分の1",
"preset.tiling": "タイル配置",
"preset.portrait": "縦向き",
"preset.landscape": "横向き"
"preset.landscape": "横向き",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "चौथाइ",
"preset.tiling": "टाइल मिलान",
"preset.portrait": "ठाडो",
"preset.landscape": "तेर्सो"
"preset.landscape": "तेर्सो",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "Quarto",
"preset.tiling": "Mosaico",
"preset.portrait": "Retrato",
"preset.landscape": "Paisagem"
"preset.landscape": "Paisagem",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "หนึ่งส่วนสี่",
"preset.tiling": "เรียงกระเบื้อง",
"preset.portrait": "แนวตั้ง",
"preset.landscape": "แนวนอน"
"preset.landscape": "แนวนอน",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "kipisi pi tu tu",
"preset.tiling": "kulupu leko",
"preset.portrait": "poka sewi",
"preset.landscape": "poka lili"
"preset.landscape": "poka lili",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -496,5 +496,9 @@
"preset.quarter": "四分之一",
"preset.tiling": "平铺",
"preset.portrait": "纵向",
"preset.landscape": "横向"
"preset.landscape": "横向",
"ui.reference-picture": "Reference picture",
"ui.load-reference-picture": "Load reference picture…",
"ui.input-reference-pictures": "Input reference pictures",
"ui.reference-picture-help": "Local reference only; never sent to the device. Uses this layer’s source format, crop, zoom, fit and mask. Choose a capture input to replace it."
}

View File

@ -473,7 +473,141 @@ impl Editor {
preview.guides = opacity / 100.0;
}
}
fn set_reference_picture(&mut self, i: usize, reference: super::super::reference::Reference) {
let id = self.layout.layer_info(i).id;
if let Some(choice) = self.device.source_choices.get_mut(&id) {
choice.follow = false;
}
self.capture.layers.insert(
id,
LayerPreview {
input: 0,
host: String::new(),
source: self.layout.formats().source_for[i],
texture: Some(reference.texture),
bytes: reference.bytes,
paused: false,
automatic: false,
capture_on_selection: false,
name: reference.name,
enabled: true,
guides: 0.35,
},
);
}
fn import_layer_reference_picture(&mut self, ctx: &egui::Context, i: usize) {
if let Some(path) = rfd::FileDialog::new()
.set_title(tr(ctx, "ui.reference-picture"))
.add_filter("Pictures", &["png", "jpg", "jpeg", "bmp"])
.pick_file()
{
match super::super::reference::Reference::load(
ctx,
&egui::DroppedFile {
path: Some(path),
..Default::default()
},
) {
Ok(reference) => {
self.set_reference_picture(i, reference);
self.capture.error.clear();
}
Err(error) => self.capture.error = error,
}
}
}
pub(super) fn input_reference_count(&self) -> usize {
(0..self.layout.layers().len())
.filter(|&i| {
self.capture
.layers
.get(&self.layout.layer_info(i).id)
.is_some_and(|p| {
p.texture.is_some() && p.source == self.layout.formats().source_for[i]
})
})
.count()
}
pub(super) fn input_reference_rows(&mut self, ui: &mut egui::Ui, ink: &Ink, y: f32, w: f32) {
if self.input_reference_count() == 0 {
return;
}
let palette = theme::palette(ui.ctx());
ink.text(
0.,
y + 8.,
&tr(ui.ctx(), "ui.input-reference-pictures"),
11.,
palette.purple,
);
let mut row = 0;
let mut remove = None;
let mut selected = false;
for i in 0..self.layout.layers().len() {
let info = self.layout.layer_info(i);
let id = info.id;
let layer_name = info.name.clone();
let Some(p) =
self.capture.layers.get_mut(&id).filter(|p| {
p.texture.is_some() && p.source == self.layout.formats().source_for[i]
})
else {
continue;
};
let row_y = y + 44. + row as f32 * 44.;
row += 1;
let rect = Rect::from_min_size(ink.point(0., row_y), Vec2::new(w, 38.));
ui.scope_builder(egui::UiBuilder::new().max_rect(rect), |ui| {
ui.horizontal(|ui| {
ui.checkbox(&mut p.enabled, "")
.on_hover_text(tr(ui.ctx(), "ui.enable-input-preview"));
let label = format!("{}: {}", layer_name, p.name);
if ui
.add_sized(
Vec2::new((ui.available_width() - 32.0).max(20.0), 24.0),
egui::Button::new(label).truncate(),
)
.on_hover_text(tr(ui.ctx(), "ui.reference-picture-help"))
.clicked()
{
self.layout.select(Some(i));
self.reference_selected = None;
self.edit_content = false;
selected = true;
}
if ui
.small_button("×")
.on_hover_text(tr(ui.ctx(), "ui.remove-reference-image"))
.clicked()
{
remove = Some(id);
}
});
});
}
if let Some(id) = remove {
self.capture.layers.remove(&id);
}
if selected {
self.sync();
}
}
pub(super) fn crop_preview_controls(&mut self, ui: &mut egui::Ui, i: usize) {
if ui
.button(tr(ui.ctx(), "ui.load-reference-picture"))
.on_hover_text(tr(ui.ctx(), "ui.reference-picture-help"))
.clicked()
{
self.import_layer_reference_picture(ui.ctx(), i);
}
let local_picture = self
.capture
.layers
.get(&self.layout.layer_info(i).id)
.is_some_and(|p| p.input == 0 && !p.bytes.is_empty());
if local_picture {
ui.small(tr(ui.ctx(), "ui.reference-picture-help"));
}
let id = self.layout.layer_info(i).id;
let source = self.layout.formats().source_for[i];
let inputs = self
@ -569,9 +703,9 @@ impl Editor {
preview.guides = opacity / 100.0;
}
if !preview.name.is_empty() { ui.small(&preview.name); }
if !online { ui.small(tr(ui.ctx(), "ui.connect-to-a-device-to-choose-and-capture-an-input")); }
else if preview.input == 0 { ui.small(tr(ui.ctx(), "ui.choose-an-input-to-capture-its-first-still-automatically")); }
else if !available { ui.small(tr(ui.ctx(), "ui.input-snapshots-are-unavailable-enable-snapshots-in-webrcs-then-refresh-the-connectio")); }
if !online && !local_picture { ui.small(tr(ui.ctx(), "ui.connect-to-a-device-to-choose-and-capture-an-input")); }
else if preview.input == 0 && !local_picture { ui.small(tr(ui.ctx(), "ui.choose-an-input-to-capture-its-first-still-automatically")); }
else if !available && !local_picture { ui.small(tr(ui.ctx(), "ui.input-snapshots-are-unavailable-enable-snapshots-in-webrcs-then-refresh-the-connectio")); }
if !idle { ui.spinner(); ui.small(tr(ui.ctx(), "ui.fetching-an-input-snapshot")); }
else if cooling { ui.small(tr(ui.ctx(), "ui.capture-available-again-shortly")); ui.ctx().request_repaint_after(std::time::Duration::from_secs(1)); }
});
@ -897,6 +1031,85 @@ mod tests {
use super::super::tests::{click_text, frame};
use super::*;
#[test]
fn local_reference_picture_survives_project_roundtrip_without_capture_or_geometry_changes() {
let ctx = egui::Context::default();
let mut app = Editor::default();
let before = app.layout.layers().clone();
let formats = app.layout.formats().clone();
let reference = tiny_reference(&ctx, "offline.png");
let bytes = reference.bytes.clone();
let id = app.layout.layer_info(0).id;
app.device.source_choices.insert(
id,
super::super::device_ui::SourceChoice {
input: 4,
change: false,
follow: true,
},
);
app.set_reference_picture(0, reference);
app.sync_followed_inputs();
assert!(!app.device.source_choices[&id].follow);
assert!(!app.device.source_choices[&id].change);
assert_eq!(*app.layout.layers(), before);
assert_eq!(*app.layout.formats(), formats);
assert_eq!(app.input_reference_count(), 1);
assert!(app.layer_preview(0).is_some());
assert!(app.next_selection_capture().is_none());
assert!(app.next_preview_refresh().is_none());
let saved = app.project();
let json = saved.to_json().unwrap();
let mut restored = Editor::default();
restored
.install_project(
&ctx,
canvas_pilot::project::Project::from_json(&json).unwrap(),
)
.unwrap();
assert_eq!(restored.project(), saved);
assert_eq!(restored.capture.layers[&id].bytes, bytes);
assert_eq!(restored.capture.layers[&id].input, 0);
assert!(restored.capture.layers[&id].host.is_empty());
assert!(!restored.capture.layers[&id].automatic);
assert!(restored.layer_preview(0).is_some());
restored.capture.layers.get_mut(&id).unwrap().enabled = false;
assert!(restored.layer_preview(0).is_none());
assert_eq!(restored.input_reference_count(), 1);
restored.capture.layers.remove(&id);
assert_eq!(restored.input_reference_count(), 0);
}
#[test]
fn input_reference_list_selects_its_layer() {
let ctx = egui::Context::default();
let mut app = Editor::default();
app.set_reference_picture(0, tiny_reference(&ctx, "offline.png"));
app.layout.select(Some(1));
let label = format!("{}: offline.png", app.layout.layer_info(0).name);
click_text(&mut app, &ctx, &label);
assert_eq!(app.layout.selected(), Some(0));
assert!(app.reference_selected.is_none());
assert!(app.layer_preview(0).is_some());
}
#[test]
fn local_reference_rejects_a_capture_already_in_flight() {
let ctx = egui::Context::default();
let mut app = Editor::default();
let id = app.layout.layer_info(0).id;
let source = app.layout.formats().source_for[0];
let (tx, rx) = std::sync::mpsc::channel();
app.capture.pending = Some(rx);
app.capture.pending_layer = Some((id, source, 4, app.device.host.clone()));
app.set_reference_picture(0, tiny_reference(&ctx, "offline.png"));
tx.send(Ok(tiny_reference(&ctx, "late device image")))
.unwrap();
app.capture_window(&ctx);
assert_eq!(app.capture.layers[&id].name, "offline.png");
assert!(app.capture.pending.is_none());
}
#[test]
fn following_source_invalidates_old_capture_without_routing() {
let mut app = Editor::default();

View File

@ -1582,7 +1582,13 @@ impl Editor {
fn inspector(&mut self, ui: &mut egui::Ui) {
let rows = self.layer_tree_rows();
let layer_row_count = rows.len();
let input_reference_rows = self.input_reference_count();
let row_count = layer_row_count
+ if input_reference_rows == 0 {
0
} else {
input_reference_rows + 1
}
+ 1
+ if self.references.is_empty() {
0
@ -1810,6 +1816,17 @@ impl Editor {
true,
);
self.reference_rows(ui, &ink, 28.0 + (layer_row_count + 1) as f32 * 44.0, w);
let fixed_reference_rows = if self.references.is_empty() {
0
} else {
self.references.len() + 1
};
self.input_reference_rows(
ui,
&ink,
28.0 + (layer_row_count + 1 + fixed_reference_rows) as f32 * 44.0,
w,
);
let r = r.translate(Vec2::new(0.0, (row_count as f32 - 2.0) * 44.0));
let ink = Ink::new(ui, r);
ink.line([0.0, 130.0], [w, 130.0], palette.line);

View File

@ -4,6 +4,13 @@
## Unreleased
### v0.9.71
- Load a local reference picture into a layer’s input preview for offline crop,
zoom, fit and mask preparation. Pictures never upload to a switcher.
- List input reference pictures separately beneath fixed reference pictures.
Local pictures save with projects and disable capture refresh/follow for that layer.
### v0.9.70
- Stop treating unrecognized clipboard text as draw.io XML. Unsupported text now

View File

@ -96,6 +96,27 @@ See [hardware checks and limitations](../device-check.md) and the
[device exchange contract](../../interfaces/midra-device-send.md). The footer's ONLINE
status reflects the last successful read, not continuous polling.
## Reference picture for offline cropping
In a layer’s **Preview** controls, choose **Load reference picture…** and select a
PNG, JPEG or BMP. The picture replaces that layer’s captured preview and follows
its crop, zoom, fit and mask, just like an input preview. It is **local only**:
Canvas Pilot never sends the picture to a switcher or changes device input routing.
The picture represents the full source frame using the layer’s configured source
format. Set that format to match the intended input; loading a picture does not
change source dimensions or geometry automatically. Capture auto-refresh and
**Follow input** are disabled when loading it. Choosing a physical input again
replaces the local picture with a device capture.
**Input reference pictures**, beneath the fixed reference pictures in the layer
list, lists both local pictures and captured layer previews. Click a row to select
its layer; use the checkbox to hide/show its preview or × to remove it. Fixed
reference pictures remain independent canvas overlays.
Save a `.pilot.json` project to retain pictures and preview visibility offline.
As with captured previews, geometry-only exports do not retain these pictures.
## Capture an input still from Midra
For a layer, open **Layer properties → Preview** (or right-click → **Configure capture…**):

View File

@ -322,6 +322,15 @@ original image bytes, empty if no still has been captured). Stale assignments fo
removed layers or changed editor input slots are omitted on save. Captures retain
original bytes, without recompression or display rounding.
Local **Reference pictures** reuse this existing preview representation: `input: 0`,
`host: ""`, `automatic: false`, and nonempty `image_base64`/`name`. No schema change
is needed. They use the layer’s source frame and the same crop/fit/mask renderer;
loading one changes neither geometry nor source dimensions nor device routing.
Follow-input and pending selection refresh are disabled for that layer, and late
physical-input captures cannot replace the local image. Selecting a physical input
explicitly replaces the local picture. Preview images, including local ones, are
not included in geometry-only exports.
Reference and preview images together must fit within **64 MiB decoded base64
bytes**, and the whole JSON within 96 MiB. Decoding uses the same bounded image
loader as references, on the project-loading worker. Malformed images abort the

View File

@ -143,3 +143,14 @@ request is created. Opening/enabling these controls does not fetch the preselect
input; Capture still or explicit auto-refresh starts capture. The preselected input is saved with the preview assignment in project version 11.
Aquilon uses the same capture scheduling and image limits through its own discovery profile (`IN_n`, layer source `LIVE_n`, up to 256 discovered inputs). See [Aquilon discovery and endpoints](device-adapters.md). Midra retains its 16-input limit; family-specific IDs are not guessed from a saved layer number.
## Local reference pictures
Layer previews also accept PNG/JPEG/BMP files through **Load reference picture**.
They use the bounded reference decoder and the existing layer preview rendering;
there is no network upload. A local picture clears the physical input and host,
disables automatic capture and Follow input, and rejects late capture results.
The project preview representation retains the original image bytes (see the
[project contract](canvas-pilot.md)). The layer inspector lists these
and captured previews under **Input reference pictures**, below fixed references.