beffroi-tools/docs/midra-layout-plan.md

458 lines
28 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Midra layout helper: plan and visual proposals
Status: the [Rust/egui positioning draft](midra-layout.md) supports draw.io
geometry paste with review, two-layer multi-selection and group movement, box
selection, explicit-target alignment, numeric corner positioning/size, undo/redo
and camera zoom/pan. Concept A is user-approved.
Reference images, output presets/custom resolution, source/input formats and
advanced crop-backed content zoom/offset are implemented offline. Device mapping
verification, connectivity and REST calls remain deferred. Checklist gates
that combine implemented and outstanding work remain open.
Target: Analog Way Eikos 4K, running the latest firmware according to the user.
Record the actual installed version during compatibility checks; do not infer it
from a different Midra generation's documentation. Working name: **Midra Layout**.
## Intended workflow
Set the output format and preset or custom screen resolution, optionally load a reference picture, then place one or
two rectangular live layers visually or by entering exact offsets from a chosen
screen corner. Read or copy their equivalent WebRCS position values. Select layers
on the canvas or in the right-hand list. All work is local and offline.
For this plan, “window” means the application window and “layer” means a video
rectangle within one screen viewport. There is no additional window/group object.
The first release supports two layers; the underlying collection should support
more later without promising that every device configuration supports them.
The reference image is an editing aid, not an additional hardware layer. Screen
resolution is independent of the application window size, zoom and image size.
Source routing, transitions, playback, masking, rotation, keying, live preview
and device control remain outside the offline implementation scope. Device writes
are a later stretch goal (step 10), not authorized work on the connected unit.
Input cropping and virtual layer
resolutions are now planned, with support deferred until after the first draft.
Their coordinate spaces must be designed from the start. See the
[resolution and advanced-cropping specification](midra-layout-resolutions.md).
## Ordered task list
Complete each acceptance gate before broadening the scope. The first usable
positioning draft comprises steps 1–6, including source/output format selection
and custom viewport dimensions, and the applicable checks in step 8. Steps 7
and 9 follow the draft. Step 9 is committed scope; its device compatibility
checks must pass before presenting advanced-crop values as usable in WebRCS.
### 1. Establish the Eikos 4K coordinate contract
- [ ] Record device model, exact firmware and relevant WebRCS field labels.
- [ ] Check horizontal and vertical origins, signs, pixel units, numeric step,
accepted size/position ranges and off-screen placement in that WebRCS version.
- [ ] Verify whether the editable bounds refer to the layer rectangle rather
than cropped source content. Start with an unmasked, uncropped live layer.
- [ ] Check odd widths/heights and half-pixel centers; do not round silently.
- [ ] Establish input-crop and layer-crop coordinate spaces, units, processing
order, limits and Centered aspect behavior for the later resolution milestone.
Separate final visible bounds from any device layer container; do not infer
container size or position from the uncropped image's projected dimensions.
- [ ] Record examples at all four screen corners and at screen center using a
simulator or manual entry in an appropriate test setup, without API calls.
Gate: a small verified input/output table establishes what the application may
label as WebRCS-compatible. The example math below is provisional until then.
### 2. Review visual direction and prove the viewport
- [x] Choose a direction from the three proposals below. Approved: A, with
B's optional rulers and C's contextual alignment panel later.
- [ ] Build a disposable viewport interaction spike with two rectangles before
committing to the full UI. Rust/egui is the preferred starting point because
the repository already uses it; validate interaction quality before settling.
- [ ] Prove pointer-centered zoom, pan, selection, hit testing, keyboard nudge,
image rendering and display scaling on the target operator laptop.
- [ ] Try a dense reference image at Fit, 100%, 400% and 1600% zoom and at several
OS display scales. Zooming must not move layer coordinates.
- [ ] Review at 1440 × 900 and 1024 × 768 logical window sizes, including enlarged
UI text. Keep numerical controls usable with a scrollable right inspector.
Gate: the viewport feels predictable and shows exact edges clearly. If the
toolkit cannot meet these needs, resolve that before building the rest.
### 3. Implement geometry independently of the UI
- [x] Add workspace member `crates/midra-layout/` with shared geometry
and a desktop binary.
- [ ] Model screen dimensions and stable layer IDs with canonical top-left
position and final visible size. Keep selection, camera and reference image separate.
- [ ] Define source format/native dimensions, input crop, advanced crop and
derived device geometry as distinct concepts from day one. Store crop edges
and final size as authoritative geometry; content zoom/offset are derived
controls. Do not implement crop interactions until after the first draft.
- [ ] Implement four-corner offset conversion and inverse WebRCS conversion
under the verified device contract.
- [ ] Switching corner changes the displayed offsets without moving the layer.
Resizing keeps the chosen corner and its offsets fixed; moving changes offsets.
- [ ] Preserve precision internally. Validate finite numbers, positive sizes and
device limits; report unsupported values without silently clamping them.
- [ ] Test all corners, inverse round trips, odd dimensions, negative offsets,
non-16:9 screens, exact screen edges and resizing from each anchor.
Gate: viewport scale and UI state cannot affect geometry or conversion results.
### 4. Build the two-layer positioning workspace
- [ ] Add screen resolution controls, one/two-layer setup, persistent layer IDs
and names, and a right-hand layer list with a selected-layer inspector.
- [ ] Add source format choices (including 4:3, 16:9 and 16:10) with explicit
native pixel dimensions; an aspect ratio alone is not a pixel resolution.
- [ ] Add output format choices and clearly labeled viewport resolution presets
(HD 1280 × 720, Full HD 1920 × 1080, UHD 4K 3840 × 2160), plus manual width
and height. Derive the output ratio from custom dimensions, never silently
stretch them to match a preset. Other formats can be added after validation.
- [ ] Changing output dimensions preserves existing layer pixel bounds, flags
out-of-screen layers and rechecks reference-image mapping. Validate positive
integer dimensions independently of verified device output support; custom
planning geometry must not imply that Eikos accepts every output mode.
- [ ] Support click selection, Shift-click multi-selection, clear selection and
list selection for overlapped or off-screen rectangles. Define deterministic
overlap hit testing; list order represents preview stacking, not device routing.
- [ ] Add dragging, exact width/height fields, four-corner selection and inward
offsets. Prefer numeric sizing first; add resize handles only with anchor
behavior consistent with the geometry contract.
- [ ] Show WebRCS horizontal/vertical center values beside corner inputs; make
output values read-only initially. Offer per-value copy and labeled plain text
for a whole layer. Do not imply plain text is a WebRCS-native clipboard format.
- [ ] Add arrow nudges of 1 screen pixel and Shift-arrow nudges of 10; focused
text fields must receive these keys instead of moving the selection.
- [ ] Implement Space-drag/middle-button pan, pointer-centered zoom, Fit screen,
Fit selection, explicit zoom percentage and a clearly defined 1:1 pixel view.
- [ ] Keep handles/hit areas legible at every zoom; distinguish UI logical points,
screen pixels and physical display pixels on HiDPI screens.
- [ ] Add visible optional snapping to pixel edges, screen edges and other layer
edges. Exact numeric entry bypasses proximity snapping. Show the active guide
and allow a temporary snap override; toggling snapping must not move anything.
- [ ] Add undo/redo for geometry changes, one undo step per drag or committed
field edit, and Escape to cancel an in-progress drag or uncommitted edit.
Gate: place a layer 120 px from the right and 180 px from the top, change zoom,
switch anchors and move it by one pixel without losing track of the real values.
Selection and dragging never alter unrelated layers. No selection shows screen
and reference settings; multi-selection shows mixed values explicitly.
### 5. Add accurate reference-picture placement
- [ ] Load PNG/JPEG through a file picker or drag/drop; report decoding failures
while preserving the current image. Keep source files unchanged.
- [ ] Use a matching-resolution image at screen origin with 1 image pixel per
screen pixel. For mismatched dimensions, display the mismatch and require an
explicit mapping choice: native pixels with offset, fit preserving aspect, or
stretch to screen. Show scale, offsets and any letterboxing.
- [ ] Lock the reference by default, add hide/show and opacity controls, and
offer outline-only layer display so image edges remain visible.
- [ ] Use sharp nearest-neighbor inspection at high zoom and an appropriate
downsampled view when zoomed out. Do not confuse filtering with coordinate
precision. Reveal the screen-pixel grid only when it is legible.
- [ ] Decode large images away from the input/render loop, cap supported image
dimensions/memory with useful errors, and preserve viewport responsiveness.
Gate: a known rectangle in a 1920 × 1080 test image can be matched exactly by a
layer; changing image opacity, visibility, zoom or filtering changes no geometry.
### 6. Add extensible clipboard position import
- [ ] Start with an explicit, documented plain-text geometry format and a
paste-preview panel. For example: `x=1160; y=180; width=640; height=360; origin=top-left; unit=px`.
- [ ] Preview parsed values, origin, units, target layer and before/after bounds.
Allow position-only import; importing size must be an explicit choice.
- [ ] Treat Ctrl/Cmd-V in a focused text input as normal text paste. Elsewhere,
open geometry preview; do not mutate the document merely by reading clipboard.
- [ ] Define adapters by source format/version. Collect actual clipboard samples
from each requested software before claiming support. The first adapter now
supports the supplied draw.io mxGraphModel clipboard example. Do not guess an
application from an unlabeled number tuple.
- [ ] Make points, inches, percentages, source canvas origins, decimal conventions
and scale explicit. Ambiguity produces an explanation, not a guessed placement.
- [ ] Validate the complete proposed edit before applying it atomically, with one
undo step. Unknown formats, missing fields and invalid values preserve state.
- [ ] Test real fixtures for every added adapter and clipboard failure behavior
on supported desktop platforms, including Wayland where applicable.
Gate: supported data places the selected rectangle correctly; malformed or
ambiguous data makes no changes. Ship the basic format first if no external
software format has yet been selected; do not claim that completes an adapter.
### 7. Explicit multi-layer alignment
- [x] Add Align top and Align left; show the alignment target explicitly:
screen, selection bounds or a named reference layer.
- [x] For a reference layer, keep that layer fixed and move only the others.
For selection bounds, align to the minimum top/left edge. For the screen,
align to coordinate zero on the selected axis.
- [x] Preserve sizes and the other axis, preview the affected selection, and
make each alignment a single undoable operation.
- [x] Add right/bottom/center alignment using the same explicit targets.
- [x] Add Shift-click toggling in the viewport/list, left-drag box selection,
Shift-drag additive selection and Escape cancellation. Group movement/nudging
preserves spacing and uses one undo step per action.
- [ ] Extend beyond two layers under step 11. No distribution or constraint system yet.
Gate: two differently sized layers align exactly against the chosen target,
regardless of selection order except when explicitly choosing a reference layer.
### 8. Harden and package the first release
- [ ] Include empty, invalid-input, reference-load-error, off-screen and clipboard
error states in UI review, with an obvious way to recover or frame the selection.
- [ ] Verify tab order, keyboard focus, numeric input commit/cancel, contrast and
selection indicators that do not rely on color alone.
- [ ] Manually exercise screen/reference setup → corner positioning → nudge →
paste preview → copy values → undo. Verify results in the target WebRCS setup.
- [ ] Add geometry/import regression tests and interaction tests for camera
transforms and undo. Establish a smooth-drag baseline on the operator laptop;
target a 60 Hz interaction budget, measuring rather than assuming performance.
- [ ] Document supported firmware, coordinate conventions, clipboard formats,
limits and shortcuts. Keep the root README to a short link/quick start.
- [ ] Integrate the actual crate into workspace CI, coverage and manual packaging;
run the repository Rust gates, whole-application coverage and relevant packaging
tests. Document unavailable device or desktop checks.
- [ ] Commit each completed task; no automatic release publishing.
Session-file save/load can follow if needed; it is not required to demonstrate
precise positioning. Networking is deferred to the separate stretch goal below.
### 9. After the first draft: input and layer resolutions
- [ ] Implement the separate input-crop and advanced-crop stages defined in the
[resolution specification](midra-layout-resolutions.md), with no device calls.
- [ ] Keep Source format selected when Input resolution is Custom. Support exact
input dimensions and crop placement, defaulting to equal opposite-side crops.
- [ ] Expose Layer resolution as the final visible width/height, with optional
aspect lock. Add Content zoom and Offset X/Y controls and an explicit content
framing mode; moving the layer and moving its content must be distinct actions.
- [ ] Keep the final box fixed during content framing; store four crop edges and
final size, derive zoom/offset, and compose input/layer crops without applying
either stage twice. Preserve source-level sharing when two layers use one input.
- [ ] Test the documented numerical examples, inverse transforms, offset limits,
extreme ratios, fractional crops, resizing, shared inputs and undo behavior.
- [ ] Verify the device adapter's crop units, Centered mode, layer container size
and center position with the actual firmware. Show rounding error; block values
that cannot preserve the requested box within the verified device precision.
- [ ] Add UI review, documentation and release checks for these features before
claiming them as supported. Existing A/B/C mockups depict the positioning draft.
Gate: the user's 1920 × 1080 source → 1920 × 1000 input → 300 × 250 layer
workflow retains a 300 × 250 selectable box through zoom, content offset and
placement. The WebRCS settings reproduce the same visible result without masking.
### 10. Later stretch goal: apply layer format through the WebRCS REST API
Status: requested for later; do not implement or exercise device connectivity
during the current offline work. The user's currently connected WebRCS controls
a **live Eikos 4K**. Do not probe, connect to or send commands to that unit.
This backlog entry does not authorize future live-device testing or writes.
- [ ] After offline positioning and advanced cropping are validated, research
the supported API for the exact model/firmware using documentation and an
isolated simulator or explicitly designated non-live test device. Verify
endpoint availability, authentication, units and field semantics rather than
assuming an endpoint from the WebRCS UI.
- [ ] Define the smallest supported operation for applying a layer's format:
dimensions and, where needed for the virtual format, crop/aspect settings from
the verified device adapter. Identify any required position correction and
include it explicitly in the proposed change. Do not alter source routing,
transitions or trigger a Take as an implicit side effect.
- [ ] Keep offline mode as the default. Selecting a device or editing, dragging,
pasting, undoing or redoing locally must never automatically send commands.
- [ ] Present a concrete before/after preview with the device identity, firmware,
screen, layer, Program/Preview target and every setting to be changed. Require
an explicit user action to send that reviewed change; do not infer that a
Preview target is harmless or available without verifying device behavior.
- [ ] Detect stale device state before applying; require a refreshed review if
relevant settings changed. Document whether the API can apply the change
atomically. Report partial failures and uncertain outcomes without blind
retries or automatic rollback commands against a live system.
- [ ] Test against mocked transport first, then an explicitly authorized isolated
target. Verify that no writes occur on startup, connection, local edits,
cancellation or errors, and that only reviewed fields are sent.
Gate: a reviewed layer-format change can be applied and checked on a designated
non-live target. Any use on the current live installation requires separate,
explicit authorization at that time. No API calls are part of this planning task.
### 11. Later stretch goal: layer management and export scope
Status: per-layer visibility and locking are implemented for the two current layers;
the remaining management features are requested for later. This is local document organization; it can be
implemented independently of step 10 and does not authorize device connectivity.
Folders are an optional follow-up, not a prerequisite for adding/hiding layers.
- [ ] Replace the fixed two-layer document with a collection using stable local
IDs. Add an **Add layer** button and editable names. New local layers start
visible and unmapped; adding a planning layer does not create a hardware layer.
- [x] Add per-layer **Show / Hide** controls in the layer list. Hidden layers
keep their geometry and mapping, remain accessible in the list, and are excluded
from canvas drawing and hit testing. Hiding is local only: never send a device
hide, clear, opacity or routing command as a side effect.
- [ ] Add an **Options → Layer mapping** section that maps each local layer to
an explicit device/profile, screen and Midra layer ID. Keep local names and list
order independent of hardware identity; reordering or hiding must not remap.
- [ ] Add an export-scope choice: **Specific layer…** or **Visible layers only**.
The specific-layer picker uses local names and shows the mapped destination;
allow intentionally exporting a hidden layer through this explicit selection.
Visible means enabled in the layer list, not currently inside the viewport,
uncovered by another layer or selected on the canvas.
- [ ] Preview the exact included layers, geometry and destination mappings before
export. An empty visible set gives an explanation and produces no output.
Preserve deterministic list order and never include excluded layers implicitly.
- [ ] For device-targeted exports or a future API send, require mappings for every
included layer and reject unsupported destinations or duplicate destination
mappings within that export. Alternative local layouts may share a destination
when only one is included. Generic coordinate copies may remain unmapped.
- [ ] Keep export scope distinct from transport: exporting/copying a configuration
does not send it. Future device writes still use step 10's separate reviewed,
explicit send action, including an explicit Program/Preview target.
- [ ] Make layer creation and visibility changes undoable. When session save/load
is added, preserve stable IDs, names, geometry, visibility and mappings. Extend
selection, paste targeting and undo so they no longer depend on array indices.
- [ ] Optional: add named folders with expand/collapse and drag-to-organize.
Folders organize local layers; they do not introduce geometry transforms or
hardware groups. Collapsing a folder changes only the list display. If folder
visibility is added, effective visibility is the layer's own setting AND all
ancestor-folder settings; hiding a folder must preserve child visibility choices.
- [ ] Test adding layers, hidden-layer selection through the list, stable mappings
after reorder, paste/undo targeting, both export scopes, empty scopes, unmapped
or conflicting targets, and optional nested-folder visibility. Confirm that
organization and export preparation produce no device calls.
Gate: maintain several local layout alternatives, map them explicitly in Options,
then export one named layer or exactly the visible set without changing local
geometry, overwriting a conflicting destination silently, or contacting a device.
## Coordinate contract to verify
Proposed internal model: screen origin at its top-left, X increases rightward,
Y increases downward, and a layer's final visible box is `(x, y, width, height)`. Positive corner
offsets point inward. A corner means the matching corner of the layer measured
from that screen corner; there are not two independent anchor selectors in v1.
For screen size `W × H`, layer size `w × h`, and offsets `dx, dy`:
| Selected corner | Layer left `x` | Layer top `y` |
| --- | --- | --- |
| Top left | `dx` | `dy` |
| Top right | `W - dx - w` | `dy` |
| Bottom left | `dx` | `H - dy - h` |
| Bottom right | `W - dx - w` | `H - dy - h` |
For the uncropped positioning draft, under the user's described center-position convention, WebRCS values are
`horizontal = x + w/2`, `vertical = y + h/2`. The inverse subtracts the half-size.
The corresponding inward margins are `left=x`, `top=y`, `right=W-x-w` and
`bottom=H-y-h`. Switching corner just chooses which margins to display.
Example used in every mockup: on a **1920 × 1080** screen, a **640 × 360** layer
**120 px from the right** and **180 px from the top** has top-left **1160, 180**
and center **1480, 360**. The other layer is at **120, 180**, center **440, 360**.
An odd width such as 641 produces a half-pixel center when the left edge is an
integer; hardware acceptance/rounding is a compatibility question, not something
to hide with display formatting. Advanced cropping must retain these visible-box
coordinates while deriving any distinct device container through its adapter.
## Visual proposals
These are original, editable SVG interface mockups, not application screenshots.
Open them in a browser or image viewer to inspect at full size. The example
background is illustrative artwork standing in for an imported picture. Labels
such as “Copy” and “Paste” illustrate planned controls; nothing is interactive yet.
### A. Operator desk — recommended
Dark slate surfaces, cyan selection, amber second-layer identity and a restrained
purple primary action consistent with existing repository interfaces. A large
viewport occupies the left; the fixed right inspector groups the layer list,
corner selector, offsets, size and WebRCS outputs in reading order. Thin outlines,
corner markers, center crosshairs and dimension guides make selection unambiguous.
![Concept A: dark operator workspace](design/midra-layout/a-concept.svg)
Best for working beside show-control tools. The tradeoff is less contrast against
dark reference images; add a contrasting outline halo and an outline-only mode.
Keep the output card visible while upper inspector sections scroll. A compact
bottom status line carries camera/pixel information and shortcut hints.
### B. Precision desk
A light inspector and drafting surface with screen-coordinate rulers. The same
placement workflow emphasizes measurements and field readability. The artwork
retains its own colors independently of the UI theme.
![Concept B: light workspace with coordinate rulers](design/midra-layout/b-concept.svg)
Best for daytime preparation and detailed artwork matching. It uses more visible
measurement marks and can feel busier; rulers and guides should be optional. Its
useful features can be incorporated into A without maintaining two different apps.
### C. Focus desk
A wider viewport and narrower contextual inspector. This proposal illustrates the
multi-selection state: a named reference layer, explicit alignment actions
and a compact per-layer coordinate table. The layer list remains on the right.
![Concept C: compact workspace with two selected layers](design/midra-layout/c-concept.svg)
Best for repeated placement and comparison once the basic workflow is familiar.
Less space for numeric explanations makes it weaker as the initial default.
The alignment state is now available under step 7; the SVG remains a design concept.
### Shared interaction and responsive behavior
Keep the screen aspect ratio exact, surround it with a neutral pan area, and use
consistent screen-space selection markers. Display only relevant guides, with
labels in screen pixels. Layer identity and selected state must be distinct.
Always allow selecting a layer from the list when its outline is hard to hit.
The desktop window should retain a right inspector at normal working sizes.
At narrow widths, reduce decorative padding and collapse secondary sections;
below the proven minimum width, use a right drawer rather than crushing numeric
fields. At short heights, scroll sections while retaining access to conversion
outputs. Mouse and keyboard interaction are the first priority; a mobile UI is
not part of this desktop tool.
## Research and open decisions
The [Analog Way Midra 4K V3.2 manual, July 2024, pp. 83–85](https://www.bhphotovideo.com/lit_files/1168243.pdf#page=83)
documents layer alignment, snapping, position and size controls. This supports
the workflow research; it does not establish the precise coordinate convention
or current firmware behavior for the user's installation. Avoid portraying all
WebRCS versions as lacking alignment. The proposed improvement is more explicit
corner-based positioning and reference-image inspection.
[Figma's zoom/view documentation](https://help.figma.com/hc/en-us/articles/360041065034-Adjust-your-zoom-and-view-options)
provides useful precedents for fit-to-view, selection zoom and pixel inspection.
Borrow those interaction ideas, without introducing vector-editor features.
Analog Way's [Midra 4K FAQ](https://www.analogway.com/midra-4k-faqs) identifies an
offline simulator that may help verify behavior without connecting this helper
to hardware. Simulator agreement should still be checked against the installed
WebRCS version.
Open compatibility decisions: exact installed firmware and primary operator
OS/display scale. Concept A is approved and draw.io is the first clipboard source,
with the user-supplied percent-encoded rectangle retained as a regression fixture.
Reference-picture implementation update: PNG/JPEG/BMP import and file drop are
available, with fill-screen or proportion-preserving fit, opacity and visibility.
Resolution scaling produces a warning; aspect distortion a severe warning. Both
remain non-blocking. Image offsets and persistent sessions remain future work.
Corner-resizing implementation update: selected layers have interactive corner
handles with a fixed opposite anchor, ratio locking enabled by default, an explicit
free-resize toggle, Escape cancellation and one undo record per gesture. Numeric
width/height edits follow the same ratio toggle; pasted source sizes remain exact.
Formats/framing implementation update: dynamic output presets/custom resolution,
source aspect/native dimensions, shared input crops, virtual visible layer sizes,
advanced crop-backed zoom/offset, content drag/nudge, test-pattern preview and
provisional calculation reports are implemented. Resize/source edits preserve
normalized content centers, with explicit review of any required adjustment.
Document history includes source, output and layer crops atomically. Firmware
container verification, crop quantization and hardware integration remain deferred.