beffroi-tools/docs/midra-layout-plan.md

28 KiB
Raw Blame History

Midra layout helper: plan and visual proposals

Status: the Rust/egui positioning draft supports draw.io geometry paste with review, two-layer multi-selection and group movement, box selection, explicit-target alignment, numeric corner positioning/size, undo/redo and camera zoom/pan. Concept A is user-approved. Reference images, output presets/custom resolution, source/input formats and advanced crop-backed content zoom/offset are implemented offline. Device mapping verification, connectivity and REST calls remain deferred. Checklist gates that combine implemented and outstanding work remain open. Target: Analog Way Eikos 4K, running the latest firmware according to the user. Record the actual installed version during compatibility checks; do not infer it from a different Midra generation's documentation. Working name: Midra Layout.

Intended workflow

Set the output format and preset or custom screen resolution, optionally load a reference picture, then place one or two rectangular live layers visually or by entering exact offsets from a chosen screen corner. Read or copy their equivalent WebRCS position values. Select layers on the canvas or in the right-hand list. All work is local and offline.

For this plan, “window” means the application window and “layer” means a video rectangle within one screen viewport. There is no additional window/group object. The first release supports two layers; the underlying collection should support more later without promising that every device configuration supports them.

The reference image is an editing aid, not an additional hardware layer. Screen resolution is independent of the application window size, zoom and image size. Source routing, transitions, playback, masking, rotation, keying, live preview and device control remain outside the offline implementation scope. Device writes are a later stretch goal (step 10), not authorized work on the connected unit. Input cropping and virtual layer resolutions are now planned, with support deferred until after the first draft. Their coordinate spaces must be designed from the start. See the resolution and advanced-cropping specification.

Ordered task list

Complete each acceptance gate before broadening the scope. The first usable positioning draft comprises steps 1–6, including source/output format selection and custom viewport dimensions, and the applicable checks in step 8. Steps 7 and 9 follow the draft. Step 9 is committed scope; its device compatibility checks must pass before presenting advanced-crop values as usable in WebRCS.

1. Establish the Eikos 4K coordinate contract

  • Record device model, exact firmware and relevant WebRCS field labels.
  • Check horizontal and vertical origins, signs, pixel units, numeric step, accepted size/position ranges and off-screen placement in that WebRCS version.
  • Verify whether the editable bounds refer to the layer rectangle rather than cropped source content. Start with an unmasked, uncropped live layer.
  • Check odd widths/heights and half-pixel centers; do not round silently.
  • Establish input-crop and layer-crop coordinate spaces, units, processing order, limits and Centered aspect behavior for the later resolution milestone. Separate final visible bounds from any device layer container; do not infer container size or position from the uncropped image's projected dimensions.
  • Record examples at all four screen corners and at screen center using a simulator or manual entry in an appropriate test setup, without API calls.

Gate: a small verified input/output table establishes what the application may label as WebRCS-compatible. The example math below is provisional until then.

2. Review visual direction and prove the viewport

  • Choose a direction from the three proposals below. Approved: A, with B's optional rulers and C's contextual alignment panel later.
  • Build a disposable viewport interaction spike with two rectangles before committing to the full UI. Rust/egui is the preferred starting point because the repository already uses it; validate interaction quality before settling.
  • Prove pointer-centered zoom, pan, selection, hit testing, keyboard nudge, image rendering and display scaling on the target operator laptop.
  • Try a dense reference image at Fit, 100%, 400% and 1600% zoom and at several OS display scales. Zooming must not move layer coordinates.
  • Review at 1440 × 900 and 1024 × 768 logical window sizes, including enlarged UI text. Keep numerical controls usable with a scrollable right inspector.

Gate: the viewport feels predictable and shows exact edges clearly. If the toolkit cannot meet these needs, resolve that before building the rest.

3. Implement geometry independently of the UI

  • Add workspace member crates/midra-layout/ with shared geometry and a desktop binary.
  • Model screen dimensions and stable layer IDs with canonical top-left position and final visible size. Keep selection, camera and reference image separate.
  • Define source format/native dimensions, input crop, advanced crop and derived device geometry as distinct concepts from day one. Store crop edges and final size as authoritative geometry; content zoom/offset are derived controls. Do not implement crop interactions until after the first draft.
  • Implement four-corner offset conversion and inverse WebRCS conversion under the verified device contract.
  • Switching corner changes the displayed offsets without moving the layer. Resizing keeps the chosen corner and its offsets fixed; moving changes offsets.
  • Preserve precision internally. Validate finite numbers, positive sizes and device limits; report unsupported values without silently clamping them.
  • Test all corners, inverse round trips, odd dimensions, negative offsets, non-16:9 screens, exact screen edges and resizing from each anchor.

Gate: viewport scale and UI state cannot affect geometry or conversion results.

4. Build the two-layer positioning workspace

  • Add screen resolution controls, one/two-layer setup, persistent layer IDs and names, and a right-hand layer list with a selected-layer inspector.
  • Add source format choices (including 4:3, 16:9 and 16:10) with explicit native pixel dimensions; an aspect ratio alone is not a pixel resolution.
  • Add output format choices and clearly labeled viewport resolution presets (HD 1280 × 720, Full HD 1920 × 1080, UHD 4K 3840 × 2160), plus manual width and height. Derive the output ratio from custom dimensions, never silently stretch them to match a preset. Other formats can be added after validation.
  • Changing output dimensions preserves existing layer pixel bounds, flags out-of-screen layers and rechecks reference-image mapping. Validate positive integer dimensions independently of verified device output support; custom planning geometry must not imply that Eikos accepts every output mode.
  • Support click selection, Shift-click multi-selection, clear selection and list selection for overlapped or off-screen rectangles. Define deterministic overlap hit testing; list order represents preview stacking, not device routing.
  • Add dragging, exact width/height fields, four-corner selection and inward offsets. Prefer numeric sizing first; add resize handles only with anchor behavior consistent with the geometry contract.
  • Show WebRCS horizontal/vertical center values beside corner inputs; make output values read-only initially. Offer per-value copy and labeled plain text for a whole layer. Do not imply plain text is a WebRCS-native clipboard format.
  • Add arrow nudges of 1 screen pixel and Shift-arrow nudges of 10; focused text fields must receive these keys instead of moving the selection.
  • Implement Space-drag/middle-button pan, pointer-centered zoom, Fit screen, Fit selection, explicit zoom percentage and a clearly defined 1:1 pixel view.
  • Keep handles/hit areas legible at every zoom; distinguish UI logical points, screen pixels and physical display pixels on HiDPI screens.
  • Add visible optional snapping to pixel edges, screen edges and other layer edges. Exact numeric entry bypasses proximity snapping. Show the active guide and allow a temporary snap override; toggling snapping must not move anything.
  • Add undo/redo for geometry changes, one undo step per drag or committed field edit, and Escape to cancel an in-progress drag or uncommitted edit.

Gate: place a layer 120 px from the right and 180 px from the top, change zoom, switch anchors and move it by one pixel without losing track of the real values. Selection and dragging never alter unrelated layers. No selection shows screen and reference settings; multi-selection shows mixed values explicitly.

5. Add accurate reference-picture placement

  • Load PNG/JPEG through a file picker or drag/drop; report decoding failures while preserving the current image. Keep source files unchanged.
  • Use a matching-resolution image at screen origin with 1 image pixel per screen pixel. For mismatched dimensions, display the mismatch and require an explicit mapping choice: native pixels with offset, fit preserving aspect, or stretch to screen. Show scale, offsets and any letterboxing.
  • Lock the reference by default, add hide/show and opacity controls, and offer outline-only layer display so image edges remain visible.
  • Use sharp nearest-neighbor inspection at high zoom and an appropriate downsampled view when zoomed out. Do not confuse filtering with coordinate precision. Reveal the screen-pixel grid only when it is legible.
  • Decode large images away from the input/render loop, cap supported image dimensions/memory with useful errors, and preserve viewport responsiveness.

Gate: a known rectangle in a 1920 × 1080 test image can be matched exactly by a layer; changing image opacity, visibility, zoom or filtering changes no geometry.

6. Add extensible clipboard position import

  • Start with an explicit, documented plain-text geometry format and a paste-preview panel. For example: x=1160; y=180; width=640; height=360; origin=top-left; unit=px.
  • Preview parsed values, origin, units, target layer and before/after bounds. Allow position-only import; importing size must be an explicit choice.
  • Treat Ctrl/Cmd-V in a focused text input as normal text paste. Elsewhere, open geometry preview; do not mutate the document merely by reading clipboard.
  • Define adapters by source format/version. Collect actual clipboard samples from each requested software before claiming support. The first adapter now supports the supplied draw.io mxGraphModel clipboard example. Do not guess an application from an unlabeled number tuple.
  • Make points, inches, percentages, source canvas origins, decimal conventions and scale explicit. Ambiguity produces an explanation, not a guessed placement.
  • Validate the complete proposed edit before applying it atomically, with one undo step. Unknown formats, missing fields and invalid values preserve state.
  • Test real fixtures for every added adapter and clipboard failure behavior on supported desktop platforms, including Wayland where applicable.

Gate: supported data places the selected rectangle correctly; malformed or ambiguous data makes no changes. Ship the basic format first if no external software format has yet been selected; do not claim that completes an adapter.

7. Explicit multi-layer alignment

  • Add Align top and Align left; show the alignment target explicitly: screen, selection bounds or a named reference layer.
  • For a reference layer, keep that layer fixed and move only the others. For selection bounds, align to the minimum top/left edge. For the screen, align to coordinate zero on the selected axis.
  • Preserve sizes and the other axis, preview the affected selection, and make each alignment a single undoable operation.
  • Add right/bottom/center alignment using the same explicit targets.
  • Add Shift-click toggling in the viewport/list, left-drag box selection, Shift-drag additive selection and Escape cancellation. Group movement/nudging preserves spacing and uses one undo step per action.
  • Extend beyond two layers under step 11. No distribution or constraint system yet.

Gate: two differently sized layers align exactly against the chosen target, regardless of selection order except when explicitly choosing a reference layer.

8. Harden and package the first release

  • Include empty, invalid-input, reference-load-error, off-screen and clipboard error states in UI review, with an obvious way to recover or frame the selection.
  • Verify tab order, keyboard focus, numeric input commit/cancel, contrast and selection indicators that do not rely on color alone.
  • Manually exercise screen/reference setup → corner positioning → nudge → paste preview → copy values → undo. Verify results in the target WebRCS setup.
  • Add geometry/import regression tests and interaction tests for camera transforms and undo. Establish a smooth-drag baseline on the operator laptop; target a 60 Hz interaction budget, measuring rather than assuming performance.
  • Document supported firmware, coordinate conventions, clipboard formats, limits and shortcuts. Keep the root README to a short link/quick start.
  • Integrate the actual crate into workspace CI, coverage and manual packaging; run the repository Rust gates, whole-application coverage and relevant packaging tests. Document unavailable device or desktop checks.
  • Commit each completed task; no automatic release publishing.

Session-file save/load can follow if needed; it is not required to demonstrate precise positioning. Networking is deferred to the separate stretch goal below.

9. After the first draft: input and layer resolutions

  • Implement the separate input-crop and advanced-crop stages defined in the resolution specification, with no device calls.
  • Keep Source format selected when Input resolution is Custom. Support exact input dimensions and crop placement, defaulting to equal opposite-side crops.
  • Expose Layer resolution as the final visible width/height, with optional aspect lock. Add Content zoom and Offset X/Y controls and an explicit content framing mode; moving the layer and moving its content must be distinct actions.
  • Keep the final box fixed during content framing; store four crop edges and final size, derive zoom/offset, and compose input/layer crops without applying either stage twice. Preserve source-level sharing when two layers use one input.
  • Test the documented numerical examples, inverse transforms, offset limits, extreme ratios, fractional crops, resizing, shared inputs and undo behavior.
  • Verify the device adapter's crop units, Centered mode, layer container size and center position with the actual firmware. Show rounding error; block values that cannot preserve the requested box within the verified device precision.
  • Add UI review, documentation and release checks for these features before claiming them as supported. Existing A/B/C mockups depict the positioning draft.

Gate: the user's 1920 × 1080 source → 1920 × 1000 input → 300 × 250 layer workflow retains a 300 × 250 selectable box through zoom, content offset and placement. The WebRCS settings reproduce the same visible result without masking.

10. Later stretch goal: apply layer format through the WebRCS REST API

Status: requested for later; do not implement or exercise device connectivity during the current offline work. The user's currently connected WebRCS controls a live Eikos 4K. Do not probe, connect to or send commands to that unit. This backlog entry does not authorize future live-device testing or writes.

  • After offline positioning and advanced cropping are validated, research the supported API for the exact model/firmware using documentation and an isolated simulator or explicitly designated non-live test device. Verify endpoint availability, authentication, units and field semantics rather than assuming an endpoint from the WebRCS UI.
  • Define the smallest supported operation for applying a layer's format: dimensions and, where needed for the virtual format, crop/aspect settings from the verified device adapter. Identify any required position correction and include it explicitly in the proposed change. Do not alter source routing, transitions or trigger a Take as an implicit side effect.
  • Keep offline mode as the default. Selecting a device or editing, dragging, pasting, undoing or redoing locally must never automatically send commands.
  • Present a concrete before/after preview with the device identity, firmware, screen, layer, Program/Preview target and every setting to be changed. Require an explicit user action to send that reviewed change; do not infer that a Preview target is harmless or available without verifying device behavior.
  • Detect stale device state before applying; require a refreshed review if relevant settings changed. Document whether the API can apply the change atomically. Report partial failures and uncertain outcomes without blind retries or automatic rollback commands against a live system.
  • Test against mocked transport first, then an explicitly authorized isolated target. Verify that no writes occur on startup, connection, local edits, cancellation or errors, and that only reviewed fields are sent.

Gate: a reviewed layer-format change can be applied and checked on a designated non-live target. Any use on the current live installation requires separate, explicit authorization at that time. No API calls are part of this planning task.

11. Later stretch goal: layer management and export scope

Status: per-layer visibility and locking are implemented for the two current layers; the remaining management features are requested for later. This is local document organization; it can be implemented independently of step 10 and does not authorize device connectivity. Folders are an optional follow-up, not a prerequisite for adding/hiding layers.

  • Replace the fixed two-layer document with a collection using stable local IDs. Add an Add layer button and editable names. New local layers start visible and unmapped; adding a planning layer does not create a hardware layer.
  • Add per-layer Show / Hide controls in the layer list. Hidden layers keep their geometry and mapping, remain accessible in the list, and are excluded from canvas drawing and hit testing. Hiding is local only: never send a device hide, clear, opacity or routing command as a side effect.
  • Add an Options → Layer mapping section that maps each local layer to an explicit device/profile, screen and Midra layer ID. Keep local names and list order independent of hardware identity; reordering or hiding must not remap.
  • Add an export-scope choice: Specific layer… or Visible layers only. The specific-layer picker uses local names and shows the mapped destination; allow intentionally exporting a hidden layer through this explicit selection. Visible means enabled in the layer list, not currently inside the viewport, uncovered by another layer or selected on the canvas.
  • Preview the exact included layers, geometry and destination mappings before export. An empty visible set gives an explanation and produces no output. Preserve deterministic list order and never include excluded layers implicitly.
  • For device-targeted exports or a future API send, require mappings for every included layer and reject unsupported destinations or duplicate destination mappings within that export. Alternative local layouts may share a destination when only one is included. Generic coordinate copies may remain unmapped.
  • Keep export scope distinct from transport: exporting/copying a configuration does not send it. Future device writes still use step 10's separate reviewed, explicit send action, including an explicit Program/Preview target.
  • Make layer creation and visibility changes undoable. When session save/load is added, preserve stable IDs, names, geometry, visibility and mappings. Extend selection, paste targeting and undo so they no longer depend on array indices.
  • Optional: add named folders with expand/collapse and drag-to-organize. Folders organize local layers; they do not introduce geometry transforms or hardware groups. Collapsing a folder changes only the list display. If folder visibility is added, effective visibility is the layer's own setting AND all ancestor-folder settings; hiding a folder must preserve child visibility choices.
  • Test adding layers, hidden-layer selection through the list, stable mappings after reorder, paste/undo targeting, both export scopes, empty scopes, unmapped or conflicting targets, and optional nested-folder visibility. Confirm that organization and export preparation produce no device calls.

Gate: maintain several local layout alternatives, map them explicitly in Options, then export one named layer or exactly the visible set without changing local geometry, overwriting a conflicting destination silently, or contacting a device.

Coordinate contract to verify

Proposed internal model: screen origin at its top-left, X increases rightward, Y increases downward, and a layer's final visible box is (x, y, width, height). Positive corner offsets point inward. A corner means the matching corner of the layer measured from that screen corner; there are not two independent anchor selectors in v1.

For screen size W × H, layer size w × h, and offsets dx, dy:

Selected corner Layer left x Layer top y
Top left dx dy
Top right W - dx - w dy
Bottom left dx H - dy - h
Bottom right W - dx - w H - dy - h

For the uncropped positioning draft, under the user's described center-position convention, WebRCS values are horizontal = x + w/2, vertical = y + h/2. The inverse subtracts the half-size. The corresponding inward margins are left=x, top=y, right=W-x-w and bottom=H-y-h. Switching corner just chooses which margins to display.

Example used in every mockup: on a 1920 × 1080 screen, a 640 × 360 layer 120 px from the right and 180 px from the top has top-left 1160, 180 and center 1480, 360. The other layer is at 120, 180, center 440, 360. An odd width such as 641 produces a half-pixel center when the left edge is an integer; hardware acceptance/rounding is a compatibility question, not something to hide with display formatting. Advanced cropping must retain these visible-box coordinates while deriving any distinct device container through its adapter.

Visual proposals

These are original, editable SVG interface mockups, not application screenshots. Open them in a browser or image viewer to inspect at full size. The example background is illustrative artwork standing in for an imported picture. Labels such as “Copy” and “Paste” illustrate planned controls; nothing is interactive yet.

Dark slate surfaces, cyan selection, amber second-layer identity and a restrained purple primary action consistent with existing repository interfaces. A large viewport occupies the left; the fixed right inspector groups the layer list, corner selector, offsets, size and WebRCS outputs in reading order. Thin outlines, corner markers, center crosshairs and dimension guides make selection unambiguous.

Concept A: dark operator workspace

Best for working beside show-control tools. The tradeoff is less contrast against dark reference images; add a contrasting outline halo and an outline-only mode. Keep the output card visible while upper inspector sections scroll. A compact bottom status line carries camera/pixel information and shortcut hints.

B. Precision desk

A light inspector and drafting surface with screen-coordinate rulers. The same placement workflow emphasizes measurements and field readability. The artwork retains its own colors independently of the UI theme.

Concept B: light workspace with coordinate rulers

Best for daytime preparation and detailed artwork matching. It uses more visible measurement marks and can feel busier; rulers and guides should be optional. Its useful features can be incorporated into A without maintaining two different apps.

C. Focus desk

A wider viewport and narrower contextual inspector. This proposal illustrates the multi-selection state: a named reference layer, explicit alignment actions and a compact per-layer coordinate table. The layer list remains on the right.

Concept C: compact workspace with two selected layers

Best for repeated placement and comparison once the basic workflow is familiar. Less space for numeric explanations makes it weaker as the initial default. The alignment state is now available under step 7; the SVG remains a design concept.

Shared interaction and responsive behavior

Keep the screen aspect ratio exact, surround it with a neutral pan area, and use consistent screen-space selection markers. Display only relevant guides, with labels in screen pixels. Layer identity and selected state must be distinct. Always allow selecting a layer from the list when its outline is hard to hit.

The desktop window should retain a right inspector at normal working sizes. At narrow widths, reduce decorative padding and collapse secondary sections; below the proven minimum width, use a right drawer rather than crushing numeric fields. At short heights, scroll sections while retaining access to conversion outputs. Mouse and keyboard interaction are the first priority; a mobile UI is not part of this desktop tool.

Research and open decisions

The Analog Way Midra 4K V3.2 manual, July 2024, pp. 83–85 documents layer alignment, snapping, position and size controls. This supports the workflow research; it does not establish the precise coordinate convention or current firmware behavior for the user's installation. Avoid portraying all WebRCS versions as lacking alignment. The proposed improvement is more explicit corner-based positioning and reference-image inspection.

Figma's zoom/view documentation provides useful precedents for fit-to-view, selection zoom and pixel inspection. Borrow those interaction ideas, without introducing vector-editor features.

Analog Way's Midra 4K FAQ identifies an offline simulator that may help verify behavior without connecting this helper to hardware. Simulator agreement should still be checked against the installed WebRCS version.

Open compatibility decisions: exact installed firmware and primary operator OS/display scale. Concept A is approved and draw.io is the first clipboard source, with the user-supplied percent-encoded rectangle retained as a regression fixture.

Reference-picture implementation update: PNG/JPEG/BMP import and file drop are available, with fill-screen or proportion-preserving fit, opacity and visibility. Resolution scaling produces a warning; aspect distortion a severe warning. Both remain non-blocking. Image offsets and persistent sessions remain future work.

Corner-resizing implementation update: selected layers have interactive corner handles with a fixed opposite anchor, ratio locking enabled by default, an explicit free-resize toggle, Escape cancellation and one undo record per gesture. Numeric width/height edits follow the same ratio toggle; pasted source sizes remain exact.

Formats/framing implementation update: dynamic output presets/custom resolution, source aspect/native dimensions, shared input crops, virtual visible layer sizes, advanced crop-backed zoom/offset, content drag/nudge, test-pattern preview and provisional calculation reports are implemented. Resize/source edits preserve normalized content centers, with explicit review of any required adjustment. Document history includes source, output and layer crops atomically. Firmware container verification, crop quantization and hardware integration remain deferred.