beffroi-tools/docs/rundown-converter.md

21 KiB
Raw Blame History

Rundown converter

Status: first implementation, available from source. A Rust core powers both the CLI and egui desktop app, styled like the PPTX reporter. It reads XLSX and UTF-8 CSV/TSV, infers columns, preserves untimed gaps and exports Ontime JSON. Ambiguous documents stop for clarification. See the development plan and sample findings.

Quick start

Run from the repository root:

cargo run --locked -p rundown-converter --bin rundown-converter-gui
cargo run --locked -p rundown-converter --bin rundown-converter -- inspect show.xlsx
cargo run --locked -p rundown-converter --bin rundown-converter -- validate show.xlsx --sheet "Main"
cargo run --locked -p rundown-converter --bin rundown-converter -- convert show.xlsx --sheet "Main" --output show.ontime.json

Try the included synthetic lunch example:

cargo run --locked -p rundown-converter --bin rundown-converter -- convert crates/rundown-converter/tests/fixtures/lunch.csv --output /tmp/lunch.ontime.json

Select a sheet explicitly when a workbook has several. inspect --json exposes original cells, formulas, formats relevant to timing, and merged ranges. validate --json gives structured decisions, diagnostics and the IR when available. convert --neutral exports that neutral IR instead of Ontime JSON. Existing output requires --overwrite; the source itself cannot be the output. Parent output directories must already exist. Exit codes: 0 success, 1 read/profile/write failure, 2 unresolved rundown diagnostics (also used by clap for invalid command syntax). CLI logs are plain text on stderr; JSON results go to stdout.

In the desktop app, choose or drop one file, read its sheets, select a sheet and click Analyze rundown. Setup exposes two General options, also saved in resolution profiles and usable from the CLI:

  • Ignore all warnings (ignore_warnings): try excluding rows with missing start/duration/title, untimed content, orphan notes or unresolved breaks, then rebuild and validate. Column ambiguity, malformed timings, overlaps and other inconsistencies still block export. Manual row and duration decisions are kept. Automatically excluded rows are counted in the summary and remain visible in All rows. Disable the option and recheck to restore them.
  • Treat untimed rows as notes (untimed_as_notes): attach untimed rows to the preceding event. When both options are enabled, this runs before warning exclusions so notes are kept wherever possible.

Changes take effect on analysis / recheck. A checklist shows Column detection, Gap detection, Schedule construction, and Consistency checks. Checked green boxes mean Passed; other steps explicitly say Needs attention, Recheck required, or Not checked. Blocking column errors open Columns automatically. Row corrections become available after the timing mapping is unambiguous and rechecked. Spreadsheet cells marked as times display as HH:MM:SS in samples and the inspector; the original stored value remains available in the inspector. When several columns could have the same timing role, the error gives a short instruction and highlights all candidate columns in amber immediately. The Choose … column… button scrolls to that candidate group and gives it one slow brightness pulse with a fixed border, which distinguishes the group when several mapping issues are present. The short message and button share a line when space permits. The original inference explanation remains under the subdued Technical details. Review has two views:

The main navigation and Review view tabs use a subtle purple tint so they remain visually distinct from labels and ordinary actions; the active tab uses the stronger accent.

  • Schedule & corrections combines the schedule preview and row editing in one table. Start/end/duration, titles, colored issues and row types are visible together. Click anywhere on a source row or inferred gap to select it; a subtle tint marks the selection while retaining its status color. Edit row… opens the selected source row editor, including start, end and duration corrections. Edit gap… opens an inferred gap's timing, diagnostics and allocation choice even though it has no source cells. All rows includes ignored rows; Included rows hides ignored rows and rows containing only ignored columns, while keeping notes, markers and breaks; Needs attention shows rows with diagnostics from the last analysis and gaps for context. Ignored rows can be restored using All rows. Notes attached to another event keep their own source row and identify their parent timer instead of duplicating it. Amber means unresolved, red means an error, dim gray means ignored, teal marks gaps, and medium-gray timing values were inferred rather than read directly from the rundown. Start, end and duration are colored independently, so a row can show exactly which values were calculated. An overlap marks both the later cue and its predecessor in red because either timing may need correction; each row's hover leads with the warning or error severity, code, source location, full explanation and a concrete way to resolve it. Timing inference is conveyed by the table colors instead of repeated in the hover. Row status and editing hints follow as secondary information; raw source cells remain in the double-click inspector. For overlaps, both affected rows explain their side of the conflict. Gaps appear in schedule order with their start, end and elapsed duration (no timer). Named breaks use their source row; inferred gaps update after recheck. Each gap offers Keep gap (default, no timer), Extend previous timing (extend the preceding event's end), or Add to next timing (start the following event earlier, keeping its end). Apply & recheck updates durations and Ontime output. Allocated intervals remain in the preview for review and undo; choose Keep gap to restore original timings. The selected event must touch the gap, and unresolved/overlapping intervals cannot be absorbed. Choices are saved in resolution profiles as gap_resolutions, keyed by exact gap boundaries. If other corrections change those boundaries, remove the stale choice in Setup or reset decisions and recheck. Unknown values show ?. Unresolved source rows offer Ignore to set the ignore resolution and Details to open the row editor. Ignoring requires recheck and can be undone under All rows. Ignore all unresolved (N) applies this to unresolved source rows and timed rows missing a title across the sheet, regardless of the current filter. Other timed rows, timing errors and existing manual corrections are excluded. Hover over a title or details button for row status and full diagnostic explanations. Double-click anywhere on a source row, or click its Details / Error button, to open the same editor (inline dropdowns keep their normal interaction). It shows timing, editable row type and start/end/duration corrections, full diagnostics, timing decisions and a formatted table of original cells (including empty values and formulas). Pending edits are distinguished from the last analyzed timing. Issues opens the issue navigator, including sheet-level and gap errors. The table fills the remaining space; its title column expands with the window. Preview cells align at the top; blank title lines are omitted without changing original values in the inspector or exports. On narrow windows, timing values stack so corrections remain accessible. Changes require recheck; the timing preview and diagnostics continue to show the previous analysis.
  • Columns shows a concise verified mapping summary, green detected roles, per-column role dropdowns, and sample values. Expand Detection evidence for full inference and validation details; hover the Columns tab for a compact explanation of the role choices. Selecting one duration column explicitly resolves duplicate TIMER headers. Metadata keeps a column as Ontime notes without treating its values as timing or titles; Ignore excludes it from inference and Ontime output. Several columns can be metadata or ignored. Auto restores inference; other columns remain included as metadata by default. Manual header mapping remains available in Setup for the header row and column choices. The separate Advanced settings section contains duration units and profile JSON.

The row type selector shows the detected type in parentheses, such as Auto (Event). Auto (?) means unresolved or awaiting recheck.

Click Apply & recheck after corrections. Pending edits retain the issue list and previous preview so you can resolve several rows without losing your place; export and remembering a resolution remain blocked until validation succeeds. Blank timing corrections restore source or calculated values. Start and end accept clock times such as 09:30 or 09:30:00; duration accepts explicit values such as 2 min or 00:02:00. Missing timings, overlaps and untimed row decisions can be resolved here. Unsupported interpretations and unresolved conflicts retain explicit errors; correct the source when available controls cannot express the intended fix. CLI untimed-row errors also include a labeled source preview.

Save resolutions… opens the save/load/remember controls without taking space away from the correction table. Results offers Open, Open output folder and Copy path. Details contains status, completed analysis stage count and the full IR, including original rows. Cancel is observed between analysis stages and before publishing an export; reading a workbook or an active stage finishes first.

Purpose

Convert differently formatted conduites (rundowns) into an Ontime project without manually rebuilding each document. Detect tables, headers, event titles and timing columns using explainable heuristics. Preserve source order and scheduled start times, especially when a session resumes after a break.

When the source is ambiguous, stop export with a precise explanation and ask for clearer headers or an explicit column mapping. Never silently swap start times and durations, discard questionable rows, or fill unknown durations with zero.

Workflow

Open or drop a document and select its sheet/table, then follow five stages:

  1. Infer column meanings from data across rows. An ambiguous cell can be clarified by other rows, but one meaning must fit the whole column; otherwise return an error asking for clearer headers, corrected values or an explicit mapping.
  2. Identify untimed gaps and incorporate them into the draft intermediate representation (IR). These breaks occupy schedule time without a shown timer.
  3. Construct the complete IR from times and gap constraints, preserving original and inferred values. Ask for clarification when required facts remain unknown.
  4. Check the complete schedule for inconsistencies, including backward times, negative durations, overlaps and contradictory start/end/duration values.
  5. Generate native Ontime JSON from the validated IR, with optional neutral JSON for inspection or reuse. Open the result or its output folder.

Review and correction controls expose each stage's decisions and errors. CLI and GUI follow this same sequence; export cannot bypass unresolved errors.

The CLI and GUI use the same detection, validation and export code. CLI failures identify the source location and return a nonzero exit status. Inputs stay unchanged; existing output requires explicit overwrite permission. Processing is local. XLSX preserves merged ranges and cached formula values; formulas and macros are never executed. Recalculate and save formula-based workbooks before importing. The converter cannot determine whether cached values are stale. CSV/TSV uses UTF-8, quoted fields and comma/semicolon/tab detection. Legacy XLS, ODS, Word, PDF and scans are not yet supported. A PDF feasibility trial recovered table text from the supplied PDFs but could not establish a complete timer schedule; PDF importer work is paused for now. Input files are limited to 32 MiB; XLSX expanded content is limited to 128 MiB.

Timing and breaks

Explicit start times are schedule anchors. Durations can come from a duration column or an unambiguous start/end pair. The preview distinguishes original values from calculated values, unresolved values and gaps.

Source evidence Interpretation
Cue at 11:45 lasting 15 minutes; next cue at 13:30 Preserve a 12:00–13:30 gap and the 13:30 start
Named lunch row from 12:00 to 13:30 Preserve an explicit break with its original label
Cue at 11:45; next cue at 13:30; no end or duration Event duration versus break is unknown; do not invent a lunch break
Start 11:45, end 12:00, duration 20 minutes Report conflicting timing and request clarification
Final cue has only a start Request its end/duration or an explicit non-timed classification

Inferring a duration from the next start is allowed only when the source or a user-selected policy establishes consecutive events. That policy and each inference must remain visible. A long interval alone does not prove a break. Do not extend the preceding timer through an established gap or shift an explicit start to make durations fit. Overlaps, day changes and parallel stages need explicit interpretation rather than automatic sorting or midnight wrapping.

Ontime documents independent starts, linked starts and separate duration/end strategies in Smart time entry. Its delay documentation also describes schedule gaps absorbing delays. The export retains a known unnamed break as a gap, with the following start unlinked. Untimed gaps remain in the IR without generating a synthetic timer or displaying a timer duration. An explicitly timed break cue remains distinct from such gaps. Schedule preservation does not imply automatic playback or guarantee that an operator will start an event on time.

Output choice

Native Ontime JSON is the primary target: it lets the exporter control time relationships and project structure without a second column-mapping step. Ontime supports project files and spreadsheet interchange; see its import/export documentation. Stretch goals are a Typst report (editable source and PDF) and XLS/ODS re-export with normalized columns directly suitable for Ontime import. Actual file-format and column compatibility must be verified against the target release; any required format conversion will be documented. These outputs share the validated IR and are not intermediate steps for native JSON export.

The adapter targets Ontime 4 native project JSON. Its events and schedule metadata were checked with the Ontime 4.15.1 native parser; see the verification record. Import the generated project through Ontime's project manager. All starts are currently exported unlinked, including derived starts, so later anchors survive edits to earlier cues. Timers use duration lock and manual end actions.

Clarification profiles

Pass --profile mapping.json, or load/edit the same JSON in the desktop app:

{
  "header_row": 1,
  "columns": {"start": "A", "duration": "B", "title": "G"},
  "column_usage": {"C": "metadata", "F": "ignore"},
  "expected_headers": {"A": "Heure", "B": "Durée"},
  "untimed_as_notes": true,
  "row_kinds": {"20": "marker"},
  "starts": {"8": "09:30"},
  "ends": {"8": "09:32"},
  "durations": {"8": "2 min"}
}

This illustrates options, not a profile to apply blindly to the supplied files. Rows are one-based original worksheet rows; columns use Excel letters. Supported roles are start, end, duration and title. duration_unit accepts minutes, seconds, hh:mm, mm:ss or excel. Cell number formats establish Excel time units where available; otherwise units must be explicit or consistently supported by independent start/end pairs. Unknown profile keys and changed expected headers are errors. starts, ends and durations are keyed by one-based source row; corrected cells are excluded from whole-column validation while all remaining explicit mappings still undergo it.

column_usage maps column letters to metadata or ignore. Both choices prevent that column from being inferred as a start, end, duration or title, even when its header suggests one of those roles. A column cannot also have an explicit timing or title mapping. Metadata appears in Ontime event notes, including metadata from rows attached as notes or merged continuations. Ignore removes those cell values from Ontime notes; rows containing only ignored values need no clarification. Original cells remain available in the review and neutral IR for retained rows. These choices are saved in profiles and remembered resolutions; use Apply & recheck before export. The default inclusion of other source columns is unchanged.

row_kinds accepts event, gap, marker, note and ignore. Notes attach to the preceding event. Untimed markers remain in the IR but are not exported as timers. An ignored row is recorded as an explicit exclusion. Duration overrides retain original cell values and the fact that a correction was requested. Use expected_headers to guard reusable profiles against changed layouts.

Under Save resolutions…, the two main actions are Save resolution and Reuse remembered resolution. Advanced contains file management, reset, and forget controls.

Save resolution as… (under Advanced) writes a portable profile JSON, usable with CLI --profile. Save (overwrite resolution) replaces the loaded/saved profile; Save as refuses existing files unless its replacement checkbox is enabled. These actions save decisions, not a rewritten spreadsheet or an Ontime export. Profiles may contain unfinished decisions; loading one always requires a fresh analysis. Explicitly loaded profiles use worksheet row numbers, so review their applicability when the document has changed.

Save resolution stores a validated profile in the platform's user configuration directory, under rundown-converter/resolutions (on Linux, $XDG_CONFIG_HOME/rundown-converter/resolutions, normally ~/.config/rundown-converter/resolutions). The status displays the exact path. Reuse remembered resolution becomes available for an identical sheet and loads its decisions for revalidation; Forget removes that remembered file. Matching includes every cell, timing format, formula, merge, sheet name and row order, so changed documents cannot silently inherit row corrections. This is explicit reuse rather than automatic application. Changing file or sheet clears current decisions. Remembered files contain profile decisions and a hash in their filename, not a copy of the workbook; they remain local to this machine.

Current limits

One selected sheet represents one sequential, single-day schedule. Parallel tracks, midnight rollover and hierarchical parent/child timers need separate interpretations; the converter does not sort or flatten them automatically. A merged clock window plus individual row durations creates bounded sequential cues. A shared merged duration creates one timer with the other rows retained as notes. Unmerged untimed directions need explicit classification.

A full-width merged text band is treated as a section/footnote and recorded in the IR. Nonempty unmerged text in a time column is a contradiction, not silently ignored. Missing end/duration in a start-only schedule remains unresolved. Recent-file persistence, direct start/end/title editing and portable release bundles remain future work. JSON profiles provide the same supported resolution decisions in both interfaces.

Stretch goal: reserve buffer time

An optional correction mode will shorten timed rows to create a requested buffer in an overcrowded schedule. It will offer equal or proportional cuts, minimum durations and protected rows, with a before/after preview. Original values remain available. Untimed breaks do not contribute time cuts.

Saved time can stay as slack after each event, preserving starts, or be collected before a chosen anchor if the user explicitly allows intermediate starts to move. Fixed restart times stay protected. Impossible requests must report the available capacity; existing overlaps must be accounted for before claiming new buffer. Every adjusted schedule must pass validation again before export. See the buffer design.

Expanding coverage

The sample findings list the supplied documents, public workbooks and unresolved interpretation questions. Further examples should keep table structure, cell formats and representative time values when redacted. Private originals stay out of Git; regression fixtures are synthetic.

See all tools and repository development.