362 lines
22 KiB
Markdown
362 lines
22 KiB
Markdown
# Rundown converter
|
||
|
||
Status: first implementation, available from source. A Rust core powers both the
|
||
CLI and egui desktop app, styled like the [PPTX reporter](rust-reporter.md).
|
||
It reads XLSX and UTF-8 CSV/TSV, infers columns, preserves untimed gaps and exports
|
||
Ontime JSON or normalized XLSX. Ambiguous documents stop for clarification. See the
|
||
[development plan](rundown-converter-plan.md) and [sample findings](rundown-corpus.md).
|
||
|
||
## Quick start
|
||
|
||
Run from the repository root:
|
||
|
||
```sh
|
||
cargo run --locked -p rundown-converter --bin rundown-converter-gui
|
||
cargo run --locked -p rundown-converter --bin rundown-converter -- inspect show.xlsx
|
||
cargo run --locked -p rundown-converter --bin rundown-converter -- validate show.xlsx --sheet "Main"
|
||
cargo run --locked -p rundown-converter --bin rundown-converter -- convert show.xlsx --sheet "Main" --output show.ontime.json
|
||
cargo run --locked -p rundown-converter --bin rundown-converter -- convert show.xlsx --sheet "Main" --output show.ontime.xlsx
|
||
```
|
||
|
||
Try the included synthetic lunch example:
|
||
|
||
```sh
|
||
cargo run --locked -p rundown-converter --bin rundown-converter -- convert crates/rundown-converter/tests/fixtures/lunch.csv --output /tmp/lunch.ontime.json
|
||
```
|
||
|
||
Select a sheet explicitly when a workbook has several. `inspect --json` exposes
|
||
original cells, formulas, formats relevant to timing, and merged ranges.
|
||
`validate --json` gives structured decisions, diagnostics and the IR when available.
|
||
The output extension selects native `.json` or spreadsheet `.xlsx` export.
|
||
`convert --neutral` exports the neutral IR to JSON instead of an Ontime project.
|
||
Existing output requires `--overwrite`; the source itself cannot be the output. Parent output
|
||
directories must already exist. Exit codes: 0 success, 1 read/profile/write failure,
|
||
2 unresolved rundown diagnostics (also used by clap for invalid command syntax).
|
||
CLI logs are plain text on stderr; JSON results go to stdout.
|
||
|
||
In the desktop app, choose or drop one file, read its sheets, select a sheet and
|
||
click **Analyze rundown**. Setup exposes two **General options**, also saved in
|
||
resolution profiles and usable from the CLI:
|
||
|
||
- **Ignore all warnings** (`ignore_warnings`): try excluding rows with missing
|
||
start/duration/title, untimed content, orphan notes or unresolved breaks, then
|
||
rebuild and validate. Column ambiguity, malformed timings, overlaps and other
|
||
inconsistencies still block export. Manual row and duration decisions are kept.
|
||
Automatically excluded rows are counted in the summary and remain visible in
|
||
**All rows**. Disable the option and recheck to restore them.
|
||
- **Treat rows without a timing as notes** (`untimed_as_notes`): attach rows without a timing to the
|
||
preceding event. When both options are enabled, this runs before warning
|
||
exclusions so notes are kept wherever possible.
|
||
|
||
Changes take effect on analysis / recheck. A checklist shows **Column detection**, **Gap detection**, **Schedule construction**,
|
||
and **Consistency checks**. Checked green boxes mean Passed; other steps explicitly
|
||
say Needs attention, Recheck required, or Not checked.
|
||
Blocking column errors open **Columns** automatically. Row corrections become
|
||
available after the timing mapping is unambiguous and rechecked. Spreadsheet
|
||
cells marked as times display as `HH:MM:SS` in samples and the inspector; the
|
||
original stored value remains available in the inspector.
|
||
When several columns could have the same timing role, the error gives a short
|
||
instruction and highlights all candidate columns in amber immediately. The
|
||
**Choose … column…** button scrolls to that candidate group and gives it one slow
|
||
brightness pulse with a fixed border, which distinguishes the group when several
|
||
mapping issues are present. The short message and button share a line when space
|
||
permits. The original inference explanation remains under the subdued **Technical details**.
|
||
Review has two views:
|
||
|
||
The main navigation and Review view tabs use a subtle purple tint so they remain
|
||
visually distinct from labels and ordinary actions; the active tab uses the stronger accent.
|
||
|
||
- **Schedule & corrections** combines the schedule preview and row editing in one
|
||
table. Start/end/duration, titles, colored issues and row types are visible
|
||
together. Click anywhere on a source row or inferred gap to select it; a subtle tint marks
|
||
the selection while retaining its status color. **Edit row…** opens the selected source
|
||
row editor, including title, start, end and duration corrections. **Edit gap…** opens an inferred gap's timing,
|
||
diagnostics and allocation choice even though it has no source cells. **All rows** includes ignored rows;
|
||
**Included rows** hides ignored rows and rows containing only ignored columns,
|
||
while keeping notes, markers and breaks; **Needs attention** shows rows with
|
||
diagnostics from the last analysis and gaps for context. Ignored rows can be restored using All rows.
|
||
Notes attached to another event keep their own source row and identify their
|
||
parent timer instead of duplicating it. Amber means unresolved, red means an
|
||
error, dim gray means ignored, teal marks gaps, and medium-gray timing values were
|
||
inferred rather than read directly from the rundown. Start, end and duration are
|
||
colored independently, so a row can show exactly which values were calculated. An overlap marks both the later
|
||
cue and its predecessor in red because either timing may need correction; each
|
||
row's hover leads with the warning or error severity, code, source location,
|
||
full explanation and a concrete way to resolve it. Timing inference is conveyed
|
||
by the table colors instead of repeated in the hover. Row status and editing hints
|
||
follow as secondary information; raw source cells remain in the double-click
|
||
inspector. For overlaps, both affected rows explain their side of the conflict.
|
||
Gaps appear in schedule order with their start, end and elapsed duration (no timer).
|
||
Named breaks use their source row; inferred gaps update after recheck. Each gap offers **Keep gap**
|
||
(default, no timer), **Extend previous timing** (extend the preceding event's end),
|
||
or **Add to next timing** (start the following event earlier, keeping its end).
|
||
Apply & recheck updates durations and Ontime output. Allocated intervals remain
|
||
in the preview for review and undo; choose Keep gap to restore original timings.
|
||
The selected event must touch the gap, and unresolved/overlapping intervals
|
||
cannot be absorbed. Choices are saved in resolution profiles as `gap_resolutions`,
|
||
keyed by exact gap boundaries. If other corrections change those boundaries,
|
||
remove the stale choice in Setup or reset decisions and recheck.
|
||
Unknown values show `?`.
|
||
Unresolved source rows offer **Ignore** to set the ignore resolution and
|
||
**Details** to open the row editor. Ignoring requires recheck and can be undone
|
||
under All rows. **Ignore all unresolved (N)** applies this to unresolved source
|
||
rows and timed rows missing a title across the sheet, regardless of the current
|
||
filter. Other timed rows, timing errors and existing manual corrections are excluded.
|
||
Hover over a title or details button for row status and full diagnostic explanations.
|
||
Double-click anywhere on a source row, or click its **Details** / **Error** button,
|
||
to open the same editor (inline dropdowns keep their normal interaction).
|
||
It shows timing, editable row type and start/end/duration corrections, full diagnostics, timing
|
||
decisions and a formatted table of original cells (including empty values and
|
||
formulas). Pending edits are distinguished from the last analyzed timing.
|
||
**Issues** opens the issue navigator, including sheet-level and gap errors.
|
||
The table fills the remaining space; its title column expands with the window.
|
||
Preview cells align at the top; blank title lines are omitted without changing
|
||
original values in the inspector or exports. On narrow windows,
|
||
timing values stack so corrections remain accessible. Changes require recheck;
|
||
the timing preview and diagnostics continue to show the previous analysis.
|
||
- **Columns** shows a concise verified mapping summary, green detected roles,
|
||
per-column role dropdowns, and sample values. Expand **Detection evidence** for
|
||
full inference and validation details; hover the **Columns** tab for a compact
|
||
explanation of the role choices. Selecting one duration column explicitly resolves duplicate TIMER
|
||
headers. **Notes** appends a column's values to Ontime event notes. **Metadata**
|
||
creates an Ontime custom field named from the source header. **Ignore** excludes
|
||
the column from inference and Ontime output. Several columns can use any of these
|
||
destinations. **Auto** restores inference: production headers such as video,
|
||
light/lumière, sound/son, PowerPoint and slides become metadata; note, comment,
|
||
remark and instruction headers become notes; other unused columns are ignored.
|
||
**Manual header mapping** remains
|
||
available in Setup for the header row and column choices. The separate
|
||
**Advanced settings** section contains duration units and profile JSON.
|
||
|
||
The row type selector shows the detected type in parentheses, such as **Auto (Event)**.
|
||
**Auto (?)** means unresolved or awaiting recheck.
|
||
|
||
Click **Apply & recheck** after corrections. Pending edits retain the issue list
|
||
and previous preview so you can resolve several rows without losing your place;
|
||
export and remembering a resolution remain blocked until validation succeeds.
|
||
Blank title and timing corrections restore source or calculated values. Start and end accept
|
||
clock times such as `09:30` or `09:30:00`; duration accepts explicit values such as
|
||
`2 min` or `00:02:00`. Missing timings, overlaps and decisions for rows without a timing can be
|
||
resolved here. Unsupported interpretations and unresolved conflicts retain explicit
|
||
errors; correct the source when available controls cannot express the intended fix.
|
||
CLI errors for rows without a timing also include a labeled source preview.
|
||
|
||
**Save resolutions…** opens the save/load/remember controls without taking space away
|
||
from the correction table. **Results** offers Open, Open output folder and Copy
|
||
path. **Details** contains status, completed analysis stage count and the full IR,
|
||
including original rows. Cancel is observed between analysis stages and before
|
||
publishing an export; reading a workbook or an active stage finishes first.
|
||
|
||
## Purpose
|
||
|
||
Convert differently formatted conduites (rundowns) into an Ontime project without
|
||
manually rebuilding each document. Detect tables, headers, event titles and timing
|
||
columns using explainable heuristics. Preserve source order and scheduled start
|
||
times, especially when a session resumes after a break.
|
||
|
||
When the source is ambiguous, stop export with a precise explanation and ask for
|
||
clearer headers or an explicit column mapping. Never silently swap start times
|
||
and durations, discard questionable rows, or fill unknown durations with zero.
|
||
|
||
## Workflow
|
||
|
||
Open or drop a document and select its sheet/table, then follow five stages:
|
||
|
||
1. Infer column meanings from data across rows. An ambiguous cell can be clarified
|
||
by other rows, but one meaning must fit the whole column; otherwise return an
|
||
error asking for clearer headers, corrected values or an explicit mapping.
|
||
2. Identify untimed gaps and incorporate them into the draft intermediate
|
||
representation (IR). These breaks occupy schedule time without a shown timer.
|
||
3. Construct the complete IR from times and gap constraints, preserving original
|
||
and inferred values. Ask for clarification when required facts remain unknown.
|
||
4. Check the complete schedule for inconsistencies, including backward times,
|
||
negative durations, overlaps and contradictory start/end/duration values.
|
||
5. Generate native Ontime JSON or an Ontime-ready XLSX from the validated IR,
|
||
with optional neutral JSON for inspection or reuse. Open the result or its
|
||
output folder.
|
||
|
||
Review and correction controls expose each stage's decisions and errors. CLI and
|
||
GUI follow this same sequence; export cannot bypass unresolved errors.
|
||
|
||
The CLI and GUI use the same detection, validation and export code. CLI
|
||
failures identify the source location and return a nonzero exit status.
|
||
Inputs stay unchanged; existing output requires explicit overwrite permission.
|
||
Processing is local. XLSX preserves merged ranges and cached formula values.
|
||
When a cached result is missing, the converter safely calculates arithmetic made
|
||
of numbers, same-sheet cell references, parentheses and `+`, `-`, `*` or `/`.
|
||
It does not run spreadsheet functions or macros. Remaining formulas are grouped
|
||
into one review issue with the affected cells; recalculate and save those files
|
||
in Excel or LibreOffice. The converter cannot determine whether cached values are
|
||
stale. CSV/TSV uses UTF-8, quoted fields and comma/semicolon/tab detection.
|
||
Legacy XLS, ODS, Word, PDF and scans are not yet supported. A
|
||
[PDF feasibility trial](rundown-corpus.md#pdf-feasibility-trial) recovered table
|
||
text from the supplied PDFs but could not establish a complete timer schedule;
|
||
PDF importer work is paused for now. Input files are limited
|
||
to 32 MiB; XLSX expanded content is limited to 128 MiB.
|
||
|
||
## Timing and breaks
|
||
|
||
Explicit start times are schedule anchors. Durations can come from a duration
|
||
column or an unambiguous start/end pair. The preview distinguishes original
|
||
values from calculated values, unresolved values and gaps.
|
||
|
||
| Source evidence | Interpretation |
|
||
| --- | --- |
|
||
| Cue at 11:45 lasting 15 minutes; next cue at 13:30 | Preserve a 12:00–13:30 gap and the 13:30 start |
|
||
| Named lunch row from 12:00 to 13:30 | Preserve an explicit break with its original label |
|
||
| Cue at 11:45; next cue at 13:30; no end or duration | Event duration versus break is unknown; do not invent a lunch break |
|
||
| Start 11:45, end 12:00, duration 20 minutes | Report conflicting timing and request clarification |
|
||
| Final cue has only a start | Request its end/duration or an explicit non-timed classification |
|
||
|
||
Inferring a duration from the next start is allowed only when the source or a
|
||
user-selected policy establishes consecutive events. That policy and each
|
||
inference must remain visible. A long interval alone does not prove a break.
|
||
Do not extend the preceding timer through an established gap or shift an
|
||
explicit start to make durations fit. Overlaps, day changes and parallel stages
|
||
need explicit interpretation rather than automatic sorting or midnight wrapping.
|
||
|
||
Ontime documents independent starts, linked starts and separate duration/end
|
||
strategies in [Smart time entry](https://docs.getontime.no/quick-tips/smart-time-entry/).
|
||
Its [delay documentation](https://docs.getontime.no/quick-tips/managing-delays/)
|
||
also describes schedule gaps absorbing delays. The export retains a known unnamed break as a gap, with the following start unlinked.
|
||
Untimed gaps remain in the IR without generating a synthetic timer or displaying
|
||
a timer duration. An explicitly timed break cue remains distinct from such gaps.
|
||
Schedule preservation does not imply automatic playback or guarantee that an
|
||
operator will start an event on time.
|
||
|
||
## Output choice
|
||
|
||
Native Ontime JSON is the primary target: it lets the exporter control time
|
||
relationships and project structure without a second column-mapping step.
|
||
XLSX export writes one `Ontime` worksheet with `Start`, `End`, `Duration`, `Cue`,
|
||
`Title`, `Note` and `Timer type` columns, followed by selected metadata columns.
|
||
Times use `HH:MM:SS` strings accepted by Ontime. Import the file with Ontime's
|
||
[spreadsheet importer](https://docs.getontime.no/features/import-spreadsheet/)
|
||
and map the displayed columns. Schedule gaps remain visible through absolute
|
||
event starts; the exporter does not invent timer rows for them. Ontime's current
|
||
file importer accepts XLSX and does not expose ODS upload, so ODS is not generated.
|
||
These outputs share the validated IR and are not intermediate steps for native
|
||
JSON export. A Typst source/PDF report remains a stretch goal.
|
||
|
||
The adapter targets Ontime 4 native project JSON. Its events and schedule metadata
|
||
were checked with the Ontime 4.15.1 native parser; see the
|
||
[verification record](rundown-corpus.md#ontime-verification). Import the generated
|
||
project through Ontime's project manager. All starts are currently exported
|
||
unlinked, including derived starts, so later anchors survive edits to earlier
|
||
cues. Timers use duration lock and manual end actions.
|
||
|
||
## Clarification profiles
|
||
|
||
Pass `--profile mapping.json`, or load/edit the same JSON in the desktop app:
|
||
|
||
```json
|
||
{
|
||
"header_row": 1,
|
||
"columns": {"start": "A", "duration": "B", "title": "G"},
|
||
"column_usage": {"C": "notes", "D": "metadata", "F": "ignore"},
|
||
"expected_headers": {"A": "Heure", "B": "Durée"},
|
||
"untimed_as_notes": true,
|
||
"row_kinds": {"20": "marker"},
|
||
"titles": {"8": "Opening remarks"},
|
||
"starts": {"8": "09:30"},
|
||
"ends": {"8": "09:32"},
|
||
"durations": {"8": "2 min"}
|
||
}
|
||
```
|
||
|
||
This illustrates options, not a profile to apply blindly to the supplied files.
|
||
Rows are one-based original worksheet rows; columns use Excel letters. Supported
|
||
roles are `start`, `end`, `duration` and `title`. `duration_unit` accepts `minutes`,
|
||
`seconds`, `hh:mm`, `mm:ss` or `excel`. Cell number formats establish Excel time
|
||
units where available; otherwise units must be explicit or consistently supported
|
||
by independent start/end pairs. Unknown profile keys and changed expected headers
|
||
are errors. `titles`, `starts`, `ends` and `durations` are keyed by one-based source row;
|
||
corrected cells are excluded from whole-column validation while all remaining
|
||
explicit mappings still undergo it.
|
||
|
||
`column_usage` maps column letters to `notes`, `metadata` or `ignore`. Every choice
|
||
prevents that column from being inferred as a start, end, duration or title, and a
|
||
column cannot also have an explicit timing or title mapping. Notes values are joined
|
||
into the Ontime event note, including values from attached note rows and merged
|
||
continuations. Metadata creates a project-level Ontime text custom field and stores
|
||
each event's value in its `custom` object. Ignore removes the values from Ontime;
|
||
rows containing only ignored values need no clarification. Inferred timing and
|
||
correction provenance remain visible in the neutral IR and review UI but are not
|
||
added to Ontime notes.
|
||
|
||
With no manual choice, timing and title inference runs first. Remaining columns use
|
||
conservative header rules: recognised production departments and media become
|
||
metadata, recognised note-like headers become notes, and everything else is ignored.
|
||
Original cells remain available in the review and neutral IR for retained rows.
|
||
These choices are saved in profiles and remembered resolutions; use **Apply &
|
||
recheck** before export.
|
||
|
||
`row_kinds` accepts `event`, `gap`, `marker`, `note` and `ignore`. Notes attach to
|
||
the preceding event. Untimed markers remain in the IR but are not exported as
|
||
timers. An ignored row is recorded as an explicit exclusion. Duration overrides
|
||
retain original cell values and the fact that a correction was requested. Use
|
||
`expected_headers` to guard reusable profiles against changed layouts.
|
||
|
||
Under **Save resolutions…**, the two main actions are **Save resolution** and
|
||
**Reuse remembered resolution**. **Advanced** contains file management, reset,
|
||
and forget controls.
|
||
|
||
**Save resolution as…** (under Advanced) writes a portable profile JSON, usable with CLI
|
||
`--profile`. **Save (overwrite resolution)** replaces the loaded/saved profile;
|
||
Save as refuses existing files unless its replacement checkbox is enabled.
|
||
These actions save decisions, not a rewritten spreadsheet or an Ontime export.
|
||
Profiles may contain unfinished decisions; loading one always requires a fresh
|
||
analysis. Explicitly loaded profiles use worksheet row numbers, so review their
|
||
applicability when the document has changed.
|
||
|
||
**Save resolution** stores a validated profile in the platform's user
|
||
configuration directory, under `rundown-converter/resolutions` (on Linux,
|
||
`$XDG_CONFIG_HOME/rundown-converter/resolutions`, normally
|
||
`~/.config/rundown-converter/resolutions`). The status displays the exact path.
|
||
**Reuse remembered resolution** becomes available for an identical sheet and
|
||
loads its decisions for revalidation; **Forget** removes that remembered file.
|
||
Matching includes every cell, timing format, formula, merge, sheet name and row
|
||
order, so changed documents cannot silently inherit row corrections. This is
|
||
explicit reuse rather than automatic application. Changing file or sheet clears
|
||
current decisions. Remembered files contain profile decisions and a hash in their
|
||
filename, not a copy of the workbook; they remain local to this machine.
|
||
|
||
## Current limits
|
||
|
||
One selected sheet represents one sequential, single-day schedule. Parallel
|
||
tracks, midnight rollover and hierarchical parent/child timers need separate
|
||
interpretations; the converter does not sort or flatten them automatically.
|
||
A merged clock window plus individual row durations creates bounded sequential
|
||
cues. A shared merged duration creates one timer with the other rows retained
|
||
as notes. Unmerged untimed directions need explicit classification.
|
||
|
||
A full-width merged text band is treated as a section/footnote and recorded in
|
||
the IR. Nonempty unmerged text in a time column is a contradiction, not silently
|
||
ignored. Missing end/duration in a start-only schedule remains unresolved.
|
||
Recent-file persistence, direct start/end/title editing and portable release
|
||
bundles remain future work. JSON profiles provide the same supported resolution
|
||
decisions in both interfaces.
|
||
|
||
## Stretch goal: reserve buffer time
|
||
|
||
An optional correction mode will shorten timed rows to create a requested buffer
|
||
in an overcrowded schedule. It will offer equal or proportional cuts, minimum
|
||
durations and protected rows, with a before/after preview. Original values remain
|
||
available. Untimed breaks do not contribute time cuts.
|
||
|
||
Saved time can stay as slack after each event, preserving starts, or be collected
|
||
before a chosen anchor if the user explicitly allows intermediate starts to move.
|
||
Fixed restart times stay protected. Impossible requests must report the available
|
||
capacity; existing overlaps must be accounted for before claiming new buffer.
|
||
Every adjusted schedule must pass validation again before export. See the
|
||
[buffer design](rundown-converter-plan.md#optional-buffer-adjustment).
|
||
|
||
## Expanding coverage
|
||
|
||
The [sample findings](rundown-corpus.md) list the supplied documents, public
|
||
workbooks and unresolved interpretation questions. Further examples should keep
|
||
table structure, cell formats and representative time values when redacted.
|
||
Private originals stay out of Git; regression fixtures are synthetic.
|
||
|
||
See [all tools](../README.md) and [repository development](development.md).
|