22 KiB
Rundown converter development plan
Status: the XLSX/CSV release candidate is available with a shared core, CLI and egui interface. User-facing scope and current limitations are in Rundown converter; corpus findings record real inputs and unresolved questions. Unchecked tasks below remain future work; broader design requirements are not claims of implemented support.
Completion audit
Audited against the 0.3.0-rc.1 source and documentation on 2026-11-03. The
five required workflow stages and the requested shared CLI/GUI review experience
are implemented. Remaining work is acceptance hardening, explicitly unsupported
schedule models, packaging and stretch goals.
| Area | Status | Evidence or remaining work |
|---|---|---|
| 1. Whole-column inference | Complete | Header/data evidence, manual mapping and whole-column contradictions are implemented and tested. Explicit roles can select the start component of a mixed range and apply one documented extraction policy to a mixed duration column. |
| 2. Untimed gaps | Complete | Named and inferred gaps enter the IR; keep/extend previous/add to next resolutions are available. |
| 3. Complete IR | Complete for supported single-day schedules | Explicit and inferred timing provenance, notes, metadata, corrections and exclusions are retained. |
| 4. Consistency checks | Complete for supported single-track schedules | Conflicting triples, backward/negative timing, overlaps and gap boundaries block export. |
| 5. Ontime output | Complete | Native Ontime JSON, neutral JSON and normalized XLSX share the validated IR; native parsing was checked against Ontime 4.15.1. |
| CLI and desktop review | Complete for the release-candidate workflow | Inspect/validate/convert, Setup/Review/Results, mapping, row editing, profiles, remembered resolutions and pending/recheck behavior are present. Recent-file persistence and broader native/platform interaction checks remain. |
| Public evaluation corpus | Expanded, still not a benchmark | Ten external files now cover XLSX and CSV successes, deliberate failures and manual-resolution cases. A fully reviewed expected-result matrix and held-out accuracy exercise remain. |
| PDF input | Trial complete; importer paused | The bounded 17-page experiment recovered tables but could not establish dependable sequential timers. |
| Parallel tracks and midnight/day rollover | Unsupported by design | These require explicit models; current behavior stops instead of guessing. |
| Portable release bundles | Open | Workspace builds include the converter, but release packaging is still reporter-oriented. |
| XLSX re-export stretch goal | Complete | Writes Ontime-importable normalized columns. |
| Typst/PDF report stretch goal | Open | No report exporter is implemented. |
| Buffer-adjustment stretch goal | Open | No proposal, feasibility or redistribution workflow is implemented. |
| ODS | Intentionally excluded | Ontime's file importer does not expose ODS upload; XLSX is the supported spreadsheet export. |
Architecture
Workspace member: crates/rundown-converter/, with binaries
rundown-converter and rundown-converter-gui. It uses the reporter's
Rust/clap/eframe approach and optional desktop feature.
A Python prototype is optional for investigating a difficult format, not a
second maintained implementation.
Source extraction prepares tables before the five distinct processing stages:
source readers -> source tables
1. infer and verify column meanings across rows
2. identify untimed gaps and record them in the draft IR
3. construct the complete IR from source times and gap constraints
4. validate schedule consistency
5. export the validated IR to Ontime JSON
Each stage has a separate result and diagnostics. A later stage must not silently reinterpret an earlier decision. Unresolved column meanings block stage 2; unresolved gap boundaries remain explicit constraints for stage 3. If a complete schedule cannot be established, return an actionable error rather than exporting partial guesses. Stage 4 is an independent validation pass over the complete IR, not a side effect of export. All exporters consume the same validated IR.
Readers retain original cell text, typed values, number formats, merged-cell ranges and source coordinates when available. A table model separates extraction from interpretation so format-specific quirks do not leak into the Ontime adapter. PDF extraction must retain page/position information and expose unreliable reading order; image-only documents must produce an actionable unsupported/extraction error until a tested reader exists. Never execute document macros or spreadsheet functions. A bounded arithmetic parser may calculate same-sheet references when cached results are missing; unsupported or stale results require explicit handling.
Planned core modules cover readers, detection, neutral model, timing, diagnostics, export and orchestration. GUI and CLI are thin clients of the same core; neither contains independent parsing rules. Extract shared reporter UI utilities only when useful code reuse is demonstrated; do not couple the new model to videos, FFprobe, Typst or the reporter's sequential timer assumptions.
Neutral representation
Use a versioned serializable model independent of Ontime fields and units. Represent unresolved facts explicitly; validation produces an exportable model only when required decisions are resolved.
| Entity | Information to retain |
|---|---|
| Document | Source identity, reader, sheet/table choices, model version, interpretation settings |
| Source reference | File, sheet/page, row/cell or bounding region, original text |
| Rundown/session | Ordered entries, label, date/day context, stage/track where established |
| Entry | Stable identity, cue, title, notes, custom-field metadata, additional original columns, event/break/marker kind |
| Timing | Optional explicit start/end, duration, day offset, fixed/relative intent, inferred/resolved values |
| Gap | Untimed break, boundary references/constraints, resolved interval when known, provenance, no timer display |
| Decision | Selected mapping/rule, evidence, alternatives, user overrides |
| Diagnostic | Stable code, severity, source reference, explanation, corrective action |
Use integer time units, not floating-point arithmetic, for normalized durations and schedule calculations. Keep wall-clock times plus date/day offsets separate from elapsed durations. Preserve given and derived values so conflicts are reportable rather than overwritten. Convert to Ontime milliseconds only in the adapter. Stable identities should be reproducible for unchanged input; do not promise identity stability across arbitrary source edits.
1. Infer and verify column meanings
Combine normalized French/English header aliases, cell types, number formats, value patterns and consistency across rows. Normalize accents, casing, spacing and line breaks without losing original text. Evaluate candidate header bands and mappings jointly instead of assigning columns from individual keywords. Infer meaning primarily from data across the entire column: one ambiguous cell can be resolved by evidence in other rows. A column must retain one semantic role and a compatible unit interpretation throughout the selected table. Different textual formats may express the same role; roles must not switch row by row.
Require the chosen interpretation to fit every relevant nonempty data cell, not
just a majority or a sample. If no single meaning fits, return
INCONSISTENT_COLUMN_MEANING with conflicting cell references and ask the user to
clarify headers, correct values or explicitly separate distinct tables. An
explicit mapping must also pass this check. Empty cells provide no evidence;
recognized headings/totals are structural rows with recorded classifications,
not convenient exceptions used to hide contradictory event data. Check semantic
consistency here; chronological contradictions belong to stage 4.
Account for title blocks above tables, merged/multiline headers, repeated headers,
blank separators, continuation rows, section labels, footnotes and totals.
Account for clock forms such as 09:30/9h30, explicit units such as 15 min,
spreadsheet time serials, formula results, decimal commas and CSV delimiters.
An unlabelled 10:30 or bare 15 is not sufficient evidence for a time role/unit.
Formatting and keywords are evidence, not proof.
Calibrate acceptance thresholds and the separation between competing mappings against examples. Do not present heuristic scores as statistical probabilities. Report the winning evidence and meaningful alternatives. If required roles or row classifications remain ambiguous, block export; nonessential columns can remain preserved metadata with a warning. Never silently drop a possible cue.
Example diagnostic, with proposed wording:
AMBIGUOUS_TIME_COLUMN — Sheet “Jour 1”, C4:C28, header “Temps”
Values such as “10:30” could be start times or durations.
Clarify the header to “Début” or “Durée (mm:ss)”, or map column C and its units.
No Ontime file was written.
Mappings must be possible without modifying originals. CLI options/config and a GUI mapping panel should select table, header rows, column roles and units. Reusable mapping profiles should verify expected headers before applying; a changed document must not silently inherit an unsuitable profile.
2. Identify gaps and incorporate them into the IR
Identify breaks with no timer to display before deriving event durations. Use source labels, structural evidence and independently established event ends and following starts. Record each gap in the draft IR with its boundary references, known times and unresolved constraints. A named untimed lunch row is a gap, not an event with a missing duration. A break explicitly intended as a timed cue remains a different entry kind.
Gaps consume schedule time but never create a timer or show a duration as an event. Their intervals may appear in the diagnostic preview for explanation. Resolve unknown boundaries in stage 3 when sufficient evidence exists; otherwise ask for clarification. A long start-to-start interval alone is insufficient to identify a break. No optional synthetic break timer is generated for these gaps.
3. Construct the complete IR from times
Apply the policies in the tool overview. Solve constraints within a session while preserving explicit anchors. An explicit start/end pair yields duration; a start/duration pair yields end. If all three are present, retain all three plus calculated values for stage 4 to compare. Complete the ordered entries and gap intervals, preserving provenance and explicit versus inferred facts. Missing facts that cannot be derived must produce a clarification error; do not call an unresolved schedule complete.
4. Check schedule consistency
Validate the complete IR independently of the reader and export target. Check start/end/duration agreement using a documented precision tolerance, time going backward within each session/track, negative durations, overlaps and gap boundaries. Keep source order; never sort away a backward time or repair it by assuming a new day. Validate that gaps do not overlap events or accidentally consume cue time.
Do not infer an end across an unknown break, another session, another day or a parallel track. Midnight rollover must be supported by evidence or an explicit setting. Treat unresolved overlap, negative duration, invalid time and unavailable final duration as actionable errors. Define zero-duration markers separately from missing-duration events. Never silently clamp or wrap unsupported target values.
5. Generate importable output
The Ontime adapter will map absolute starts to independent schedule entries and link only entries explicitly intended to follow their predecessor. Duration-lock, end-lock and count-to-end playback are distinct choices: preserving a scheduled start must not silently enable count-to-end or automated playback. Keep default end actions manual. Validate target limits, ordering, grouping and project schema. An established untimed gap remains empty; do not use a delay entry or synthetic timer to represent it. Preserve the independent start after it.
Export via a temporary file and publish only after complete validation, with explicit overwrite behavior and cleanup on errors/cancellation. Produce neutral JSON separately from native Ontime JSON with unmistakable filenames. Diagnostic preview may remain available on failure, but an invalid project must not be published. Keep local source paths out of exported project notes by default.
Stretch goals
Additional exports
Generate a Typst report (editable source and rendered PDF) from the validated IR. XLSX re-export provides explicit, normalized columns suitable for Ontime import; ODS remains unavailable because Ontime's current file importer does not expose ODS upload. Keep untimed gaps free of timer rows in every output. These exports do not become intermediate steps for JSON export.
Optional buffer adjustment
Offer an explicit correction mode that removes time from event rows to reserve buffer in a cramped schedule. Keep the original IR and create a separate proposed revision with a before/after preview and a per-row adjustment record. Never apply this automatically during inference, validation or export.
Let the user choose a target buffer (duration or percentage), scope/session, distribution (equal cuts or proportional cuts), minimum event durations and protected rows. By default distribute cuts across all eligible timed rows in the selected scope; untimed gaps, markers and protected rows contribute nothing. Reject infeasible requests with the maximum achievable buffer. Allocate integer rounding remainders deterministically so cuts sum exactly to the requested buffer.
Define where the saved time goes: either slack after each shortened event while keeping its start, or a consolidated buffer before a selected anchor. Consolidation requires explicit permission to move affected intermediate starts; preserve fixed session/restart anchors and protected timings. Never lengthen an existing break by accident or count existing slack as newly recovered time. If a source already overlaps, distinguish cuts needed to eliminate the overlap from additional cuts needed to create the requested buffer. Column ambiguities must be resolved first; correction must not conceal malformed times or silently choose their meaning.
Run the complete stage-4 validation again on any proposed revision before stage-5 export. Report remaining conflicts and export only a valid selected revision. Test insufficient capacity, minimum durations, protected rows, anchor preservation, rounding, existing gaps, overlap repair and GUI/CLI parity.
CLI and desktop experience
Proposed CLI operations are inspect, validate and convert; finalize syntax after
sample review. Support explicit output, sheet/table selection, mappings, units,
neutral JSON and overwrite. Provide useful progress, plain redirected logs,
NO_COLOR, documented exit codes and structured diagnostics for automation.
CLI operation must not depend on GUI availability or interactive prompts.
Match the reporter's compact egui layout, purple primary action, native file picker, drag/drop, recent inputs, output controls, Settings, fixed bottom progress and Cancel, scrollable content, and Results with Open/Open folder/Copy path. Add a Review pane for source samples, mappings and the resolved schedule; show anchored starts, inferred durations, breaks and blockers clearly. Keep controls usable in small windows and at larger text sizes. Conversion runs on a worker thread; controls reflect running/cancelled/completed states. Parsing and export errors must remain visible with direct corrective actions.
Ordered task list and acceptance gates
- Inspect existing CLI, GUI, Ontime export and repository documentation.
- Check official Ontime documentation for schedule gaps and import options.
- Specify neutral model, ambiguity policy, start-time priority and break rules.
- Add separate tool documentation and link it from the repository overview.
- Inspect both supplied XLSX workbooks and both PDFs; download five public workbook layouts with a provenance manifest. Record findings separately.
- Run a bounded PDF guessing experiment over all 17 supplied pages using text coordinates and vector table borders. Record recovery and validator results in the PDF feasibility trial. Pause importer work: column recovery succeeds, but missing durations and simultaneous tracks prevent dependable whole-document timer conversion.
- Add bounded XLSX and UTF-8 CSV/TSV readers, synthetic regression fixtures, merged-block handling and source-located formula errors.
- Verify exported event times and a 90-minute gap through the Ontime 4.15.1 native rundown parser and metadata calculation at the recorded source commit.
- Add initial Setup/Review/Results GUI, source preview, column/unit selectors, shared profile controls and worker/cancellation tests.
- Keep inference evidence and the source table on Review after failures; add column-role dropdowns, row classifications and title/start/end/duration corrections.
- Save/overwrite/save-as portable resolution profiles and explicitly remember, reuse or forget validated resolutions matched to the complete sheet.
- Redesign Review into Resolve rows / Schedule preview / Columns, with one issue at a time, row navigation/filtering, a full-height correction table and separate saved-resolution controls. Preserve pending edits and previous results until recheck; verify compact layout and dropdown interaction.
- Merge schedule preview and row corrections with All rows / Included rows / Needs attention filters, preserving ignored rows for restoration and displaying attached notes individually without duplicating their timers.
- Add recent-file persistence and comprehensive native GUI interaction checks.
- Inventory supplied examples by format/layout; record intended interpretations and the user's Ontime version. Decide first-release input coverage from these.
- Build sanitized fixtures and an expected-result matrix before tuning rules. Include difficult examples and some held-out layouts to detect overfitting.
- Confirm a pinned Ontime schema/release and create a minimal import experiment with independent starts and a lunch gap; verify actual schedule and playback.
- Scaffold the Rust member, core model, optional GUI and CLI; retain reporter defaults. Include the new member in workspace CI, documentation and coverage.
- Implement the required source readers and extraction diagnostics. Test unsupported/encrypted/corrupt inputs and formula/merged-cell behavior as relevant.
- Stage 1: implement whole-column inference, explicit mappings and located errors. Test ambiguous cells clarified by other rows, incompatible meanings in one column, structural rows and misleading headers/units.
- Keep one Duration column role while offering nested whole-cell, minimum, maximum, upper median and sum extraction policies, plus independent margin and split toggles. Let a Start mapping select only the first clock in a mixed range; retain margin gaps and split entries in the IR.
- Add a secondary title role with Either, Merge, Notes and Metadata handling, shared across CLI profiles, GUI review and exports.
- Stage 2: identify untimed gaps and populate draft IR constraints. Test named and unnamed breaks, unknown boundaries and distinction from timed break cues.
- Stage 3: construct the complete IR from times and gap constraints. Test start-only schedules, missing final duration and provenance of inferred values.
- Stage 4: independently validate backward times, negative durations, contradictory triples, overlaps, gap boundaries and explicit markers.
- Add explicit midnight/day and multiple-track interpretations; current behavior rejects these rather than guessing.
- Implement inspect/validate/convert CLI and atomic exports. Test CLI status, output collisions, cancellation, source preservation and deterministic results.
- Stage 5: implement the Ontime adapter and neutral JSON export. Check native imports against the pinned parser and manually inspect real Ontime behavior; don't rely solely on JSON snapshots. Verify that later anchors survive earlier edits and gaps remain visible in the schedule; document playback implications.
- Implement desktop Setup/Review/Results, mapping corrections and diagnostics. Verify GUI/CLI parity; manually inspect representative and ambiguous inputs, resized windows, larger text, long paths, progress and cancellation.
- Run the supported corpus end to end: each source cue must be represented or have an explicit reviewed exclusion; every unresolved ambiguity must block export.
- Complete usage examples, supported-format matrix, limitations and illustrated GUI instructions. Add runnable quick-start commands only once they exist.
- Run appropriate Python CI/packaging tests and all Rust gates listed in Development, plus whole-workspace coverage. Preserve production GUI coverage and reporter behavior. Record unavailable manual checks.
- Extend manual packaging/release preparation for both binaries and supported platforms, including only dependencies this tool needs. CI must not publish releases.
- Stretch: add XLSX re-export with verified Ontime-compatible columns and documented format support.
- Stretch: add Typst source/PDF reports.
- Stretch: add explicit buffer adjustment, before/after review, feasibility checks and revalidation, with the cases listed above.
- Commit each completed implementation task with its validation summary.
Documentation conventions
Keep README as a short repository entry point with quick-start examples and links. Each tool has its own usage page; detailed design/task tracking stays separate. Use one H1 per page, sentence-case headings, fenced code with a language, ordinary relative Markdown links and consistent tables/lists. Label planned features and commands clearly. The existing illustrated guide remains explicitly scoped to the PPTX reporter; add a separate converter guide when a working GUI can be captured.