Separate rundown processing stages and plan buffer corrections
This commit is contained in:
parent
48414bf279
commit
5a28720fea
|
|
@ -12,13 +12,24 @@ Use the reporter's Rust/clap/eframe approach and optional desktop feature.
|
|||
A Python prototype is optional for investigating a difficult format, not a
|
||||
second maintained implementation.
|
||||
|
||||
The processing pipeline is:
|
||||
Source extraction prepares tables before the five distinct processing stages:
|
||||
|
||||
```text
|
||||
source readers -> source tables -> header/column detection -> neutral rundown
|
||||
-> timing resolution and validation -> Ontime adapter -> file
|
||||
source readers -> source tables
|
||||
1. infer and verify column meanings across rows
|
||||
2. identify untimed gaps and record them in the draft IR
|
||||
3. construct the complete IR from source times and gap constraints
|
||||
4. validate schedule consistency
|
||||
5. export the validated IR to Ontime JSON
|
||||
```
|
||||
|
||||
Each stage has a separate result and diagnostics. A later stage must not silently
|
||||
reinterpret an earlier decision. Unresolved column meanings block stage 2;
|
||||
unresolved gap boundaries remain explicit constraints for stage 3. If a complete
|
||||
schedule cannot be established, return an actionable error rather than exporting
|
||||
partial guesses. Stage 4 is an independent validation pass over the complete IR,
|
||||
not a side effect of export. All exporters consume the same validated IR.
|
||||
|
||||
Readers retain original cell text, typed values, number formats, merged-cell
|
||||
ranges and source coordinates when available. A table model separates extraction
|
||||
from interpretation so format-specific quirks do not leak into the Ontime adapter.
|
||||
|
|
@ -46,7 +57,7 @@ only when required decisions are resolved.
|
|||
| Rundown/session | Ordered entries, label, date/day context, stage/track where established |
|
||||
| Entry | Stable identity, cue, title, speaker, notes, additional original columns, event/break/marker kind |
|
||||
| Timing | Optional explicit start/end, duration, day offset, fixed/relative intent, inferred/resolved values |
|
||||
| Gap | Known previous end and next start, interval, provenance, explicit or inferred classification |
|
||||
| Gap | Untimed break, boundary references/constraints, resolved interval when known, provenance, no timer display |
|
||||
| Decision | Selected mapping/rule, evidence, alternatives, user overrides |
|
||||
| Diagnostic | Stable code, severity, source reference, explanation, corrective action |
|
||||
|
||||
|
|
@ -57,12 +68,25 @@ reportable rather than overwritten. Convert to Ontime milliseconds only in the
|
|||
adapter. Stable identities should be reproducible for unchanged input; do not
|
||||
promise identity stability across arbitrary source edits.
|
||||
|
||||
## Detection and clarification
|
||||
## 1. Infer and verify column meanings
|
||||
|
||||
Combine normalized French/English header aliases, cell types, number formats,
|
||||
value patterns and consistency across rows. Normalize accents, casing, spacing
|
||||
and line breaks without losing original text. Evaluate candidate header bands
|
||||
and mappings jointly instead of assigning columns from individual keywords.
|
||||
Infer meaning primarily from data across the entire column: one ambiguous cell
|
||||
can be resolved by evidence in other rows. A column must retain one semantic role
|
||||
and a compatible unit interpretation throughout the selected table. Different
|
||||
textual formats may express the same role; roles must not switch row by row.
|
||||
|
||||
Require the chosen interpretation to fit every relevant nonempty data cell, not
|
||||
just a majority or a sample. If no single meaning fits, return
|
||||
`INCONSISTENT_COLUMN_MEANING` with conflicting cell references and ask the user to
|
||||
clarify headers, correct values or explicitly separate distinct tables. An
|
||||
explicit mapping must also pass this check. Empty cells provide no evidence;
|
||||
recognized headings/totals are structural rows with recorded classifications,
|
||||
not convenient exceptions used to hide contradictory event data. Check semantic
|
||||
consistency here; chronological contradictions belong to stage 4.
|
||||
|
||||
Account for title blocks above tables, merged/multiline headers, repeated headers,
|
||||
blank separators, continuation rows, section labels, footnotes and totals.
|
||||
|
|
@ -91,12 +115,38 @@ GUI mapping panel should select table, header rows, column roles and units.
|
|||
Reusable mapping profiles should verify expected headers before applying; a
|
||||
changed document must not silently inherit an unsuitable profile.
|
||||
|
||||
## Timing resolution and export contract
|
||||
## 2. Identify gaps and incorporate them into the IR
|
||||
|
||||
Identify breaks with no timer to display before deriving event durations. Use
|
||||
source labels, structural evidence and independently established event ends and
|
||||
following starts. Record each gap in the draft IR with its boundary references,
|
||||
known times and unresolved constraints. A named untimed lunch row is a gap, not
|
||||
an event with a missing duration. A break explicitly intended as a timed cue
|
||||
remains a different entry kind.
|
||||
|
||||
Gaps consume schedule time but never create a timer or show a duration as an
|
||||
event. Their intervals may appear in the diagnostic preview for explanation.
|
||||
Resolve unknown boundaries in stage 3 when sufficient evidence exists; otherwise
|
||||
ask for clarification. A long start-to-start interval alone is insufficient to
|
||||
identify a break. No optional synthetic break timer is generated for these gaps.
|
||||
|
||||
## 3. Construct the complete IR from times
|
||||
|
||||
Apply the policies in the [tool overview](rundown-converter.md#timing-and-breaks).
|
||||
Solve constraints within a session while preserving explicit anchors. An explicit
|
||||
start/end pair yields duration; a start/duration pair yields end. If all three
|
||||
are present, validate agreement using a documented precision tolerance.
|
||||
are present, retain all three plus calculated values for stage 4 to compare.
|
||||
Complete the ordered entries and gap intervals, preserving provenance and
|
||||
explicit versus inferred facts. Missing facts that cannot be derived must produce
|
||||
a clarification error; do not call an unresolved schedule complete.
|
||||
|
||||
## 4. Check schedule consistency
|
||||
|
||||
Validate the complete IR independently of the reader and export target. Check
|
||||
start/end/duration agreement using a documented precision tolerance, time going
|
||||
backward within each session/track, negative durations, overlaps and gap boundaries.
|
||||
Keep source order; never sort away a backward time or repair it by assuming a new
|
||||
day. Validate that gaps do not overlap events or accidentally consume cue time.
|
||||
|
||||
Do not infer an end across an unknown break, another session, another day or a
|
||||
parallel track. Midnight rollover must be supported by evidence or an explicit
|
||||
|
|
@ -104,14 +154,15 @@ setting. Treat unresolved overlap, negative duration, invalid time and unavailab
|
|||
final duration as actionable errors. Define zero-duration markers separately from
|
||||
missing-duration events. Never silently clamp or wrap unsupported target values.
|
||||
|
||||
## 5. Generate importable output
|
||||
|
||||
The Ontime adapter will map absolute starts to independent schedule entries and
|
||||
link only entries explicitly intended to follow their predecessor. Duration-lock,
|
||||
end-lock and count-to-end playback are distinct choices: preserving a scheduled
|
||||
start must not silently enable count-to-end or automated playback. Keep default
|
||||
end actions manual. Validate target limits, ordering, grouping and project schema.
|
||||
An established gap remains empty by default; do not use a delay entry to represent
|
||||
an ordinary lunch break. If visible inferred break cues are implemented, label
|
||||
and track them separately from source events.
|
||||
An established untimed gap remains empty; do not use a delay entry or synthetic
|
||||
timer to represent it. Preserve the independent start after it.
|
||||
|
||||
Export via a temporary file and publish only after complete validation, with
|
||||
explicit overwrite behavior and cleanup on errors/cancellation. Produce neutral
|
||||
|
|
@ -119,6 +170,46 @@ JSON separately from native Ontime JSON with unmistakable filenames. Diagnostic
|
|||
preview may remain available on failure, but an invalid project must not be
|
||||
published. Keep local source paths out of exported project notes by default.
|
||||
|
||||
## Stretch goals
|
||||
|
||||
### Additional exports
|
||||
|
||||
Generate a Typst report (editable source and rendered PDF) from the validated IR.
|
||||
Re-export XLS/ODS spreadsheets with explicit, normalized columns suitable for
|
||||
Ontime import. Verify accepted file types and exact header/unit requirements
|
||||
against the target Ontime release before claiming direct import support; if a
|
||||
requested format needs conversion, document that limitation explicitly. Include
|
||||
round-trip import tests for timing and gaps. Keep untimed gaps free of timer rows
|
||||
in every output. These exports do not become intermediate steps for JSON export.
|
||||
|
||||
### Optional buffer adjustment
|
||||
|
||||
Offer an explicit correction mode that removes time from event rows to reserve
|
||||
buffer in a cramped schedule. Keep the original IR and create a separate proposed
|
||||
revision with a before/after preview and a per-row adjustment record. Never apply
|
||||
this automatically during inference, validation or export.
|
||||
|
||||
Let the user choose a target buffer (duration or percentage), scope/session,
|
||||
distribution (equal cuts or proportional cuts), minimum event durations and
|
||||
protected rows. By default distribute cuts across all eligible timed rows in the
|
||||
selected scope; untimed gaps, markers and protected rows contribute nothing.
|
||||
Reject infeasible requests with the maximum achievable buffer. Allocate integer
|
||||
rounding remainders deterministically so cuts sum exactly to the requested buffer.
|
||||
|
||||
Define where the saved time goes: either slack after each shortened event while
|
||||
keeping its start, or a consolidated buffer before a selected anchor. Consolidation
|
||||
requires explicit permission to move affected intermediate starts; preserve fixed
|
||||
session/restart anchors and protected timings. Never lengthen an existing break
|
||||
by accident or count existing slack as newly recovered time. If a source already
|
||||
overlaps, distinguish cuts needed to eliminate the overlap from additional cuts
|
||||
needed to create the requested buffer. Column ambiguities must be resolved first;
|
||||
correction must not conceal malformed times or silently choose their meaning.
|
||||
|
||||
Run the complete stage-4 validation again on any proposed revision before stage-5
|
||||
export. Report remaining conflicts and export only a valid selected revision.
|
||||
Test insufficient capacity, minimum durations, protected rows, anchor preservation,
|
||||
rounding, existing gaps, overlap repair and GUI/CLI parity.
|
||||
|
||||
## CLI and desktop experience
|
||||
|
||||
Proposed CLI operations are inspect, validate and convert; finalize syntax after
|
||||
|
|
@ -152,14 +243,19 @@ errors must remain visible with direct corrective actions.
|
|||
defaults. Include the new member in workspace CI, documentation and coverage.
|
||||
- [ ] Implement the required source readers and extraction diagnostics. Test
|
||||
unsupported/encrypted/corrupt inputs and formula/merged-cell behavior as relevant.
|
||||
- [ ] Implement detection, explicit mappings and source-located errors; verify
|
||||
competing headers and misleading time formats fail clearly.
|
||||
- [ ] Implement timing constraints and gaps. Test known/unknown lunch gaps,
|
||||
start-only schedules, contradictory triples, missing final duration, overlaps,
|
||||
midnight/day transitions, zero-duration markers and multiple sessions/tracks.
|
||||
- [ ] Stage 1: implement whole-column inference, explicit mappings and located
|
||||
errors. Test ambiguous cells clarified by other rows, incompatible meanings in
|
||||
one column, structural rows and misleading headers/units.
|
||||
- [ ] Stage 2: identify untimed gaps and populate draft IR constraints. Test named
|
||||
and unnamed breaks, unknown boundaries and distinction from timed break cues.
|
||||
- [ ] Stage 3: construct the complete IR from times and gap constraints. Test
|
||||
start-only schedules, missing final duration and provenance of inferred values.
|
||||
- [ ] Stage 4: independently validate backward times, negative durations,
|
||||
contradictory triples, overlaps, gap boundaries, midnight/day transitions,
|
||||
zero-duration markers and multiple sessions/tracks.
|
||||
- [ ] Implement inspect/validate/convert CLI and atomic exports. Test CLI status,
|
||||
output collisions, cancellation, source preservation and deterministic results.
|
||||
- [ ] Implement the Ontime adapter and neutral JSON export. Check native imports
|
||||
- [ ] Stage 5: implement the Ontime adapter and neutral JSON export. Check native imports
|
||||
against the pinned parser and manually inspect real Ontime behavior; don't rely
|
||||
solely on JSON snapshots. Verify that later anchors survive earlier edits and
|
||||
gaps remain visible in the schedule; document playback implications.
|
||||
|
|
@ -175,6 +271,10 @@ errors must remain visible with direct corrective actions.
|
|||
production GUI coverage and reporter behavior. Record unavailable manual checks.
|
||||
- [ ] Extend manual packaging/release preparation for both binaries and supported
|
||||
platforms, including only dependencies this tool needs. CI must not publish releases.
|
||||
- [ ] Stretch: add Typst source/PDF reports and XLS/ODS re-export with verified
|
||||
Ontime-compatible columns and documented format support.
|
||||
- [ ] Stretch: add explicit buffer adjustment, before/after review, feasibility
|
||||
checks and revalidation, with the cases listed above.
|
||||
- [ ] Commit each completed implementation task with its validation summary.
|
||||
|
||||
## Documentation conventions
|
||||
|
|
|
|||
|
|
@ -19,12 +19,22 @@ and durations, discard questionable rows, or fill unknown durations with zero.
|
|||
|
||||
## Planned workflow
|
||||
|
||||
1. Open or drop a document, then select a sheet or table if several are plausible.
|
||||
2. Inspect detected headers, sample rows and a normalized schedule preview.
|
||||
3. Resolve highlighted ambiguities through column mappings or clearer source
|
||||
headers. Ordinary unambiguous files should require no manual mapping.
|
||||
4. Export native Ontime JSON and optionally a neutral JSON representation for
|
||||
inspection or reuse. Open the result or its output folder.
|
||||
Open or drop a document and select its sheet/table, then follow five stages:
|
||||
|
||||
1. Infer column meanings from data across rows. An ambiguous cell can be clarified
|
||||
by other rows, but one meaning must fit the whole column; otherwise return an
|
||||
error asking for clearer headers, corrected values or an explicit mapping.
|
||||
2. Identify untimed gaps and incorporate them into the draft intermediate
|
||||
representation (IR). These breaks occupy schedule time without a shown timer.
|
||||
3. Construct the complete IR from times and gap constraints, preserving original
|
||||
and inferred values. Ask for clarification when required facts remain unknown.
|
||||
4. Check the complete schedule for inconsistencies, including backward times,
|
||||
negative durations, overlaps and contradictory start/end/duration values.
|
||||
5. Generate native Ontime JSON from the validated IR, with optional neutral JSON
|
||||
for inspection or reuse. Open the result or its output folder.
|
||||
|
||||
Review and correction controls expose each stage's decisions and errors. CLI and
|
||||
GUI follow this same sequence; export cannot bypass unresolved errors.
|
||||
|
||||
The CLI and GUI will use the same detection, validation and export code. CLI
|
||||
failures will identify the source location and return a nonzero exit status.
|
||||
|
|
@ -59,8 +69,8 @@ strategies in [Smart time entry](https://docs.getontime.no/quick-tips/smart-time
|
|||
Its [delay documentation](https://docs.getontime.no/quick-tips/managing-delays/)
|
||||
also describes schedule gaps absorbing delays. The planned default export will
|
||||
retain a known unnamed break as a gap, with the following start unlinked.
|
||||
An optional visible “Break (inferred)” cue can be evaluated during implementation;
|
||||
it must be visibly synthetic and must not change the following scheduled start.
|
||||
Untimed gaps remain in the IR without generating a synthetic timer or displaying
|
||||
a timer duration. An explicitly timed break cue remains distinct from such gaps.
|
||||
Schedule preservation does not imply automatic playback or guarantee that an
|
||||
operator will start an event on time.
|
||||
|
||||
|
|
@ -70,14 +80,31 @@ Native Ontime JSON is the primary target: it lets the exporter control time
|
|||
relationships and project structure without a second column-mapping step.
|
||||
Ontime supports project files and spreadsheet interchange; see its
|
||||
[import/export documentation](https://docs.getontime.no/quick-tips/import-export/).
|
||||
An XLSX export is a possible later convenience for human editing, not a required
|
||||
intermediate format. Legacy XLS output is not planned.
|
||||
Stretch goals are a Typst report (editable source and PDF) and XLS/ODS re-export
|
||||
with normalized columns directly suitable for Ontime import. Actual file-format
|
||||
and column compatibility must be verified against the target release; any required
|
||||
format conversion will be documented. These outputs share the validated IR and
|
||||
are not intermediate steps for native JSON export.
|
||||
|
||||
The existing reporter targets Ontime 4. The new adapter will initially target
|
||||
that family, with the exact tested release/schema recorded before implementation
|
||||
is accepted. Import and timing behavior must be verified in that release;
|
||||
consulting documentation alone does not establish compatibility.
|
||||
|
||||
## Stretch goal: reserve buffer time
|
||||
|
||||
An optional correction mode will shorten timed rows to create a requested buffer
|
||||
in an overcrowded schedule. It will offer equal or proportional cuts, minimum
|
||||
durations and protected rows, with a before/after preview. Original values remain
|
||||
available. Untimed breaks do not contribute time cuts.
|
||||
|
||||
Saved time can stay as slack after each event, preserving starts, or be collected
|
||||
before a chosen anchor if the user explicitly allows intermediate starts to move.
|
||||
Fixed restart times stay protected. Impossible requests must report the available
|
||||
capacity; existing overlaps must be accounted for before claiming new buffer.
|
||||
Every adjusted schedule must pass validation again before export. See the
|
||||
[buffer design](rundown-converter-plan.md#optional-buffer-adjustment).
|
||||
|
||||
## Examples needed next
|
||||
|
||||
Provide representative originals, including difficult or ambiguous layouts and
|
||||
|
|
|
|||
Loading…
Reference in New Issue