Record PDF rundown guessing trial and pause unreliable import

This commit is contained in:
Dandaedre 2026-11-03 13:38:10 +01:00
parent b0fb3b6996
commit b5555b2cf2
3 changed files with 85 additions and 1 deletions

View File

@ -237,6 +237,11 @@ errors must remain visible with direct corrective actions.
- [x] Add separate tool documentation and link it from the repository overview.
- [x] Inspect both supplied XLSX workbooks and both PDFs; download five public
workbook layouts with a provenance manifest. Record findings separately.
- [x] Run a bounded PDF guessing experiment over all 17 supplied pages using
text coordinates and vector table borders. Record recovery and validator
results in the [PDF feasibility trial](rundown-corpus.md#pdf-feasibility-trial).
Pause importer work: column recovery succeeds, but missing durations and
simultaneous tracks prevent dependable whole-document timer conversion.
- [x] Add bounded XLSX and UTF-8 CSV/TSV readers, synthetic regression fixtures,
merged-block handling and source-located formula errors.
- [x] Verify exported event times and a 90-minute gap through the Ontime 4.15.1

View File

@ -77,7 +77,10 @@ Processing is local. XLSX preserves merged ranges and cached formula values;
formulas and macros are never executed. Recalculate and save formula-based
workbooks before importing. The converter cannot determine whether cached values
are stale. CSV/TSV uses UTF-8, quoted fields and comma/semicolon/tab detection.
Legacy XLS, ODS, Word, PDF and scans are not yet supported. Input files are limited
Legacy XLS, ODS, Word, PDF and scans are not yet supported. A
[PDF feasibility trial](rundown-corpus.md#pdf-feasibility-trial) recovered table
text from the supplied PDFs but could not establish a complete timer schedule;
PDF importer work is paused for now. Input files are limited
to 32 MiB; XLSX expanded content is limited to 128 MiB.
## Timing and breaks

View File

@ -26,6 +26,82 @@ do not reproduce those details. An unchanged local file can be inspected with
`rundown-converter inspect INPUT --json`; this output contains the original cells,
so treat it like the source document.
## PDF feasibility trial
A bounded follow-up trial attempted automatic table recovery from all 17 pages
of the two supplied PDFs. Result: useful text/table recovery, but no dependable
whole-document timer conversion. PDF importer work is paused; the application
continues to return `PDF_REQUIRES_REVIEW`. This is a finding about these samples,
not evidence that all PDF rundowns are unsuitable.
### Extraction attempts
First compare `pdftotext -layout` with word coordinates from
`pdftotext -bbox-layout`. Plain reading order sometimes joins a clock and its
activity, while multiline cells and vertically centred text prevent reliable
row detection from baselines alone. The rendered first page of each document
was inspected to compare the recovered coordinates with the visible grid.
Then use `pdftocairo -svg` to recover the table borders. In these files the borders
are thin filled rectangles. A temporary Python standard-library prototype
clusters their coordinates into column boundaries and row bands, assigns words
by their bounding-box centres, and records missing horizontal borders as possible
vertical cell spans. This is specific to ruled, text-based PDFs; it does not
establish support for scans, borderless tables or other PDF drawing conventions.
The first geometry pass lost a column on two Montage continuation pages where
the left border was absent. It also missed text above the first visible border
on some pages. A second pass aligns column boundaries across the document and
extends the outer row bounds to include body text. It recovers four columns for
Montage and five for Exploitation on every page.
| Measurement | Montage | Exploitation/démontage |
| --- | --- | --- |
| Pages | 4 | 13 |
| Nonempty geometric row bands | 33 | 123 |
| Body words assigned to cells | 714 | 2,645 |
| Repeated page-heading words excluded | 16 | 39 |
| Populated time cells | 13 | 46 |
| Explicit start/end ranges | 1 | 15 |
| Start-only cells | 9 | 31 |
| Other time-cell values | `TBC`, `01H/2H`, `2H MAX` | None |
All non-heading words were assigned in this second pass. That is extraction
coverage, **not** proof that every cell boundary, inherited start or event grouping
is correct. Row bands are geometric subdivisions, not a count of semantic events.
Missing grid borders and cross-page continuations still need interpretation.
### Timing experiment and stop decision
For a deliberately generous trial, write the recovered rows to temporary CSVs
with guessed Start/Title/Owner/Location/Notes headers and enable the existing
`untimed_as_notes` policy. No start or duration is filled from the next row.
Keep page/bounding-box and possible-merge evidence in separate temporary JSON;
this CSV trial does not resolve or automatically apply those ambiguous spans.
Montage stops at column validation on its three unresolved time-cell values.
Exploitation reaches IR construction but reports 31 missing durations. Its 12
orphan-note diagnostics are downstream consequences of unresolved preceding
events, not 12 additional extraction failures. A shared time cell can cover
several simultaneous tasks; treating them as consecutive cues is unjustified.
As a diagnostic only, test the explicit-range subset separately. Montage's single
range validates, but that excludes almost the entire document. Exploitation's
15 ranges yield 14 candidate timers and one untimed break, with six overlapping
predecessor pairs and three gap-overlap diagnostics. This subset also omits the
31 start-only entries, so it is not an acceptable converted rundown even if
those conflicts were removed. **No partial Ontime project was published.**
The remaining obstacles are semantic: selecting simultaneous tracks, deciding
whether untimed rows are instructions or events, distinguishing arrivals and
cutoff markers from timers, and interpreting uncertain overnight deadlines.
Neither a following start nor a missing table border supplies that information.
Per the requested stop condition, no experimental reader or new runtime dependency
was added. Revisit with a sequential PDF containing explicit durations/end times,
or after adding explicit track and row-role selection to the neutral model.
Temporary extractions and the prototype stay outside Git; no private PDF content
or contact details are redistributed.
## Public examples
| Source | Result and useful variation |