Record PDF rundown guessing trial and pause unreliable import
This commit is contained in:
parent
b0fb3b6996
commit
b5555b2cf2
|
|
@ -237,6 +237,11 @@ errors must remain visible with direct corrective actions.
|
|||
- [x] Add separate tool documentation and link it from the repository overview.
|
||||
- [x] Inspect both supplied XLSX workbooks and both PDFs; download five public
|
||||
workbook layouts with a provenance manifest. Record findings separately.
|
||||
- [x] Run a bounded PDF guessing experiment over all 17 supplied pages using
|
||||
text coordinates and vector table borders. Record recovery and validator
|
||||
results in the [PDF feasibility trial](rundown-corpus.md#pdf-feasibility-trial).
|
||||
Pause importer work: column recovery succeeds, but missing durations and
|
||||
simultaneous tracks prevent dependable whole-document timer conversion.
|
||||
- [x] Add bounded XLSX and UTF-8 CSV/TSV readers, synthetic regression fixtures,
|
||||
merged-block handling and source-located formula errors.
|
||||
- [x] Verify exported event times and a 90-minute gap through the Ontime 4.15.1
|
||||
|
|
|
|||
|
|
@ -77,7 +77,10 @@ Processing is local. XLSX preserves merged ranges and cached formula values;
|
|||
formulas and macros are never executed. Recalculate and save formula-based
|
||||
workbooks before importing. The converter cannot determine whether cached values
|
||||
are stale. CSV/TSV uses UTF-8, quoted fields and comma/semicolon/tab detection.
|
||||
Legacy XLS, ODS, Word, PDF and scans are not yet supported. Input files are limited
|
||||
Legacy XLS, ODS, Word, PDF and scans are not yet supported. A
|
||||
[PDF feasibility trial](rundown-corpus.md#pdf-feasibility-trial) recovered table
|
||||
text from the supplied PDFs but could not establish a complete timer schedule;
|
||||
PDF importer work is paused for now. Input files are limited
|
||||
to 32 MiB; XLSX expanded content is limited to 128 MiB.
|
||||
|
||||
## Timing and breaks
|
||||
|
|
|
|||
|
|
@ -26,6 +26,82 @@ do not reproduce those details. An unchanged local file can be inspected with
|
|||
`rundown-converter inspect INPUT --json`; this output contains the original cells,
|
||||
so treat it like the source document.
|
||||
|
||||
## PDF feasibility trial
|
||||
|
||||
A bounded follow-up trial attempted automatic table recovery from all 17 pages
|
||||
of the two supplied PDFs. Result: useful text/table recovery, but no dependable
|
||||
whole-document timer conversion. PDF importer work is paused; the application
|
||||
continues to return `PDF_REQUIRES_REVIEW`. This is a finding about these samples,
|
||||
not evidence that all PDF rundowns are unsuitable.
|
||||
|
||||
### Extraction attempts
|
||||
|
||||
First compare `pdftotext -layout` with word coordinates from
|
||||
`pdftotext -bbox-layout`. Plain reading order sometimes joins a clock and its
|
||||
activity, while multiline cells and vertically centred text prevent reliable
|
||||
row detection from baselines alone. The rendered first page of each document
|
||||
was inspected to compare the recovered coordinates with the visible grid.
|
||||
|
||||
Then use `pdftocairo -svg` to recover the table borders. In these files the borders
|
||||
are thin filled rectangles. A temporary Python standard-library prototype
|
||||
clusters their coordinates into column boundaries and row bands, assigns words
|
||||
by their bounding-box centres, and records missing horizontal borders as possible
|
||||
vertical cell spans. This is specific to ruled, text-based PDFs; it does not
|
||||
establish support for scans, borderless tables or other PDF drawing conventions.
|
||||
|
||||
The first geometry pass lost a column on two Montage continuation pages where
|
||||
the left border was absent. It also missed text above the first visible border
|
||||
on some pages. A second pass aligns column boundaries across the document and
|
||||
extends the outer row bounds to include body text. It recovers four columns for
|
||||
Montage and five for Exploitation on every page.
|
||||
|
||||
| Measurement | Montage | Exploitation/démontage |
|
||||
| --- | --- | --- |
|
||||
| Pages | 4 | 13 |
|
||||
| Nonempty geometric row bands | 33 | 123 |
|
||||
| Body words assigned to cells | 714 | 2,645 |
|
||||
| Repeated page-heading words excluded | 16 | 39 |
|
||||
| Populated time cells | 13 | 46 |
|
||||
| Explicit start/end ranges | 1 | 15 |
|
||||
| Start-only cells | 9 | 31 |
|
||||
| Other time-cell values | `TBC`, `01H/2H`, `2H MAX` | None |
|
||||
|
||||
All non-heading words were assigned in this second pass. That is extraction
|
||||
coverage, **not** proof that every cell boundary, inherited start or event grouping
|
||||
is correct. Row bands are geometric subdivisions, not a count of semantic events.
|
||||
Missing grid borders and cross-page continuations still need interpretation.
|
||||
|
||||
### Timing experiment and stop decision
|
||||
|
||||
For a deliberately generous trial, write the recovered rows to temporary CSVs
|
||||
with guessed Start/Title/Owner/Location/Notes headers and enable the existing
|
||||
`untimed_as_notes` policy. No start or duration is filled from the next row.
|
||||
Keep page/bounding-box and possible-merge evidence in separate temporary JSON;
|
||||
this CSV trial does not resolve or automatically apply those ambiguous spans.
|
||||
|
||||
Montage stops at column validation on its three unresolved time-cell values.
|
||||
Exploitation reaches IR construction but reports 31 missing durations. Its 12
|
||||
orphan-note diagnostics are downstream consequences of unresolved preceding
|
||||
events, not 12 additional extraction failures. A shared time cell can cover
|
||||
several simultaneous tasks; treating them as consecutive cues is unjustified.
|
||||
|
||||
As a diagnostic only, test the explicit-range subset separately. Montage's single
|
||||
range validates, but that excludes almost the entire document. Exploitation's
|
||||
15 ranges yield 14 candidate timers and one untimed break, with six overlapping
|
||||
predecessor pairs and three gap-overlap diagnostics. This subset also omits the
|
||||
31 start-only entries, so it is not an acceptable converted rundown even if
|
||||
those conflicts were removed. **No partial Ontime project was published.**
|
||||
|
||||
The remaining obstacles are semantic: selecting simultaneous tracks, deciding
|
||||
whether untimed rows are instructions or events, distinguishing arrivals and
|
||||
cutoff markers from timers, and interpreting uncertain overnight deadlines.
|
||||
Neither a following start nor a missing table border supplies that information.
|
||||
Per the requested stop condition, no experimental reader or new runtime dependency
|
||||
was added. Revisit with a sequential PDF containing explicit durations/end times,
|
||||
or after adding explicit track and row-role selection to the neutral model.
|
||||
Temporary extractions and the prototype stay outside Git; no private PDF content
|
||||
or contact details are redistributed.
|
||||
|
||||
## Public examples
|
||||
|
||||
| Source | Result and useful variation |
|
||||
|
|
|
|||
Loading…
Reference in New Issue