Skip to content

Record spans in the event API; recovery keeps the record after a missing ; (0.11.4) - #21

Merged
GeneralPawz merged 3 commits into
mainfrom
fix/recovery-spans-and-swallowed-ids
Nov 6, 2026
Merged

GeneralPawz merged 3 commits into
mainfrom
fix/recovery-spans-and-swallowed-ids

Conversation

@GeneralPawz

Copy link
Copy Markdown
Contributor

Closes #18. Closes #20.

#18. Adds a provided method, EventSink::data_record(record, span). The parser calls it for each data record with the span from # through ;. The default forwards to event(Event::DataRecord(record)), so existing sinks and closures keep working.

#20. Recovery now also stops before the next #id = outside a literal, so a record missing its ; no longer swallows the next record. That record is parsed and kept, which recovers more than listing the swallowed ids would. It takes the issue's first option, with one difference: no whitespace is needed before the #, because #1=A(1)#2=B(); loses #2 in the same way. a_skip_over_several_records_names_only_the_first is replaced accordingly.

Lenient output changes for such files: more records, the same or more diagnostics, and shorter spans. With the reference check on, a duplicate after the skip is now reported, which is the first repro in openbimrs/ifc#668. The second repro also names the swallowed #1 now, because that record is kept.

Remaining limits, documented on Diagnostic::instance and pinned by a test. Both stay inside the skip's span:

  • a following record whose own = is missing;
  • a record that an unbalanced quote turned into string text, which cannot be told from a string that really holds record syntax.

record_head_at reads comments as resync_from does rather than calling the lexer. The two disagree on a comment opener split by a control, and with the lexer a crafted input made every # scan to the end of the file. A time-bounded test covers this: with the lexer-based check it was still running at 150 s.

Verification

  • scripts/gate.sh passes; cargo semver-checks against 0.11.3 reports no semver update required.
  • Parallel equals sequential with records missing their ; (three separators), and with each ; removed in turn from the trap file.
  • Lenient record and diagnostic counts match main on 626 local files, 5 of which have skips.
  • examples/recovery_property: 0 fabricated records, 0 span faults.
  • Callgrind, strict event parse of a 3 MB file: 221,029,873 → 221,055,449 instructions (+0.01%).

A consumer of the event API could not say where a record it rejects sits
in the file: a record the IFC model cannot represent was reported without
a position (openbimrs/ifc#614, #616). The parser already computes the span
for the reference check. A provided method hands it over without changing
any sink that only implements `event`.

Closes #18.
Recovery resynchronized at the next `;`, so a record missing its `;` took
the following record with it, and no diagnostic named that record. A
consumer that reserves skipped ids could reuse the id, and a conflicting
duplicate went unreported (openbimrs/ifc#668).

The scan now also stops before a `#id =` outside a literal, so the next
record is parsed and kept, which recovers more than listing the swallowed
ids would. It reads comments the way the scan does rather than through the
lexer: the two disagree on a comment opener split by a control, and that
made every `#` look ahead to the end of the file.

Closes #20.
@GeneralPawz
GeneralPawz merged commit 33f802b into main Nov 6, 2026
2 checks passed
@GeneralPawz
GeneralPawz deleted the fix/recovery-spans-and-swallowed-ids branch November 6, 2026 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report every record id swallowed by one skipped-record recovery Byte spans of data records in the event API

1 participant