Skip to content

Close native formula coverage gaps and measure the frozen corpus - #85

Merged
tcballard merged 9 commits into
mainfrom
codex/formula-coverage
Sep 11, 2026
Merged

tcballard merged 9 commits into
mainfrom
codex/formula-coverage

Conversation

@tcballard

@tcballard tcballard commented Sep 8, 2026

Copy link
Copy Markdown
Owner

OmaSheets’ native formula coverage was 89.26% on the frozen 1,000-workbook corpus. This PR closes ordinary syntax and function gaps and records a fresh before/after measurement without changing the sample, tolerance, or denominator.

It adds bounded array constants for aggregates and lookups, sheet-qualified defined names, known modern function prefixes, qualified/deleted range endpoints, and eight financial/statistical function names. The registry now contains 104 names. Tests cover reference values, errors, cached XLSX values, recalculation, and stable bindings through sorting and reopening.

Frozen corpus metric Base ecf0036 Implementation 12d1c8c
Workbooks opened 996 / 1,000 996 / 1,000
Formula denominator 924,235 924,235
Compiled and compared 825,016 (89.26%) 829,529 (89.75%)
Stored values matched 823,932 828,437
Stored values mismatched 1,084 1,092
Match rate of compared 99.87% 99.87%

That is 4,513 more compiled formulas and 4,505 more stored-value matches. The eight additional mismatches remain reported. Aggregate results cannot establish that every previously matched formula is unchanged.

The new workflow builds baseline and candidate in separate directories and scores them sequentially on one runner. It retains aggregate evidence and fixed token-class diagnostics only. The frozen corpus comparison passed; summary and delta are committed under corpus/sources/ with the scored engine revision. An earlier workflow revision incorrectly reused the baseline binary; its candidate result is superseded and is not used in the committed evidence.

All 12 implementation CI jobs passed, including 151 Rust tests, workspace clippy with warnings denied, Qt checks and both Arch installation/acceptance jobs. The final commit changes documentation and aggregate evidence only; its six evidence-validation tests passed locally, and GitHub has started the automatic checks again.

The remaining 94,706 refusals include 47,007 external workbook references and 21,737 proprietary add-in calls. Parsing farther also exposes OFFSET and invalid-reference failures that were previously classified as syntax/name errors. These counts remain in raw coverage. The 97% release gate is unchanged and is not marked passed.

@tcballard
tcballard marked this pull request as ready for review September 11, 2026 10:15
@tcballard
tcballard merged commit 93fc27b into main Sep 11, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant