How should headless spreadsheet engines handle stale XLSX cached values?

[gregkonush]

When people test a headless spreadsheet engine against XLSX files, it is easy to compare against the cached formula values embedded in the file.

That cache is useful, but it can be stale. It should not be treated as the truth for an accuracy claim.

The rule I am using for @bilig/headless now is stricter:

• cache comparison is a diagnostic only
• a Bilig accuracy bug needs a fresh recalculation oracle
• the preferred oracle is Microsoft Excel: open the workbook, force full recalculation, save a recalculated copy, then compare Bilig against that copy
• if Excel is unavailable or Excel rewrites the original formula into an unsupported UDF wrapper, the result is marked missing_excel_oracle instead of being promoted to a bug

The harness commands are:

OUT=.cache/excel-oracle-evaluation
pnpm workpaper:xlsx-oracle -- prepare-oracle /path/to/workbooks "$OUT"
pnpm workpaper:xlsx-oracle -- evaluate-cache /path/to/workbooks "$OUT"
pnpm workpaper:xlsx-oracle -- evaluate-oracle /path/to/workbooks "$OUT/recalculated" "$OUT"
pnpm workpaper:xlsx-oracle -- summarize "$OUT"

Docs: https://proompteng.github.io/bilig/xlsx-corpus-verifier-walkthrough.html
Source: https://github.com/proompteng/bilig/blob/main/docs/xlsx-corpus-verifier-walkthrough.md

I would like feedback from people who evaluate spreadsheet engines or workbook automation libraries:

1. Is Microsoft Excel recalculation the right default oracle for this kind of harness?
2. What should the report include before you would trust a formula mismatch enough to open a bug?
3. Are there workbook cases where cache-only diagnostics are still useful enough to keep separate?
4. What private-corpus workflow would make this practical in CI without leaking customer workbook names, paths, or formulas?

The goal is not to claim blanket Excel compatibility. The goal is to stop stale caches from producing fake wins or fake bugs.

[gregkonush]

Update: @bilig/headless@0.14.9 is now published with the cache/oracle distinction in the npm README.

The short version is unchanged: embedded XLSX cached formula values are useful triage data, but not an accuracy verdict. For a correctness claim, the expected value should come from a fresh Microsoft Excel recalculation, and workbooks without that oracle should stay labeled missing_excel_oracle.

Useful links:

• npm package: https://www.npmjs.com/package/@bilig/headless
• Excel oracle harness docs: https://github.com/proompteng/bilig/blob/main/docs/xlsx-corpus-verifier-walkthrough.md#run-the-excel-oracle-harness
• release: https://github.com/proompteng/bilig/releases/tag/libraries-v0.14.9

If you have a workbook corpus where cached values and recalculated Excel values disagree, sanitized examples would help tune the report format and issue templates.

[gregkonush]

Update, May 14: @bilig/headless@0.14.11 is now live on npm with the rewritten README and the cache/oracle policy in the package page itself.

The current split is:

• evaluate-cache writes cache-diagnostic.json and stays non-authoritative.
• prepare-oracle uses Excel automation when available and writes recalculated copies without touching originals.
• evaluate-oracle writes excel-oracle-report.json; only those fresh Excel values can support a Bilig accuracy bug.
• summarize writes summary.md plus sanitized issue drafts for true Excel-oracle mismatches.

Links:

• npm README: https://www.npmjs.com/package/@bilig/headless
• harness walkthrough: https://proompteng.github.io/bilig/xlsx-corpus-verifier-walkthrough.html
• source docs: https://github.com/proompteng/bilig/blob/main/docs/xlsx-corpus-verifier-walkthrough.md#run-the-excel-oracle-harness
• contributor starter for a sanitized oracle summary sample: docs(xlsx): add a sanitized Excel oracle summary sample #383

The part I still want feedback on: what fields should be mandatory in a public issue before a formula mismatch is credible enough to call a library bug? Right now I am treating formula text, fresh Excel expected value, Bilig actual value, workbook context, and repro notes as the minimum.