Skip to content

test: port legacy reader workloads to the modern generator - #62

Merged
rliao147 merged 1 commit into
mainfrom
stack/port-legacy-dat-tests
Sep 28, 2026
Merged

rliao147 merged 1 commit into
mainfrom
stack/port-legacy-dat-tests

Conversation

@rliao147

@rliao147 rliao147 commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This change retires the legacy Python DAT reader generator and use the modern Scala workload generator as the source of public DAT workloads. Most legacy scenarios were already covered by modern suites. This adds the missing checkpoint and protocol workloads:

  • CheckpointsSuite.prior_commits_removed
  • CheckpointsSuite.struct_stats_only
  • CheckpointsSuite.stats_disabled
  • ProtocolVersionsSuite.iceberg_compat_v1

The legacy Python generator, PySpark reader test, Python-only dependencies, and obsolete CI commands are removed.

How was this patch tested?

Self-validation for all newly added workloads

Does this require an update to the documentation?

No

@rliao147
rliao147 requested review from OussamaSaoudi and removed request for OussamaSaoudi September 26, 2026 00:21
@rliao147
rliao147 force-pushed the stack/port-legacy-dat-tests branch from 6fd294d to deca116 Compare September 26, 2026 00:51
@rliao147
rliao147 merged commit 5173797 into main Sep 28, 2026
1 check passed
@rliao147
rliao147 deleted the stack/port-legacy-dat-tests branch September 28, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants