Skip to content

Optimize large S3 dataset listings - #416

Open
robinskil wants to merge 26 commits into
mainfrom
features/optimize-large-listings-s3
Open

robinskil wants to merge 26 commits into
mainfrom
features/optimize-large-listings-s3

Merge branch 'main' into features/optimize-large-listings-s3

8c75f2d
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded Sep 4, 2026 in 0s

83.27% (+0.03%) compared to 294e328

View this Pull Request on Codecov

83.27% (+0.03%) compared to 294e328

Details

Codecov Report

❌ Patch coverage is 85.61439% with 144 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.27%. Comparing base (294e328) to head (8c75f2d).

Files with missing lines Patch % Lines
...acon-db/beacon-datafusion-ext/src/big_page_list.rs 85.57% 46 Missing ⚠️
beacon-server/beacon-server/src/server/catalog.rs 54.54% 20 Missing ⚠️
...n-db/beacon-functions/src/listing/list_datasets.rs 69.84% 19 Missing ⚠️
...on-db/beacon-datafusion-ext/src/listing_factory.rs 90.90% 17 Missing ⚠️
...db/beacon-functions/src/listing/browse_datasets.rs 51.51% 16 Missing ⚠️
beacon-db/beacon-functions/src/listing/provider.rs 89.07% 13 Missing ⚠️
beacon-db/beacon-functions/src/listing/exec.rs 93.10% 8 Missing ⚠️
beacon-db/beacon-functions/src/listing/args.rs 80.00% 3 Missing ⚠️
.../beacon-datafusion-ext/src/listing_url_resolver.rs 95.23% 1 Missing ⚠️
beacon-server/beacon-server/src/server/mod.rs 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #416      +/-   ##
==========================================
+ Coverage   83.23%   83.27%   +0.03%     
==========================================
  Files         373      378       +5     
  Lines       62526    63354     +828     
==========================================
+ Hits        52044    52757     +713     
- Misses      10482    10597     +115     
Files with missing lines Coverage Δ
beacon-db/beacon-core/src/crawler/engine.rs 87.35% <ø> (ø)
beacon-db/beacon-core/src/file_stats.rs 95.33% <100.00%> (ø)
beacon-db/beacon-core/src/runtime_builder.rs 92.22% <100.00%> (-0.02%) ⬇️
beacon-db/beacon-datafusion-ext/src/format_ext.rs 95.88% <100.00%> (+0.19%) ⬆️
...beacon-datafusion-ext/src/object_store_registry.rs 79.89% <100.00%> (ø)
beacon-db/beacon-db-py/src/connection.rs 0.00% <ø> (ø)
...le-formats/beacon-arrow-zarr/src/datafusion/mod.rs 92.53% <100.00%> (-0.04%) ⬇️
.../beacon-file-formats/beacon-arrow-zarr/src/util.rs 93.65% <100.00%> (+0.68%) ⬆️
beacon-db/beacon-functions/src/file_formats/mod.rs 100.00% <100.00%> (ø)
beacon-db/beacon-functions/src/lib.rs 100.00% <ø> (ø)
... and 13 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.