feat: add cadcat_preview catalog for sup3rcc preview datasets#757
Draft
neilSchroeder wants to merge 1 commit into
Draft
feat: add cadcat_preview catalog for sup3rcc preview datasets#757neilSchroeder wants to merge 1 commit into
neilSchroeder wants to merge 1 commit into
Conversation
Adds a new 'cadcat_preview' catalog alongside cadcat, renewables, and hdp for internal/pre-release datasets. First member is sup3rcc (NREL super- resolved 4km CONUS data; EC-Earth3-CC historical + ssp245, 13 variables). - New CATALOG_CADCAT_PREVIEW constant and CADCAT_PREVIEW_CATALOG_URL path - Bundled intake-esm collection (cae-preview-collection.json + cae-preview-zarr.csv) under climakitae/data/; resolved via _package_file_path at load time. The URL may later be flipped to an s3:// URL so AWS credentials gate access. - DataCatalog loads preview with silent failure (logger.debug) so users without access to internal data see no warnings; adds .preview property and merge_catalogs inclusion mirroring HDP/renewables patterns. - New CadcatPreviewValidator opts out of warming_level, bias_adjust_model_to_station, and filter_unadjusted_models processors (cadcat/WRF/LOCA2-specific). Uses the same activity_id/table_id/grid_label/variable_id required keys. - variable_descriptions.csv: 9 sup3r-unique vars (swddif, swddni, swdnb, wd10/100/200, ws10/100/200). Shared vars (t2, prec, psfc, rh) reuse existing Dynamical rows via first-match lookup semantics. - 15 new validator tests + updated data_access tests for the 4th catalog.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes and related issue
Adds a new 'cadcat_preview' catalog alongside cadcat, renewables, and hdp for internal/pre-release datasets. First member is sup3rcc (NREL super- resolved 4km CONUS data; EC-Earth3-CC historical + ssp245, 13 variables).
Testing
How to Test
sup3r_preview_demo.ipynb
Download this notebook ^, checkout this branch and install locally with
pip install -e ., then play around with loading and manipulating the Sup3rCC dataset withclimakitae.Documentation
Code Quality
_Review Process
Administrative Reminders