Skip to content

adds cbrfc_wsv_esp data to local warehouse for crmms test locations - #67

Open
samland1116 wants to merge 2 commits into
mainfrom
20-add-cbrfc-wsv-esp-to-warehouse
Open

adds cbrfc_wsv_esp data to local warehouse for crmms test locations#67
samland1116 wants to merge 2 commits into
mainfrom
20-add-cbrfc-wsv-esp-to-warehouse

Conversation

@samland1116

Copy link
Copy Markdown
Collaborator
  • Restructures the warehouse/local dir to include a utils folder to store the setup_utils.py and any other ingest specific utils. These utils will move to a prefect workflow but are stored here temporarily to avoid blocking progress on frontend-development

  • CBRFC WSV ESP Test Data:

    • adds notebook setup for CBRFC water supply volume ESP forecasts for three CRMMS locations where we had a crosswalk
    • queries the CBRFC forecast data for an archived period to compare to USGS volumes/NWM-retrospective volumes for an overlapping period (2016-01-01 to 2022-11-30)

Will transition into prefect workflow as opposed to this scratch-work implementation once we have the full locations list

Closes #20

@samland1116 samland1116 self-assigned this Aug 20, 2026
@samland1116 samland1116 added the enhancement New feature or request label Aug 20, 2026
@samland1116 samland1116 linked an issue Aug 20, 2026 that may be closed by this pull request
@mgdenno

mgdenno commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Mostly same comments at #68

Overall I think we need to coordinate the secondary IDs, crosswalks and variable names.

I also think we want to keep things split up so domain adding code separate from fetching code, etc. but we can evolve the process as we go as long as we commit to doing so and continuing to refactor.

Just a note - don't need to address here: Some of this code loading the domains, locations, crosswalks, etc. is really data migrations (vs database schema migrations) - ultimately, we probably need to figure out a better pattern for handling "data migrations" which seems to be a pain point across many projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add CBRFC WSV ESP to warehouse

2 participants