Repository navigation
Enable STOFS Pacific System - #74
Open
SorooshMani-NOAA wants to merge 4 commits into
Open
SorooshMani-NOAA wants to merge 4 commits into
SorooshMani-NOAA wants to merge 4 commits into
Conversation
SorooshMani-NOAA
changed the title
Collaborator
Author
|
I'd like to note here that in ops v3.1 Pacific system ADT processing Pacific workflow sets a scale factor of 0.001, where in Atlantic we have 1. I hope this don't cause problem in our ADT processing here. |
This commit introduces the necessary structural changes to support the
Pacific domain's operational processing requirements attempting to avoid
breaking Atlantic parity.
* nos_utils/config.py:
- Decouple time steps by adding explicit `obc_dt` and `nudging_dt`.
- Add config flags for `obc_time_smoothing` and `nudging_time_smoothing`
- Add `rtofs_daily_3z` flag to support variable-split daily 3D files.
- Expose `obc_min_timesteps` to be configurable via YAML.
* nos_utils/coords.py:
- Add `shift_grid_0360` utility to cleanly stitch and sort global
[-180, 180] grids across the antimeridian for Pacific subsets.
* nos_utils/forcing/adt.py:
- Remove hardcoded Atlantic domain bounding boxes (`ADT_LON_MIN`, etc.).
- Dynamically read destination bounds from `self.config.domain`.
- Apply `shift_grid_0360` during `_subset_ops_roi` and `_read_adt` to
seamlessly support 0-360 meshes.
* nos_utils/forcing/rtofs.py & nos_utils/forcing/nudging.py:
- Replace hardcoded 21600.0s time steps with `config.obc_dt` and
`config.nudging_dt`.
- Add `groupby` logic to process variable-split daily files (`t3z`,
`s3z`) simultaneously so the time axis is not corrupted.
- Implement a configurable 2-step temporal moving average
`(data[:-1] + data[1:]) / 2` for `elev2D`, `TEM_3D`, `SAL_3D`,
and nudging arrays to match ops Pacific time smoothing.
- Use dynamically configured `obc_min_timesteps` for the phase plan
and QC checks.
Adds the Surface Restore processor to replicate the operational STOFS-3D-PAC surface temperature and salinity relaxation pipeline, and refactors orchestrator archiving to dynamically reflect configuration toggles. * nos_utils/forcing/surface_restore.py: - Create `SurfaceRestoreProcessor` to extract and interpolate RTOFS surface T/S to target nodes. - Implement a configurable 2-step temporal moving average for surface restore outputs. - Write results to standard `surface_restore.nc`. * nos_utils/config.py: - Add `surface_restore_enabled`, `surface_restore_time_smoothing`, and `surface_restore_weight_file` to `ForcingConfig`. * nos_utils/orchestrator.py: - Integrate `_run_surface_restore` into the main prep pipeline. - Add `surface_restore.nc` to the `_OPS_RUN_INPUTS` completeness check. - Refactor `_OBC_ARCHIVE_NAMES` into a dynamic `@property` to automatically include nudging and surface restore files in QA/QC gates and fallback staging when enabled. - Update tarball generation blocks (`payload_files`, `obc_files`) to dynamically read from `_OBC_ARCHIVE_NAMES.keys()`.
Comment on lines
+1
to
+10
| """ | ||
| Surface Restore processor for SCHISM. | ||
|
|
||
| Interpolates RTOFS surface temperature and salinity to boundary/restore nodes | ||
| and applies a 2-step temporal moving average, replicating the ops pipeline: | ||
| ncks -O -d time,0,3 ... ncks -O -d time,1,4 ... ncecat ... ncwa | ||
|
|
||
| Output: | ||
| - surface_restore.nc | ||
| """ |
Collaborator
Author
There was a problem hiding this comment.
Not tested yet, needs unit testing
| "uv3D.th.nc": "uv3dth", | ||
| } | ||
| @property | ||
| def _OBC_ARCHIVE_NAMES(self) -> Dict[str, str]: |
Collaborator
Author
There was a problem hiding this comment.
@mansurjisan I'm not sure if this was the best idea, please let me know what you think?
| "elev2D.th.nc", "TEM_3D.th.nc", "SAL_3D.th.nc", | ||
| "uv3D.th.nc", "TEM_nu.nc", "SAL_nu.nc", | ||
| ], | ||
| "payload_files": list(self._OBC_ARCHIVE_NAMES.keys()), |
Collaborator
Author
There was a problem hiding this comment.
@mansurjisan to make the list dynamic I just get it from the archive names property, please let me know if that makes sense
| obc_files = ["elev2D.th.nc", "TEM_3D.th.nc", "SAL_3D.th.nc", "uv3D.th.nc", | ||
| "TEM_nu.nc", "SAL_nu.nc"] | ||
| # each with all active files (boundary + nudging + surface restore). | ||
| obc_files = list(self._OBC_ARCHIVE_NAMES.keys()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Make changes so that Pacific system inputs can be processed
Type of change
Change characteristics
How has this been tested?
not tested yet
Checklist