Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 29 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,50 @@

[![Docs](https://readthedocs.org/projects/nos-utils/badge/?version=latest)](https://nos-utils.readthedocs.io/en/latest/)

Python forcing generators for NOAA National Ocean Service's operational ocean forecast systems.
nos-utils is a Python package that generates the forcing and run-configuration files for NOAA National Ocean Service coastal forecast systems, and some of their post-processing products. It replaces parts of the COMF and STOFS Fortran/shell prep chains with testable Python modules. [nos-workflow](https://github.com/mansurjisan/nos-workflow) uses it as a submodule (`ush/python/nos-utils`) to run the prep and post stages.

**Documentation**: [nos-utils.readthedocs.io](https://nos-utils.readthedocs.io/en/latest/)
## Systems

## Install
| System | Model | What nos-utils provides |
|---|---|---|
| SECOFS | SCHISM, DATM+SCHISM, DATM+SCHISM+WW3 (UFS-Coastal) | sflux or DATM atmospheric forcing, RTOFS open boundary, river, tides, nudging, param.nml, UFS-Coastal configuration; for WW3, the GFS-Wave `nest.ww3` boundary and WAV PET bounds / coupling interval |
| STOFS-3D-ATL | SCHISM standalone, DATM+SCHISM; ported to match operational v3.1 | the same set plus ADT and tide-gauge SSH adjustment, NWM and St. Lawrence rivers, and the post products |
| STOFS-2D-Global | ADCIRC | GFS meteorological forcing in ADCIRC OWI NetCDF form (NWS=14) |

## Install, test, docs

```bash
pip install -e ".[full]" # numpy, netCDF4, scipy
pip install -e ".[full,dev]" # full: netCDF4, scipy, cfgrib, pyyaml; dev: pytest
pytest # unit tests; synthetic fixtures, no data downloads
```

## Usage
Documentation: [nos-utils.readthedocs.io](https://nos-utils.readthedocs.io/en/latest/). Example:

```python
from nos_utils.config import ForcingConfig
from nos_utils.forcing import GFSProcessor

config = ForcingConfig.for_secofs(pdy="20260324", cyc=12)
gfs = GFSProcessor(config, input_path="/data/gfs/v16.3", output_path="/data/sflux")
result = gfs.process()
result = GFSProcessor(config, input_path="/data/gfs", output_path="/work/sflux").process()
```

CLI:
```bash
nos-utils prep --ofs secofs --pdy 20260324 --cyc 12 --gfs /data/gfs --output /work/
```
A prep cycle runs through `PrepOrchestrator(config, paths, skip_legacy=False)`, which nos-workflow reaches via `nco_bridge.run_prep(skip_legacy=False)`. The `nos-utils prep` CLI (see `nos-utils --help`) keeps the default `skip_legacy=True`, which skips the SECOFS RTOFS and NWM steps.

## Processors
## Main modules

| Processor | Input | Output |
|-----------|-------|--------|
| GFSProcessor | GFS 0.25° GRIB2 | sflux or DATM |
| HRRRProcessor | HRRR 3km GRIB2 | sflux (secondary) |
| GEFSProcessor | GEFS ensemble GRIB2 | sflux per member |
| RTOFSProcessor | RTOFS NetCDF | elev2D, TEM/SAL_3D, uv3D |
| NWMProcessor | NWM channel_rt | vsource.th, msource.th |
| TidalProcessor | TPXO9 template | bctides.in |
| ParamNmlProcessor | param.nml template | param.nml |
| HotstartProcessor | restart archive | hotstart.nc |
| PartitionProcessor | hgrid.gr3 | partition.prop |
| ESMFMeshProcessor | forcing grid | esmf_mesh.nc |
| BlenderProcessor | GFS+HRRR sflux | datm_forcing.nc |
| Purpose | Modules in `nos_utils.forcing` |
|---|---|
| Atmospheric | `gfs`, `hrrr`, `gefs`, `blender` (HRRR+GFS), `sflux_writer`, `datm_writer`, `forcing_writer` |
| Ocean boundary | `rtofs`, `adt`, `dynamic_adjust`, `wl_bias`, `obc_ops_interp`, `nudging`, `nudge_ops` |
| River | `nwm`, `river_clim`, `st_lawrence` |
| Tides | `tidal` (bctides.in), `adcirc_tides` (ADCIRC nodal factors) |
| Restart, partition, UFS | `hotstart`, `partition`, `param_nml`, `esmf_mesh`, `ufs_config` |
| Waves | `wave_boundary` (GFS-Wave spectra to WW3 `nest.ww3`) |
| ADCIRC | `adcirc_met` (GFS to OWI NetCDF, fort.22), `adcirc_tides` |
| Post-processing | `nos_utils.post`: `adcirc`, `geopkg`, `maxele`, `profiles`, `slab2d`, `stations` |

## Test
Other packages: `nos_utils.interp` (structured and precomputed-weight interpolation), `nos_utils.io` (GRIB2, NetCDF and SCHISM grid readers), `nos_utils.config` (`ForcingConfig`), `nos_utils.orchestrator`, `nos_utils.nco_bridge` (NCO environment variables), `nos_utils.cli`.

```bash
pytest -v # 206 unit tests, no data needed
```
## Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an "as is" basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.
7 changes: 7 additions & 0 deletions docs/api/coords.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Longitude conventions
=====================

.. automodule:: nos_utils.coords
:members:
:undoc-members:
:show-inheritance:
110 changes: 99 additions & 11 deletions docs/api/forcing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@ Forcing Processors
==================

.. automodule:: nos_utils.forcing
:members:
:undoc-members:
:no-members:

Base classes
------------
Expand Down Expand Up @@ -40,6 +39,14 @@ GEFS
:undoc-members:
:show-inheritance:

HRRR and GFS blender
~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.blender
:members:
:undoc-members:
:show-inheritance:

Ocean boundary
--------------

Expand All @@ -59,6 +66,46 @@ ADT satellite SSH blending
:undoc-members:
:show-inheritance:

Dynamic SSH adjustment
~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.dynamic_adjust
:members:
:undoc-members:
:show-inheritance:

Water-level bias correction
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.wl_bias
:members:
:undoc-members:
:show-inheritance:

Operational RTOFS interpolation (3-D open boundary)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.obc_ops_interp
:members:
:undoc-members:
:show-inheritance:

T/S interior nudging
~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.nudging
:members:
:undoc-members:
:show-inheritance:

Operational nudging records
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.nudge_ops
:members:
:undoc-members:
:show-inheritance:

River
-----

Expand All @@ -78,18 +125,29 @@ River climatology
:undoc-members:
:show-inheritance:

St. Lawrence River
~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.st_lawrence
:members:
:undoc-members:
:show-inheritance:

Tidal
-----

SCHISM bctides.in
~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.tidal
:members:
:undoc-members:
:show-inheritance:

Nudging
-------
ADCIRC tidal parameters
~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.nudging
.. automodule:: nos_utils.forcing.adcirc_tides
:members:
:undoc-members:
:show-inheritance:
Expand Down Expand Up @@ -122,7 +180,7 @@ Partition
:show-inheritance:

UFS-Coastal
------------
-----------

ESMF mesh
~~~~~~~~~
Expand All @@ -132,10 +190,32 @@ ESMF mesh
:undoc-members:
:show-inheritance:

Blender
~~~~~~~
UFS-Coastal configuration files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.blender
.. automodule:: nos_utils.forcing.ufs_config
:members:
:undoc-members:
:show-inheritance:

Waves
-----

GFS-Wave boundary spectra
~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.wave_boundary
:members:
:undoc-members:
:show-inheritance:

ADCIRC (STOFS-2D-Global)
------------------------

GFS to ADCIRC OWI NetCDF
~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.adcirc_met
:members:
:undoc-members:
:show-inheritance:
Expand All @@ -144,17 +224,25 @@ Writers
-------

Sflux writer
~~~~~~~~~~~~~
~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.sflux_writer
:members:
:undoc-members:
:show-inheritance:

DATM writer
~~~~~~~~~~~~
~~~~~~~~~~~

.. automodule:: nos_utils.forcing.datm_writer
:members:
:undoc-members:
:show-inheritance:

GFS and HRRR forcing writer
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. automodule:: nos_utils.forcing.forcing_writer
:members:
:undoc-members:
:show-inheritance:
2 changes: 2 additions & 0 deletions docs/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ API Reference
:maxdepth: 2

config
coords
forcing
post
interp
io
orchestrator
Expand Down
11 changes: 6 additions & 5 deletions docs/api/interp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@ Interpolation
=============

.. automodule:: nos_utils.interp
:members:
:undoc-members:
:no-members:

Overview
--------
Expand All @@ -28,9 +27,11 @@ Precomputed weight workflow

There are two steps:

.. currentmodule:: nos_utils.interp.precomputed_weights

1. **One-time build** (offline, after any grid change): run the Fortran
prep once with ``NOS_EXPORT_WEIGHTS=YES`` to dump ``*_remesh_export.txt``
text files, then convert them to NPZ using :func:`build_ssh_npz`,
text files, then convert them to NPZ using :func:`build_npz`,
:func:`build_nudge_npz`, or :func:`build_3d_npz`. The NPZ is committed
to the FIX directory and keyed to the RTOFS grid via an MD5 checksum.

Expand All @@ -48,12 +49,12 @@ Example
import numpy as np
from pathlib import Path
from nos_utils.interp.precomputed_weights import (
build_ssh_npz,
build_npz,
apply_precomputed_ssh,
)

# --- One-time build (offline) ---
build_ssh_npz(
build_npz(
export_txt=Path("obc_ssh_remesh_export.txt"),
rtofs_lon_2d=rtofs_lon, # (ny, nx) from rtofs_glo_2ds
rtofs_lat_2d=rtofs_lat,
Expand Down
3 changes: 1 addition & 2 deletions docs/api/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@ I/O Utilities
=============

.. automodule:: nos_utils.io
:members:
:undoc-members:
:no-members:

GRIB2 extraction
-----------------
Expand Down
61 changes: 61 additions & 0 deletions docs/api/post.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
Post-processing
===============

.. automodule:: nos_utils.post
:no-members:

ADCIRC-format water level
-------------------------

.. automodule:: nos_utils.post.adcirc
:members:
:undoc-members:
:show-inheritance:

Disturbance GeoPackage
----------------------

.. automodule:: nos_utils.post.geopkg
:members:
:undoc-members:
:show-inheritance:

Maximum water level
-------------------

.. automodule:: nos_utils.post.maxele
:members:
:undoc-members:
:show-inheritance:

Station vertical profiles
-------------------------

.. automodule:: nos_utils.post.profiles
:members:
:undoc-members:
:show-inheritance:

2-D slab fields
---------------

.. automodule:: nos_utils.post.slab2d
:members:
:undoc-members:
:show-inheritance:

Station time series
-------------------

.. automodule:: nos_utils.post.stations
:members:
:undoc-members:
:show-inheritance:

Mesh helpers
------------

.. automodule:: nos_utils.post.mesh
:members:
:undoc-members:
:show-inheritance:
4 changes: 3 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@
"show-inheritance": True,
}
autodoc_member_order = "bysource"
autodoc_mock_imports = ["numpy", "netCDF4", "scipy", "cfgrib", "xarray", "nco", "cdo"]
# numpy is a core dependency and is installed on Read the Docs; mocking it breaks
# module-level numpy arithmetic in forcing.nudge_ops. MJ (10/19/26)
autodoc_mock_imports = ["netCDF4", "scipy", "cfgrib", "xarray", "nco", "cdo"]

# Suppress duplicate object warnings from dataclass fields re-exported via __init__
suppress_warnings = ["duplicate.object.description"]
Expand Down
Loading
Loading