[mansurjisan]

Description

The STOFS-2D-Global ops-mode surface forcing builder (build_sfcf_forcing) can now read the public GFS flux files (gfs.tHHz.sfluxgrbfFFF.grib2), not only the native-grid gfs.tHHz.sfcfFFF.nc files. The GFS sfcf netCDF files are public only to f012, so a machine without the operational GFS COM could not build the 180 h forecast forcing. The flux files are on AWS (noaa-gfs-bdp-pds) hourly to f120, then 3-hourly to f384, back to 2021-04. They use the same native Gaussian grid and hold the four fields: PRES:surface, UGRD/VGRD at 10 m, and ICEC:surface.

• build_sfcf_forcing(..., source="sfcf"|"sflux"). The default sfcf is unchanged: its output is byte-identical to 5496da0.
  • sflux wraps each decoded GRIB file so it looks like an sfcf dataset. The existing build loop, windows, getges search and _interp_bracket therefore run unchanged.
  • The output has the sfcf variable and global attributes and types, read from real sfcf files.
• sflux_path, SFLUX_FIELDS (the wgrib2 match strings and the sfcf names).
• gfs_input_files(comin_gfs, cycle, segment, start=None, source=...) lists the exact files the builder reads, for a downloader or a preflight. For a 12z ncst: 06z f000-f005 and 12z f000. For fcst: f000-f120 hourly and both brackets of the interpolated hours past f120.
• Decoding is one wgrib2 call per file: -order raw -no_header, an -if/-bin/-fi block per field, and -grid.
  • Latitudes come from Gaussian quadrature and longitudes from the grid header, with no flip: GRIB and sfcf both run north to south, lon 0-360.
  • The decode refuses any scan order other than WE:NS/WE:SN and times out after 300 s.
  • A two-entry cache decodes each file once per build.
• ICEC has no bitmap in the flux files; if one ever appears, icec is set to 0.0 there, as land is in sfcf. Any other field with undefined points raises.
• Readiness: a flux file is ready when it is non-empty and its .idx exists, or when it is older than the settle time; the getges search skips flux files that are not ready, so a half-downloaded file never wins.
• The files read are reported through log_input_files (GFS_SFCF / GFS_SFLUX, category atmospheric), as the other forcing processors do, for both sources.

Type of change

• New feature (non-breaking change that adds functionality)

Change characteristics

• Changes outputs (values, COMOUT files or names, archives)? NO for the default source="sfcf". source="sflux" is new and opt-in.
• Breaking change to existing functionality? NO.
• Requires a nos-utils submodule bump (nos-workflow) or a paired nos-workflow change (nos-utils)? YES: the nos-workflow PR that adds the Hercules download and ADCIRC_GFS_SOURCE bumps to this. Its ops-copy staging helper needs ncattrs/getncattr on its dataset shim, which that PR adds.
• Requires new or regenerated fix files on the machine? NO.
• Requires a re-prep or a new hotstart before the next cycle? NO.
• Machines affected: both (the builder is shared; flux files are used where sfcf is not available).

How has this been tested?

• pytest: 1298 passed, 9 skipped, rc 0. The new tests cover:

  • the sflux build matching the sfcf build in structure and values;
  • the brackets;
  • a contract test that fills a tank with exactly gfs_input_files and checks the manifest equals it, for ncst and fcst and both sources;
  • orientation, the icec rule, the exact attributes and types;
  • the missing-file and missing-wgrib2 errors;
  • the scan-order and timeout errors;
  • the decode cache and readiness.

• Real files, 20261012 12z:

  • The grid is the same 3072 x 1536 as sfcf. Latitudes match sfcf to 8.5e-14 deg, longitudes exactly.
  • The real _decode_sflux with wgrib2 2.0.8 and 3.7.0 is bitwise identical to a pygrib decode, and the field means equal wgrib2's own -stats.

• End to end against the operational ncst forcing for 20261012 12z (stofs_2d_glo_ncst.{221,222,225}.nc from the public NCO bucket):

  • The structure is the same: dims, variables, dtypes, attributes, chunking. The only difference is the NCO history/NCO global attributes.
  • Values are not bitwise, because GRIB2 packing differs from the netCDF sfcf.

  Field	max |diff|	mean |diff|
  10 m wind	0.005 m/s (0.05 m/s in the coarsely packed 06z f000-f002 files)	0.0025 m/s
  icec	5e-4	1.3e-5
  surface pressure	270 Pa near steep terrain (terrain-correlated)	2.9 Pa; 0.7 Pa over ocean

• A full 181-record forecast build: 581 s and 1.2 GB peak memory.

• Not yet run on Hercules or WCOSS2.

Checklist

• Dependent PRs merged and published (nos-utils before any submodule bump): none; this is the nos-utils side
• pytest suite passes
• Existing operational behavior preserved, or the change is called out above
• No edits to a shared FIXofs while it is cycling
• Self-review done; yaml comments and docstrings updated where behavior changed
• No markdown files added to the repo (this template is the only exception)

[Copilot]

🟡 Changes recommended

Sflux fallback can decode incomplete files, and consumed flux inputs bypass centralized prep input capture.

2 open findings

• Require sflux candidates to pass readiness checks before decoding · New
• Log de-duplicated sflux GRIB inputs in the input manifest · New

What changed in this PR

Adds opt-in public GFS flux-file support for STOFS-2D-Global surface forcing while retaining the existing sfcf default.

Changes:

• Adds sflux path discovery, readiness checks, decoding, caching, and dataset adaptation.
• Adds exact GFS input-file enumeration and source-aware manifests.
• Adds comprehensive sflux behavior and compatibility tests.

File	Description
nos_utils/​forcing/​adcirc_met.py	Implements sflux forcing support and input discovery.
tests/​test_adcirc_met.py	Tests decoding, structure, interpolation, readiness, caching, and errors.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.