Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 62 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,68 @@ All notable changes to seisfetch are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
[Semantic Versioning](https://semver.org).

## Unreleased
## 0.4.1 — 2026-08-17

The release NoisePy's obspy-free data path depends on: it makes seisfetch
installable alongside noisepy's pinned S3 stack, and closes the last two gaps
between the numpy pre-processing chain and noisepy's obspy one.

### Added

- `contrib.obspy_ports.gps2dist_azimuth_np`: port of
`obspy.geodetics.base.gps2dist_azimuth` (the Vincenty-inverse branch, the
one that runs in a default install). noisepy's `cc_parameters` writes
dist/azi/baz into every saved cross-correlation, so this was the last
obspy call on the cross-correlation path. Exactly equal to obspy's own
Vincenty on 2000 random coordinate pairs plus station-like, equatorial,
identical-point and longitude-wrapping cases; see the geographiclib note
under Fixed for the one install where "exactly" becomes "to ~5e-6 m".

### Fixed

- `contrib.noisepy_adapter.preprocess_raw_np` now rounds the merged start
time to whole microseconds, as obspy's miniSEED reader does, so noisepy's
sub-sample correction (derived from `UTCDateTime.microsecond`) cannot see a
different value than the obspy chain would. The rounding uses integer
arithmetic: at epoch-nanosecond magnitudes a float quotient carries a
0.25 ulp, which quantizes the sub-microsecond part to 250 ns steps and
picks a different microsecond for ~13 % of arbitrary inputs, and `round()`
is half-to-even besides.

Honest scope: this is a guard, not an observed repair. Scanning 3026
segments across all cached SCEDC, NCEDC and EarthScope day files found
**zero** with a sub-microsecond start — pymseed 0.9.4 reports whole
microseconds on every one, so the line is a no-op on all data available
here and both roundings agree with obspy on every fixture. It costs
nothing and closes the hole if sub-microsecond starts ever appear.
- The `gps2dist_azimuth_np` equivalence tests asserted exact equality against
whichever branch obspy happened to take, so they failed on any machine with
`geographiclib` installed — obspy delegates to it when present and only
falls back to its own Vincenty otherwise. The exact-equality tests now skip
when `HAS_GEOGRAPHICLIB`, and a tolerance test runs either way so coverage
is not silently lost. The two branches differ by ~5e-6 m over ~500 km, and
geographiclib spells a due-south back-azimuth `0.0` where Vincenty says
`360.0`; both are far below what CCF metadata resolves, but it means
bit-identity holds against a default obspy install rather than every one.
- `preprocess_raw_np(..., pretrimmed=False)` skips the sample-grid alignment
guard. The guard exists for callers that pre-trim segments with
`TraceBundle.trim` (inside-window) before pre-processing; a caller that
passes whole segments — exactly what `obspy.read` hands noisepy — is
trimmed only by `trim_pad0_np` at the end of the chain, which IS obspy's
nearest-sample trim, so unaligned windows are handled identically. NoisePy
asserts bit-identity on a window 0.218 samples off the grid.

### Changed

- `boto3` lower bound relaxed to `>=1.26` (was `>=1.28`). NoisePy cannot
install seisfetch otherwise: `noisepy-seis-io` pins `s3fs==2023.4.0`,
which forces `aiobotocore 2.5.2` and `botocore<1.29.162`, a range
`boto3>=1.28` excludes. `seisfetch/s3.py` uses only `boto3.client`,
`boto3.Session`, list-objects paginators, `botocore.UNSIGNED` and
`ClientError` — all long-stable. Verified on boto3 1.26.161: live
SCEDC/NCEDC/EarthScope day-file pulls and the full precision suite
(56 passed, including both NoisePy CCF equivalence harnesses,
bit-identical).

## 0.4.0 — 2026-08-16

Expand Down
37 changes: 37 additions & 0 deletions docs/noisepy-obspy-replacement-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,43 @@ on the S3 path, and `num_segments` counted records. Fixes:
guard; lazy transport imports (`import seisfetch` no longer pulls boto3);
- CI (there was none), five small committed fixtures, 20+ new tests.

## Status (2026-08-16): the migration this report recommended is implemented

All three steps below are done, on branches: `seisfetch` 0.4.1,
`noisepy-io` `feat/seisfetch-input-layer`, `NoisePy` `feat/seisfetch-data-path`.

| Step | State | Evidence |
|---|---|---|
| (i) adopt the numpy ports behind the `rm_resp` switch | done — `noise_module.preprocess_raw_np`, dispatched in `correlate.preprocess`; the ports are IMPORTED from seisfetch, not copied, so NoisePy (MIT) stays free of the LGPL file | `NoisePy tests/test_preprocess_equivalence.py`: bit-identical to the obspy chain on both fixture sets x 40/20/1 Hz |
| (ii) array-backed `ChannelData` with a lazy `stream` | done — new `TraceSegment`; `ChannelData.from_segments`/`from_array`; `stream` built on first access | `noisepy-io tests/test_seisfetchstore.py` |
| (iii) demote obspy to an extra + split the input layer | done — every obspy/pyasdf import in noisepy-io and NoisePy is now lazy; new `noisepy.seis.io.seisfetchstore` (routing owned by `seisfetch.s3`, coordinates from the FDSN text service); `obspy`/`asdf` extras in both packages | `NoisePy tests/test_no_obspy.py` runs read -> preprocess -> FFT -> cross-correlate in a subprocess where importing obspy RAISES |

Two gaps this report did not anticipate, both found and closed during the
migration:

- **`cc_parameters` called `obspy.geodetics.base.gps2dist_azimuth`** for every
station pair, writing dist/azi/baz into every saved CCF. Ported to
`contrib.obspy_ports.gps2dist_azimuth_np` (Vincenty inverse), exactly equal
to obspy on 2000 random pairs. `noise_module.taper` likewise called obspy's
hann entry point; it resolves to `scipy.signal.windows.hann`, now called
directly.
- **Microsecond rounding.** obspy's miniSEED reader rounds sample times to whole
microseconds and noisepy reads `fric` off `UTCDateTime.microsecond`; pymseed
keeps exact nanoseconds. On a day file starting at 00:00:00.019537920 the two
chains differed by ~1e-8 relative — not bit-identical. `preprocess_raw_np`
now rounds to microseconds, and the sample-grid alignment guard applies only
to pre-trimmed input (`pretrimmed=False` for whole segments, which is what
the obspy path gets from `obspy.read`).

The **two-env caveat is now retired**: `tests/test_no_obspy.py` blocks obspy at
import time and still completes a cross-correlation, so the claim is no longer
"the data path does not use obspy" but "the pipeline runs without it installed".

One dependency consequence: `noisepy-seis-io` pins `s3fs==2023.4.0`, which
forces `botocore<1.29.162`, so seisfetch's `boto3>=1.28` floor made the two
uninstallable together. Relaxed to `>=1.26` in 0.4.1 (verified on boto3
1.26.161 against live SCEDC/NCEDC/EarthScope).

## Architecture recommendation

1. **seisfetch becomes the sole owner of data-center knowledge.** Today
Expand Down
Loading
Loading