Release 0.4.0 + pymseed 0.9.5 evaluation - #6
Merged
Merged
Conversation
Cut the Unreleased section — GeoNet, SeisfetchLiveSource, the NCEDC region fix — as 0.4.0, and bring the docs up to what the code does. README was written for three archives and the pre-live Earth2Studio adapters: no GeoNet anywhere, no SeisfetchLiveSource, NCEDC listed in us-east-2, and the notebook list stopped at 04 while 05 (response removal) and 06 (three-archive cross-correlation) went unlinked. Adds an "ObsPy-free, and what that buys" section so the footprint case and the bit-identical NoisePy equivalence are visible from the front page instead of only in benchmarks/. Also exports SeisfetchLiveSource at the top level, where the other two Earth2Studio adapters already live, and gives GeoNet an entry in THIRD_PARTY_NOTICES (which also still said NCEDC was us-east-2). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It was one sentence inside the obspy-free pitch, which undersells the piece of work that actually unblocked the migration — obspy has no pure-Python response evaluator at all, so contrib.response had to re-derive every stage and then match compiled evalresp. Adds the measured numbers (1.6e-10 vs evalresp, 6.6e-16 of peak vs Trace.remove_response, 1.9s vs 3.6s), both evaluation modes with the paz error envelope stated, both deconvolution styles, the loud-failure posture, and the not-implemented list. Also scopes the bit-identity claim: the 0.0 CCF result is the rm_resp=NO chain, and response removal is separate evidence. That was implicit and a reader could reasonably have merged the two. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chad shipped take_np_datasamples() in 0.9.5 the day after the 0.9.4 keepalive landed, and asked on the issue for help testing it — he recommends it over the record_list path for the common case. Adds two profile_parse cases: the plain no-copy view (safe since 0.9.4) and the new take. Findings in docs/pymseed-issue-draft.md. It is correct — bit-identical to the record-list decode, survives GC of the trace list, one-shot and destructive (segment goes to numsamples 0). But on this channel-day it is 22.4 ms against 21.5 ms for the record-list path, so it is not a win for us: the record-list build we would be avoiding is the same pass that hands us the per-record encoding, and asking for both is 27.8 ms. The native tier is also the wrong place to judge a memory-traffic change; the cgroup tiers where the original issue lived need docker, which was unavailable. Separately confirms 0.4.0 is safe to publish: parse_mseed output is bit-identical between 0.9.4 and 0.9.5 on every fixture, so the noisepy equivalence carries over by construction, and both suites are green on 0.9.5. Pin >=0.6,<0.10 stands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The native pass could not separate take_np_datasamples() from the record-list path, and native is the wrong tier for a memory-traffic question anyway. A first container pass was unusable — at 0.5 cpu the means ran 2-4x the mins and tl_unpack+view swung 40 to 88 ms between runs of identical code. Adds benchmarks/ab_pymseed_paths.py: only the four cases that matter, shuffled and interleaved each round so quota refill and host drift cancel instead of biasing whichever case runs first. Two containers x 15 reps per version per tier. That resolves it. take_np_datasamples() is indistinguishable from the plain 0.9.4 safe view (27.6 vs 27.7 ms fargate, 28.6 vs 28.3 lambda) — which makes sense, since the keepalive already made the view zero-copy, so the new call changes ownership rather than data movement. Its value is lifetime semantics, not throughput. The record-list path stays fastest in both tiers and leads by ~25% there against ~1 ms natively. Caveats recorded: single trace/segment file, and Docker Desktop on macOS is cgroup-in-VM rather than bare Linux. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The single-segment 11 MB channel-day is the friendliest case for the unpack paths, so it could not answer whether the record-list lead holds on what EarthScope actually serves. ab_pymseed_paths.py now takes a glob and times the whole set per round. Re-run over II.PFO station-days (44 traceids / ~50 segments each): the record-list lead grows from ~1.3x to ~2.1x — 216 vs 447 ms on 194 segments where it was 21.6 vs 27.7 on one. That fits the mechanism, since mstl3_unpack_recordlist decodes into one numpy-owned allocation per segment while unpack_data=True wraps libmseed's buffers and pays per-segment overhead. So the path seisfetch already uses wins by the widest margin exactly where our heaviest objects live. One counterpoint recorded honestly: take_np did beat the plain view at lambda-1g with 97 segments (341.6 vs 384.2), consistent across min/p25/median but a single run, so noted as suggestive only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR cuts the 0.4.0 release by bumping package versions and aligning project documentation with current capabilities (notably GeoNet support and Earth2Studio “live” sourcing), and it records/extends benchmark evaluation of pymseed 0.9.5 decode-path behavior.
Changes:
- Release bookkeeping: version bump to 0.4.0 and changelog entry for the 0.4.0 release.
- Documentation updates to reflect four S3 archives (adds GeoNet) and expanded Earth2Studio integration details.
- Benchmark additions/updates for evaluating pymseed 0.9.5 (
take_np_datasamples()and related decode paths).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
THIRD_PARTY_NOTICES.md |
Updates NCEDC region and adds GeoNet attribution details. |
seisfetch/__init__.py |
Updates top-level docs/version fallback and exports SeisfetchLiveSource. |
README.md |
Expands docs for GeoNet, ObsPy-free rationale, response removal, and Earth2Studio live source usage. |
pyproject.toml |
Bumps project version to 0.4.0. |
docs/pymseed-issue-draft.md |
Adds recorded evaluation notes for pymseed 0.9.5 take_np_datasamples() behavior/perf. |
CHANGELOG.md |
Adds 0.4.0 release notes and highlights GeoNet + live Earth2Studio source. |
benchmarks/profile_parse.py |
Adds benchmark cases for unpack+view and unpack+take_np paths. |
benchmarks/ab_pymseed_paths.py |
Adds a focused A/B benchmark harness for pymseed decode paths under cgroup limits. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+116
to
+121
| for tid in tl: | ||
| for seg in tid: | ||
| arr = seg.take_np_datasamples() | ||
| n += arr.shape[0] | ||
| return n | ||
|
|
__all__ listed the four Earth2Studio adapter names unconditionally while they are only bound when the optional seisfetch.earth2 import succeeds, so `from seisfetch import *` raised AttributeError on a minimal install. Reproduced by forcing that import to fail — three of the four names predate this branch, and adding SeisfetchLiveSource widened it. __all__ now gains those names only when the import succeeded, which also keeps __all__ an honest description of what the install actually provides. The import needs an explicit noqa: F401 now that ruff can no longer see the re-export through a dynamic __all__. profile_parse's take_np case called take_np_datasamples() unguarded. run_table catches per-case failures, but run_rss does not, so `--rss tl_unpack+take_np` on an older pymseed printed a bare AttributeError rather than the version reason. Now raises with the requirement and the installed version named; table mode still degrades to one FAILED row, with a better message than before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts 0.4.0 and brings the docs up to what the code already does, then evaluates pymseed 0.9.5 before publishing against it.
Release 0.4.0
CHANGELOG
Unreleasedbecomes0.4.0; version bumped inpyproject.tomland the__init__fallback, which was stale at 0.3.0.README was written for three archives and the pre-live Earth2Studio adapters — no GeoNet anywhere, no
SeisfetchLiveSource, NCEDC listed in us-east-2, and the notebook list stopped at 04 while 05 and 06 went unlinked. Adds an ObsPy-free section so the footprint case and the bit-identical NoisePy result are visible from the front page, and gives instrument response removal its own section: obspy has no pure-Python response evaluator, socontrib.responsehad to re-derive every stage and then match compiled evalresp, which the one-line mention undersold.Also scopes the bit-identity claim — the
0.0CCF result is therm_resp=NOchain and response removal is separate evidence. That was implicit and a reader could reasonably have merged the two.SeisfetchLiveSourceis now exported at top level alongside the other two Earth2Studio adapters. THIRD_PARTY_NOTICES gains a GeoNet entry and loses the same stale NCEDC region.pymseed 0.9.5
The
>=0.6,<0.10pin already admits 0.9.5, so publishing 0.4.0 ships a package whose fresh installs resolve to a pymseed that had not been tested. It has been now:parse_mseedoutput is bit-identical between 0.9.4 and 0.9.5 across every fixture and across 163 traces in 15 real cached day files from all three archives, including the 48-channel EarthScope station-day topology. Both suites green on 0.9.5. The noisepy equivalence therefore carries over by construction, since path B starts atparse_mseed. Pin stands.take_np_datasamples()(the API Chad added in 0.9.5 and asked for help testing, np_datasamples forces a full-size copy that dominates parse time in cgroup-limited containers — proposal for a safe owned-array accessor EarthScope/pymseed#6) is correct — bit-identical to the record-list decode, survives GC, one-shot and destructive — but is indistinguishable from the plain 0.9.4 safe view on throughput. The keepalive already made the view zero-copy, so the new call changes ownership rather than data movement.Full tables and caveats in
docs/pymseed-issue-draft.md; harness inbenchmarks/ab_pymseed_paths.py. Numbers are cgroup-in-VM (Docker Desktop on macOS), which is what the original issue's numbers were too.Verification
270 unit tests passing, 53 precision tests passing, on both 0.9.4 and 0.9.5. Lint and format clean.
Follow-up, not in this PR: a reply to EarthScope/pymseed#6 reporting the above, leaving the issue open for Chad's docs proposal.
🤖 Generated with Claude Code