Skip to content

Integrate master into dev - #748

Merged
d-chambers merged 57 commits into
devfrom
dev_integrate
Jul 20, 2026
Merged

Integrate master into dev#748
d-chambers merged 57 commits into
devfrom
dev_integrate

Conversation

@d-chambers

@d-chambers d-chambers commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Integrates the latest master (through #744) into dev. Because master's history was rebased at some point, its commits no longer share ancestry with the copies dev previously absorbed via the "Merge master into dev" squashes, so a flatten-rebase is unsafe. Instead this is a merge of master into dev with conflicts resolved in dev's favor (merge -X ours), keeping dev's feature/architecture work while pulling in master's non-conflicting changes. origin/dev is an ancestor of this branch, so landing it is a fast-forward.

What's included

  • Merge commit integrating all 54 master PRs since the branches diverged (new transforms STALTA/FBE/kurtosis, Febus BSL/T1/mtx and SR4731 readers, spool read/merge speedups, specplot, numerous regression fixes, etc.).
  • Deleted-in-dev modules stay deleted per dev's API cleanup: transform/fft.py and transform/spectro.py (replaced by fourier.py / viz.spectrogram).
  • One follow-up fix: always specify an explicit ns unit when constructing np.datetime64 / np.timedelta64, closing the remaining unitless call sites (notably the flat-dump null-step sentinel in core/summary.py and the 0-D degenerate-array branches in utils/time.py). This avoids the numpy 2.5 unitless-NaT inconsistency.

Verification

Summary by CodeRabbit

  • Bug Fixes

    • Improved consistency when handling dates and time durations by using nanosecond precision.
    • Improved timestamp parsing from supported filenames.
    • Empty HDF5 files without a detectable format now return a clear format error.
  • Tests

    • Added coverage for format detection on empty HDF5 files.

Changelog

none

d-chambers and others added 30 commits March 26, 2026 12:29
* workaround for dasvader HDF2 compatibility issue.

* address first review

* address review 2
Co-authored-by: Derrick UCD <derrick.chambers@ucd.ie>
* allow array_likes to pass through patch creation

* clean up

* address review
* fix hampel bottleneck median centering

* Address moving median review feedback
* added support for OptoDAS v11. That format has no changes relevant to DAScore, so this is a simple fix

* added the V11 class import, and added to plugins

* Added missing OptoDASV9 to the __init__.py
* fixed sampling rate determination for older versions <1

* set 'data_units' and 'data_type' attributes

* removed superflous empty lines

* fixed again for Valencia data. different approach now to fix pre-2021 formats

* try a different approach to satisfy CODEDEV checks
* Fix #662

* address review

* address review
* Added typical IDE project files to be ignored by git

* Update .gitignore

* removed duplicate PyCharm ".idea/" reference in gitignore
* axis lables are now capitalised, and unit in square brackets

* fixed time axis label and test suite

* updated waterfall test to be case-insentive
* new patch functions for demean and demedian

* Update basic.py

* fixed docstring

* suppress output in examples to satisfy testing

* incorporated axis-improvements suggested by Coderabbit

* Update tests for demean/demedian to make it independent of dimension

* fixed bug that was suggested by coderabbit

* fix comment median >> mean

* minor cleanup of redundand np.array call

* fixed indetation issue (hopefully)

* fixed typo in comment
* fix minor febus linting issues

* ensure lint runs on PR
* small tweaks for agents

* address review
* add *args and **kwargs to rolling.apply

* add example to apply for rolling
* add more details about making a release

* address review
Added STALTA, FBE, and Kurtosis transforms

This works now also the with latest upgrade of the "rolling" function to allow for arguments being passed 
#689

numba implementation to speed up significantly


---------

Co-authored-by: Derrick Chambers <d-chambers@users.noreply.github.com>
* waterfall now has default colormaps for various datatypes. Colorbar also shows extend triangles if appropriate

* fixed typo in docstring

* default cmap

---------

Co-authored-by: Derrick Chambers <chambers.ja.derrick@gmail.com>
* add cbar back to waterfall, cmap=None default again

* address review
…) (#696)

* Coerce numeric relative offsets for timedelta64 coordinates
Logic for loading Febus T1 data

---------

Co-authored-by: Shawn Boltz <mboltz@cdc.gov>
* febus BSL h5 read support

* Address BSL review comments

* Rename BSL HDF5 reader class
* added output type to dtf; data_type is now 'fourier transform'

* applied coderabits suggestions

* fixed maybe_units labels for plots with ft-axis

* Fix DFT spectral output scaling

* address review, ensure lint runs only once

* refactor, cover missing line

* keep one-sided spectra

* minor typo in error message

* minor typo in error message in test_fourier

---------

Co-authored-by: Derrick Chambers <chambers.ja.derrick@gmail.com>
d-chambers and others added 12 commits July 1, 2026 11:11
* Fix DASDAE FileSpool patch attribute loading

* Ignore null DASDAE attr filters
* Defer matplotlib import by lazily loading dascore.viz
Integrate latest master (through #744) into the dev branch. Conflicts were
resolved giving dev's feature/architecture changes priority (merge -X ours),
while incorporating master's non-conflicting work. The deprecated
transform/fft.py and transform/spectro.py (removed in dev's API cleanup and
replaced by fourier.py / viz.spectrogram) were kept deleted per dev's
architecture.
Add an explicit ns precision to the remaining np.datetime64/np.timedelta64
call sites that lacked one. Notably summary.py's flat-dump null-step sentinel
used a bare timedelta64(NaT) (numpy 2.5 treats unitless NaT inconsistently),
and the 0-D degenerate-array branches in utils/time.py returned datetime64/
timedelta64 with whatever precision the input carried instead of normalizing
to ns like the sibling array branches.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4894743a-edf9-4af2-b94d-c001e02cead7

📥 Commits

Reviewing files that changed from the base of the PR and between 96edf7a and 3dd0f91.

📒 Files selected for processing (3)
  • dascore/core/summary.py
  • dascore/utils/time.py
  • tests/test_io/test_io_core.py
💤 Files with no reviewable changes (1)
  • dascore/utils/time.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • dascore/core/summary.py

📝 Walkthrough

Walkthrough

The changes enforce explicit nanosecond precision across NumPy datetime and timedelta conversions, summary sentinels, and Sentek filename parsing. A test also verifies unknown-format handling for empty HDF5 files.

Changes

Nanosecond precision and format detection

Layer / File(s) Summary
Time conversion utilities use explicit ns units
dascore/utils/time.py
Datetime and timedelta conversion paths now construct NumPy values with explicit nanosecond precision.
Summary and Sentek timestamps use ns units
dascore/core/summary.py, dascore/io/sentek/utils.py
The missing-step NaT sentinel and parsed Sentek filename timestamps now use explicit nanosecond units.
Empty HDF5 format detection coverage
tests/test_io/test_io_core.py
Adds coverage confirming empty HDF5 files raise UnknownFiberFormatError when no fiber format is detected.

Suggested labels: IO, patch

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the main change: merging master into dev.
Description check ✅ Passed The description covers purpose, included changes, and verification, though it doesn't follow the template headings exactly.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev_integrate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added bug Something isn't working IO Work for reading/writing different formats patch related to Patch class transform Related to transform operations labels Jul 9, 2026
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.96%. Comparing base (d3d6520) to head (3dd0f91).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #748   +/-   ##
=======================================
  Coverage   99.96%   99.96%           
=======================================
  Files         163      163           
  Lines       17087    17087           
=======================================
  Hits        17081    17081           
  Misses          6        6           
Flag Coverage Δ
network 48.76% <33.33%> (ø)
unittests 99.95% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The added 'if not keys: continue' check is unreachable: the preceding
'if not unsorted: continue' already skips formats with no loaded fiber_ios
(_format_version is a defaultdict(dict), so a failed/absent format yields an
empty dict). Dropping the dead branch restores full patch coverage.
Bring the branch up to date with dev (#749-#759) so CI runs against the
current tip before landing the master integration.
@coderabbitai coderabbitai Bot removed bug Something isn't working transform Related to transform operations labels Jul 20, 2026
@d-chambers
d-chambers merged commit 93d2707 into dev Jul 20, 2026
29 checks passed
@d-chambers
d-chambers deleted the dev_integrate branch July 20, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

IO Work for reading/writing different formats patch related to Patch class

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants