Repository navigation
Add petrolib.data_qc_io: cleaning, scaling, filtering, signal, synth, IO (PR-A, final domain) - #157
Merged
Conversation
Final plan domain (LIBRARY_MERGE_PLAN.md data_qc_io). Package with six submodules, each grounded in a corpus survey of the srcYYYY_MM article forms and pre-verified bit-exact against the delegation targets: - clean: sentinels_to_nan, impute_gaps, outlier_mask (zscore/iqr/mad, sided), MAD despike (interp/median/nan), closure_residual, renormalize, relative_discrepancy (both corpus arrangements round identically) - scale: normalize_to_reference (Shier two-point, percentile or min/max endpoints), match_moments - filt: smooth (boxcar/gaussian/median), moving_stat (centered or causal; mean/std/var/mad/cv), tool_response (Gaussian vertical response), median_filter2d, window_features (edge-padded stacks), detect_bed_boundaries (|gradient|*sqrt(var) local maxima) - signal: snr_db (noise record or noise_std, VMD guard), stack_gain, block_stack, add_gaussian_noise (sigma = fraction * mean|x|) - synth: blocky_log, log_suite (verbatim CPA draw sequence), shifted_pair, gaussian_mixture_spectrum, sphere_pack_volume, disk_image (verbatim SEM draw sequence) - io: WellboreData (Bradley et al. 2023 universal JSON container, plus from_json) Deliberately not duplicated: unit conversions (petrolib.units), metrics and zscore/minmax (petrolib.ml_stats), xcorr/DTW alignment (petrolib.depth_matching), pay flags and net-to-gross (petrolib.porosity_lithology). 37/37 bit-exact pre-checks against article bodies; ruff + mypy clean; 252 tests pass (12 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B2BX1XL8422gebCU1oVoQC
ingehap
marked this pull request as ready for review
July 10, 2026 08:53
This was referenced Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Canonical module PR (PR-A) for data_qc_io — the final domain of LIBRARY_MERGE_PLAN.md. Adds the
petrolib/data_qc_io/package with six submodules, each function grounded in a four-agent corpus survey of thesrcYYYY_MMarticle forms:sentinels_to_nan(np.isclose matching, src2021_12/article01),impute_gaps(linear interp over NaN runs, src2018_12/article11),outlier_mask(zscore/iqr/madwithside='both'|'upper'|'lower'),despike(MAD·1.4826;replace='interp'|'median'|'nan'; the upper/median combination reproduces src2024_08/magnetic_permeability bit-exactly),closure_residual,renormalize,relative_discrepancy(the two corpus arrangements2|a-b|/(a+b)and|a-b|/(0.5(a+b))round identically in IEEE, so one canonical body serves both).normalize_to_reference(Shier two-point map; explicit endpoints, percentile default (5,95), orpct=Nonemin/max — covers src2021_12normalize_reference, src2016_12histogram_normalize(opposite arg order, mapped by keyword), and src2015_04histogram_scale),match_moments(src2016_12affine_normalize).smooth(boxcar/gaussian/median with per-kind corpus edge conventions),moving_stat(centered-shrinking or causal windows; mean/std/var/mad/cv, all ddof=0; reproducesrolling_cv,moving_std, andmoving_msdbit-exactly),tool_response(verbatim src2023_10/article_09 Gaussian vertical response incl. the odd-length|1kernel),median_filter2d(generalized src2023_06 3×3 stack median),window_features(edge-paddedsliding_window_viewstacks of the induction papers),detect_bed_boundaries(verbatim src2023_08/article5|gradient|·sqrt(var)local-max pick).snr_db(noise record ornoise_std;guardreproduces the VMD+1e-30convention),stack_gain,block_stack(block-mean, trailing remainder dropped),add_gaussian_noise(verbatim src2021_12/article01,sigma = fraction·mean|x|, deterministic default rng).blocky_log,log_suite(verbatim src2022_08 draw sequence; returns a curve dict + boundaries + facies),shifted_pair,gaussian_mixture_spectrum(the-0.5standard-deviation form),sphere_pack_volume,disk_image(verbatim src2024_04 SEM draw sequence).WellboreData, the Bradley et al. (2023) universal wellbore JSON container (verbatim semantics from src2023_12/bradley_wellbore_format.py, plus afrom_jsonclassmethod).Registered
data_qc_ioin the PEP 562 lazy-load list ofpetrolib/__init__.py.Scope decisions vs the plan sketch
Four planned files are deliberately omitted because the canon already exists:
units.py→petrolib.units.convert+petrolib.constantsalready cover the psi/bar/MPa/ft families from the same sources.metrics.py→petrolib.ml_stats(rmse/mae/mape/r2_score/pearson_r) and its zscore/minmax scaling.align.py→petrolib.depth_matching(xcorr_shift, dtw, shifts);detect_bed_boundarieslives infiltinstead.pay.py→petrolib.porosity_lithology(pay_flag,interval_thickness, 3-argnet_to_gross).Verification
np.varvsmean((x-mean)²)for the causal MSD, reshape-mean vs slice-mean block stacking, both relative-discrepancy arrangements, the fullsynth_log_suite/synthetic_sem_imageRNG draw sequences, and byte-identicalWellboreData.to_jsonoutput.ruff check+ruff format --checkclean;python3 -m mypy petrolibclean (45 source files).python3 -m pytest tests/petrolib: 252 passed (12 new intests/petrolib/test_data_qc_io.py).🤖 Generated with Claude Code
https://claude.ai/code/session_01B2BX1XL8422gebCU1oVoQC
Generated by Claude Code