Repository navigation
Conversation
New exported read_plate_dataset(path, date_processed, max_file_bytes) reads one plate dataset (the Parquet file the parser writes for one plate, one row per well per tick) at plate dataset schema version 1: - reads the file as data only: no eval, parse or unserialize of file content, Arrow/R metadata in the file ignored, attributes dropped; - refuses an unknown or missing plate_dataset_schema_version, any missing, extra, reordered or mistyped column, and any value outside the schema's rules (one value per plate/well/measurement/tick, 96 wells A01..H12, every well at every tick, lowercase sha256 plate id, ISO 8601 UTC date_run), with size, row and string-length caps; - converts O2 emissions to O2 (mmHg) (Stern-Volmer, uncorrected Ksv) and pH emissions to pH (line through the calibration pH at the target emission); - returns seahtrue's plate structure with our names and runs validate_preprocessed() itself; assay_info carries plate_dataset_schema_version and plate_dataset_schema_sha256. Adds nanoparquet (>= 0.5.1) to Imports; it also runs under webR. Tests use synthetic plate datasets only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ooter-cap tests - Refuse a non-finite value (Inf, -Inf, NaN) in any double column, and a non-finite O2_mmHg, pH_em_corr_corr or pH after conversion (a zero emission, F0, Ksv or calibration emission), so validation never passes over NaN silently. - Schema v1 copy follows the parser's updated file (sha256 8ffe7fdd..., parser commit f6d95e1): new required BOOLEAN column plate_flagged_well after flagged_well. Both manual flags are carried in raw_data as data; no well is dropped or excluded by either, and background wells are averaged whether flagged or not. - Tests for the footer row-count cap, rows read differing from the footer, and the uncompressed-size cap (footer reader mocked), for the non-finite refusals, and for the two flags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Revise round (Vincent at acceptance: "4 revise, F2 ok"), commit 54868e8:
On Vincent's word ("skip the tarball rebuild"), R CMD check and the webR load check were not re-run this round. Both must be re-run once before this PR leaves draft. 🤖 Generated with Claude Code |
…round Vincent: "flagged background should be excluded!". A "Background" well flagged in either flagged_well or plate_flagged_well is left out of the background average; a plate whose every Background well is flagged is refused rather than given a background from nothing. The flags exclude nothing else: every well stays in raw_data, with both flag columns as data. Tests: a background well flagged under each flag column is left out (the flagged well, H01, changes the mean, so the test tells exclusion from no exclusion); the all-flagged refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vincent: "accept no refusal". A plate whose every Background well is flagged (in either flag column) now gets missing background columns, as a plate with no Background well does, instead of being refused. The flag rule itself is unchanged: a Background well flagged in either column is left out of the background average. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eft out Code review CR4-F3: the flag tests now also expect pH_em_corr_bkg to leave out the flagged well H01 (whose pH emission differs from the other three Background wells), under each flag column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NEWS.md: keep the development-version entry for read_plate_dataset() above devel's 1.7.2 entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before this PR leaves draft
R CMD check --no-manualon the head commit. It was skipped for54868e8,400f7bc,2915b2aandc261d83on Vincent's word ("skip the tarball rebuild"); last pass was one9906ed.library(seahtrue)andread_plate_dataset()under webR 0.6.0). It was skipped for54868e8,400f7bc,2915b2aandc261d83; last pass was one9906ed.plate_dataset_schema_v1.jsonis on swamp_seahtrue's main branch, and its sha256 still equalsPLATE_DATASET_SCHEMA_V1_SHA256.Summary
Adds an exported Parquet reader for the plate dataset,
read_plate_dataset(). A plate dataset is the Parquet file the lab's.asyrparser writes for one plate, one row per well per tick.The reader:
Reads the file as data only. Nothing from the file is evaluated or deserialised. Arrow and R metadata in the file are ignored (
use_arrow_metadata = FALSE), and every column is reduced to a plain atomic vector.Rejects an unknown or missing
plate_dataset_schema_version. Only"1"is known.Refuses anything outside schema version 1:
date_runthat is not ISO 8601 UTC.File size, row count, uncompressed size and string length are all capped. Refusal messages quote file values only truncated and escaped.
Converts emissions. O2 emission becomes O2 (mmHg) by Stern-Volmer with F0 and the uncorrected Ksv. pH emission becomes pH on a line through the calibration pH at the pH target emission.
Keeps every well.
flagged_wellandplate_flagged_wellare manual flags a person set while looking at the data, both carried as data. Their one effect: a Background well flagged in either is left out of the background average. When every Background well is flagged, or there is none, the background columns are missing.Returns seahtrue's plate structure with our names and runs
validate_preprocessed()itself.assay_infocarriesplate_dataset_schema_versionandplate_dataset_schema_sha256, so callers can check which schema the plate was read with without using internals.rate_datahas zero rows: no OCR or ECAR yet.Adds nanoparquet (>= 0.5.1) to Imports. It has no dependencies and runs under webR; arrow does not.
The schema is a copy of the parser's
plate_dataset_schema_v1.json: 34 columns, sha2568ffe7fdd…4c5b, parser commitf6d95e1. The parser is still in review, so this PR stays a draft.Checks
devtools::test()atc261d83: 171 expectations pass, 0 failures, 89 of them for the new reader on synthetic plate datasets. No real plate data is in this repository.e9906ed:R CMD check --no-manual: 0 errors, 0 warnings, 2 notes. One note is new: Imports now has 21 non-default packages.library(seahtrue)and the reader works.c261d83(see the checklist at the top).Do not merge or tag from this PR; Vincent decides the release.
🤖 Generated with Claude Code