Add data-reuse summaries for datasetPapers

datasetPapers/examples/griswold_2025/FINDINGS.md

@@ -0,0 +1,116 @@
+# Findings — Griswold 2025 Worked Example
+
+This report captures what was awkward, what was missing, and what should change before writing any NDI schemas. Scored against the user-facing criteria: **flexible, standardized, easy to create, easy to search, intuitive**.
+
+---
+
+## Summary scorecard
+
+| Criterion | Verdict | Notes |
+|---|---|---|
+| **Flexible** | ✅ Strong | The openMINDS `customPropertySet` escape hatch plus `stimulation:Stimulus.specification` handled every per-paradigm parameter without schema gymnastics. |
+| **Standardized** | ⚠️ Mixed | 90 % of fields mapped to existing openMINDS types or controlled terms, but the factorial-stimulus pattern and sample-aligned epochs have no obvious openMINDS home and had to be NDI-native. |
+| **Easy to create** | ❌ Not yet | ~28 openMINDS + ~16 NDI documents for one session of one subject. No researcher will author this by hand. Templates + a helper tool are needed. |
+| **Easy to search** | ⚠️ Mixed | Simple per-document queries resolve cleanly (Queries 1, 4). Multi-hop queries through the provenance DAG (Queries 2, 3, 5) require careful walking. |
+| **Intuitive** | ⚠️ Mixed | openMINDS's protocol/execution and subject/state patterns are conceptually clean but unfamiliar. A reader new to openMINDS needs about 15 minutes of orientation before the file tree makes sense. |
+
+---
+
+## Concrete findings (ordered by severity)
+
+### F1 — Multi-hop queries through the state chain are awkward (severe)
+
+**Evidence:** Query 3 (`find recordings ≥28 days post eye opening`) required four join steps: `ElectrodeArrayUsage` → `SubjectState` → walk `descendedFrom` → identify eye-opening `ProtocolExecution` → age arithmetic.
+
+**Implication:** For any question that spans more than one openMINDS entity, naïve implementations will be slow and bug-prone. This is *not* a flaw in openMINDS — PROV-style DAGs always have this shape — but it does mean NDI consumer tooling must provide:
+
+- A `walk_state_chain(subject)` helper that returns states in order with their associated protocol executions.
+- A `days_since(subject_state, event_type)` helper for age arithmetic.
+- A cached "materialized view" of flattened per-session metadata for common query patterns.
+
+**Recommendation:** Before writing schemas, prototype a tiny query helper (Python or MATLAB, ~200 lines) that answers all five queries against the worked example. If the helper can't keep them under 20 lines each, either the graph shape is wrong or the helper API is wrong.
+
+### F2 — Analysis provenance is under-specified (severe)
+
+**Evidence:** Query 5 could not trace which individual tuning-curve documents fed into the mixed-effects analysis. The analysis doc lists `included_subjects` and `included_sessions`, but not the unit-level input documents.
+
+**Implication:** Reproducibility breaks. If someone wants to re-run the mixed-effects fit on a filtered subset of units (e.g. excluding low-SNR ones) they can't, because the original input set isn't enumerated.
+
+**Recommendation:** The `analysis_output` schema should require an explicit `inputs[]` list of document IDs (or a query spec that resolves to a stable ID list). For Griswold, the analysis doc would reference ~600 `analysis_output(indices)` documents — one per unit across all animals. That's a lot, but it's the correct structure.
+
+### F3 — No home for factorial stimulus design in openMINDS (moderate)
+
+**Evidence:** Stimulus Set 1 is an 8×4×6 = 192-cell factorial with shared blank control. openMINDS's `stimulation:Stimulus` is one-stimulus-per-document. We handled this by making an NDI-native `factor_design` doc plus one openMINDS `Stimulus` per *example* cell — but fully representing the experiment would need 192 `Stimulus` documents for Set 1 alone.
+
+**Implication:** Authoring 192 near-identical openMINDS documents is prohibitive. Either (a) the factorial is represented entirely NDI-side with no openMINDS individual-stimulus documents (but then openMINDS-level search loses stimulus-level granularity), or (b) we propose a `stimulation:StimulusFactorDesign` upstream contribution.
+
+**Recommendation:** For now, keep the factorial in NDI and reference each `StimulationActivity` → one `factor_design` via `customPropertySet.factorDesignRef` (as demonstrated). Consider an upstream contribution to openMINDS later.
+
+### F4 — "Placement" vs "ElectrodeArrayUsage" — the openMINDS pattern is awkward for chronic/longitudinal studies (moderate)
+
+**Evidence:** In Griswold the probe is used in one terminal session, so `ElectrodeArrayUsage` trivially spans one session. In Reikersdorfer (the carbon-fiber MEA paper), the same probe is chronically implanted for 11 months and records across hundreds of sessions. openMINDS's `DeviceUsage` has no explicit longitudinal model.
+
+**Implication:** Not a problem for Griswold specifically, but when we do the Bhar stress-test example (or a hypothetical chronic example) we'll need to decide: one `ElectrodeArrayUsage` per session, or one persistent `ElectrodeArrayUsage` with many sessions pointing at it?
+
+**Recommendation:** Defer to the Bhar / Reikersdorfer analysis. Flag this for the cyclic/longitudinal findings.
+
+### F5 — Controlled terms that don't yet exist (moderate)
+
+**Evidence:** The documents reference `controlledTerms:SubjectAttribute/eyes-closed`, `/one-eye-open-right`, `/anesthetized`, `/paralyzed`, `/artificially-ventilated`, `controlledTerms:VisualStimulusType/ambient-scene`, `controlledTerms:Technique/forceps-dissection`, `/receptive-field-mapping`, and `controlledTerms:UnitOfMeasurement/postnatal-day`. Not all of these exist in the openMINDS controlled-term instance sets.
+
+**Implication:** For the worked example these are annotations-that-aspire-to-be-standard. For a real deployment, the project needs a policy: (a) submit missing terms upstream and wait, (b) fork controlledTerms locally, or (c) fall back to free text with ontology-IRI hints.
+
+**Recommendation:** Adopt policy (a) as the default, with (c) as the transitional fallback. Keep a running list (this paper alone generated 10+ candidate terms).
+
+### F6 — Time representation is inconsistent (moderate)
+
+**Evidence:** openMINDS `Activity.startTime` / `endTime` are wall-clock ISO 8601. NDI `epoch.sample_range` is sample-indexed. NDI `session.date` is a calendar date. NDI `recording.duration_seconds` is a float. There is no single normalised time axis.
+
+**Implication:** A query like "what happened in the recording at wall-clock 14:15:00" requires the consumer to compose `session.date` + offset-from-start. A query "what was the subject state at sample 7575000" requires resolving recording_start wall clock → sample 0.
+
+**Recommendation:** The NDI `recording` doc should carry an explicit `start_time_utc` field aligned to sample 0, and the NDI `epoch` should be both `sample_range` and `wall_clock_range` (computed). This is probably a small additional field rather than a new schema.
+
+### F7 — The openMINDS → NDI link direction is one-way (minor)
+
+**Evidence:** NDI documents reference openMINDS instances by IRI. openMINDS documents do not reference NDI documents. This means given only an openMINDS `SubjectState`, you cannot discover which NDI `session`s used it without an external index.
+
+**Implication:** A reverse index must be maintained outside either schema. For NDI Cloud this is a cloud-level concern, not a schema concern, but it should be called out.
+
+**Recommendation:** NDI Cloud (or the consumer library) builds and maintains the reverse index. Do not try to solve this at the schema level.
+
+### F8 — Filenames do not help the reader understand document relationships (minor)
+
+**Evidence:** The numbered prefixes (`01_`, `10_`, `30_`) in both folders are only a human convenience. There's no machine-readable ordering or grouping.
+
+**Implication:** Readers lean on the filenames for orientation; without them, finding a specific document in a flat directory of 40+ files is hard.
+
+**Recommendation:** For archival deposits, keep the flat structure but also emit a `MANIFEST.json` that indexes all documents by classname, subject, session, and timestamp. Not a schema issue.
+
+### F9 — Easy-to-create will require a template engine (minor)
+
+**Evidence:** Looking at the 28 openMINDS documents, maybe 60 % of the content is boilerplate that could be auto-filled from a handful of paradigm-level inputs ("this is an in-vivo visual ephys experiment, ferret, Brandeis lab, published in eLife 2025, three-group design"). Researchers should not be hand-typing these.
+
+**Recommendation:** A future CLI like `ndi new experiment --from-template visual-ephys-vertebrate` is the right shape. Scope for a later iteration — not needed before the schemas are agreed.
+
+---
+
+## What to change in the synthesis doc
+
+Based on these findings, two amendments to `datasetPapers/summaries/00_schema_design_synthesis.md`:
+
+1. **Open question #3 (how much should `analysis_output` be schema-constrained)** — Finding F2 says: require an `inputs[]` list of document IDs. This is the minimum for reproducibility.
+2. **New open question:** What helper API does a consumer library need to expose so that typical researcher questions (Queries 1–5) take <20 lines to answer? This is not a schema question per se, but the schema shape constrains what's possible.
+
+---
+
+## What changes the Bhar example will likely expose that this one did not
+
+Based on what Bhar has that Griswold doesn't:
+
+- **Populational subjects** — `SubjectGroup` / `SubjectGroupState` should handle this, but the Bhar exchange assay (naïve worms transferred onto plates that held trained worms) is a cross-group provenance pattern we haven't tested.
+- **Cyclic protocols** — the `Protocol.steps` + `depends_on` composition design from the synthesis doc needs to be actually rendered.
+- **Substrate-as-measured-object** — the plate carries LC-MS observations that have no subject attached. Will require either `core:TissueSample` (awkward) or a new NDI-native `substrate` doc.
+- **No data deposit** — "data availability: contact the authors" maps to what `core:UsageAgreement` in openMINDS; we didn't test this.
+- **Cross-species exchange** — a single plate holds a history involving both *C. elegans* and *C. briggsae*, which is an inherently cross-reference provenance pattern.
+
+If Bhar can be expressed cleanly with no more than one new NDI document type and no more than three new findings of F1/F2 severity, the overall design is validated. If it cannot, iterate.

datasetPapers/examples/griswold_2025/QUERIES.md

@@ -0,0 +1,107 @@
+# Queries — Griswold 2025 Worked Example
+
+Five questions a researcher might want to ask against this dataset, each traced through the document graph. The resolution steps below are what a query engine or consumer library would have to do.
+
+The goal is to find out where the **query surface is clean** and where it is **awkward or broken** — those failures go into `FINDINGS.md`.
+
+---
+
+## Query 1 — "What species was studied, and was it sex-balanced?"
+
+**Resolution:**
+
+1. Load all `core:SubjectGroup` documents for the dataset → `20_subject_group_control.json`, `21_subject_group_EO1.json`, `22_subject_group_EO2.json`.
+2. Read `species` field → resolves to `openminds:species-mustela-putorius-furo` → *Mustela putorius furo* (ferret).
+3. Read `biologicalSex` → all three groups: `controlledTerms:BiologicalSex/female`.
+
+**Answer:** Ferret (*Mustela putorius furo*), female only.
+
+**Assessment:** Clean. One field on each group resolves the question immediately. No NDI documents needed.
+
+---
+
+## Query 2 — "For subject EO1-01, what manipulations preceded the terminal recording, in order?"
+
+**Resolution:**
+
+1. Load `30_subject_EO1_01.json`.
+2. Read `studiedState` → four state IRIs.
+3. For each state, resolve and read `descendedFrom` to build the chain:
+   - `P25-pre` → (none)
+   - `P25-postopen` → descended from `P25-pre`
+   - `P28-postexposure` → descended from `P25-postopen`
+   - `P55-terminal` → descended from `P28-postexposure`
+4. For each state transition, find the `core:ProtocolExecution` whose `input` is the earlier state and `output` is the later state:
+   - `P25-pre` → `P25-postopen`: `protex-eyeopen-EO1-01` (protocol = `premature-eye-opening`)
+   - `P25-postopen` → `P28-postexposure`: `protex-exposure-EO1-01` (protocol = `visual-exposure`)
+   - `P28-postexposure` → `P55-terminal`: `protex-surgery-EO1-01` (protocol = `terminal-surgery`)
+5. Order by `startTime` on each execution.
+
+**Answer:** (1) P25 premature right-eye opening (2024-10-15), (2) P25–P28 unguided visual exposure (2 h/day × 4 days), (3) P55 terminal surgery (2024-11-14).
+
+**Assessment:** Clean but **two-step**: first walk the state chain, then match executions by input/output. Consumers will want a helper that does this in one call. The openMINDS "everything is an Activity with input/output" pattern makes this work, but naive users will not discover it immediately.
+
+---
+
+## Query 3 — "Find all recordings that used a Plexon S probe in monocular V1 of a ferret with ≥28 days post-eye-opening."
+
+**Resolution:**
+
+1. Find all `ephys:ElectrodeArrayUsage` documents whose `device` is `openminds:griswold-device-plexon-s-probe`.
+2. For each such usage, read `usedSpecimen` (a `SubjectState`) and `anatomicalLocationOfArray`.
+3. Filter by anatomical target = V1 monocular (e.g. contains `openminds:griswold-target-v1-monocular`).
+4. For each remaining usage, walk the state chain backwards through `descendedFrom` until reaching a state that is a `ProtocolExecution.output` whose protocol is `premature-eye-opening` or `natural-eye-opening`.
+5. Compute the age at recording state minus the age at the eye-opening state. Filter by ≥ 28 days.
+
+**Answer:** (for this example) `51_electrode_array_usage_EO1_01.json` — 30 days post-opening.
+
+**Assessment:** **Awkward.** This works but is a multi-hop query requiring:
+
+- Traversal of the `SubjectState.descendedFrom` chain.
+- Identifying which state-transition was "eye opening" vs another manipulation.
+- Age arithmetic with unit awareness.
+
+This is the kind of query a neuroscientist would reasonably ask but that our document graph does not make easy. **Finding #1 in FINDINGS.md.**
+
+---
+
+## Query 4 — "For unit-001 in session EO1-01, give me all tuning curves and the derived selectivity indices."
+
+**Resolution:**
+
+1. From session `ndi-session-griswold-EO1-01`, find the `spikesort_output` via `depends_on` → `ndi-spikesort-griswold-EO1-01-session-01`.
+2. Verify `unit-001` ∈ `unit_ids`.
+3. Find all documents that `depends_on` the spikesort_output and have `classname` in {`stimulus_tuningcurve`, `fit_curve`, `analysis_output`} and reference `unit_id: "unit-001"`.
+4. Return them.
+
+**Answer:** 4 tuning curves (ori, dir, SF, TF), 3 fits (Movshon SF, Movshon TF, Naka-Rushton contrast), 1 indices doc.
+
+**Assessment:** Clean. All within the NDI layer. This is the core use case that the existing `stimulus_tuningcurve` / `data/fitcurve` schemas already support.
+
+---
+
+## Query 5 — "What is the across-animal effect of premature eye opening on direction selectivity, and which documents are the evidence?"
+
+**Resolution:**
+
+1. Find all `analysis_output` documents with `method: "linear_mixed_effects"` and a name matching "direction selectivity" → `60_analysis_mixed_effects_direction_selectivity.json`.
+2. Read `fixed_effects` → group coefficients with p-values.
+3. For provenance, walk `included_subjects`, `included_sessions`, and the per-unit `tuning_curve` documents that would have gone in. (These are implicit, not explicit.)
+
+**Answer:** Control intercept 0.155 (p<1e-11); EO1contra +0.036 (p=0.24, n.s.); EO1ipsi −0.048 (p=0.047, *); EO2 +0.021 (p=0.61, n.s.).
+
+**Assessment:** The *answer* is clean. The **provenance trace is broken**: the analysis doc lists `included_subjects` and `included_sessions` but does *not* enumerate which `tuning_curve` / `analysis_output(indices)` documents were the inputs. A reader cannot recreate the dataset of 1−DCV values from the document graph alone. **Finding #2 in FINDINGS.md.**
+
+---
+
+## Summary of assessment
+
+| Query | Works cleanly? | Issue |
+|---|---|---|
+| 1. Species, sex | ✅ | — |
+| 2. Manipulation timeline | ⚠️ | Requires two-step traversal |
+| 3. Probe + anatomy + age filter | ❌ | Multi-hop through state chain; no direct "days post eye opening" field |
+| 4. Unit tuning + fits + indices | ✅ | — |
+| 5. Across-animal analysis provenance | ❌ | Analysis doc does not enumerate input documents |
+
+Three out of five queries expose friction. The specifics are captured in `FINDINGS.md`.