[SatyaA27]

Task C10 — Dataset Quality Gate. Owner: SatyaA27. Reviewer: Mohamed Sayed (@panthera-momagdii). Target: 16 October 2026.

DRAFT — SYNTHETIC QA PASS; REAL_EPISODE_NOT_VERIFIED; PRODUCTION ACCEPTANCE BLOCKED.

This adds a deterministic, read-only C9 conformance reference checker. Structural success without required timing, native artifact, calibration/configuration/profile, release or recording evidence cannot open the training gate. PASS, REJECT and NOT_VERIFIED remain separate; exit codes stay 0, 1 and 2. No recorder, training runtime or second native dataset format is introduced.

Evidence-handoff review follow-up

Addresses Mohamed's scoped review at 8440f4b818f4bd0e6bbbbcc1538eee0555e0012a. New signed commit 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d changes only the workflow, evidence generator/tests and handoff documentation. The accepted C9 schema, checker/semantic logic, synthetic test policy, original 94 C10 tests, dependency manifests and existing ROS gates are unchanged by this follow-up.

Accessible bundle: download C10 CLI report and native-suite output from successful exact-head C10 run 38938591049. Artifact: c10-evidence-34fd6de3fa9735ee550c2da4da28b48e31bc1f9d-1, 16 files, retained 30 days (expires 15 November 2026). GitHub authentication may be needed. Reviewers no longer need access to ignored local paths.

The downloaded artifact was independently verified after CI:

• synthetic/evidence.json: actual, expected and after-run checker SHA all 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d; source tree clean before/after; immutable checkout verified.
• Exact checker argv, shell-readable command and cwd; Python/package versions; all 7 source hashes and 6 generated dataset hashes checked against the checkout/archive, before and after execution.
• Dataset: openamrobot/synthetic-c9-contract-fixture, revision synthetic-fixture-1, episode synthetic-episode-000000, index 0, origin synthetic.
• synthetic/c10-report.json: unchanged raw CLI NOT_VERIFIED / exit 2, training_ready=false, production_acceptance=NOT_VERIFIED; 134 PASS + 22 NOT_VERIFIED findings, native fixture coverage 2/2 and selected-time skew 0 ns on two synthetic frames.
• Raw report SHA-256 d5280ecbfaa4ccd5da4d7f72aac24b6adb3a78ef295a524544df19114389abbe, identical to the clean-head local report. Archive SHA-256 matches GitHub's published digest: 03659517404adfad6d2cb604136c92a49af2b55981716b8259b822eb67d489f8.
• core-suite.log, native-suite.log, evidence-suite.log: 63 / 31 / 23 tests, each OK, zero unittest skips. The native log confirms the actual synthetic MP4 creation/decode/corruption branch; this is not a robot recording.
• generation.log, original CLI stdout/stderr, revert-negative output, native Parquet, matching C9 sidecar and synthetic policy are included.

The helper's exit 0 means the expected blocked reporting path was observed. It does not rewrite the report or convert exit 2 into training readiness. Unexpected PASS/REJECT, inconsistent or missing reports, source/data drift and wrong/dirty expected checkouts fail the helper and retain available raw evidence.

Captured test/helper pipelines use pipefail; later evidence steps run after earlier failures unless cancelled, and upload uses if: always() with hidden files included. Local regression tests execute all five captured command pipelines with injected exit 23, verify logs remain and exit 23 is preserved, and enforce unconditional upload configuration. No continue-on-error or blanket || true is used. The successful hosted run proves accessible upload on this head; no deliberately failing hosted run is claimed.

Not verified

• REAL_EPISODE_NOT_VERIFIED: no actual recording was supplied or validated. The fixture/report is synthetic; no real dataset revision or hardware/calibration acceptance is claimed.
• Work-package owner's explicit adopt / adapt / reject decision is still pending. The review recommends conformance-reference placement while runtime ownership is agreed; it does not approve a runtime home, release pin or numeric policy.
• Runtime adoption, validated LeRobot release, timing/FPS/freshness/action/source profiles, authoritative bounds and calibration/configuration source artifacts/criteria remain unresolved. Existing baseline defaults are not substituted.
• Private contributor-agreement and other rights-holder authorizations remain unverified. No attestation is invented.
• Local Mac has no ROS Jazzy/colcon or ffmpeg/ffprobe. Local explicit unavailable branches are not media/ROS execution. Hosted results below supply synthetic media and full Jazzy evidence, without real-episode or production acceptance.

The full October 16 deliverable, including one real episode and production acceptance, is not satisfied by this handoff.

Integration Gate

Searched the authorized workspace, accessible organization repository list, organization C10/dataset issues and PR references, interfaces open PRs, accepted C9 issue #18/comments (including 13 October), merged PR #16, organization AGENTS/decisions and the existing baseline. No overlapping open interfaces PR or approved Embodied AI/C10 runtime home was found. Search absence is not proof that private decisions do not exist.

Reused unchanged: accepted C9 Draft 2020-12 schema; existing lint_schemas.load_data duplicate-key/nonfinite validation path; existing unittest runner/zero-test/no-skip guard; native LeRobot format/artifact bindings. The positive fixture is derived from the real repository C9 YAML and changes only its temporary copied synthetic data/policy, including the event interval needed to contain linked frame 0.

Inspected read-only: 4u-79/openamrobot-c-baseline at a29ce7c89b7c42c9147ad81e6a751de9dad045c9, especially both validators, eight-fixture harness, info, requirements and README. Useful corruption concepts informed independent checks. No baseline code was copied or changed: that commit has no tracked source licence. Its 19-channel historical fixed-mast constants, camera aliases, epoch comparisons, 30 ms/95% FPS defaults and nonblocking stale-image flags were rejected as accepted C9 policy.

Placement: interfaces remains canonical contract owner; C10 is a consumer under existing tools/ and tests/ as a draft conformance reference pending architecture review. No runtime ownership is invented; no ROS recorder, training, motion, safety or governance implementation is added. C9 schema/docs/fixtures, native format and frozen contracts are unchanged.

Follow-up reuse: the existing C9-derived fixture/native writer supplies the evidence inputs unchanged. The generator launches the original checker as a subprocess and inventories exact input/output bytes. No accepted contract, policy values, runtime ownership, dependency pin or baseline source was changed in response to the review.

Compatibility and unresolved decisions

The compatibility matrix compares accepted C9 and the baseline across cameras, ordered/versioned vectors and units, clock domains/mapping, sample time, skew/action latency, release provenance, calibration/configuration hashes and native bindings.

• C9 head/base are required; camera_scene/camera_base_nav are never silently aliased.
• 16-channel C9 example and historical 19-channel profile remain distinct. Profiles supply exact names/order/units/frames; bounds require explicit reviewed versioned evidence.
• Recording-relative integer ns and per-stream raw-unit affine mapping are recomputed with inclusive validity and exact decimal ties-to-even rounding. Observation skew excludes action; explicit action semantics/reference-frame offset govern separate latency.
• Pending/unmapped/missing evidence remains NOT_VERIFIED. A declared C9 tolerance must match the externally reviewed policy; no default is assumed.
• Inspected LeRobot v0.6.1/v3.0 and baseline lerobot==0.4.1 are not approved project release pins. External snapshot identity evidence can bind standard native metadata without injecting nonnative fields into info.json.

File-by-file changes

File	Purpose
tools/check_c10_dataset.py	CLI, native v3/legacy metadata and Parquet IO, video count/decode/pixel evidence, exact-byte hashes, calibration/release/real-episode evidence and strict contextual JSON/console report.
tools/c10_semantics.py	18 semantic diagnostic families: native bindings/features/tasks/coverage, clocks/selection/health/sequences/skew/sync/FPS, vector/profile/bounds/action latency, events/outcomes/completeness.
tools/generate_c10_evidence.py	Fresh reproducible synthetic CLI bundle: exact Git SHA/command, unchanged raw report/logs, source/native/report hashes, environment and strict expected-verdict checks.
tests/test_c10_evidence.py	23 evidence integration/regression tests, including output retention, source/data drift and actual shell failure preservation.
docs/c10-evidence-handoff.md	Download/reproduction instructions and pending owner decision checklist.
tools/prove_c10_negative.py	Temporary precise revert experiment; unchanged corruption test must fail for the intended diagnostic.
tests/c10_fixture.py	Independent synthetic sidecar/native/profile policy derived from canonical C9; native artifacts generated only in temporary locations.
tests/test_c10_dataset.py	63 dependency-light semantic/corruption/boundary/pending tests; C10-only mutations retain C9 structural validity.
tests/test_c10_native.py	31 explicit optional-native integration tests including actual truncated/corrupt Parquet, strict report/exit/read-only checks, and explicit video capability handling.
tools/requirements-c10.txt	Standalone install declarations for existing schema dependencies only.
tools/requirements-c10-native.txt	Optional Parquet dependency.
tools/verify.sh	Adds core C10 suite to existing shared verification without weakening existing gates.
.github/workflows/c10-dataset-quality.yml	Checks out the exact PR head, captures all suites/proof/CLI output and uploads evidence with always(), including after failures; read-only fork PR permissions.
docs/c10-readme.md	Mac install/run/configuration, report/exit meanings, recorder/C11 handoff and limitations.
docs/c10-compatibility-matrix.md	Accepted C9 versus existing baseline and unresolved decisions.
docs/c10-provenance.md	Search/reuse/licensing/placement evidence and dependency sources.
docs/c10-check-fixture-matrix.md	46 named C9-valid semantic corruptions plus 21 wrapper cases covering all 28 diagnostic families.
docs/c10-verification.md	Exact execution commands/counts/environment, synthetic report results, revert evidence and explicit unsuccessful/unavailable checks.

Tests

Branch base: 87ab8a936c97ab55ee74b8988ca3d7659160ea6a.
Follow-up parent: 8440f4b818f4bd0e6bbbbcc1538eee0555e0012a.
Current head: 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d.
Branch: SatyaA27:c10-dataset-quality-gate → openAMRobot/openamrobot-interfaces:main.

Verified hosted results for current head

Check	Result
C10 reference	SUCCESS: 117 tests (63 core + 31 native + 23 evidence), original arithmetic revert proof, clean exact-SHA CLI generation, and artifact upload. Artifact was downloaded and verified as described above.
Full ROS Jazzy quality/build	SUCCESS: 15 schema tests, 63 C10 core tests, 14 compatibility tests, generated/install/consumer/DDS stages and expected exit-42 reverted-interface rejection; log ends PASS: all implemented verification stages.
Repository quality	SUCCESS. Its separate applicability-only quality/test job is SKIPPED and is not counted as test evidence.
DCO	SUCCESS.

C10 workflow checks out the explicit PR-head SHA and requires it in the generator, rather than describing a merge checkout as the tested checker revision. Hosted C10 environment: Python 3.12.15 on Ubuntu 24.04; jsonschema 4.26.0, PyYAML 6.0.3, pyarrow 26.0.0. The artifact lists complete installed package versions. Local environment: macOS 26.5.1 arm64, Python 3.12.13 with the same three package versions.

Exact local commands executed from repository root:

/private/tmp/openamrobot-c10-venv/bin/python tools/run_verification_tests.py tests 'test_c10*.py'
# PASS: 117 tests, no skips
/private/tmp/openamrobot-c10-venv/bin/python tools/run_verification_tests.py tests test_compatibility.py
# PASS: 14 tests
/private/tmp/openamrobot-c10-venv/bin/python -m unittest discover -s tests -p test_schema_validation.py -k SchemaTests -v
# PASS: 10 tests
/private/tmp/openamrobot-c10-venv/bin/python -c 'from pathlib import Path; import sys; sys.path.insert(0,"tools"); from lint_schemas import validate_schemas; print(validate_schemas(Path.cwd()))'
# PASS: 1 schema / 10 registered C9 assets
/private/tmp/openamrobot-c10-venv/bin/python tools/prove_c10_negative.py
# PASS proof: current test passes; temporary reverted arithmetic rule gives exactly 1 expected assertion failure
/private/tmp/openamrobot-c10-venv/bin/python tools/generate_c10_evidence.py \
  --output-dir .verification/c10-review-evidence/exact-head-synthetic \
  --expected-checker-sha 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d
# helper 0; raw gate NOT_VERIFIED / exit 2; source clean; report matches CI byte-for-byte

The full existing schema suite was also executed locally with .../python tools/run_verification_tests.py tests test_schema_validation.py: 15 tests, FAILED with 3 missing-catkin_pkg errors. It is not a local pass. All 15 passed in hosted Jazzy. Local bash tools/verify.sh was not run without Jazzy/colcon; hosted quality/build ran that full command successfully.

Additional follow-up revert proof: in a temporary helper source copy only, replace if not expected_gate_result(report, manifest["gate"]["process_exit_code"]): with if False: and run unchanged C10EvidenceTests.test_rejects_unexpected_gate_pass. It failed exactly once at assertNotEqual(helper_exit, 0) because the reverted guard accepted an unexpected PASS. No import errors/skips; original tracked source and test expectations were preserved. The 23-test suite includes that unchanged negative test on the real helper, source/data/HEAD drift, exact exit-code types, report reproduction, fresh-directory protection and retained failure evidence.

Exact checker command captured by the hosted manifest (cwd /home/runner/work/openamrobot-interfaces/openamrobot-interfaces):

/opt/hostedtoolcache/Python/3.12.15/x64/bin/python /home/runner/work/openamrobot-interfaces/openamrobot-interfaces/tools/check_c10_dataset.py --dataset-path /home/runner/work/openamrobot-interfaces/openamrobot-interfaces/.verification/c10-evidence/synthetic/dataset --policy /home/runner/work/openamrobot-interfaces/openamrobot-interfaces/.verification/c10-evidence/synthetic/dataset/synthetic-policy.json --report /home/runner/work/openamrobot-interfaces/openamrobot-interfaces/.verification/c10-evidence/synthetic/c10-report.json

For another machine, use a clean checkout at 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d, install the recorded dependency versions, then run:

python tools/generate_c10_evidence.py \
  --output-dir .verification/c10-evidence/synthetic \
  --expected-checker-sha 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d

Output must be fresh. The generated manifest captures the new machine's actual interpreter/paths; raw report and fixture reproduction remain separate from invocation-specific timestamp/path fields. Handoff documentation describes every artifact and failure behavior. Earlier local ignored evidence remains supplemental, not the reviewer handoff.

Dependencies and licences

No training stack/LeRobot release pin or unrelated dependency was changed. jsonschema>=4.18,<5 (PyPI, MIT) and PyYAML>=6,<7 (PyPI, MIT) expose existing standalone schema prerequisites. Optional pyarrow>=17,<27 (PyPI/Apache Arrow, Apache-2.0) supports native inspection. External ffmpeg/ffprobe are distribution-installed only for media tests (LGPL/GPL depending on build); no binary vendored. Software is MIT and documentation CC BY 4.0 under the existing licence map. Baseline source was not copied because licence permission is unresolved. Provenance and modifications are documented.

This follow-up adds no dependency or manifest change; it reuses the installed native-test environment and standard library. New helper/tests carry MIT SPDX headers; handoff documentation is CC BY 4.0 under the existing licence map.

Intellectual property and provenance

• All commits carry DCO sign-off using the contributor's actual configured identity, as explicitly requested.
• Applicable accepted Individual/Corporate Contributor Agreement coverage verified.
• Required employer/university/other rights-holder authorizations verified.
• Material AI assistance and inspected third-party sources disclosed; no baseline source copied.
• No intentionally included credentials, private documents or real-recording payloads.

AI disclosure

OpenAI Codex produced the draft checker, synthetic fixtures/tests, CI wiring and documentation with multi-agent implementation and adversarial review. As requested, the semantic agent used gpt-6-astra with high reasoning; test work used gpt-6.1-sol with high reasoning; audit/documentation used gpt-6.1-sol with medium reasoning. Human approval, hardware evidence, contributor-agreement acceptance and release acceptance were not generated or inferred.

Follow-up implementation also used OpenAI Codex agents for the evidence generator, 23 regression tests, workflow/documentation updates and adversarial review. No human adoption decision, release/timing/action/calibration approval, real recording or agreement attestation was inferred.

Owner and unresolved decisions

One checklist remains open; no pending item is filled with baseline defaults:

• SatyaA27: record an explicit adopt / adapt / reject decision in this PR thread.
• Architecture/runtime owners: confirm runtime repository ownership and adoption scope. Conformance-reference placement is only the scoped review recommendation.
• Release owner: provide validated LeRobot release/commit evidence.
• Recording/profile owners: approve timing/FPS/freshness and action/source/vector profiles, units/order/bounds/observation relationship.
• Calibration/configuration owners: provide immutable versioned source artifacts, exact-byte hashes and residual acceptance criteria.
• Recording owner: supply the next actual native episode revision, matching C9 sidecar/artifacts and real-episode acceptance report.

@panthera-momagdii: evidence handoff is now accessible and reproducible. Please review these outputs and pending decisions. The prior formal reviewer request was denied by GitHub permission; no reviewer assignment or approval is fabricated. Keep draft; no merge or ready-for-review transition is requested without the user's review.

[panthera-momagdii]

Initial, scoped review at 8440f4b818f4bd0e6bbbbcc1538eee0555e0012a. I inspected the checker and semantic-gate implementation, native corruption/read-only tests, the C10 workflow, and the recorded limitations. This is not a full audit of every changed file or a local execution of the suites.

I independently checked the hosted results: C10 reference run 38861626485 and its nonempty-suite step succeeded; ROS 2 Jazzy run 38861626521 and its quality/build verification step succeeded; Repository quality run 38861627006 also succeeded. These are useful implementation results, not proof of a real recorded episode or production acceptance.

The distinction between REJECT and NOT_VERIFIED, explicit policy inputs, preservation of the C9 contract, and keeping synthetic data out of training-ready acceptance are the right direction. For the placement discussion, my recommendation is to keep this a conformance-reference consumer under tools/ and tests/ while runtime ownership is agreed separately. This comment does not approve a runtime home, numeric policy, release pin or source profile.

Please keep the PR draft. Next steps:

1. Record the work-package owner's explicit adopt/adapt/reject decision and the unresolved owner decisions in one short checklist. Do not fill missing release/timing/action/calibration policies with baseline defaults.
2. Attach one accessible, reproducible C10 CLI report with the exact command, checker SHA and dataset revision. A synthetic report is useful for reviewing the reporting path, but must remain labelled synthetic and NOT_VERIFIED. The next real-episode acceptance report must include its matching native artifacts and C9 sidecar.
3. Consider retaining that report and the native-suite test output as workflow artifacts, including on failure; the C10 workflow currently runs the suite but has no upload step. Local ignored evidence paths are not a handoff other reviewers can retrieve.

No merge approval or completed-C10 claim is given here. Review assisted by ChatGPT; no local ROS, native-video, dataset or hardware run was performed in this review.

[panthera-momagdii]

Reviewed the evidence-handoff follow-up from 8440f4b to 34fd6de3fa9735ee550c2da4da28b48e31bc1f9d.

Review basis: openAMRobot/openamrobot-interfaces, branch c10-dataset-quality-gate, head 34fd6de, base 87ab8a9. Expected outcome: close the evidence-access follow-up while keeping C10 acceptance and owner adoption explicit.

The five-file follow-up addresses the requested report access and retained test output. The helper preserves the checker's raw NOT_VERIFIED result and exit code 2, records the reviewed commit and available inputs, and retains pipeline failures rather than turning a successful tee into a passing test. The core checker, C9 contract, and dependencies are unchanged in this delta.

I inspected the hosted C10 job: 63 core tests, 31 native tests, and 23 evidence tests completed successfully, including the expected arithmetic-revert negative control. The evidence artifact is available at run 38938591049, artifact 12062027077. The separate Jazzy verification run also passed, including its expected consumer failure after the revert. I verified the job output and artifact availability; I did not independently unpack or hash the uploaded ZIP.

No new blocking finding in this evidence-only follow-up. Please keep the PR draft until the work-package owner records adopt/adapt/reject and the runtime, release/profile, and calibration decisions are settled. Acceptance still needs an actual recorded episode with its native artifacts and matching C9 sidecar. The synthetic handoff verifies the reporting path; its NOT_VERIFIED result does not establish real-episode or production acceptance.

ChatGPT-assisted source and hosted-evidence review; no local ROS, hardware, or real-dataset execution.