Source: codebase audit, 2026-06-11 (audit finding F5, Tier 2)
Severity: Medium
Category: Invariant surface
Problem
`submit_response` accepts the client's `shown_questions` array without validating it against the survey definition.
A misbehaving (or adversarial) client could submit a shown-set that includes question names that don't exist in the definition, silently violating CLAUDE.md invariant 3 ("shown-set captured at submit time from the SurveyJS engine; never reconstruct routing by re-evaluating visibleIf in SQL").
The dbt `shown_set_integrity` test catches mart-side fallout, but only at the next dbt run. An ingest-time check localizes the failure to the submitting request, giving a clear 422.
Suggested approach
After the hash check and before the `RawResponse` insert in `submit_response`:
- Build the set of valid question names from `survey.definition_json`. Reuse `_iter_elements` or similar helpers in api/survey_engine/validation.py.
- The set must include both plain question names AND matrix/panel cell names per the documented `m.r.c` / `p.element` composition (CLAUDE.md invariant 4; dbt `subquestion_name` macro).
- Validate: `set(shown_questions) ⊆ defined_question_names ∪ defined_cell_names`.
- We do NOT require it to be a subset of `payload.keys()` — a shown-but-skipped question won't appear in the payload (that's the whole point of capturing shown-set separately).
- On violation → raise `SubmissionRejected("shown_questions references undefined questions: ...")` → 422.
Tests
- Regression test in api/tests/test_responses.py: post with `shown_questions=["q_does_not_exist"]` → expect 422 with the rejection message.
- Positive case: matrix cell name in `shown_questions` (e.g. `"matrix.row1"`) is accepted.
- Positive case: panel cell name (`"panel.0.element"`) is accepted.
Out of scope
The fact that the API trusts the client to correctly report which questions were shown remains by design — this issue closes the narrower "shape" gap (referenced names exist), not the "fidelity" gap (the client truthfully reports SurveyJS visibility).
Source: codebase audit, 2026-06-11 (audit finding F5, Tier 2)
Severity: Medium
Category: Invariant surface
Problem
`submit_response` accepts the client's `shown_questions` array without validating it against the survey definition.
A misbehaving (or adversarial) client could submit a shown-set that includes question names that don't exist in the definition, silently violating CLAUDE.md invariant 3 ("shown-set captured at submit time from the SurveyJS engine; never reconstruct routing by re-evaluating visibleIf in SQL").
The dbt `shown_set_integrity` test catches mart-side fallout, but only at the next dbt run. An ingest-time check localizes the failure to the submitting request, giving a clear 422.
Suggested approach
After the hash check and before the `RawResponse` insert in `submit_response`:
Tests
Out of scope
The fact that the API trusts the client to correctly report which questions were shown remains by design — this issue closes the narrower "shape" gap (referenced names exist), not the "fidelity" gap (the client truthfully reports SurveyJS visibility).