Skip to content

jev_decide: surface contradicted requirements, add opt-in escalation - #46

Closed
Rex-Gao wants to merge 1 commit into
jkudish:mainfrom
Rex-Gao:decide-contradiction-escalation
Closed

Rex-Gao wants to merge 1 commit into
jkudish:mainfrom
Rex-Gao:decide-contradiction-escalation

Conversation

@Rex-Gao

@Rex-Gao Rex-Gao commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #43. This includes the structural half of #44 (recommendation.contradicted_requirements, same shape: zero-based requirement indexes from contradictsRecommendation) and adds the opt-in escalation half that issue left as a maintainer decision — happy to defer to #44 for the structural piece if that lands first.

What

  • recommendation.contradicted_requirements: number[] now ships on every jev_decide result (empty when nothing contradicted, [] on invalid_response), so the disagreement is structural, not a warning string.
  • New optional input escalate_on_contradiction (default false, current behavior unchanged): when set, a recommendation whose own requirement checks came back contradicted returns selected: null with status: "escalate" — the jev_verify vocabulary — instead of the candidate id. probabilities and contradicted_requirements stay inspectable.
  • No re-selection, per option 3's fail-closed argument in the issue: a withdrawn recommendation is never silently replaced by the runner-up; the model's genuine pick stays visible next to the escalation.

Testing

npm run typecheck, npm run build, npm test: 245 tests. New coverage: default keeps selected with the structural array and the warning; escalate_on_contradiction withdraws a contradicted pick without promoting the runner-up; the flag keeps a non-contradicted recommendation untouched (a contradiction on a different candidate never escalates).

Docs

README jev_decide section: result example gains the field, and a bullet documents the flag and the no-re-selection rule.

A recommendation whose own requirement checks come back contradicted was
kept as selected with only a warning string, so a caller reading
recommendation.selected or status alone could act on a flagged candidate
(jkudish#43).

recommendation.contradicted_requirements now names the zero-based
requirement indexes (the data contradictsRecommendation already
computed), and the new optional escalate_on_contradiction input
withdraws a contradicted recommendation as selected: null with status
"escalate" (the jev_verify vocabulary) instead of the candidate id.
Default behavior is unchanged; no re-selection ever happens, the
probabilities stay inspectable.

jkudish commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Thank you — merged, with one docs amendment: escalation happens in addition to the warning, and the indexes describe the pick before withdrawal. Shipping in the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jev_decide: a contradicted recommendation only surfaces as a warning string

2 participants