Context: implementing equivalence-aware answer matching for Self-Consistency
(math-verify-style canonicalization) as a custom consistency_space_projection_func,
per the SC hardening doc [link].
math-verify is currently in the research extra only, not core deps:
research = [
"math-verify>=0.1.0", # For mathematical reasoning evaluation in benchmark scripts
...
]
Proposal: keep it there. The new projection function will lazy-import math-verify
inside the function body and raise a clear error ("install its_hub[research] to use
this feature") if it's missing, rather than promoting math-verify to a core
dependency. This keeps the default pip install its_hub install footprint and
behavior completely unchanged — the feature is opt-in the same way
create_regex_projection_function is today.
Open question for review: does this satisfy the doc's "no new deployment
dependency, core install only" success metric as written, or does that metric
need a footnote clarifying it means "no new required dependency" rather than
"no new dependency in the codebase at all"? Flagging since math-verify is already
present in the repo, just not in the path this feature would use it in.
Context: implementing equivalence-aware answer matching for Self-Consistency
(math-verify-style canonicalization) as a custom consistency_space_projection_func,
per the SC hardening doc [link].
math-verify is currently in the
researchextra only, not core deps:Proposal: keep it there. The new projection function will lazy-import math-verify
inside the function body and raise a clear error ("install its_hub[research] to use
this feature") if it's missing, rather than promoting math-verify to a core
dependency. This keeps the default
pip install its_hubinstall footprint andbehavior completely unchanged — the feature is opt-in the same way
create_regex_projection_function is today.
Open question for review: does this satisfy the doc's "no new deployment
dependency, core install only" success metric as written, or does that metric
need a footnote clarifying it means "no new required dependency" rather than
"no new dependency in the codebase at all"? Flagging since math-verify is already
present in the repo, just not in the path this feature would use it in.