Skip to content

Equivalence-aware SC matching. Keep math-verify optional via lazy import #264

Description

@lukeinglis

Context: implementing equivalence-aware answer matching for Self-Consistency
(math-verify-style canonicalization) as a custom consistency_space_projection_func,
per the SC hardening doc [link].

math-verify is currently in the research extra only, not core deps:

research = [
    "math-verify>=0.1.0",  # For mathematical reasoning evaluation in benchmark scripts
    ...
]

Proposal: keep it there. The new projection function will lazy-import math-verify
inside the function body and raise a clear error ("install its_hub[research] to use
this feature") if it's missing, rather than promoting math-verify to a core
dependency. This keeps the default pip install its_hub install footprint and
behavior completely unchanged — the feature is opt-in the same way
create_regex_projection_function is today.

Open question for review: does this satisfy the doc's "no new deployment
dependency, core install only" success metric as written, or does that metric
need a footnote clarifying it means "no new required dependency" rather than
"no new dependency in the codebase at all"? Flagging since math-verify is already
present in the repo, just not in the path this feature would use it in.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions