Repository navigation
Add opt-in jevgate-1 classify path - #125
Conversation
Keep lexical match_lineage as the default cascade. HYPERLEX_JEVGATE or classify --jevgate calls pinned jev-1.13.0 after a miss, abstaining unless mean P(none) is strictly below the prereg v5 tau. Co-authored-by: Daniel Meyer <scrimshawlife-ctrl@users.noreply.github.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9d516a53b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return importlib.import_module("hyperlexical.layout") | ||
| except ImportError: | ||
| path = Path(__file__).resolve().parents[3] / "scripts" / "shadow" / "hyperlexical" / "layout.py" | ||
| if not path.is_file(): | ||
| raise JevCallError("layout_unavailable", "hyperlexical.layout is not importable") from None |
There was a problem hiding this comment.
When a user invokes the documented package entry point on a registry miss (python -m hyperlex classify ... --jevgate), a normal wheel contains only packages under src and declares no hyperlexical dependency, so neither import path here exists. After making all three paid provider requests, eight_families() therefore returns layout_unavailable, and every otherwise successful Jev classification becomes none. Keep the canonical family order in an installed module or include/declare the layout dependency.
Useful? React with 👍 / 👎.
| def _is_prob(value: Any) -> bool: | ||
| if isinstance(value, bool) or not isinstance(value, (int, float)): | ||
| return False | ||
| return math.isfinite(float(value)) |
There was a problem hiding this comment.
When the provider returns malformed but finite numeric values, _is_prob accepts negatives and values above 1 as valid probabilities. For example, three responses with P(none) = -1 and an ai-native value of 2 produce an ai-native classification instead of the promised parse-error fail-closed result, because -1 < TAU; validate every probability as 0 <= value <= 1 before admitting the pass.
Useful? React with 👍 / 👎.
All-zero and all-equal family means are not a family signal. The tau gate on family is unchanged. Co-authored-by: Daniel Meyer <scrimshawlife-ctrl@users.noreply.github.com>
2 participants
Summary
Opt-in family classification for one short term: lexical
match_lineagestill decides when it fires, and the existing cascade (miss →none) stays the default.HYPERLEX_JEVGATE=1orclassify --jevgatecalls TypeSafe Jev only after a registry miss.The gate is frozen by prereg v5: model
jev-1.13.0, three identical passes, tau0.30333(a family only when meanP(none)is strictly below tau). It abstains often by design.jev_best_guess,jev_p_none, and the per-family means are low-confidence and never change the decision.jev_best_guessis null when every family mean is 0 or all eight means are equal (no family signal). Jev failures retry once, then returnnonewithjev_errorset.Prompt sha256 (
sha256(JSON.stringify(QUESTIONS))):0dab58787402cb6e5aadc92e3123a65f09d281444e100b8485ede273c7d9af53. Checked at import; a mismatch refuses the Jev path.How to enable
--no-jevgateforces the registry cascade even ifHYPERLEX_JEVGATEis set.python -m hyperlex classifyis the same command.API key
JEV_API_KEY(orTYPESAFE_AI_API_KEY). AlternativelyJEV_KEY_FILEis a path, from the environment, of a file that contains the key. There is no default path and no key in the repo.Files changed
src/hyperlex/analysis/jevgate.py— cascade, frozen prompt, HTTP client forPOST /v1/systemonesrc/hyperlex/cli.py—classifycommand and--jevgate/--no-jevgatescripts/hyperlex.py— same command on the skill CLItests/test_jevgate.py— rule precedence, tau boundary, tie-break, best-guess, flat means, retry-then-fail-closed, model-id mismatch, prompt sha (mocked client, synthetic strings)docs/jevgate.md— gate, tau, model, prompt sha, abstention, best-guessdocs/commands.md— command map rowsmkdocs.yml— nav entryTests
PYTHONPATH=src python3 -m pytest -q— 919 passed, 10 skipped.