Skip to content
This repository was archived by the owner on Oct 10, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ pipeline / run / ingest
| `sources --route live` | Preview resolve |
| `ingest "<query>" --route offline` | Ingest only (structured + fingerprint) |
| `analyze "<query>" --route offline` | Analyze without auto-receipt |
| `classify "<term>"` | Family of one short term. Registry `match_lineage`, else `none` |
| `classify "<term>" --jevgate` | Same cascade, then opt-in [jevgate-1](jevgate.md) on a miss |

Prefer **`--route offline|live|glossary|social`** over raw adapter names.
Aliases: `real`→glossary, `x`→x_search, `firecrawl`→crawl4ai (Crawl4AI 0.9.3 default; no paid Firecrawl without Danny yes), `live`→combined.
Expand Down
49 changes: 49 additions & 0 deletions docs/jevgate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# jevgate-1 (opt-in classify)

Family classification of one short term into the eight hyperlexical families or `none`.

The default path is unchanged: lexical `match_lineage` (vector re-rank off). If that rule fires, its family decides. If it misses, the cascade returns `none`.

## Enable

```bash
export HYPERLEX_JEVGATE=1
python3 scripts/hyperlex.py classify "<term>"

# or one shot, without the env var
python3 scripts/hyperlex.py classify "<term>" --jevgate
```

`--no-jevgate` forces the registry cascade even when `HYPERLEX_JEVGATE` is set.

Package entry point: `python -m hyperlex classify "<term>" --jevgate`.

## Gate

Frozen by prereg v5. Do not tune.

| Item | Value |
|------|--------|
| Model | `jev-1.13.0` (pinned; a missing or different `model` id is an error) |
| Passes | 3 identical requests; probabilities are means |
| Tau | `0.30333` |
| Decision | a family only when mean `P(none)` is **strictly below** tau; otherwise `none` |
| Ties | earliest family in `FAMILIES` from `hyperlexical.layout` |
| Prompt sha256 | `0dab58787402cb6e5aadc92e3123a65f09d281444e100b8485ede273c7d9af53` |

`sha256(JSON.stringify(QUESTIONS))` is checked when the module loads. If it differs, the Jev path is refused.

The gate **abstains often by design**. `none` is the expected output for ordinary English, names, noise, and slang from communities outside the eight.

`jev_best_guess`, `jev_p_none`, and the per-family means are low-confidence fields. They never change `family`. `jev_best_guess` is the argmax over the eight families, ignoring tau, except when Jev gives no family signal: every family mean is 0, or all eight means are equal so the argmax would be only the layout tie-break. In that case `jev_best_guess` is null. The gated `family` is unchanged. Jev's own `choice`, `in_scope`, and `any_slang` may be logged under `jev_log` and never decide.

On provider error, timeout, parse failure, model mismatch, or a missing key, the call is retried once. If it still fails, the result is `none` with `jev_error.flag` and `jev_error.reason` set. Jev failures do not raise into callers.

## API key

The key is read only from the environment:

- `JEV_API_KEY` (preferred), or `TYPESAFE_AI_API_KEY`
- `JEV_KEY_FILE` — path of a file that contains the key (no default path)

There is no key in the repo. `HYPERLEX_OFFLINE=1` refuses the network call and fail-closes to `none`.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ nav:
- Virality: modules/virality.md
- Vector DB: modules/vectordb.md
- LLM: modules/llm.md
- jevgate-1 classify: jevgate.md
- Phase 5 research: phase5.md
- Simulation modules: modules/simulation.md
- Brier design: brier-calibration.md
Expand Down
32 changes: 32 additions & 0 deletions scripts/hyperlex.py
Original file line number Diff line number Diff line change
Expand Up @@ -726,6 +726,18 @@ def _resolve_log_path(args: argparse.Namespace) -> Path:
return default_log_path()


def cmd_classify(args: argparse.Namespace) -> int:
from hyperlex.analysis.jevgate import classify_term, jevgate_from_args

term = (getattr(args, "term_pos", None) or getattr(args, "term", None) or "").strip()
if not term:
_emit({"ok": False, "command": "classify", "error": "empty term", "brier": None})
return 2
out = classify_term(term, jevgate=jevgate_from_args(args))
_emit({"ok": True, "command": "classify", **out})
return 0


def cmd_analyze(args: argparse.Namespace) -> int:
pkg, err = _import_hyperlex()
if pkg is None:
Expand Down Expand Up @@ -922,6 +934,8 @@ def cmd_commands(_args: argparse.Namespace) -> int:
{"cmd": "sources --route live", "why": "Preview resolve for a route"},
{"cmd": "ingest \"<query>\" --route offline", "why": "Ingest only (structured + fingerprint)"},
{"cmd": "analyze \"<query>\" --route offline", "why": "Analyze without auto-receipt"},
{"cmd": "classify \"<term>\"", "why": "Family cascade: match_lineage, else none (jevgate off)"},
{"cmd": "classify \"<term>\" --jevgate", "why": "Opt-in jevgate-1 after a registry miss (prereg v5)"},
],
"research": [
{"cmd": "simulate --term <t> --mode scenario", "why": "Phase 5 research (SPECULATIVE, brier null)"},
Expand Down Expand Up @@ -2528,6 +2542,24 @@ def _add_pipeline_args(p: argparse.ArgumentParser, *, default_route: str = "offl
analyze_parser.add_argument("--out")
analyze_parser.set_defaults(func=cmd_analyze)

classify_parser = subparsers.add_parser(
"classify",
help="Classify a short term into 8 hyperlexical families or none",
)
classify_parser.add_argument("term_pos", nargs="?", default="", help="Short slang term")
classify_parser.add_argument("--term", default="", help="Short slang term (flag form)")
classify_parser.add_argument(
"--jevgate",
action="store_true",
help="Opt in to jevgate-1 after match_lineage misses (prereg v5). Default off.",
)
classify_parser.add_argument(
"--no-jevgate",
action="store_true",
help="Force the registry cascade even if HYPERLEX_JEVGATE is set",
)
classify_parser.set_defaults(func=cmd_classify)

run_parser = subparsers.add_parser(
"run",
help="AUTO backend (alias of pipeline): ingest → full results packet",
Expand Down
Loading
Loading