Land evaluation sources (batch b) - #135
scrimshawlife-ctrl merged 12 commits into
Conversation
e56419a
into
cursor/select-001-holdout-land-41af
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eafb92477d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| import unicodedata | ||
| from pathlib import Path | ||
|
|
||
| from hyperlexical.unbind_screen_v3 import NUMBERS, PARTICLES, STOP, stems |
There was a problem hiding this comment.
In this commit's tree, hyperlexical.unbind_screen_v3 does not exist, so importing the newly added v4 scorer immediately raises ModuleNotFoundError; for example, PYTHONPATH=scripts/shadow pytest -q tests/shadow/test_km_candidate_evaluation.py fails during collection at this import. This also prevents the v4/v5/v6 measurement drivers from starting, so the required dependency needs to be included before these evaluation scripts are usable.
Useful? React with 👍 / 👎.
| if receipt.get("state") not in {"PREDICTIONS_FROZEN", "OPERATOR_LABELING", "LABELS_FROZEN"}: | ||
| refuse(f"cannot freeze labels from state {receipt.get('state')}") |
There was a problem hiding this comment.
When freeze_labels is called a second time before scoring, the receipt is already in LABELS_FROZEN, but this state is accepted and the existing label artifact and its recorded hash are silently overwritten. That makes a supposedly frozen operator judgment mutable and can change the resulting evaluation without leaving the original freeze protected; only the pre-freeze labeling states should permit writing this artifact.
Useful? React with 👍 / 👎.
1 participant
Lands the Spark evaluation-source scripts from land-b onto the integration branch. Byte-checked against the Spark tree. Does not change evaluation-reserve.md.