Skip to content

Add submission: 0xUrsanomics - #70

Open
0xUrsanomics wants to merge 1 commit into
8090-inc:mainfrom
0xUrsanomics:submission/0xUrsanomics
Open

0xUrsanomics wants to merge 1 commit into
8090-inc:mainfrom
0xUrsanomics:submission/0xUrsanomics

Conversation

@0xUrsanomics

Copy link
Copy Markdown

Adds submissions/0xUrsanomics/ with predictions.jsonl (5,000 of 5,000 validation cases), MEMO.md, and SUBMISSION.md. No files outside that folder are touched.

Solution repository: https://github.com/0xUrsanomics/mib-doc-challenge-solution (public, MIT, Dockerfile at root).

Score on the labelled train split via scripts/evaluate.py: 101.42 / 150 (fields 40.37/50, classification 48.94/80, calibration 12.11/20, missing-case penalty 0/10). Predictions were produced by that image under --cpus 4 --memory 8g --network none --read-only, 0.091s per PDF against a 6s budget.

No model at runtime: PyMuPDF for text, tesseract for scan pages, hand-written rules for adjudication.

The trust layer is the part of the write-up I would point at first. Across the 188 train packets carrying a planted answer key, the key's field values agree with ground truth 93.1% of the time while its adjudication agrees 0 times out of 188, so the verdict is refused unconditionally and the field values are read only as a last resort, after adjudication has already returned. Details and known failure modes in MEMO.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant