Repository navigation
Update RF100-VL skill with the 2026-08 campaign measurements - #710
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| backend = getattr(self.coco_evaluator, "last_backend", None) | ||
| if backend: | ||
| metrics["provenance/coco_backend"] = backend |
There was a problem hiding this comment.
When standalone detection validation runs with verbose output enabled by default, provenance/coco_backend reaches the inherited numeric-only metric printer, which formats every value with %.4f and emits a logging TypeError traceback to stderr.
Knowledge Base Used: Validation Pipeline
There was a problem hiding this comment.
Valid catch, resolved by removal: the branch was re-scoped after #707/#708 merged the code half from a fresher base. dev carries backend provenance as model.last_eval_backend plus the eval_backend field in the CLI JSON payload, keeping the metrics dict float-only, so the string never reaches the numeric printer. This PR no longer touches detection_validator.py.
| # opt-in only made every user pay for detection-dense datasets. Falls back | ||
| # to pycocotools automatically when the package is not installed; set False | ||
| # or LIBREYOLO_FASTER_COCO_EVAL=0 to force the reference implementation. | ||
| faster_coco_eval: bool = True |
There was a problem hiding this comment.
When libreyolo val or libreyolo train is invoked without the evaluator option, the CLI supplies an explicit False that overrides these new True config defaults, causing ordinary CLI runs to keep using pycocotools instead of the advertised default backend.
Knowledge Base Used:
There was a problem hiding this comment.
Correct on this branch's base, but already fixed on dev: #708 flipped the CLI defaults to True with --faster-coco-eval/--no-faster-coco-eval toggles in both val and train, so the CLI no longer overrides the config default. The branch was re-scoped onto current dev and no longer touches the configs.
|
Follow-up on the "Not verified" note: the full unit suite has now finished green. Run against this branch with |
Re-scoped from the original PR after #707 and #708 merged the code half (non-strict load logging, faster-coco-eval default flip with CLI toggles, backend provenance via model.last_eval_backend) from a fresher base. What remains here is the part dev does not have: the skill rewrite with measured numbers (scoring cost vs box density, OMP thread pinning, packing vs makespan, per-family VRAM measurement, disk projection constants, multi-box operations, stop-is-not-restartable, the sync-artifacts trap, the selection-vs-test divergence check) plus the fast-eval extra comment that still described the backend as opt-in.
629a47d to
f75ad8d
Compare
EHxuban11
changed the title
|
Review + re-scope (Cursor agent, on Xuban's behalf). What was wrong: this branch was cut from dev
What I did: rebuilt the branch as a single commit on current dev ( The skill content itself is the most valuable part of the original PR and survives untouched: measured scoring cost vs box density, OMP pinning, packing vs makespan, per-family VRAM measurement, disk projection constants, multi-box ops, stop-is-not-restartable, the |
1 participant
Skill and packaging half of the 2026-08 RF100-VL campaign postmortem. Harness fixes landed in vision-analysis-benchmark#7.
Re-scoped (by the Cursor agent): the original push also carried non-strict load logging, the faster-coco-eval default flip, and backend provenance, written from a base that predated #707 and #708 — both of which merged the same features into dev ~80 minutes earlier (with CLI toggles and YOLO-NAS loader coverage that this branch lacked). Those hunks are dropped; this PR now contains only what dev does not have.
What
skills/run-rf100vl-benchmark/SKILL.md: rewritten with what the campaigns actually measured — scoring cost scales with box density not image count (496 s scoring vs 112 s training ongwhd2021), OMP thread pinning when packing (11,890 threads on 128 cores, 67.6% system time), packing vs makespan, measured VRAM per family (same param count, 1.7x apart), disk projection constants (~953 KB/image at 640px), multi-box operations, "stopped is not restartable", thesync-artifactssilence trap, and the selection-vs-test divergence check that would have caught the BN eps bug in seconds.pyproject.toml: thefast-evalextra comment still said "Opt-in", which Make faster-coco-eval the default COCO metrics backend, with backend provenance #708 made stale; updated to describe the default-on-with-fallback behaviour and why it stays an extra.Check
Docs and a comment only; no code paths changed. The dropped features are covered on dev by
tests/unit/test_nonstrict_load_logging.pyandtests/unit/test_faster_coco_eval.py(includingtest_fallback_when_package_missing, which already asserts the default-on fallback this branch's extra test duplicated).Code provenance
All content written for this PR. No third-party code is copied, ported, adapted, or derived.
Greptile Summary
The PR now updates the optional fast-evaluation dependency guidance and expands the RF100-VL benchmark runbook with measured evaluator, capacity-planning, and campaign-safety guidance.
faster-coco-evalremains an optional extra despite being preferred by default.Confidence Score: 5/5
The PR appears safe to merge.
No blocking failure remains; both previously reported issues are fixed in the current code.
Important Files Changed
Reviews (2): Last reviewed commit: "Update RF100-VL skill with the 2026-08 c..." | Re-trigger Greptile