Repository navigation
Add RF100-VL training and evaluation harness - #6
Merged
Merged
Conversation
Mirror the ONNX backend path with a new `--format tensorrt` that drives
LibreYOLO's native TensorRT backend.
- cli.py: add `tensorrt` to --format choices; require --weights-dir for it.
- models.py: add resolve_tensorrt_weights() (looks for `{stem}.engine`,
clear FileNotFoundError) and load_tensorrt() returning a TensorRTBackend.
- benchmark.py: add _benchmark_tensorrt() as a faithful copy of the ONNX
black-box path (total wall time/image, no phase split, no VRAM), calling
the callable backend(image, ...) interface, plus a TensorRT warmup.
- output.py: parameterize runtime.precision (default fp32, unchanged for
pytorch/onnx); TensorRT emits format=tensorrt, precision=fp16,
provider=cuda, device=gpu.
- README: document the TensorRT/CUDA support row; drop it from out-of-scope.
PyTorch/ONNX paths unchanged.
Each result JSON now carries a `repro` block so a published number can be reproduced without guessing how it was run: - harness_commit + dirty flag (the harness side of the pin; a dirty tree is flagged rather than passed off as pinned) - verbatim argv and a copy-pasteable command line - dataset.image_id_sha256: a verifiable fingerprint of the evaluated image-id set (mini500 vs full val2017 vs any subset), plus the HF dataset label/revision - weights sha256 for user-supplied ONNX/TensorRT artifacts, plus an export manifest read from a <weights>.json sidecar and the detected ONNX opset Also records onnx + tensorrt versions in software info, adds --dataset-id and --dataset-revision flags, and bumps the harness to 2.1.0. New provenance.py is covered by tests/test_provenance.py; assemble_result threading is covered in tests/test_output.py.
Adds a "Reproducing a published result" section, requires the repro block in the output checklist (harness_commit real + not dirty, dataset image_id fingerprint, weights sha256 / export manifest for ONNX/TensorRT), and notes the <weights>.json export-manifest sidecar convention.
Fine-tuned protocol: per-dataset checkpoints via --weights-root, test-split COCO eval per dataset, unweighted mean AP across datasets, per-dataset breakdown in the submission JSON (dataset id rf100_vl). Download helper wraps the rf100vl package. Pretrained runs require an explicit flag and are marked non-submittable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
rf100vl-train: isolated dataset children, one queue consumer per GPU, atomic state/failure/rerun files, safe resume, timeouts, RF-DETR dense/OOM fallback, and 13 recipes.Code provenance
Original orchestration/cache/CLI code was written for this harness. Permissive sources documented in
THIRD_PARTY_NOTICES.txt:ac87f5077ad6b8864c2dc5e93d14cae62d1db05a, BSD-2-Clause.1c1ecd84cf2865270702121311e1d9e5f2a861aa, Apache-2.0.3e3c9dffc9e4bd41a363f1dd6ca1c736e145a5d8, MIT.How was this tested?