Add Scaled-YOLOv4 family and yolov7x pretrained models - #137
Merged
Conversation
Convert the per-output transform assignments from A[...] = RHS (which materializes RHS before copying into the view) to fused in-place .= broadcasts. Outputs verified unchanged vs stored refs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two scaling problems when many detections reach NMS: - perform_detection_nms scanned all columns once per batch (findall), then once per distinct class (Set + findall), then ran a per-group sortperm and matrix copy, all through nested index-vector views. Now all detections are ordered once by (batch, class, -score) with a single stable sortperm and one gather; every (batch, class) group is then a contiguous, already-sorted column range. - nms! allocated an index slice per suppression round to build the IoU view (O(N^2) bytes across a group). bboxiou! gained a column-index variant so rounds now read through a view of the persistent index vector with no per-round allocation. For v3_416 with all 10647 candidates kept (detect_thresh=0, overlap_thresh=1), NMS drops 14.6 ms / 53.9 MiB -> 11.6 ms / 14.2 MiB. Output content is unchanged (verified vs stored refs for v2, v2_tiny, v3_tiny, v3, plus NMS unit tests and the soft-NMS fuzz check); column order is now batch, then ascending class id, then descending score, where classes were previously in first-appearance order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
extend_for_attributes allocated a zeros block, cat copied everything, permutedims copied everything again, and the batch-number fill ran on the intermediate 5D layout. On the CPU path (Array/AllocArray) the whole sequence is now one allocation: permutedims! straight into the first `a` rows of the (a+4)-row destination, zero-fill of the 4 attribute rows, and a contiguous range fill for batch numbers on the final layout. GPU arrays keep the dense cat + permutedims path, gated by a new fast_scalar_indexing trait that CUDAExt sets to false for CuArray. "processing outputs" allocations drop 11.0 MiB -> 3.7 MiB for v3_416 with 10647 candidates. Outputs verified unchanged vs stored refs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
keepdetections previously required cat-ing all output heads into one matrix, then built a boolean mask and gathered a second copy. A new vector method counts and copies the kept columns from the per-head matrices directly (function barriers keep the loops concretely typed; GPU arrays fall back to the cat path via the fast_scalar_indexing trait). "filter detections" drops 743 us / 9.1 MiB -> 199 us / 4.5 MiB for v3_416 with all 10647 candidates kept. Outputs verified unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
At inference batchnorm is a per-channel affine map, so fold it into the conv kernel and bias when weights are loaded (as darknet's fuse_conv_batchnorm does), removing the BatchNorm layer and its full read+write pass over every conv output. Forward pass on an M2 Pro (CPU, single image, min of 15): v3_416 303.7 -> 290.4 ms, v3_tiny_416 36.0 -> 32.6 ms, v4_416 543.7 -> 514.0 ms. Numerics shift only by float rounding: max abs deviation vs the stored reference detections is < 1e-6 across v2/v2_tiny/v3_tiny/v3, far inside the test tolerance, so no reference updates are needed. Side benefit: the positional Flux.BatchNorm internal constructor -- a compat hazard across Flux versions -- is no longer used. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The forward pass is gemm-bound: AppleAccelerate is ~35% faster end-to-end than the default OpenBLAS for v3_416 on an M2 Pro, and BLAS thread count (not Julia threads) is what controls conv parallelism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The seen/seen_images header read was not guarded for the dummy case, so constructing any model without a weights file threw MethodError(read, (nothing, 8)). The dummy path is useful for cfg validation and future precompile workloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Official AlexeyAB cfgs contain trailing inline comments
("layers = 339 ###") and whitespace-only lines, both of which crashed
cfgread/cfgsplit (seen in cspx-p7-mish.cfg and yolov4-csp-x-swish.cfg).
Strip comments and whitespace-only lines up front.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New models, all loaded from official upstream weights (AlexeyAB darknet releases; yolov7x from WongKinYiu/yolov7 v0.1): - v4_csp_COCO (yolov4-csp, native 512) - v4_csp_x_swish_COCO (yolov4-csp-x-swish, native 640) - v4x_mish_COCO (yolov4x-mish, native 640) - v4_p5_COCO (yolov4-p5, native 896) - v4_p6_COCO (yolov4-p6, native 1280) - v7x_COCO (yolov7x, native 640) Pretrained models now default to their native input size via MODEL_DEFAULT_SIZE (previously-supported models keep their 416 default, unchanged). Fixed-size convenience constructors come from MODEL_CONVENIENCE_SIZES; v4_p6 sizes must be divisible by 64, the rest by 32. Weights are repackaged as lazy artifacts on the existing `weights` release tag; tarball sha256s verified against the uploaded assets, including a download round-trip. All six models verified against Darknet.jl (AlexeyAB) at native size on both test images: identical classes and boxes/confidences within 0.05. Notably this is the first coverage of the new_coords=1 decode path (csp/x-swish/x-mish/p5/p6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reference images and detection refs for all six new models on both test images, generated locally against Darknet.jl with parity asserted at generation time (12/12 combinations). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The lts/ubuntu job was killed by the runner 3 seconds into loading v4_p5: darknet's loader allocates every layer's activations eagerly at cfg size, and 896/1280-pixel inputs exhaust a 7 GB GitHub runner. Parity vs darknet holds at any valid size as long as both sides use the same one, so the big models are now tested at 512 (csp-x-swish, x-mish, p5) and 448 (p6, stride 64). v7x stays at its native 640 -- proven to fit by csp-x-swish passing at 640 on the same runner before the kill. The Darknet.jl side loads a temp cfg with width/height overridden to match. Also drop net/yolomod references before the per-model GC.gc() so the darknet C-side network is freed via its finalizer before the next model loads, halving peak C-side memory. References for the resized models regenerated at the new sizes, with darknet parity re-asserted at generation time (10/10 combinations). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reduced-size run still died at the 8th model: peak RSS accumulates across models (GC heap growth plus malloc arenas that glibc never returns). Use a full collection and malloc_trim between models, and log free memory per model to make the next failure diagnosable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repo's long-standing cfgs are inference-normalized (batch=1), but the six new cfgs kept upstream training settings (batch=64, subdivisions=8). darknet's loader allocates activations for batch/subdivisions = 8 images, which is the multi-GB load spike that was killing CI runners at v4_p5 even at reduced input sizes (the per-model free-memory instrumentation showed 3-4 GB retained per large model). Detection results are unaffected -- verified identical to the stored refs for both darknet and ObjectDetector sides. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
IanButterworth
added a commit
that referenced
this pull request
Aug 7, 2026
…r deps (#138) * Remove unused flipdict and createcountdict helpers Neither has any call site in src, test, or examples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove unused lhtan_grad activation gradient Gradient kernels are training-only; the package does no training and nothing references it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove dead cu_functional guard cu_functional() held a CUDA.allowscalar(false) call but was never invoked, so it never ran. allowscalar is a global CUDA.jl session flag, so a package should not impose it on load either -- scalar-indexing policy belongs to the user. Delete the dead function and its Ref. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove pre-1.9 extension fallback shim Base.get_extension exists since Julia 1.9 and the package requires 1.10, so the include branch was unreachable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Drop redundant using statements in prepareimage.jl The including module already imports these. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Parameterize AllocWrappedModel Concretely-typed fields avoid dynamic dispatch on every call through the wrapper, which is the default CPU entry path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Keep weights-file training metadata instead of discarding it seen/seen_images were read (necessarily, to advance the stream) and then dropped; store them in cfg where darknetversion already lives. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Key benchmark() by model name and cover all models The hardcoded constructor list with numeric select indices was stale (none of the six new models) and fragile. Iterate YOLO_MODELS keys instead; each model loads at its native default size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix benchmark table printing under PrettyTables v3 Compat allows PrettyTables 2 and 3, but v3 renamed the header kwarg to column_labels, so benchmark()'s final table throw a MethodError on any v3 resolve. Gate on pkgversion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Use Base.identity for the linear activation linear(x) = x duplicated identity; the codebase already used identity elsewhere for the same purpose. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Validate input dims against the network's actual maximum stride assertdimconform required width/height divisible by the FIRST CONV'S FILTER COUNT -- an output-channel number with no relation to spatial downsampling. It worked for the classic models only because their first conv happens to have 32 (or 16) filters, and it wrongly rejected valid sizes (yolov7x at 416, first conv filters=40). Compute the real constraint instead: walk the cfg blocks tracking each layer's cumulative downsample (conv/maxpool multiply by stride, upsample divides, reorg multiplies, routes adopt the referenced layer's scale) and require divisibility by the maximum. Verified to give 32 for all classic models and 64 for v4_p6, matching their head strides; v7x now constructs at 416 and invalid sizes still throw. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Extract shared draw_boxes geometry helper Both draw_boxes! variants recomputed the identical image/model ratio and coordinate-index mapping; factor it into _box_geometry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add a precompile workload PrecompileTools was a dependency with no workload. Use the (recently fixed) dummy-weight path to compile cfg parsing, model construction and the full inference + NMS pipeline at precompile time: time-to-first- inference in a fresh session drops from ~12.8s to ~0.7s on an M2 Pro (package load ~3.8s warm). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Move benchmark() behind a BenchmarkTools/PrettyTables extension BenchmarkTools and PrettyTables were hard dependencies used only by the benchmark() utility, taxing load time for every detection-only user. They are now weakdeps triggering a BenchmarkExt extension; calling benchmark() without them loaded raises a MethodError with a hint explaining what to load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Narrow Flux compat to tested versions 0.12/0.13 predate the extension mechanism this package relies on and have never been exercised by CI; advertising them is risk without evidence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fail on missing detection references instead of self-blessing get!(RES_REFS, key, computed) meant a new model/image combination passed trivially on its first run without comparing anything. Missing keys are now test failures, with the regeneration script referenced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Commit the test-reference generation script The script that produced resrefs.jl and the reference images lived outside the repo; anyone needing to regenerate references (new model, intentional output change) had to reconstruct it. It regenerates any subset of models, asserts darknet parity for every model/image combination before blessing, and uses the same reduced test sizes as the suite. Round-trip verified on v3_tiny (max ref drift 9e-7, within the suite's 0.05 tolerance). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add changelog entries for quality pass Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Document remaining review findings in dev/REVIEW_BACKLOG.md Everything raised across the bug/performance/model-coverage/quality review series that was not fixed in #131/#132/#137/#138, with effort estimates and the explicitly-rejected items recorded so the reasoning is not lost. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was written by Claude (Claude Code), from a model-coverage review requested and supervised by @IanButterworth.
Stacked on #132 (targets its branch; will retarget to master when it merges). Adds the six established darknet-format models identified in the coverage review — the Scaled-YOLOv4 family and yolov7x — plus two loader fixes found while validating them.
New models
All loaded from official upstream weights and repackaged as lazy artifacts on the existing
weightsrelease tag:v4_csp_COCOv4_csp_x_swish_COCOv4x_mish_COCOv4_p5_COCOv4_p6_COCOv7x_COCOPretrained constructors now default to each model's native input size via a
MODEL_DEFAULT_SIZEtable — previously-supported models keep their 416 default, unchanged. Fixed-size convenience constructors (e.g.v4_csp_512_COCO) come fromMODEL_CONVENIENCE_SIZES. Notev4_p6requires dimensions divisible by 64 (four heads, stride 64); the rest require 32.Validation
new_coords=1decode path (used by csp/x-swish/x-mish/p5/p6, exercised by nothing previously in the suite); it proved correct as-is.resrefs.jlentries) were generated locally against darknet and committed, so CI actually compares rather than self-blessing on first run.Artifacts.tomlbindings, plus a download round-trip check against the uploaded release asset.Loader fixes found during validation
weightfile=nothing) was broken for every model — theseen/seen_imagesheader read wasn't guarded for the dummy case, throwingMethodError(read, (nothing, 8)). The dummy path is useful for cfg validation and future precompile workloads.layers = 339 ###) and whitespace-only lines, both present in released cfg files, brokecfgread/cfgsplit. Comments and blank lines are now stripped up front.Notes for review
cspx-p7-mish(needs a[sam]layer and a correctedassertdimconform),yolov3-tiny-prn(route bookkeeping issue), and all post-darknet PyTorch-native YOLOs (v5+, different format entirely).🤖 Generated with Claude Code