Skip to content

feat: DINOv2 vitl14 fallback when DINOv3 gated access is pending - #15

Open
perasang wants to merge 1 commit into
shivampkumar:mainfrom
perasang:feat/dinov2-fallback
Open

feat: DINOv2 vitl14 fallback when DINOv3 gated access is pending#15
perasang wants to merge 1 commit into
shivampkumar:mainfrom
perasang:feat/dinov2-fallback

Conversation

@perasang

Copy link
Copy Markdown

Summary

  • Adds an opt-in USE_DINOV2_FALLBACK=1 env var in generate.py that swaps the DINOv3 image conditioning model for DINOv2 vitl14
  • Resizes input to 448 × 448 px (32 × 32 patches + cls = 1025 tokens), matching DINOv3 vitl16 @ 512 px exactly — no shape mismatch in the pipeline
  • Zero-diff when the env var is unset; existing behaviour is unchanged

Motivation

facebook/dinov3-vitl16-pretrain-lvd1689m is a gated Meta model that requires manual review before access is granted. New users are blocked at first run until approval arrives. This fallback lets them test the full pipeline immediately using the publicly available DINOv2 vitl14 weights (~330 MB vs ~1 GB for DINOv3).

Usage

USE_DINOV2_FALLBACK=1 python generate.py path/to/image.png

Trade-offs

  • DINOv2 and DINOv3 feature spaces differ; output quality may be slightly lower than with the intended DINOv3 model
  • Intended as a temporary workaround — switch back to DINOv3 once access is approved

Test plan

  • python generate.py (no env var) — behaviour unchanged, DINOv3 path used
  • USE_DINOV2_FALLBACK=1 python generate.py assets/shoe_input.png — pipeline loads DINOv2, runs to completion, outputs .glb

🤖 Generated with Claude Code

When DINOv3 access is pending Meta review, set USE_DINOV2_FALLBACK=1
to substitute DINOv2 vitl14 at 448px (32×32 patches = 1025 tokens),
matching DINOv3 vitl16 token count so the pipeline runs without errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant