Skip to content

aria_to_lance: 6.5x faster MANO fit, same output - #753

Open
RyanPCo wants to merge 2 commits into
ryanco/lance-migration-planfrom
ryanco/aria-mano-fit
Open

RyanPCo wants to merge 2 commits into
ryanco/lance-migration-planfrom
ryanco/aria-mano-fit

Conversation

@RyanPCo

@RyanPCo RyanPCo commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Makes the Aria MANO fit, the dominant cost of the raw Aria migration, 6.5x faster without
changing its output. On one 4,587-frame episode (2,400 iters, A40) MANO goes from 338 s to
52 s for both hands, and the keypoints match the previous fit within 1e-3 mm.

  • Batching (aria_to_lance --mano-batch, default 8192): the fit loss was normalized by the
    whole batch's valid-point count, so the batch size changed each frame's gradient scale
    and, through Adam's eps, the result. The loss is now normalized per 512-frame chunk
    (norm_group), and up to --mano-batch frames are optimized together. Same results as
    fitting each 512-frame chunk alone (~2e-4 mm); ~2.1x faster on its own.
  • Fingertip-only forward (aria_utils.mano_joints_fn): the loss uses the 16 kinematic joints
    and 5 fingertip vertices, but the stock MANO forward skins all 778 vertices every step.
    mano_joints_fn regresses the joints from the shaped template and skins only the 5 tip
    vertices, giving the same 21 points.
  • docs/zarr_to_lance_migration.md: the raw-path cost section uses the new numbers (~40 A40
    GPU-h for MANO over the 2,569 Aria episodes, down from ~235).

Testing: on a compute node, tests/unit/test_mano_fit.py (new: mano_joints_fn equals the full
forward's joints for both hands; the batched fit with per-chunk normalization equals
fitting each chunk alone) passes with the MANO weights linked in (3 passed); without them it skips. Full unit suite at the top of the stack: 1,565 passed, 5 skipped (3 of the skips are test_mano_fit without weights in the checkout; it passes with them linked in).

🤖 Generated with Claude Code

https://claude.ai/code/session_01HjCQBqQCr2igFEMR7yUovw

@RyanPCo RyanPCo left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated adversarial review (read-only; findings were checked by a second skeptical pass before posting): 1 nit.

group = norm_group or N
gid = torch.arange(N, device=device) // group
n_groups = (N + group - 1) // group
zeros = torch.zeros(n_groups, device=device)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (naming): Reusing zeros as the base for four index_add calls is correct (out-of-place), but readers have to check that. Use a small group_sum = lambda x: torch.zeros(n_groups, device=device).index_add(0, gid, x) for group_denom, group_count, pos_loss and reg_loss.

@RyanPCo
RyanPCo force-pushed the ryanco/aria-mano-fit branch from 1c4b9a0 to 5734ced Compare October 30, 2026 19:41
@RyanPCo
RyanPCo force-pushed the ryanco/lance-migration-plan branch from 72692a6 to 369caa4 Compare October 30, 2026 19:41
@RyanPCo

RyanPCo commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Review follow-up (force-pushed). The 1 finding is fixed:

  • fit_mano_to_aria_batched uses a group_sum(x) helper (fresh zeros + index_add) for denom, count, pos_loss and reg_loss instead of reusing a shared zeros base.

https://claude.ai/code/session_01HjCQBqQCr2igFEMR7yUovw

…2 chunk

The fit loss was normalized by the whole batch's valid-point count, so batch
size changed each frame's gradient scale and (through Adam eps) the result.
Normalize per 512-frame chunk instead and optimize up to --mano-batch frames
together. Results match the 512-chunk fit to ~2e-4 mm; MANO time on one
4,587-frame episode drops from 177 s to 83 s per hand on an A40.

Claude-Session: https://claude.ai/code/session_01HjCQBqQCr2igFEMR7yUovw
The fit loss uses the 16 kinematic joints and 5 fingertip vertices, but the
stock MANO forward skins all 778 vertices every step. mano_joints_fn computes
the same 21 joints from the 5 tip vertices only. On one 4,587-frame episode
(2400 iters, A40) MANO time drops from 338 s to 52 s for both hands, matching
the previous 512-chunk fit to <1e-3 mm.

Claude-Session: https://claude.ai/code/session_01HjCQBqQCr2igFEMR7yUovw
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant