Skip to content

feat: Silhouette render mode + additive directional rim#137

Open
arkavo-com wants to merge 3 commits intomainfrom
feat/additive-directional-rim
Open

feat: Silhouette render mode + additive directional rim#137
arkavo-com wants to merge 3 commits intomainfrom
feat/additive-directional-rim

Conversation

@arkavo-com
Copy link
Copy Markdown
Contributor

@arkavo-com arkavo-com commented May 3, 2026

Screenshots

Host aesthetic — body crushed to pure black, eyes self-illuminate as the composition's focal point ("ghost in the machine," appropriate when the silhouette is a UI host that needs to anchor the viewer's gaze and convey head-tracking attention):

Vita (VRM 1.0, heterochromia) Ao (VRM 1.0)

Default eyeEmissiveScale = 2.5. Lower it (~1.0) for a moodier "lit eyes" read; raise it (~4.0+) for hard-saturated white iris cores.

Earlier samples for reference (lower default eyeEmissiveScale):

AliciaSolid (VRM 0.0) VRM1_Constraint_Twist_Sample (VRM 1.0)

Jet-black silhouette with a strictly lateral warm-ember rim (config.rimLightDirection.y == 0, color * intensity ≤ 1.0 in every channel so the peak doesn't clip to white). Eye materials (VRoid English + native VRM Japanese names) self-illuminate via emissive routing — the iris texture survives the crush. Animated VRM 1.0 frame for reference:

Setup is now first-class on the renderer — call renderer.applySilhouetteMode(model:config:). Both CLIs (VRMRender, VRMVideoRenderer) call into it; defaults can be tuned via SilhouetteRenderConfig (rim direction, color, intensity, fresnel power, eye predicate, eye emissive scale).

Summary

  • Adds an opt-in stylised render path: pure-black silhouette with an additive directional fresnel rim, plus eye-material self-illumination so the iris pattern survives the crush.
  • New first-class API on VRMRenderer: applySilhouetteMode(model:config:) driven by a SilhouetteRenderConfig value type. Hosts call one method, and CLI tools (--silhouette on VRMRender and VRMVideoRenderer) use the same entry point.
  • Fixes a latent bug on the MToon path: emissiveFactor was unconditionally re-zeroed three lines after the MToon init re-propagated it, so anything routed through material.emissiveFactor (iris glow, etc.) never reached the GPU. Now consolidated into a single deterministic write.
  • .gitignore covers the local artefacts (.xcode-derived*/, *.profraw, emote-output/, the two VRMA pack folders) that were polluting git status.

Why

Silhouette + rim is a common stylised composition (menu hosts, accent shots, transition slates) that the standard MToon rim can't deliver — that rim multiplies against base albedo, so it disappears when the material is crushed to black. The new rim is additive and gated by lights, so it survives on a zero-albedo silhouette while leaving every existing render path bit-exact (defaults are off; legacy code is unchanged).

Detail

Renderer (VRMRenderer.swift)

  • disableAutoMaterialOverrides: bypasses the renderer's auto-overrides for face/eye/body/skin (white baseColor, zeroed emissive, clobbered shadeColor / shadingToony / shadingShift).
  • additiveDirectionalRimEnabled / additiveDirectionalRimPower: pipe an opt-in fresnel rim through to the MToon fragment shader.
  • The MToon emissive write is now a single branch on disableAutoMaterialOverrides, replacing two separate sites that fought each other.

Renderer extension (VRMRenderer+Silhouette.swift)

  • applySilhouetteMode(model:config:) — single entry point. Sets the renderer flags, swaps to a single warm rim light (Light 1), zeroes ambient, then walks the model:
    • Eye materials route baseColorTextureemissiveTexture so the iris pattern self-illuminates against the silhouette. eyeEmissiveScale tunes brightness; falls back to baseColorFactor literal for textureless materials.
    • Body materials are crushed (baseColor=0, shadeColor=0, emissive=0, matcap=0, parametricRim=0, giIntensity=0); only the additive directional rim contributes light.
    • Outline (MToon inverted-hull) zeroed on every material — it's albedo-independent and would otherwise pop as bright artefacts.
  • SilhouetteRenderConfig — Sendable value type with sensible review-tuned defaults:
    • rimLightDirection: (-1.0, 0.0, 0.2) — strictly lateral; any Y component would light horizontal surfaces (bow tops, shoulders) instead of the side rim.
    • rimLightColor (0.95, 0.55, 0.30) × rimLightIntensity 1.0 — every channel ≤ 1.0 so the rim peak preserves hue instead of clipping to white. Use rimFresnelPower for edge sharpness, not intensity.
    • rimFresnelPower: 14.0 — narrow grazing-angle rim; higher = tighter edge.
    • isEyeMaterial: @Sendable (String?) -> Bool — overridable predicate. Default covers VRoid English (eye / iris / sclera / pupil / highlight / eyeball) and native VRM Japanese (瞳 / 白目 / ハイライト), with lash / brow / line excluded so eyebrows / eyelashes / eyeliner are part of the body crush.

Shaders (MToonShader.metal)

  • New rim term, additive over the lit pass, gated by uniforms:
    pow(saturate(1 - N·V), max(power, 0.0001)) * saturate(N · -lightDir) * lightColor * intensity
    
    summed over the three scene lights.
  • Sits before saturate(), before the baseColor * 0.08 minLight floor (which is 0 when baseColor=0), so the rim survives on a black silhouette.
  • SkinnedShader Uniforms struct kept in sync (no fragment-side use yet); 432-byte total preserved, padding renamed not added.

CLI

  • --silhouette calls renderer.applySilhouetteMode(model:config:) — no setup duplication between the two tools.
  • --rim-power drives config.rimFresnelPower (default 14.0).
  • All other knobs are tunable on SilhouetteRenderConfig for hosts that integrate the API directly.

Test plan

  • swift build clean
  • make shaders clean (only pre-existing unrelated warnings)
  • swift test --parallel --num-workers 14 -j 16 --disable-sandbox — 1188 tests, exit 0
  • swift run VRMRender --silhouette ../Muse/Resources/VRM/AliciaSolid.vrm /tmp/v2_alicia_still.png — jet-black silhouette + lateral warm rim + glowing blue iris (see screenshot)
  • swift run VRMRender --silhouette ../three-vrm/examples/VRM1_Constraint_Twist_Sample.vrm /tmp/v2_vrm1_still.png — same path on VRM 1.0; eye-name predicate hits VRoid English tokens (see screenshot)
  • swift run VRMVideoRenderer ../three-vrm/examples/VRM1_Constraint_Twist_Sample.vrm VRMA_Locomotion_Pack/Idle.vrma /tmp/v2_vrm1_animated.mov --silhouette — silhouette + glowing eyes hold across animated frames
  • Verify legacy (non-silhouette) renders against this branch are bit-exact — defaults are off, so the code path is unchanged.
  • Verify on a VRM with native Japanese material names ( / 白目) that eye self-illumination triggers via the Japanese branch of the predicate.

Review feedback addressed in 167f7a2

  • Top-lit rimrimLightDirection.y forced to zero in the default config; light is now strictly lateral.
  • Pale-yellow / clipped rimrimLightColor * rimLightIntensity ≤ 1.0 in every channel; hue preserved at the peak. Sharpness is the fresnel exponent's job, not intensity's.
  • Pitch-black eyes → eye predicate covers Japanese material names; eye baseColorTexture is routed through the emissive sampler so the iris self-illuminates.

Notes

  • This branch is independent of fix(animation): Convert animation rest pose for VRM 0.0 retargeting #136 (orientation fix). They are sibling branches off main; either can land first.
  • The CLI tools previously had duplicated silhouette setup blocks. Both now call renderer.applySilhouetteMode(model:config:) — host apps that had similar duplication can adopt the same single entry point and inherit any future tuning automatically.

🤖 Generated with Claude Code

Adds an opt-in stylised render path designed for silhouette + rim
aesthetics (e.g. menu hosts, accent compositions) where the standard
MToon rim — which multiplies against base albedo — cannot survive a
crushed-to-black material.

Renderer (VRMRenderer):
- disableAutoMaterialOverrides: bypasses the renderer's automatic
  face/eye/body/skin overrides (white baseColor, zeroed emissive,
  clobbered shadeColor/shadingToony/shadingShift) so callers retain
  full CPU-side material control. Default false; legacy paths
  unchanged.
- additiveDirectionalRimEnabled / additiveDirectionalRimPower:
  pipe an opt-in fresnel rim through to the MToon fragment shader.
- Final emissive write on the MToon path now consolidated into a
  single deterministic branch — the prior code re-zeroed emissive
  three lines after restoring it, breaking iris-glow routing.

Shaders (MToonShader.metal):
- New rim term, additive over the lit pass, gated by uniforms:
    pow(saturate(1 - N·V), max(power, 0.0001))
      * saturate(N · -lightDir) * lightColor * intensity
  summed over the three scene lights. Sits before saturate(),
  before the baseColor*0.08 floor (which is 0 when baseColor=0),
  so it survives on a black silhouette.
- SkinnedShader Uniforms struct kept in sync (no fragment-side use
  yet); 432-byte total preserved.

CLI (VRMRender, VRMVideoRenderer):
- --silhouette wires the renderer flags, crushes baseColorFactor,
  shadeColorFactor, parametricRimColorFactor and matcapFactor to
  zero per material, kills ambient and the rim/back light, and
  warms the key light. Result: jet-black silhouette with a warm
  directional rim.
- --rim-power tunes the fresnel exponent (default 5).

.gitignore: cover xcode-derived dirs, *.profraw, emote-output, and
the VRMA pack folders so common local artefacts stop showing in
git status.

Verified visually with VRMRender (still PNG) and VRMVideoRenderer
(.mov) against AliciaSolid (VRM 0.0) and VRM1_Constraint_Twist_Sample
(VRM 1.0). All 1188 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
arkavo-com and others added 2 commits May 3, 2026 16:46
…eral rim defaults

Promotes the silhouette setup from duplicated CLI shims into a
first-class VRMRenderer extension (VRMRenderer+Silhouette.swift)
and applies three review fixes to the defaults:

1. Light vector — was (-0.2, 0.5, -0.85) (top-lit), now strictly
   lateral (-1.0, 0.0, 0.2). Y dominance was lighting the tops of
   horizontal surfaces (bow, shoulders) instead of the side rim.

2. Rim brightness — was color (1.0, 0.9, 0.7) × intensity 1.0,
   reading as pale yellow / off-white. Now warm ember (0.95, 0.55,
   0.30) × intensity 1.0 — every channel ≤ 1.0 so the rasterizer
   doesn't clip the peak to white. Fresnel exponent does the edge-
   sharpness work, not intensity.

3. Eye preservation — previously every material's emissive and
   baseColor were crushed to zero, blacking out eyes. The extension
   now routes each eye material's baseColorTexture through the
   emissive sampler so the iris pattern self-illuminates against
   the silhouette. Eye-name predicate covers VRoid (English) and
   native VRM (Japanese: 瞳, 白目, ハイライト) naming, with eyebrow /
   eyelash / eyeliner explicitly excluded.

The extension also kills MToon's inverted-hull outline on every
material (it's albedo-independent and would otherwise pop as bright
artefacts against the crushed body), and zeroes shadeColor /
parametricRim / matcap / giIntensity so only the additive
directional rim contributes light to non-eye geometry.

CLI tools now call `renderer.applySilhouetteMode(model:config:)`
instead of duplicating the setup. `--rim-power` continues to drive
`config.rimFresnelPower`. All other defaults are tunable on
`SilhouetteRenderConfig` for callers that want a different aesthetic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…etic

The previous default (1.0) emitted the iris texture at its authored
brightness, which on a fully crushed silhouette body read as "lit eyes"
rather than "luminous eyes" — the iris hue showed but the eye didn't
function as a focal point.

2.5 saturates the brighter regions of the iris and blows the sclera
out to white, giving the eyes the "ghost in the machine / host that's
watching" presence appropriate when the silhouette is composed against
a UI (menu host, transition slates) and needs to anchor the user's
gaze. Body stays bit-for-bit crushed; only the eye-material emissive
contribution changes.

Documented the curve at the call site:
  - 1.0  = subtle, lit-eye read
  - 2.5  = host / hologram (default)
  - 4.0+ = hard-saturated white iris cores

Verified visually on AliciaSolid (VRM 0.0), VRM1_Constraint_Twist_Sample
(VRM 1.0), Ao_dress (VRM 1.0), and Vita_clothing (VRM 1.0,
heterochromia) — green and blue irises both register as distinct
glowing pin-points.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant