Questions about dual expert architecture

[Kkkkkz21]

Hello! After reading the paper, I got the impression that the "dual expert architecture" mentioned here refers to two independent Transformers. However, do these two Transformers interact with each other? How are their features fused or exchanged? Is it similar to the approach used in Bagel? This is what I'm confused about. Thanks for your help!

[fengyifu2000]

Hi, thanks for the question! The dual expert architecture in Lance does not mean two fully isolated Transformers. The two streams are used to decouple understanding-oriented tokens and generation-oriented VAE tokens, while their interaction still happens through the shared interleaved multimodal context and generalized 3D causal attention. At a high level, it is similar to BAGEL: both decouple heterogeneous representations while keeping them in a unified model. Lance follows this idea and extends it to broader image/video X2T, X2I, and X2V tasks, with MaPE introduced to better distinguish different visual token groups.

…

From: ***@***.***> Date: Wed, May 27, 2026, 21:03 Subject: [bytedance/Lance] Questions about dual expert architecture (Issue #31) To: ***@***.***> Cc: ***@***.***> [image]Kkkkkz21 created an issue (bytedance/Lance#31)<#31> Hello! After reading the paper, I got the impression that the "dual expert architecture" mentioned here refers to two independent Transformers. However, do these two Transformers interact with each other? How are their features fused or exchanged? Is it similar to the approach used in Bagel? This is what I'm confused about. Thanks for your help! — Reply to this email directly, view it on GitHub<#31?email_source=notifications&email_token=CD5KRLAILH2AP372ZEOQ3KD443RQPA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NBVGMZDKMJRG4YTBJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/CD5KRLHBDTVCNC5FKKY4LB3443RQPAVCNFSM6AAAAACZPGHIF6VHI2DSMVQWIX3LMV43ASLTON2WKOZUGUZTENJRGE3TCMA>. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS<https://github.com/notifications/mobile/ios/CD5KRLBQDMZOVRABGY4HKP3443RQPA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NBVGMZDKMJRG4YTBJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android<https://github.com/notifications/mobile/android/CD5KRLHQO76KUU3BOW2OJW3443RQPA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NBVGMZDKMJRG4YTBJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you are subscribed to this thread.[image]Message ID: ***@***.***>

[Kkkkkz21]

Hi, thanks for the question!

The dual expert architecture in Lance does not mean two fully isolated Transformers. The two streams are used to decouple understanding-oriented tokens and generation-oriented VAE tokens, while their interaction still happens through the shared interleaved multimodal context and generalized 3D causal attention.

At a high level, it is similar to BAGEL: both decouple heterogeneous representations while keeping them in a unified model. Lance follows this idea and extends it to broader image/video X2T, X2I, and X2V tasks, with MaPE introduced to better distinguish different visual token groups.
…

Thanks for the great work and the helpful reply! Really looking forward to the release of the training code!

[chun-chenlin-cpu]

I'm wondering that: do two experts (LLM_und and LLM_gen) need to share the same architecture (e.g., Qwen2.5-VL-3B)?