Question About the Limited Motion-Style Variation When Adjusting `latent_eps` in BBC

[Wan020607]

Hello,

While using the BBC training and evaluation policies in this project, I encountered an issue related to the continuous style latent variable latent_eps. I would like to ask how it is expected to work and whether I may have misunderstood or used it incorrectly.

Background

I mainly referred to the bbc module in the repository. Based on my understanding of the code, the policy input includes:

• latent_c: a discrete gait category, such as walk / pace / trot / canter / jump
• latent_eps: a continuous style latent variable in the range [-1, 1]

During training, latent_eps is randomly sampled:

self.latent_eps[env_ids, :] = torch.tensor(
    np.random.rand(len(env_ids), 1) * 2. - 1.,
    dtype=torch.float32,
    device=self.device
)

At the same time, the discriminator contains an encoder_eps, which predicts latent_eps from motion clips:

eps = self.encoder_eps(x)
reward_us = -self.L1Loss(eps, label_eps)

The training loss also includes:

us_loss = self.L1Loss(eps, policy_latent_eps)
loss = ... + self.us_coef * us_loss + ...

Therefore, my understanding is that latent_eps should control continuous style variations within the same gait category, such as stride length, step frequency, body posture amplitude, or the degree of bouncing.

Issue

After training a policy with bbc, I manually adjusted latent_eps in play.py, for example from -1 to +1. However, the resulting changes in the robot's motion were not visually obvious.

I also tested the provided policy:

tsc/weights/bbc/model.pt

but the style variation was still not very noticeable.

More specifically:

• Switching latent_c produces clear changes in gait category.
• Adjusting only latent_eps results in relatively weak style changes within the same gait.
• It is difficult to observe stable and interpretable differences between latent_eps = -1, latent_eps = 0, and latent_eps = 1.

I will appreciate for your help !

[tRNAoO]

Thank you for your detailed question.

Your understanding is generally correct: latent_c represents the discrete gait category, while latent_eps is designed as a continuous latent variable that may capture style variations within the same gait category.

However, since latent_eps is learned in an unsupervised manner, its semantics are not explicitly constrained. Therefore, the interpretability of changes in latent_eps is usually much weaker than that of latent_c. In some cases, the variations induced by different latent_eps values may be subtle, entangled with other factors, or difficult to distinguish by visual inspection alone. As a result, it is possible that setting latent_eps to -1, 0, or 1 does not produce clearly visible or stable style differences in the robot motion.

In contrast, latent_c is associated with discrete gait labels, so switching latent_c is expected to produce much more obvious changes in gait category.

If you would like to further investigate the effect of the latent variables, we recommend performing quantitative analysis instead of relying only on visual observation. For example, you could fix one latent variable and vary the other, then compute statistics such as joint angles, joint range of motion, foot clearance, step frequency, stride length, body height, or torso orientation. These measurements may make the differences easier to observe than simply watching the motion.

For studying the continuous style variable specifically, you may fix latent_c to a certain gait and vary latent_eps, then compare these motion statistics across different latent_eps values.