Hello Habitat maintainers,
I’m running reinforcement learning experiments using Habitat and encountering a severe performance bottleneck in a single-GPU setup.
Despite switching from PPO to DDPPO, there is almost no improvement in training throughput, and the GPU remains largely underutilized.
I would appreciate any insights on whether this behavior is expected or if there is a configuration issue on my side.
- Training Setup
Framework: Habitat DDPPO
Training mode: Single GPU, multi-process rollout
GPU: NVIDIA L20 (46GB VRAM)
CPU: 15-core container limit
Comparison: PPO vs DDPPO
- Observed Performance
Overall training throughput is stuck at ~20 FPS, with a clear CPU/GPU imbalance.
GPU metrics
Utilization: 0–5% (brief spikes only during updates)
Memory usage: ~31GB / 46GB
Power draw: ~80–90W / 350W max
CPU metrics
CPU load: 25–28
Sustained high CPU usage with apparent scheduling pressure
3. Key Symptoms
- GPU is mostly idle
The GPU spends most of its time waiting for rollout batches rather than performing computation.
- PPO ≈ DDPPO (no meaningful speedup)
Performance comparison:
PPO: ~20 FPS
DDPPO: ~20 FPS
Even after increasing the number of rollout workers, there is no noticeable improvement in throughput.
- Initial Diagnosis
The system appears to be clearly CPU/simulator-bound rather than GPU-bound.
Likely bottlenecks:
High environment simulation cost per step
Python multiprocessing + IPC overhead
Worker oversubscription beyond available CPU cores
Rollout generation significantly slower than policy inference
Result:
GPU starvation — the GPU is frequently waiting for data.
- Questions
- Is Habitat fundamentally CPU-bound in this setup?
In single-GPU training, is it expected that CPU becomes the main bottleneck?
- Should DDPPO provide no speedup in CPU-bound regimes?
Is the following behavior expected?
simulator bottleneck → PPO ≈ DDPPO
- Recommended configuration for 15 CPU cores
What are reasonable starting values for:
NUM_ENVIRONMENTS
NUM_PROCESSES
rollout workers
How should one avoid CPU oversubscription in practice?
- Would multi-GPU DDPPO help in this case?
Given that CPU appears to be the bottleneck:
Would multi-GPU training still improve throughput?
Or is simulator/CPU optimization required first before scaling GPUs?
🙏 Any guidance or best practices would be greatly appreciated!
Hello Habitat maintainers,
I’m running reinforcement learning experiments using Habitat and encountering a severe performance bottleneck in a single-GPU setup.
Despite switching from PPO to DDPPO, there is almost no improvement in training throughput, and the GPU remains largely underutilized.
I would appreciate any insights on whether this behavior is expected or if there is a configuration issue on my side.
Framework: Habitat DDPPO
Training mode: Single GPU, multi-process rollout
GPU: NVIDIA L20 (46GB VRAM)
CPU: 15-core container limit
Comparison: PPO vs DDPPO
Overall training throughput is stuck at ~20 FPS, with a clear CPU/GPU imbalance.
GPU metrics
Utilization: 0–5% (brief spikes only during updates)
Memory usage: ~31GB / 46GB
Power draw: ~80–90W / 350W max
CPU metrics
CPU load: 25–28
Sustained high CPU usage with apparent scheduling pressure
3. Key Symptoms
The GPU spends most of its time waiting for rollout batches rather than performing computation.
Performance comparison:
PPO: ~20 FPS
DDPPO: ~20 FPS
Even after increasing the number of rollout workers, there is no noticeable improvement in throughput.
The system appears to be clearly CPU/simulator-bound rather than GPU-bound.
Likely bottlenecks:
High environment simulation cost per step
Python multiprocessing + IPC overhead
Worker oversubscription beyond available CPU cores
Rollout generation significantly slower than policy inference
Result:
GPU starvation — the GPU is frequently waiting for data.
In single-GPU training, is it expected that CPU becomes the main bottleneck?
Is the following behavior expected?
simulator bottleneck → PPO ≈ DDPPO
What are reasonable starting values for:
NUM_ENVIRONMENTS
NUM_PROCESSES
rollout workers
How should one avoid CPU oversubscription in practice?
Given that CPU appears to be the bottleneck:
Would multi-GPU training still improve throughput?
Or is simulator/CPU optimization required first before scaling GPUs?
🙏 Any guidance or best practices would be greatly appreciated!