Infant Spontaneous Movement Noise Improves Exploration in Deep RL

This repository extends the pink-noise-rl code of Eberhard et al. (ICLR, 2023) by introducing baby noise: a temporally correlated noise process whose color evolves over time, inspired by infant motor behavior.

---

Installation

Clone the repository:

git clone https://github.com/fcomlop/baby-noise-rl.git
cd baby-noise-rl

Create a conda environment:

conda create -n babynoise python=3.12
conda activate babynoise

Install requirements and package:

pip install -r requirements.txt
pip install -e .

Import in python code:

import babynoise

---

Structure

baby-noise-rl/
├── babynoise/
│   ├── __init__.py
│   ├── banner.png
│   ├── cnrl.py
│   ├── colorednoise.py
│   └── sb3.py
│
├── examples/
│   ├── sac.py
│   ├── td3.py
│   └── ddpg.py
│
├── .gitignore
├── LICENSE
├── pyproject.toml
├── README.md
└── requirements.txt

---

Usage

SAC (stochastic policies)

import gymnasium as gym
from stable_baselines3 import SAC
from babynoise import BabyNoiseDist, NoiseScheduleCallback

env = gym.make("MountainCarContinuous-v0")

model = SAC("MlpPolicy", env)

model.actor.action_dist = BabyNoiseDist(
    total_steps=100_000,
    block_steps=10_000,
    action_dim=env.action_space.shape[-1],
)
callback = NoiseScheduleCallback()
model.learn(
    total_timesteps=100_000,
    callback=callback,
)

TD3 / DDPG (deterministic policies)

import gymnasium as gym
from stable_baselines3 import TD3
from babynoise import BabyActionNoise, NoiseScheduleCallback

env = gym.make("MountainCarContinuous-v0")

action_noise = BabyActionNoise(
    sigma=0.1,
    total_steps=100_000,
    block_steps=10_000,
    action_dim=env.action_space.shape[-1],
)

model = TD3("MlpPolicy", env, action_noise=action_noise)
model.learn(total_timesteps=100_000)

Comparison with other noise types

White noise:

from babynoise import ColoredNoiseDist

model.actor.action_dist = ColoredNoiseDist(
    beta=0.0,  # white noise has beta=0
    seq_len=10_000,
    action_dim=action_dim,
)

Pink noise:

from babynoise import ColoredNoiseDist

model.actor.action_dist = ColoredNoiseDist(
    beta=1.0,  # pink noise has beta=1
    seq_len=10_000,
    action_dim=action_dim,
)

---

Detailed Examples

See:

examples/sac.py
examples/td3.py
examples/ddpg.py

---

Acknowledgements

This repository is based on the code from Eberhard et al. (ICLR 2023): https://github.com/martius-lab/pink-noise-rl.

---

Citation

If you use this code, please cite:

@inproceedings{lopez2026infant,
  title = {Infant Spontaneous Movement Noise Improves Exploration in Deep RL},
  author = {L'{o}pez, Francisco M. and Ernst, Markus R., and Cruz, Francisco and Hoffmann, Matej, and Triesch, Jochen},
  year = {2026}
}

---

Feel free to open an issue if you find bugs, want to make suggestions, or have any research questions.