CleanRL example bug?

[samlobel]

Hi all,

Getting started with the rlexplore_with_cleanrl.py example and I think there's an important typo, introduced here (line 254).

The first block, which overwrites next_obs as returned by the environment, shouldn't be there. It's throwing a shape error because line 256 is trying to put an array of shape (128, 8, 4, 84, 84) into a slot of shape (8, 4, 84, 84).

"real_next_obs" is the one that's used, and that one looks right, except that the next_obs referenced there should be the environment's next_obs, not the one modified above.

Thanks!
Sam

[friedrichyuan]

Hi Sam, I just updated the code and marked u as a co-author; see if it works now. Thanks!

[samlobel]

Sorry for the late response -- success! Works now, thanks.