The issue observed in https://github.com/ByteArena/ba/issues/45 is likely (to be confirmed) is that the items in the perception are *not in a consistent order*. That prevents the ability the scan consistently the vision of the agent.