Reinforcement learning plays Super Mario Bros (NES)

Introduction

Using my code for training PPO I was able to solve 27/32 levels in retro Mario env. I believe 4 out of 5 these levels are still possible to solve, just need to train or tune longer.

Unsolved levels:

• 4-4 - a labyrinth, for some reason environment restrictions are not good enough to prevent agent from running into loops.
• 5-3 and 6-3 - difficult platforming.
• 8-1 - long level, there are 2 hard segments near the end of the level agent struggle to solve.
• 8-4 - very hard labyrinth.

Sample results

Instructions

It is possible to train a new model and watch how it plays the game with this code. To run this code call following command first:

bash setup.sh

Training

The most important training parameters are learning rate and entropy. They are both accessible via args. For training new model, run the command

python src/train.py -w {world} -l {level}

Testing

Checkpoints of trained models are located at the google drive. For testing the checkpoint, place it in checkpoints folder and run the command:

python src/test.py -w {world} -l {level} -r

Acknowledgement

I took the environment, idea and inspiration from this repo.

Further plans

It is interesting to re-produce results of the ICM paper site, paper 2 where an RL agent learns how to explore the environment without reward and solves 11 levels.