Deep Q Network

Implementing Mnih et al. 2015

Alexandre Beaulne ID 260234941
Amina Madzhun ID 260844950
Apr 26 2018

Introduction

• "Human-level control through deep reinforcement learning", Mnih et al., Nature 2015
• Get RL agent to play Atari games with no knowledge beside screen action and rewards

Motivation

• Learn about applying RL to more complex problems from an engineering perspective
• Learn how deep learning and RL tie up in practice

Results

Before training:	After training:

Results

Before training	After training

Results

Conclusion

• Deep RL is hard!
• Debugging is difficult
• Convergence of learning very sensitive to hyperparameters (notably ε)
• Even when good hyperparameters are found, training takes a lot of time

Future Work

• Learn what to look for when training: how to debug and measure progress
• How to craft more sensible action space
• Use ConvNet with more capacity
• Use RNN (e.g. LSTM) instead of stacked frames to learn causation