The mountain car problem is a standard domain used in reinforcement learning, described in detail in R. Sutton and A. Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. The objective in this problem is to drive an underpowered car up a steep mountain road. The difficulty is that gravity is stronger than the engine, and even at full throttle the car cannot accelerate up the steep slope. The solution is to first move away from the goal and up the opposite slope on the left. Then, by applying full throttle the car can build up enough inertia to carry it up the steep slope even though it is slowing down the whole way.
The original Mountain Car simulation that we used as a starting poinnt was developed at the University of Alberta. The current version, however, is different because it allows a sensorimotor delay. More specifically, during one decision epoch for the agent, the environment actually executes some number of steps n (set to 20 in the current distribution). Also, instead of observing the current state of the environment, the agent observes the state with a delay of k environment steps (this parameter is set to 10 in the current distribution). Different values of the parameters may be used in the competition. These values are not communicated to the agent.
The main binary distribution package contains all files needed to run the simulator and to create and run agents. Template agents are provided in C and Java. These are random agents and can be used to start the development. As the simulator is written in Java, there is only one download for all platforms.
Please refer to the included readme file for specific system requirements and usage instructions.
Additional documents:
The files above are sufficient to use both the simulator and the random agents. However, we also provide the source code, for anyone interested:
For support, please e-mail Monica Dinculescu (monica.dinculescu at mail dot mcgill dot ca) and cc Doina Precup (dprecup at cs dot mcgill dot ca)