Skip to content. Skip to navigation
McGill Home SOCS Home
Personal tools
You are here: Home Announcements and Events Seminars profile

Fall 2014 Schedule
Winter 2015 Schedule

None, MC103, 12 - 12:30

General MDP feature construction using bisimulation
Gheorghe Comanici , McGill, SOCS, Reasoning and Learning Lab

Area: AI


We consider the process of generating feature-based representations for Markov Decision Processes (MDPs), a powerful mathematical framework for modelling stochastic sequential decision making. We look at a standard approach for learning good policies in reinforcement learning domains, which involves first the computation of a value function that associates states with expected returns. State features are then used to provide approximation schemes for value function computation. In this talk, I will present a systematic way of generating alternative representations by analyzing behavioural similarities on the set of states (i.e. using bisimulation metrics); I will present theoretical guarantees of corresponding approximation schemes; I will describe its relationship to other pragmatic methods, such as spectral clustering and Bellman Error Basis Functions; lastly, I will discuss its computational tractability.