Definition
Markov Decision Process
A Markov decision process (MDP) is the tuple
modelling an agent interacting with an environment over discrete time, where the environment’s response satisfies the Markov property.
: state space. : action space (or when state-dependent). : transition probability of reaching after action in . : immediate reward on transitioning from to under . : discount factor. At step
, the agent in state selects via a policy , then receives and transitions to . The goal is to choose maximising the expected discounted return