On-Policy Action-Value Function

reinforcement-learning

Definition

On-Policy Action-Value Function

The on-policy action-value function $Q^{\pi }(s,a)$ gives the expected return if you start in state $s$, take and arbitrary action $a$ (not necessarily from the policy), and then forever after act according to policy $\pi$:

$$Q^{\pi }(s,a)={\mathbb{E}}_{\tau \sim \pi }[R(\tau )\mid s_0=s,a_0=a]$$