reinforcement-learning Definition On-Policy Action-Value Function The on-policy action-value function gives the expected return if you start in state , take and arbitrary action (not necessarily from the policy), and then forever after act according to policy :