Discount Factor

reinforcement-learning

Definition

Discount Factor

The discount factor $\gamma \in [0,1]$ weights future rewards in the return:

$$G_t=\sum _{k=0}^{\infty }{\gamma }^{k}R(s_{t+k},a_{t+k}).$$

• $\gamma =0$: the agent is myopic, maximising only the immediate reward.
• $\gamma \to 1$: the agent is farsighted, valuing distant rewards almost as much as immediate ones.
• $\gamma <1$ also ensures $G_t$ remains finite for unbounded or continuing tasks, acting as a soft horizon.