Continual Learning

machine-learning

Definition

Continual Learning

Continual learning is about learning new data distributions without the necessity of re-training the entire model, which is more resource-efficient than existing foundational approaches.

Formally, an incoming batch of training samples belonging to a task $t$ can be represented as ${\mathcal{D}}_{t,b}=\{{\mathcal{X}}_{t,b},{\mathcal{Y}}_{t,b}\}$, where ${\mathcal{X}}_{t,b}$ is the input data, ${\mathcal{Y}}_{t,b}$ are the data labels, $t\in \mathcal{T}=\{1,\dots ,k\}$ is the task identity and $b\in {\mathcal{B}}_t$ is the batch index. Here, “task” refers to training samples ${\mathcal{D}}_t$ following the distribution ${\mathbb{D}}_t:=p({\mathcal{X}}_t,{\mathcal{Y}}_t)$. Under realistic constraints, however, the data labels ${\mathcal{Y}}_t$ and the task identity $t$ might not be always available.

¹

Catastrophic Forgetting

Definition

Catastrophic Forgetting

Catastrophic forgetting is a problem in continual learning where learning from dynamic data distributions results in a reduced ability to capture the old ones. ¹

Footnotes

1. A Comprehensive Survey of Continual Learning - Theory, Method and Application ↩

Link to original

Plasticity vs. Memory Stability

Plasticity, the ability to adapt to new data distribution, and memory stability, the ability to recover old data distributions, largely compromise each other. ¹

¹

---

Typical Scenarios

¹

Approaches

¹

Regularisation-based Approach

Regularisation-based Continual Learning

Regularisation-based approaches add regularisation terms with reference to the old model.

Replay-based Approach

Replay-based Continual Learning

Replay-based approaches approximate and recover old data distributions. ¹

Optimisation-based Approach

Optimisation-based Continual Learning

Optimisation-based approaches explicitly design and manipulate optimisation programs. ¹

Representation-based Approach

Representation-based Continual Learning

Representation-based approaches create robust and well-distributed representations. ¹

Architecture-based Approach

Architecture-based Continual Learning

Architecture-based approaches construct task-adaptive parameters with a properly-designed architectures. ¹

Footnotes

1. A Comprehensive Survey of Continual Learning - Theory, Method and Application ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹