Chain of Thought

machine-learning nlp

Definition

Chain of Thought

Chain of thought is a prompting technique for large language models in which the model is instructed to generate an intermediate reasoning sequence before producing a final answer. Instead of predicting the output directly, the model produces a step-by-step derivation that bridges the problem statement to the solution.

Formally, for a question $q$ and answer $a$, the model learns to generate a rationale $r=(r_1,r_2,\dots ,r_k)$ such that:

$$P(a\mid q)\approx P(a\mid r_k)\cdot \prod _{i=1}^{k}P(r_i\mid q,r_1,\dots ,r_{i-1})$$

The model is typically prompted with examples that include explicit reasoning chains.

Mechanism

Chain of thought prompting improves performance on tasks requiring multi-step reasoning (arithmetic, commonsense, symbolic) by:

1. Decomposition: Breaking a complex problem into smaller, tractable subproblems.
2. Explicit intermediate states: Making the model’s internal reasoning visible, which reduces error accumulation.
3. Few-shot conditioning: Providing exemplar reasoning traces in the prompt teaches the model the desired output structure.

Properties

Model scale dependence

Chain of thought yields significant gains primarily for sufficiently large models (typically billions of parameters). Smaller models often fail to generate coherent reasoning traces even when instructed.

Sensitivity to exemplars

The quality of the generated reasoning depends on the relevance and correctness of the few-shot examples. Poorly chosen exemplars can mislead the model.