n-gram

Definition

N-gram

Let $T=(t_1,\dots ,t_L)$ be a sequence of tokens and let $n\ge 1$. An n-gram is a contiguous subsequence of exactly $n$ tokens. Its occurrences in $T$ are

$$g_{i,n}=(t_i,t_{i+1},\dots ,t_{i+n-1}),1\le i\le L-n+1.$$

The corresponding collection is

$${\mathcal{G}}_n(T)=(g_{1,n},g_{2,n},\dots ,g_{L-n+1,n}).$$

The parameter $n$ fixes the length of the context: $n=1$ gives a unigram, $n=2$ a bigram, and $n=3$ a trigram. The items may be words, characters, or subword tokens, depending on the chosen tokenisation. Because the subsequence is contiguous, order and adjacency are preserved; repeated n-grams remain distinct occurrences in the collection.

Examples

Bigrams

For the token sequence $T=(\text{the},\text{cat},\text{sat})$,

$${\mathcal{G}}_2(T)=((\text{the},\text{cat}),(\text{cat},\text{sat})).$$

The pair $(\text{the},\text{sat})$ is not a bigram because the tokens are not adjacent.

Unigrams and trigrams

For $T=(\text{the},\text{cat},\text{sat})$, the unigrams are the individual tokens:

$${\mathcal{G}}_1(T)=(\text{the},\text{cat},\text{sat}).$$

The trigrams contain all three tokens because the sequence has length three:

$${\mathcal{G}}_3(T)=((\text{the},\text{cat},\text{sat})).$$

Repeated occurrences

For $T=(\text{the},\text{cat},\text{the},\text{cat})$,

$${\mathcal{G}}_2(T)=((\text{the},\text{cat}),(\text{cat},\text{the}),(\text{the},\text{cat})).$$

The bigram $(\text{the},\text{cat})$ occurs twice. Its two occurrences have different positions even though their token values are equal.

Character n-grams

N-grams need not contain words. For the character sequence $T=(\text{c},\text{a},\text{t})$,

$${\mathcal{G}}_2(T)=((\text{c},\text{a}),(\text{a},\text{t})).$$

A character n-gram model therefore represents text through short character sequences rather than word sequences.