Lukas' Notes

Definition

Stemming

Stemming reduces word forms to shared matching forms, called stems, using language-dependent rules such as removing prefixes or suffixes. A stem need not be a dictionary word or a true linguistic root.

Comparison with Lemmatisation

Lemmatisation seeks the appropriate dictionary headword. Stemming instead uses approximate transformations of the written form, without requiring the result to be a dictionary entry.

Use in Retrieval

Stems can serve as indexed terms. Apply the same stemming rules to indexed text and query words so intended variants match.

Example

A stem rather than a headword

A stemmer may produce

The shared output automat is a matching form, not the dictionary headword of each input.

A shared stem

Here, the shared stem also happens to be a dictionary word.