We might expect to understand a text before we can extract useful information from it. Yet even in an unfamiliar language, we can compare repeated symbols, count their occurrences, and notice which sequences appear together. Not understanding the text does not make its patterns uninformative.
Text is not random
Word order and word choice constrain which sequences we encounter. In ordinary English, I drive a red car is more likely than Car red a drive I. The words are the same, but their arrangement is not equally plausible.
These regularities let us learn which words and sequences are likely without first assigning an explicit meaning to every sentence.
Shared contexts reveal similarities
Consider I drive a truck, I drive a car, and I drive the bus. The surrounding words place truck, car, and bus in similar roles. Repeated across many texts, such patterns provide evidence that their meanings are related. This is the distributional hypothesis.
The evidence is not conclusive: hot and cold also occur in similar contexts, despite being opposites. Shared usage can reveal a common semantic dimension without making two words interchangeable.
For information retrieval, this offers a way to compare texts beyond exact word matches. Usage patterns provide clues to meaning, even when meaning is not explicitly represented.
Source: Lecture 1 – Foundations of Information Retrieval (2026), slides 12–13 (PDF pages 11–12).