Statistical Separability

machine-learning cognition

Definition

Statistical Separability

Statistical separability is Frank Rosenblatt’s principle that stimulus classes can become reliably distinguishable through the aggregate response of a randomly connected, adaptive system, even when its exact wiring is unknown.

Let $c:\mathcal{X}\to \{0,1\}$ assign a class to each stimulus, and let $R_{m,W}:\mathcal{X}\to \{0,1\}$ be the response of a perceptron with $m$ association units and wiring $W$. Under suitable correlations within the stimulus classes, the central claim is

$$\underset{W,X}{\mathrm{Pr}}\left(R_{m,W}(X)=c(X)\right)⟶1\text{as }m\to \infty .$$

The probability ranges over stimuli and admissible random wirings. Thus reliable discrimination depends on the statistical organisation of the network and its environment, not on one prescribed connection diagram.

How separation emerges

A perceptron transforms a stimulus in two stages:

$$X⟼A_W(X)\in \{0,1{\}}^m⟼R_{m,W}(X),$$

where $A_W(X)$ is the activity pattern across the association units.

1. Random connections project sensory activity onto many association units.
2. Similar stimuli tend to activate more association units in common than dissimilar stimuli.
3. Learning strengthens or weakens connections from active association units to response units.
4. The response aggregates many small contributions, making one response more probable for one class than for another.

No individual connection needs to encode a class. The distinction appears in the population response.

Why the claim is statistical

Rosenblatt specifies the system’s gross organisation and probability laws without fixing every microscopic connection. The object of analysis is therefore a response distribution:

$$\text{connection laws and class structure}⟼\mathrm{Pr}(R=r\mid c(X)=i).$$

Increasing the number of association units reduces the influence of any particular random connection and stabilises the class-level difference. Probability theory can therefore establish reliable behaviour across possible networks without enumerating each wiring diagram.

Generalisation

Generalisation requires a differentiated environment: members of the same class must share statistical structure. A novel member then tends to evoke an association pattern that overlaps with patterns strengthened by previous members, so it produces the learned response without having been stored individually.

If stimuli are unrelated, this overlap carries no class information. The perceptron may memorise some stimulus–response associations, but it has no basis for assigning unseen stimuli to the same classes.

Distinction from linear separability

Statistical separability is not the same as linear separability. Linear separability is a geometric property of a fixed set of feature vectors: a hyperplane divides one class from another. Rosenblatt’s statistical separability concerns the probability that a family of randomly organised, adaptive networks distinguishes correlated stimulus classes. A response unit may use a linear threshold, but the guarantee is over uncertain networks and stimuli rather than one fixed dataset.