How does this clip explain the way PCA chooses which directions to keep?

Conditions

• Demonstrated on a 2D scatter plot.
• The explanation is geometric and intuitive rather than algebraic.

Reasoning, step by step

1. Start with a 2D scatter plot of green points, labeled $D = 2$.
2. Identify two candidate directions in the data cloud, shown as a blue vector and a red vector.
3. Project the points onto the blue direction and observe that the points remain spread out along that line.
4. Project the points onto the red direction and observe that the cloud collapses almost completely.
5. Conclude that PCA should keep the direction with the biggest spread and discard the rest until the desired dimensionality is reached.

Example

The narrator compares projecting onto the smaller-spread line versus the larger-spread line and concludes that PCA keeps the directions of biggest spread. The same green point cloud is first projected onto a blue line, then onto a red line, showing different degrees of information loss.

Common misconceptions

• The clip gives an intuitive description only; no covariance matrix, eigenvalue equation, or optimization objective is shown in this excerpt.

Watch the explanation

Explore next

• Why does projecting onto the blue line lose less information than projecting onto the red line?
• What does the notation $D = 2$ -> D' = 1 mean in this PCA introduction?