What does it physically mean when PCA reports 95% variance explained?
It means the original data was barely three-dimensional; the data cloud hovered extremely close to the 2D projection plane. Squishing it flat cost almost nothing because there was almost no depth or thickness to lose in the discarded dimensions.
Conditions
- The data is projected onto a lower-dimensional subspace.
- The variance explained is calculated based on the eigenvalues of the principal components.
Reasoning, step by step
- Analyze the side angle of the data cloud relative to the 2D projection plane.
- Observe that the points hover extremely close to the plane, indicating a subspace thickness of approximately zero.
- Conclude that the third dimension contained only 5% of the overall scatter.
- Interpret the 95% variance explained as the preservation of the dominant planar structure.
- Recognize that the reconstruction error is negligible because the lost depth is minimal.
Example
The video uses a diorama analogy and shows a visual of points tightly clustered on a flat plane, labeling the 3rd dimension as containing only 5% of the variance.
Common misconceptions
- Assuming that 95% variance explained means the model is 95% accurate in predicting a target variable.
- Believing that the data is truly 3D but the PCA algorithm is failing to capture 5% of the information.
Watch the explanation
Explore next
Answers are generated from source material and independently checked. Consult the original video or creator if something is unclear.