Mutual information

Mutual information measures statistical dependence as the reduction in uncertainty from observing another variable.

Different ways to understand it

No reviewed video yet. Explore the concept in a map or suggest a resource. Suggest a video

Conditional entropy
Conditional entropy averages remaining uncertainty after another variable is observed.
Relative entropy
KL divergence compares distributions and is generally asymmetric, with support conditions affecting finiteness.
Channel capacity
Capacity bounds achievable reliable information rates for a specified channel model.
Error correction bounds
Minimum distance d permits correcting up to floor((d-1)/2) arbitrary symbol errors.
Hamming distance
Hamming distance counts differing positions between equal-length strings.