Why does mutual information equal zero when variables are independent?

Q: Why does mutual information equal zero when variables are independent?

When $X$ and $Y$ are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals, $p(x,y) = p(x) \cdot p(y)$. Because there is no shared information or statistical structure beyond the product of marginals, the mutual information $I(X;Y)$ is exactly zero.

When

X

and

Y

are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals,

p(x,y) = p(x) \cdot p(y)

. Because there is no shared information or statistical structure beyond the product of marginals, the mutual information

I(X;Y)

is exactly zero.

Conditions

Applies to the case where $X$ and $Y$ are independent.
The displayed notation uses probability mass functions $p(x)$ , $p(y)$ , and $p(x,y)$ .

Reasoning, step by step

Identify that $X$ and $Y$ are independent.
State the factorization of the joint distribution: $p(x,y) = p(x) \cdot p(y)$ .
Recognize that observing $Y$ provides no reduction in uncertainty about $X$ .
Conclude that the mutual information $I(X;Y)$ is exactly zero.

Example

The video shows a square grid uniformly shaded orange under the heading 'Independent', with the formulas $p(x,y)=p(x)\cdot p(y)$ and $I(X;Y)=0$ displayed below it.

Common misconceptions

Confusing dependence with zero mutual information; dependence is paired with non-factorization and strictly positive mutual information.
Assuming that independent variables can still have some pattern in their joint distribution; the uniform grid shows no concentration pattern.

Watch the explanation

YouTubeMutual Information - Explained

2:13 – 2:28Watch this moment ↗

Connected concepts

Conditional entropy →Mutual information →

Why does mutual information equal zero when variables are independent?

Conditions

Reasoning, step by step

Example

Common misconceptions

Watch the explanation

Connected concepts

Explore next

Related questions

Why does mutual information equal zero when variables are independent?

Conditions

Reasoning, step by step

Example

Common misconceptions

Watch the explanation

Connected concepts

Explore next

Related questions

What is conditional entropy H(X|Y) and how is it defined for discrete random variables?+

Why is joint entropy H(X,Y)H(X,Y)H(X,Y) not equal to the sum of marginal entropies H(X)+H(Y)H(X) + H(Y)H(X)+H(Y)?+

Which information quantities correspond to the intersection, left and right crescents, and overall region in the Venn diagram?+

Is mutual information symmetric in X and Y?+

What does it mean when conditional entropy H(X|Y) is zero?+