What would you like to understand?

Yes, mutual information is perfectly symmetric. The formula $I(X;Y) = I(Y;X)$ holds true.

Conditions: Applies to any pair of random variables $X$ and $Y$ for which mutual information is defined.

Yes, mutual information is perfectly symmetric. The formula $I(X;Y) = I(Y;X)$ holds true.

Conditions: Applies to any pair of random variables $X$ and $Y$ for which mutual information is defined.

When $X$ and $Y$ are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals, $p(x,y) = p(x) \cdot p(y)$.

Conditions: Applies to the case where $X$ and $Y$ are independent.; The displayed notation uses probability mass functions $p(x)$, $p(y)$, and $p(x,y)$.

When $X$ and $Y$ are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals, $p(x,y) = p(x) \cdot p(y)$.

Conditions: Applies to the case where $X$ and $Y$ are independent.; The displayed notation uses probability mass functions $p(x)$, $p(y)$, and $p(x,y)$.

The KL-divergence expression is expanded by applying the standard definition of discrete KL divergence. By setting $P = p(x,y)$ and $Q = p(x)p(y)$, the divergence becomes a sum over all pairs $(x,y)$ in the joint support of the joint probability times the logarithm of the ratio of the joint probability to the product of the marginals.

Conditions: The derivation is for discrete distributions using probability mass functions.; The logarithm base is not explicitly stated in the expanded sum, though earlier entropy used base 2.; The ratio form requires $p(x,y)>0$ and $p(x)p(y)>0$ on the summed support.

The KL-divergence expression is expanded by applying the standard definition of discrete KL divergence. By setting $P = p(x,y)$ and $Q = p(x)p(y)$, the divergence becomes a sum over all pairs $(x,y)$ in the joint support of the joint probability times the logarithm of the ratio of the joint probability to the product of the marginals.

Conditions: The derivation is for discrete distributions using probability mass functions.; The logarithm base is not explicitly stated in the expanded sum, though earlier entropy used base 2.; The ratio form requires $p(x,y)>0$ and $p(x)p(y)>0$ on the summed support.

Conceptually, this formula identifies mutual information as the Kullback-Leibler (KL) divergence between the actual joint distribution $p(x,y)$ and the independent reference distribution $p(x)p(y)$. It measures how far the actual joint distribution is from pure independence.

Conditions: The displayed derivation is for discrete distributions using probability mass functions.; The ratio form requires $p(x,y)>0$ and $p(x)p(y)>0$ on the summed support.

Conceptually, this formula identifies mutual information as the Kullback-Leibler (KL) divergence between the actual joint distribution $p(x,y)$ and the independent reference distribution $p(x)p(y)$. It measures how far the actual joint distribution is from pure independence.

Conditions: The displayed derivation is for discrete distributions using probability mass functions.; The ratio form requires $p(x,y)>0$ and $p(x)p(y)>0$ on the summed support.