What would you like to understand?

Mutual information $I(X;Y)$ is related to marginal entropies $H(X)$ and $H(Y)$, and joint entropy $H(X,Y)$ by the identity $I(X;Y) = H(X) + H(Y) - H(X,Y)$. This is derived from the Venn diagram where the union of the two entropy circles is the joint entropy, and their intersection is the mutual information.

Conditions: $X$ and $Y$ are random variables with finite entropies.; The diagram treats $H(X,Y)$ as the union of the two entropy circles and $I(X;Y)$ as their intersection.

Mutual information $I(X;Y)$ is related to marginal entropies $H(X)$ and $H(Y)$, and joint entropy $H(X,Y)$ by the identity $I(X;Y) = H(X) + H(Y) - H(X,Y)$. This is derived from the Venn diagram where the union of the two entropy circles is the joint entropy, and their intersection is the mutual information.

Conditions: $X$ and $Y$ are random variables with finite entropies.; The diagram treats $H(X,Y)$ as the union of the two entropy circles and $I(X;Y)$ as their intersection.

In the entropy Venn diagram, the left circle represents the entropy $H(X)$ and the right circle represents $H(Y)$. Their intersection represents the mutual information $I(X;Y)$.

Conditions: Suitable for illustrating information decomposition between two discrete random variables $X$ and $Y$.

In the entropy Venn diagram, the left circle represents the entropy $H(X)$ and the right circle represents $H(Y)$. Their intersection represents the mutual information $I(X;Y)$.

Conditions: Suitable for illustrating information decomposition between two discrete random variables $X$ and $Y$.

Mutual information $I(X;Y)$ represents the reduction in uncertainty about $X$ after observing $Y$. $H(X)$ is the initial uncertainty about $X$, and $H(X|Y)$ is the remaining uncertainty after $Y$ is known.

Conditions: $X$ and $Y$ are random variables.; $H(X)$ and $H(X|Y)$ use the same unit system.

Mutual information $I(X;Y)$ represents the reduction in uncertainty about $X$ after observing $Y$. $H(X)$ is the initial uncertainty about $X$, and $H(X|Y)$ is the remaining uncertainty after $Y$ is known.

Conditions: $X$ and $Y$ are random variables.; $H(X)$ and $H(X|Y)$ use the same unit system.