Why is joint entropy $H(X,Y)$ not equal to the sum of marginal entropies $H(X) + H(Y)$?

Conditional entropy H(X|Y) measures the average remaining uncertainty or surprise about a random variable X after another random variable Y has been observed. It is defined as the expected value of the conditional self-information h(x|y) over the joint distribution of X and Y.

Conditions: X and Y are discrete random variables.; The sum is taken over the joint alphabet $A_x \times A_y$.

In the entropy Venn diagram, the left circle represents the entropy $H(X)$ and the right circle represents $H(Y)$. Their intersection represents the mutual information $I(X;Y)$.

Conditions: Suitable for illustrating information decomposition between two discrete random variables $X$ and $Y$.

When $X$ and $Y$ are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals, $p(x,y) = p(x) \cdot p(y)$.

Conditions: Applies to the case where $X$ and $Y$ are independent.; The displayed notation uses probability mass functions $p(x)$, $p(y)$, and $p(x,y)$.

Yes, mutual information is perfectly symmetric. The formula $I(X;Y) = I(Y;X)$ holds true.

Conditions: Applies to any pair of random variables $X$ and $Y$ for which mutual information is defined.

When conditional entropy H(X|Y) is zero, it means there is no remaining uncertainty or surprise about X once Y is known. This implies that X is a deterministic function of Y, so all information about X is completely contained within Y.

Conditions: X and Y are discrete random variables.