What would you like to understand?

Mutual information $I(X;Y)$ represents the reduction in uncertainty about $X$ after observing $Y$. $H(X)$ is the initial uncertainty about $X$, and $H(X|Y)$ is the remaining uncertainty after $Y$ is known.

Conditions: $X$ and $Y$ are random variables.; $H(X)$ and $H(X|Y)$ use the same unit system.

Mutual information $I(X;Y)$ represents the reduction in uncertainty about $X$ after observing $Y$. $H(X)$ is the initial uncertainty about $X$, and $H(X|Y)$ is the remaining uncertainty after $Y$ is known.

Conditions: $X$ and $Y$ are random variables.; $H(X)$ and $H(X|Y)$ use the same unit system.

Entropy $H(X)$ is formally defined as the expected surprise of a random variable $X$. It is calculated using the formula $H(X) = - \sum_{x} p(x) \log_2 p(x)$, where $p(x)$ is the probability mass function of $X$.

Conditions: $X$ is a discrete random variable.; The summation runs over all possible values of $x$.; $p(x)$ is the probability mass function, satisfying $p(x) \ge 0$ and $\sum_x p(x) = 1$.; Using logarithm base two yields entropy measured in bits.

Entropy $H(X)$ is formally defined as the expected surprise of a random variable $X$. It is calculated using the formula $H(X) = - \sum_{x} p(x) \log_2 p(x)$, where $p(x)$ is the probability mass function of $X$.

Conditions: $X$ is a discrete random variable.; The summation runs over all possible values of $x$.; $p(x)$ is the probability mass function, satisfying $p(x) \ge 0$ and $\sum_x p(x) = 1$.; Using logarithm base two yields entropy measured in bits.