What does it mean when conditional entropy H(X|Y) is zero?
Conditions
- X and Y are discrete random variables.
Reasoning, step by step
- Start with the definition of H(X|Y) as the average remaining surprise in X given Y.
- Set H(X|Y) = 0.
- Conclude that for every outcome y of Y, the conditional distribution p(x|y) must be concentrated on a single value of x (i.e., it has zero entropy).
- Recognize that this means knowing y uniquely determines x, making X a function of Y.
Example
The lecturer explains: 'our conditional entropy being zero simply means there is no surprise left in X once we know Y... all the information in X would be contained inside Y'. In the Venn diagram, this corresponds to the H(X|Y) crescent region vanishing entirely.
Common misconceptions
- Thinking that H(X|Y) = 0 means X and Y are independent.
- Assuming that H(X|Y) = 0 implies X and Y have the same entropy, rather than X being fully determined by Y.
Watch the explanation
Connected concepts
Explore next
Related questions
Conditional entropy H(X|Y) measures the average remaining uncertainty or surprise about a random variable X after another random variable Y has been observed. It is defined as the expected value of the conditional self-information h(x|y) over the joint distribution of X and Y.
Conditions: X and Y are discrete random variables.; The sum is taken over the joint alphabet .
Joint entropy is not equal to because adding the marginal entropies double-counts the mutual information I(X;Y) shared between the variables. The correct relationship is (X;Y).
Conditions: X and Y are random variables that may share information (i.e., they are not necessarily independent).
In the entropy Venn diagram, the left circle represents the entropy and the right circle represents . Their intersection represents the mutual information .
Conditions: Suitable for illustrating information decomposition between two discrete random variables and .
When and are independent, knowing one tells you nothing about the other. Mathematically, the joint distribution factors into the product of the marginals, .
Conditions: Applies to the case where and are independent.; The displayed notation uses probability mass functions , , and .
Yes, mutual information is perfectly symmetric. The formula holds true.
Conditions: Applies to any pair of random variables and for which mutual information is defined.
Answers are generated from source material and independently checked. Consult the original video or creator if something is unclear.