What is the Matlab exercise about conditional entropy?
Conditions
- Input p is assumed to be a 2D joint probability array .
- Students may reuse existing entropy() and jointentropy() routines.
Reasoning, step by step
- Define the function conditionalentropy(p) taking a 2D matrix p.
- Compute the joint entropy and marginal entropy using existing routines, or calculate directly from p.
- Apply the chain rule H(X|Y) = to return the result.
- Validate the function against provided test cases: conditionalentropy([0.5,0;0,0.5]) should equal 0, and conditionalentropy([0.25,0.25;0.25,0.25]) should equal 1.
Example
The exercise slide states: 'Edit the Matlab function conditionalentropy(p) to return the conditional entropy for X given Y for the joint probability p.' with printed tests conditionalentropy([0.5,0;0,0.5])=0 and conditionalentropy([0.25,0.25;0.25,0.25])=1.
Common misconceptions
- Assuming the input p is a marginal probability vector rather than a 2D joint probability matrix.
- Forgetting that conditional entropy is not symmetric, as highlighted by the Guess Who! example comparing H(sex|earrings) and H(earrings|sex).
Watch the explanation
Connected concepts
Explore next
Related questions
Conditional entropy H(X|Y) measures the average remaining uncertainty or surprise about a random variable X after another random variable Y has been observed. It is defined as the expected value of the conditional self-information h(x|y) over the joint distribution of X and Y.
Conditions: X and Y are discrete random variables.; The sum is taken over the joint alphabet .
Joint entropy is not equal to because adding the marginal entropies double-counts the mutual information I(X;Y) shared between the variables. The correct relationship is (X;Y).
Conditions: X and Y are random variables that may share information (i.e., they are not necessarily independent).
When conditional entropy H(X|Y) is zero, it means there is no remaining uncertainty or surprise about X once Y is known. This implies that X is a deterministic function of Y, so all information about X is completely contained within Y.
Conditions: X and Y are discrete random variables.
Conditional entropy is the difference between the joint entropy of X and Y and the marginal entropy of Y. This relationship is expressed by the entropy chain rule: H(X|Y) = .
Conditions: X and Y are discrete random variables with well-defined joint and marginal entropies.
Conditional entropy H(X|Y) is bounded by (X|Y) ≤ because learning the value of Y provides information that can only reduce or leave unchanged the average uncertainty about X; it cannot increase it on average.
Conditions: X and Y are discrete random variables.
Answers are generated from source material and independently checked. Consult the original video or creator if something is unclear.