Conditional entropy is the difference between the joint entropy of X and Y and the marginal entropy of Y. This relationship is expressed by the entropy chain rule: H(X|Y) = H(X,Y)−H(Y).
Conditions: X and Y are discrete random variables with well-defined joint and marginal entropies.
Conditional entropy is the difference between the joint entropy of X and Y and the marginal entropy of Y. This relationship is expressed by the entropy chain rule: H(X|Y) = H(X,Y)−H(Y).
Conditions: X and Y are discrete random variables with well-defined joint and marginal entropies.
Conditional entropy H(X|Y) measures the average remaining uncertainty or surprise about a random variable X after another random variable Y has been observed. It is defined as the expected value of the conditional self-information h(x|y) over the joint distribution of X and Y.
Conditions: X and Y are discrete random variables.; The sum is taken over the joint alphabet Ax×Ay.
Conditional entropy H(X|Y) measures the average remaining uncertainty or surprise about a random variable X after another random variable Y has been observed. It is defined as the expected value of the conditional self-information h(x|y) over the joint distribution of X and Y.
Conditions: X and Y are discrete random variables.; The sum is taken over the joint alphabet Ax×Ay.
Joint entropy H(X,Y) is not equal to H(X)+H(Y) because adding the marginal entropies double-counts the mutual information I(X;Y) shared between the variables. The correct relationship is H(X,Y)=H(X)+H(Y)−I(X;Y).
Conditions: X and Y are random variables that may share information (i.e., they are not necessarily independent).
Joint entropy H(X,Y) is not equal to H(X)+H(Y) because adding the marginal entropies double-counts the mutual information I(X;Y) shared between the variables. The correct relationship is H(X,Y)=H(X)+H(Y)−I(X;Y).
Conditions: X and Y are random variables that may share information (i.e., they are not necessarily independent).
The exercise asks students to implement a Matlab function conditionalentropy(p) that computes the conditional entropy of X given Y from a 2D joint probability matrix p(x,y). It includes specific test cases to verify correctness, such as deterministic and uniform distributions.
Conditions: Input p is assumed to be a 2D joint probability array p(x,y).; Students may reuse existing entropy() and jointentropy() routines.
The exercise asks students to implement a Matlab function conditionalentropy(p) that computes the conditional entropy of X given Y from a 2D joint probability matrix p(x,y). It includes specific test cases to verify correctness, such as deterministic and uniform distributions.
Conditions: Input p is assumed to be a 2D joint probability array p(x,y).; Students may reuse existing entropy() and jointentropy() routines.