Skip to content
START WITH A QUESTION

What would you like to understand?

Find an answer. See the moment it becomes clear. Follow the idea further.

← Concept directory

Answers for “两个圆不断增长的重叠代表什么?”

2 keyword matches

Understanding your question. You can explore the search results below now.

Meet the concept

↗

Yes, mutual information is perfectly symmetric. The formula I(X;Y)=I(Y;X)I(X;Y) = I(Y;X) holds true.

Conditions: Applies to any pair of random variables XX and YY for which mutual information is defined.

Find a method

↗

The KL-divergence expression is expanded by applying the standard definition of discrete KL divergence. By setting P=p(x,y)P = p(x,y) and Q=p(x)p(y)Q = p(x)p(y), the divergence becomes a sum over all pairs (x,y)(x,y) in the joint support of the joint probability times the logarithm of the ratio of the joint probability to the product of the marginals.

Conditions: The derivation is for discrete distributions using probability mass functions.; The logarithm base is not explicitly stated in the expanded sum, though earlier entropy used base 2.; The ratio form requires p(x,y)>0p(x,y)>0 and p(x)p(y)>0p(x)p(y)>0 on the summed support.