GATE 2025 EC Q.60: solution and answer

GATE 2025 · ECQ.60 · 2 marks · NAT

$X$ and $Y$ are Bernoulli random variables taking values in ${0, 1}$ . The joint probability mass function of the random variables is given by: $P (X = 0, Y = 0) = 0.06$ , $P (X = 0, Y = 1) = 0.14$ , $P (X = 1, Y = 0) = 0.24$ , $P (X = 1, Y = 1) = 0.56$ . The mutual information $I (X; Y)$ is ________ (rounded off to two decimal places).

See the question in the paper

Tap to open full size

Answer and solution

ANSWER

$I (X; Y) = 0.00$ bits, because $X$ and $Y$ are independent.

Official answer: 0.0Source: Official answer key, GATE 2025 EC (IIT Roorkee)

Marginal of $X$ : $P (X = 0) = 0.06 + 0.14 = 0.20$ and $P (X = 1) = 0.24 + 0.56 = 0.80$ .
Marginal of $Y$ : $P (Y = 0) = 0.06 + 0.24 = 0.30$ and $P (Y = 1) = 0.14 + 0.56 = 0.70$ .
Mutual information is $I (X; Y) = \sum_{x, y} P (x, y) lo g_{2} \frac{P ( x , y )}{P ( x ) P ( y )}$ . Each term is zero when $P (x, y) = P (x) P (y)$ , and $I (X; Y) = 0$ exactly when this holds for every pair.
Check each cell: $0.2 \times 0.3 = 0.06$ , $0.2 \times 0.7 = 0.14$ , $0.8 \times 0.3 = 0.24$ , $0.8 \times 0.7 = 0.56$ . Every product equals the given joint probability.
So $X$ and $Y$ are independent and every log term is $lo g_{2} 1 = 0$ . Hence $I (X; Y) = 0$ bits, which is $0.00$ to two decimal places.

KEY IDEA

Mutual information is zero exactly when the joint PMF equals the product of its marginals. Compute the marginals and test this before doing any entropy arithmetic.

WATCH OUT FOR

Computing $H (X)$ , $H (Y)$ and $H (X, Y)$ by hand and letting rounding errors leave a small non-zero value. Another slip is to take the joint probabilities as a dependent pair without checking the products of the marginals.

From the GATE 2025 EC paper. Every question of this paper · Download the full paper · All 9 Information theory questions, with notes

GATE 2025 EC Q.60: solution and answer

Answer and solution

More Information theory questions