GATE 2025 EC Q.54: solution and answer

GATE 2025 · ECQ.54 · 2 marks · MSQ

The random variable $X$ takes values in ${- 1, 0, 1}$ with probabilities $P (X = - 1) = P (X = 1) = α$ and $P (X = 0) = 1 - 2 α$ , where $0 < α < 1/2$ . Let $g (α)$ denote the entropy of $X$ (in bits), parameterized by $α$ . Which of the following statements is/are TRUE?

(A)
$g (0.4) > g (0.3)$
(B)
$g (0.3) > g (0.4)$
(C)
$g (0.3) > g (0.25)$
(D)
$g (0.25) > g (0.3)$

See the question in the paper

Tap to open full size

Answer and solution

ANSWER

B and C are true: $g (0.3) \approx 1.571$ bits exceeds both $g (0.4) \approx 1.522$ bits and $g (0.25) = 1.5$ bits.

Official answer: B, CSource: Official answer key, GATE 2025 EC (IIT Roorkee)

Entropy in bits: $g (α) = - 2 α lo g_{2} α - (1 - 2 α) lo g_{2} (1 - 2 α) .$
A three-symbol source has maximum entropy when all probabilities are equal, so $g$ peaks where $α = 1 - 2 α$ , i.e. $α = 1/3$ . The function $g$ is concave in $α$ , so it increases for $α < 1/3$ and decreases for $α > 1/3$ .
$g (0.3)$ : probabilities $(0.3, 0.4, 0.3)$ . $g (0.3) = - 0.6 lo g_{2} 0.3 - 0.4 lo g_{2} 0.4 = 0.6 (1.737) + 0.4 (1.322) \approx 1.571 bits .$
$g (0.4)$ : probabilities $(0.4, 0.2, 0.4)$ . $g (0.4) = - 0.8 lo g_{2} 0.4 - 0.2 lo g_{2} 0.2 = 0.8 (1.322) + 0.2 (2.322) \approx 1.522 bits .$
$g (0.25)$ : probabilities $(0.25, 0.5, 0.25)$ . $g (0.25) = 0.5 (2) + 0.5 (1) = 1.5 bits .$
Compare: $g (0.3) \approx 1.571 > g (0.4) \approx 1.522$ , so (B) is true and (A) is false. Also $g (0.3) \approx 1.571 > g (0.25) = 1.5$ , so (C) is true and (D) is false.
Check with the peak: $0.3$ is $0.033$ from $1/3$ , while $0.4$ is $0.067$ away and $0.25$ is $0.083$ away. Because $g$ is concave with its maximum at $1/3$ , $0.3$ is closest to the peak and gives the largest entropy of the three. Hence (B) and (C) are true.

KEY IDEA

Entropy of the distribution $(α, 1 - 2 α, α)$ is concave in $α$ with its maximum ( $lo g_{2} 3$ bits) at the uniform point $α = 1/3$ . The closer $α$ is to $1/3$ , the larger the entropy, so evaluate each value in $- \sum p lo g_{2} p$ .

WATCH OUT FOR

Assuming entropy increases with $α$ (so picking A), or forgetting the probability of the middle symbol is $1 - 2 α$ . Another slip is to treat this as a single-answer question and stop after finding one true statement, when MSQ needs every true option.

From the GATE 2025 EC paper. Every question of this paper · Download the full paper · All 9 Information theory questions, with notes

GATE 2025 EC Q.54: solution and answer

Answer and solution

More Information theory questions