GATE 2020 EC Q.22: solution and answer

Official answer: 0.5Source: Official answer key, GATE 2020 EC (IIT Delhi)

1. $X$ takes two values with probabilities $\alpha$ and $1-\alpha$. Entropy depends only on these probabilities, not on the values $\pm 2$.

2. Binary entropy: $H(\alpha )=-\alpha {\log }_2\alpha -(1-\alpha ){\log }_2(1-\alpha )$ bits.

3. Set the derivative to zero: $\frac{dH}{d\alpha }={\log }_2\frac{1-\alpha }{\alpha }=0$.

4. This gives $\frac{1-\alpha }{\alpha }=1$, so $\alpha =0.5$.

5. Second derivative: $\frac{d^{2}H}{d{\alpha }^2}=-\frac{1}{\ln 2}\left(\frac{1}{\alpha }+\frac{1}{1-\alpha }\right)<0$ for $0<\alpha <1$, so this is a maximum.

6. At $\alpha =0.5$, $H=1$ bit, the maximum for a binary source. To one decimal place, $\alpha =0.5$.