GATE 2024 DA Q.22: solution and answer

Official answer: ASource: Official answer key, GATE 2024 DA (IISc Bengaluru)

1. $J(u)$ is a ratio of quadratic forms (a Rayleigh quotient) and is unchanged when $u$ is scaled. At the maximiser $u^{\ast }\ne 0$ the gradient must vanish: $\nabla J(u^{\ast })=0$.

2. Differentiate with the quotient rule, using that $S_B$ and $S_W$ are symmetric: $$\nabla J=\frac{2S_{B}u(u^T{S}_{W}u)-2S_{W}u(u^T{S}_{B}u)}{(u^T{S}_{W}u{)}^2}.$$

3. Set this to zero at $u^{\ast }$ and divide by $u^{\ast T}{S}_W{u}^{\ast }$, which is positive because $S_W$ is a non-singular scatter matrix (positive definite) and $u^{\ast }\ne 0$: $$S_B{u}^{\ast }=\frac{u^{\ast T}{S}_B{u}^{\ast }}{u^{\ast T}{S}_W{u}^{\ast }}{S}_W{u}^{\ast }=\lambda S_W{u}^{\ast }.$$

4. This is the generalised eigenvalue problem $S_B{u}^{\ast }=\lambda S_W{u}^{\ast }$, where $\lambda =J(u^{\ast })$ is the maximum value of the quotient. Since $S_B\ne 0$ is positive semi-definite, some $u$ gives $u^T{S}_{B}u>0$, so $\lambda >0$.

5. Since $S_W$ is non-singular, multiply on the left by $S_W^{-1}$: $$S_W^{-1}{S}_B{u}^{\ast }=\lambda u^{\ast }.$$ So $u^{\ast }$ is the eigenvector of $S_W^{-1}{S}_B$ belonging to its largest eigenvalue $\lambda$. This is option (A).

6. Check the others. (B) has $S_W$ and $S_B$ swapped: $S_W{u}^{\ast }=\lambda S_B{u}^{\ast }$ would mean $S_B{u}^{\ast }={\lambda }^{-1}{S}_W{u}^{\ast }$, which contradicts step 3 unless ${\lambda }^2=1$. (C) has the matrices in the wrong order and no inverse, so it does not follow. (D) is not implied, because $u^{\ast }$ can be scaled freely without changing $J$, so $u^{\ast T}{u}^{\ast }$ is not fixed by $\lambda$. Only (A) must hold.