CORE JOBS
Linear Algebra interview questions
Short answers on the fundamentals, for the technical round of core-job interviews. 25 questions for DS & AI students; the numericals show the working.
Companies do not publish their interview questions; these are ours.
Vector spaces
Vector spaces notesWhat must a subset of a vector space satisfy to be a subspace?
It must contain the zero vector and be closed under addition and under scalar multiplication. Equivalently, every linear combination of its vectors must stay inside it. For example, the plane in is not a subspace, because it does not contain the zero vector.
What is the difference between a spanning set and a basis?
A spanning set reaches every vector in the space as a linear combination, but it may contain redundant vectors. A basis is a spanning set that is also linearly independent, so each vector has exactly one set of coordinates. All bases of a space have the same size, called its dimension.
Why is any set of vectors that contains the zero vector linearly dependent?
Dependence needs only one non-trivial combination that equals the zero vector. Give the zero vector the coefficient 1 and every other vector the coefficient 0. The sum is zero and not all coefficients are zero, so the set cannot be independent, whatever the other vectors are.
The vectors , and in are linearly dependent. Find .
The vectors are dependent when . The determinant of the matrix with the three vectors as rows is , which is zero only at , and then the third vector is the sum of the first two.
WORKING
- Three vectors in are dependent when the determinant of the matrix with them as rows is zero.
- .
- Set , so .
How do you find a basis for the column space of a matrix?
Row reduce the matrix to find its pivot columns, then take those same columns from the original matrix. The columns of the reduced matrix are not a basis, because row operations change the column space. The number of pivot columns is the rank, which is the dimension of the column space.
Special matrices
Special matrices notesWhat is an orthogonal matrix, and what values can its determinant and real eigenvalues take?
An orthogonal matrix is a square matrix with , so its columns are orthonormal. It preserves lengths and angles. Its determinant is or , and every real eigenvalue is or . Any complex eigenvalue has modulus 1.
What is the difference between an idempotent matrix and an orthogonal projection matrix?
An idempotent matrix satisfies . An orthogonal projection matrix is idempotent and also symmetric, . So every orthogonal projection matrix is idempotent, but an idempotent matrix can be an oblique projection that is not symmetric. Both have only 0 and 1 as eigenvalues.
Why does the rank of an idempotent matrix equal its trace?
An idempotent matrix has eigenvalues 0 or 1 only, because , and it is diagonalisable. Its rank is the number of eigenvalues equal to 1. Its trace is the sum of its eigenvalues, which is also the number of 1s. So rank and trace are equal.
A matrix is partitioned into blocks as , where is the zero block. If and , find the determinant of the whole matrix.
The determinant of the whole matrix is . A block upper triangular matrix with square diagonal blocks has determinant , whatever the block is, so the answer is .
WORKING
- The matrix is block upper triangular.
- Its determinant is , whatever is.
- .
How do you test whether a quadratic form is positive definite?
Take symmetric. The form is positive definite when all eigenvalues of are positive. Equivalently, all leading principal minors of are positive. If some eigenvalues are zero and the rest are positive, the form is only positive semidefinite.
Linear systems
Linear systems notesWhen does have a solution, and when is the solution unique?
It has a solution exactly when . If this common rank equals the number of unknowns , the solution is unique. If , there are infinitely many solutions, with free variables.
Why can Gaussian elimination use row operations without changing the solution set?
Each elementary row operation is reversible: swapping two rows, scaling a row by a non-zero number, or adding a multiple of one row to another. A reversible step turns the system into one with exactly the same solutions. Scaling by zero is not allowed, because it cannot be undone.
For which value of does the system , have infinitely many solutions?
The system has infinitely many solutions when , and none for any other . The second equation is twice the first only when . For any other the two equations describe parallel lines that never meet.
WORKING
- Subtract 2 times row 1 from row 2: the second equation becomes .
- The system is consistent only if .
- Then the rank is 1, below the 2 unknowns, so there are infinitely many solutions.
A matrix has rank 3. What is the dimension of the solution space of ?
The solution space has dimension 3. By the rank-nullity theorem, rank plus nullity equals the number of columns, so the nullity is . The number of rows does not enter.
WORKING
- Rank-nullity: rank + nullity = number of columns.
- The number of columns is 6 and the rank is 3.
- Nullity .
How do the solutions of differ from the solutions of with ?
always has the trivial solution, and its solutions form a subspace, the null space. may have no solution. If it has one, the full solution set is one particular solution plus any null space vector, a shifted copy of the null space that is not a subspace.
Eigenvalues, eigenvectors and determinants
Eigenvalues notesWhat is an eigenvector, and how are eigenvalues found?
An eigenvector of is a non-zero vector with , and is its eigenvalue. Eigenvalues are the roots of , because has a non-zero solution only when is singular.
Why is the trace of a matrix the sum of its eigenvalues and its determinant their product?
The characteristic polynomial factors as over the complex numbers. Its coefficient is and its constant term is . Comparing these with the factored form gives the sum and the product, with repeated eigenvalues counted by multiplicity.
Find the larger eigenvalue of .
The eigenvalues are 5 and 2, so the larger one is 5. The trace is 7 and the determinant is 10, so the characteristic equation is , which factors as .
WORKING
- Trace and determinant .
- The characteristic equation is .
- , so or .
How are the eigenvalues of , and related to those of ?
If , then , , and, for invertible , . The eigenvalues become , and , with the same eigenvectors. The transpose has the same eigenvalues as .
What is the difference between algebraic and geometric multiplicity, and why does it matter for diagonalisation?
Algebraic multiplicity is how often an eigenvalue is a root of the characteristic polynomial. Geometric multiplicity is the dimension of its eigenspace, and it is never larger. A matrix is diagonalisable only when the two are equal for every eigenvalue. For example, fails this test.
LU decomposition and SVD
LU & SVD notesWhat does the orthogonal projection of a vector onto a subspace give, and how is it computed?
It gives the point of the subspace closest to the vector, and the leftover part is orthogonal to the subspace. For a matrix with independent columns, the projection of onto the column space is .
Why does the least-squares solution satisfy the normal equations ?
Minimising means choosing as the closest point in the column space to . That happens when the residual is orthogonal to every column of , which is . Rearranging gives the normal equations.
Why is LU decomposition useful for solving for many right-hand sides?
The elimination is done once, giving . Each new then needs only a forward substitution and a back substitution , which cost about operations each, against about for elimination. If row swaps are needed, the factorisation is and the forward substitution solves .
In the LU decomposition of a matrix , has a unit diagonal and has diagonal entries 2, 3 and 4. Find .
The determinant of is 24. Since , . has a unit diagonal, so , and is the product of its diagonal entries, .
WORKING
- .
- is triangular with unit diagonal, so .
- is triangular, so .
What does the singular value decomposition say, and how do singular values show rank and the 2-norm?
Any matrix factors as , with and orthogonal and holding the non-negative singular values. The rank is the number of non-zero singular values, and the 2-norm is the largest one. Keeping the top singular values gives the best rank- approximation.
More interview questions: Data Structures and Algorithms · Probability and Statistics · Machine Learning · Database Management and Warehousing · Artificial Intelligence
Who hires and how they select: IT jobs for DS & AI
Linear Algebra: notes, formula sheet and past questions