CORE JOBS
Machine Learning interview questions
Short answers on the fundamentals, for the technical round of core-job interviews. 25 questions for DS & AI students; the numericals show the working.
Companies do not publish their interview questions; these are ours.
Simple, multiple and ridge regression
Regression notesWhat does ordinary least squares minimise, and what are the normal equations?
Ordinary least squares minimises the sum of squared residuals. Setting the gradient to zero gives the normal equations , so , provided is invertible, which needs the feature columns to be linearly independent.
Why does ridge regression help when the features are highly correlated?
Ridge adds to , giving . With correlated features is nearly singular and plain least-squares weights swing wildly. The penalty keeps the matrix well conditioned and shrinks the weights, trading a little bias for a large drop in variance.
Why are features usually scaled before running gradient descent on a regression problem?
Features on very different scales make the loss surface a long, narrow valley, so gradient descent needs a small learning rate and zigzags slowly. Standardising the features makes the contours rounder and speeds convergence. It also matters for ridge, because the penalty treats every weight alike.
A line is fitted by least squares to the points (1, 2), (2, 4) and (3, 5). Find the slope.
The least-squares slope is . With means and , the sum of cross deviations is and the sum of squared x deviations is , so the slope is .
WORKING
- Means: , .
- .
- .
- Slope .
Logistic regression, k-nearest neighbour and naive Bayes
Logistic, k-NN, naive Bayes notesHow does logistic regression model a class probability, and what loss is it trained on?
It models with , and is trained by minimising cross-entropy, usually with gradient descent. Its decision boundary is linear in the features.
A logistic regression model has weights 2 and -1 and a bias of -1. For the input (1, 2), find the predicted probability of class 1.
The linear score is . The sigmoid turns this score into the probability of class 1: . A negative score always gives a probability below 0.5.
WORKING
- .
- .
- , so .
What is the difference between logistic regression and k-nearest neighbour as classifiers?
Logistic regression is parametric: training learns a weight vector, prediction is fast and the boundary is linear. k-NN is non-parametric: it only stores the data and predicts from distances to stored points. Its boundary is flexible, but prediction is slow and it is sensitive to feature scale.
Why is naive Bayes called naive, and what does the assumption buy?
It assumes the features are conditionally independent given the class, so . This needs only one distribution per feature and class (simple counts for discrete features) instead of a joint distribution over all features, so training is fast and works with little data, even when the assumption holds only roughly.
LDA, SVMs and decision trees
LDA, SVM & trees notesWhat does linear discriminant analysis assume about the classes?
LDA models each class as Gaussian with its own mean but one covariance matrix shared by all classes. This makes the log-odds between classes linear in , so the boundary is a hyperplane. If the true class covariances differ a lot, the linear boundary fits poorly.
What is the margin of a support vector machine, and what does it maximise?
The margin is the gap between the two classes on either side of the separating hyperplane. With labels and constraints , its width is , so maximising it means minimising . Only the support vectors on the margin fix the plane.
What does the parameter C control in a soft-margin SVM, and why does it matter?
C sets the penalty for margin violations. A large C punishes violations heavily, giving a narrow margin that fits the training data closely and may overfit. A small C tolerates more violations, giving a wider margin and a smoother boundary that may underfit.
A node holds 8 samples, 4 of each class. A split sends 4 samples to the left child (3 of class A, 1 of class B) and 4 to the right child (1 of class A, 3 of class B). Find the information gain of the split, using entropy in bits.
The parent node, with 4 samples of each class, has entropy 1 bit. Each child holds a 3 to 1 split, with entropy bits. Both children hold half the samples, so the weighted child entropy is 0.811 and the information gain is bits.
WORKING
- Parent entropy: bit (4 and 4 out of 8).
- Each child: .
- Weighted child entropy .
- Gain bits.
Why does a deep decision tree overfit, and how does pruning help?
A tree grown until every leaf is pure memorises noise, so it has low bias but high variance. Pruning, or limiting depth or minimum leaf size, removes splits that reduce impurity very little. The simpler tree usually generalises better to unseen data.
Bias–variance trade-off and cross-validation
Bias–variance & CV notesWhat are the three parts of expected squared test error in the bias-variance decomposition?
Expected squared error equals bias squared plus variance plus irreducible noise. Bias is the error from wrong model assumptions. Variance is how much the fitted model changes from one training set to another. Noise is randomness in the target, which no model can remove.
Why is training error a poor estimate of test error?
The model was fitted to those same points, so training error is optimistic and keeps falling as the model grows more complex, even when it is memorising noise. Error on held-out data rises again past some complexity, so only held-out or cross-validated error reveals overfitting.
What is the difference between k-fold cross-validation and leave-one-out cross-validation?
In k-fold cross-validation the data is split into parts, each used once for validation, so the model is trained times. Leave-one-out is the case : fits, each on points. It uses almost all data per fit but costs far more computation.
What are precision and recall, and why are they used instead of accuracy on imbalanced data?
Precision is , the share of predicted positives that are truly positive. Recall is , the share of actual positives that were found. Accuracy can look high on imbalanced classes while missing the rare class, so precision, recall and their harmonic mean F1 are reported.
Multi-layer perceptrons and feed-forward neural networks
Neural networks notesWhy must the activation functions in a neural network be non-linear?
Without a non-linearity each layer is a linear map, and a composition of linear maps is itself one linear map, so any depth collapses to a single linear model. A non-linear activation such as ReLU or sigmoid lets the network represent non-linear functions.
What is the perceptron update rule, and when does it converge?
With labels , the perceptron predicts and, on a misclassified point, updates and . It converges in a finite number of steps if and only if the classes are linearly separable. XOR is the standard example where it never settles.
A fully connected feed-forward network has 4 inputs, two hidden layers of 5 and 3 neurons, and 1 output neuron. Every neuron has a bias. How many trainable parameters does it have?
Each layer has one weight per input for every neuron, plus one bias per neuron. The layers give , and , so the network has trainable parameters.
WORKING
- Input to first hidden layer: .
- First to second hidden layer: .
- Second hidden layer to output: .
- Total .
Why is backpropagation efficient for computing gradients in a neural network?
Backpropagation applies the chain rule layer by layer from the output backwards, reusing each layer's gradient for the layer before it. It gives the loss gradient for every weight in one backward pass, costing about as much as a forward pass, instead of one pass per weight.
Clustering
Clustering notesWhat is the difference between k-means and k-medoids?
k-means uses the mean of a cluster as its centre, which need not be a data point and is pulled by outliers. k-medoids uses an actual data point as the centre, so it is more robust to outliers and works with any distance measure, but it costs more to compute.
How does the k-means algorithm work, and what does it minimise?
k-means alternates two steps: assign each point to its nearest centre, then move each centre to the mean of its points. Neither step can increase the within-cluster sum of squared distances, so it converges, but only to a local minimum, so the result depends on the starting centres.
Dimensionality reduction and principal component analysis
PCA notesWhat does principal component analysis find, and how is it computed?
PCA finds orthogonal directions of maximum variance in centred data. They are the eigenvectors of the covariance matrix, and each eigenvalue is the variance along its direction. Projecting onto the top eigenvectors reduces the dimension while keeping as much variance as possible.
The covariance matrix of centred two-dimensional data has eigenvalues 6 and 2. What percentage of the total variance does the first principal component explain?
In PCA each eigenvalue of the covariance matrix is the variance along its principal component, and the total variance is their sum, . The first component explains of it, which is 75% of the total variance.
WORKING
- Total variance sum of eigenvalues .
- Fraction explained .
- As a percentage, 75%.
More interview questions: Data Structures and Algorithms · Probability and Statistics · Linear Algebra · Database Management and Warehousing · Artificial Intelligence
Who hires and how they select: IT jobs for DS & AI
Machine Learning: notes, formula sheet and past questions