IAE Academy

GATE 2024 DA Q.43: solution and answer

GATE 2024 · DAQ.43 · 2 marks · MCQ

Consider the two neural networks (NNs) shown in Figures 1 and 2, with activation (). denotes the set of real numbers. The connections and their corresponding weights are shown in the Figures. The biases at every neuron are set to 0. For what values of in Figure 2 are the two NNs equivalent, when are positive?

GATE 2024 DA question 43: Figure 1: a feed-forward network with inputs x1, x2, x3, a first hidden layer of 3 neurons (top, middle, bottom), a second…Tap to open full size
Figure from the GATE 2024 DA paper. It is part of the question.
Figure description

Figure 1: a feed-forward network with inputs x1, x2, x3, a first hidden layer of 3 neurons (top, middle, bottom), a second hidden layer of 2 neurons and one output neuron. Input to first layer (7 edges, each weight 1): x1 to top, x1 to middle, x1 to bottom; x2 to top, x2 to bottom; x3 to middle, x3 to bottom. There is no edge from x2 to the middle neuron and none from x3 to the top neuron. First layer to second layer: fully connected, all 6 edges weight 2. The two second-layer neurons each connect to the output neuron with weight 3. Figure 2: a single output neuron connected directly to x1, x2, x3 with weights p, q, r respectively.

  1. (A)

  2. (B)

  3. (C)

  4. (D)

Answer and solution

ANSWER

(A) p = 36, q = 24, r = 24

Official answer: ASource: Official answer key, GATE 2024 DA (IISc Bengaluru)

  1. Read the first layer of Figure 1 edge by edge, since it is not fully connected. feeds all three first-layer neurons. feeds only the top and bottom neurons. feeds only the middle and bottom neurons. Every one of these edges has weight 1, and all biases are 0.

  2. Pre-activations of the first layer: top , middle , bottom . All three are positive because , so ReLU passes each one unchanged: , , .

  3. Add them up: .

  4. The second layer is fully connected with weight 2 on every edge, so each of its two neurons has pre-activation . ReLU leaves this unchanged: .

  5. The output neuron has weight 3 from each second-layer neuron: which is positive, so a ReLU at the output would change nothing.

  6. Figure 2 computes . For the two networks to agree for every positive , the coefficients must match: , , . This is option (A).

KEY IDEA

With positive inputs, zero biases and positive weights, every pre-activation is positive. ReLU then acts as the identity, so Figure 1 collapses to a linear map. The coefficient of each input equals the sum, over all paths from that input to the output, of the product of the weights along the path. reaches 3 first-layer neurons and each first-layer neuron gives downstream, so . and each reach only 2 first-layer neurons, so .

WATCH OUT FOR

Assuming the first layer is fully connected (9 edges) because the crossing lines look dense. That gives and option (D). Count the edges leaving each input: has 3, but and have only 2 each.

From the GATE 2024 DA paper. Every question of this paper · Download the full paper · All 5 Neural networks questions, with notes

More Neural networks questions