DeepLearning

TRY YOUR SELF THIS QUESTIONS

 1.Which architecture can approximate any continuous function on [0,1]?

• (A) Single hidden layer with ReLU and enough neurons

• (B) Deep network with arbitrary width but no nonlinearity

• (C) Only networks with sigmoid activations

• (D) None of the above

2. why deep networks can be more parameter-efficient than shallow ones when approximating      complex functions?

3. What core challenge arises when generalizing bump functions to two or more inputs?

4. Analyze how many bump functions you would need (and thus neurons) to approximate a          piecewise-constant function with 10 segments. What if you use two hidden layers—how          does "depth" help?

5. Construct a network using ReLUs that approximates the triangular waveform?

6. True or False: The “bump building” method from requires exponentially many neurons        as   dimension increases?

7.Derive a 1‑hidden-layer network that emulates a step function?

 Given ReLU activation, find weights and biases 

• $w_1, w_2, b_1, b_2, v_1, v_2$ such that the network’s output approximates a step at $x = a$

$f(x) = \sin(\pi x)$ on [0, 1]. Show how to approximate it using a finite sum of         bump functions, and write the corresponding neural network formulation?

1.Momentum uses the current gradient to update the velocity, while NAG uses a lookahead gradient based on a tentative future position.