[PDF] programming exercise 4: neural networks learning

What are neural network algorithms for machine learning?

Neural-network algorithms for machine learning are inspired by the architecture and the dynamics of networks of neurons in the brain. The algorithms use highly idealised neuron models. Nevertheless, the fundamental principle is the same: arti?cial neural networks learn by changing the connections between their neurons.

How to train a neural network?

Once you have computed the gradient, you will be ableto train the neural network by minimizing the cost functionJ() using anadvanced optimizer such asfmincg. You will frst implement the backpropagation algorithm to compute thegradients for the parameters for the (unregularized) neural network.

Does a neural network with more neurons classify the input data better?

A neural network with more neurons may classify the input data better, because it more accurately represents all speci?c features of the given data set. But a different set of patterns from the same input distribution can look quite different in detail, in which case the decision boundary may not classify the new data very well (Figure6.6).

Why is convergence important in the analysis of neural-network algorithms?

This is an important question in the analysis of neural- network algorithms, because an algorithm that does not converge to a meaningful solution is useless. The standard way of analysing convergence of neural-network algorithms is to de?ne anenergy function H(s) that has a minimum at the desired solution,s=x(1) say.