Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“One of the grotesqueries of present-day American life is the amount of reasoning that goes into displaying the wisdom secreted in bad movies while proving that modern art is meaningless.... They have put into practise the notion that a bad art work cleverly interpreted according to some obscure Method is more rewarding than a masterpiece wrapped in silence.”
—Harold Rosenberg (1906–1978)
“There is no method but to be very intelligent.”
—T.S. (Thomas Stearns)
“I am not afraid of the priests in the long-run. Scientific method is the white ant which will slowly but surely destroy their fortifications. And the importance of scientific method in modern practical life—always growing and increasing—is the guarantee for the gradual emancipation of the ignorant upper and lower classes, the former of whom especially are the strength of the priests.”
—Thomas Henry Huxley (1825–95)