Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“As a science of the unconscious it is a therapeutic method, in the grand style, a method overarching the individual case. Call this, if you choose, a poet’s utopia.”
—Thomas Mann (1875–1955)
“Direct action ... is the logical, consistent method of Anarchism.”
—Emma Goldman (1869–1940)
“The insidiousness of science lies in its claim to be not a subject, but a method. You could ignore a subject; no subject is all-inclusive. But a method can plausibly be applied to anything within the field of consciousness.”
—Katharine Fullerton Gerould (1879–1944)