Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“The insidiousness of science lies in its claim to be not a subject, but a method. You could ignore a subject; no subject is all-inclusive. But a method can plausibly be applied to anything within the field of consciousness.”
—Katharine Fullerton Gerould (1879–1944)
““English! they are barbarians; they don’t believe in the great God.” I told him, “Excuse me, Sir. We do believe in God, and in Jesus Christ too.” “Um,” says he, “and in the Pope?” “No.” “And why?” This was a puzzling question in these circumstances.... I thought I would try a method of my own, and very gravely replied, “Because we are too far off.” A very new argument against the universal infallibility of the Pope.”
—James Boswell (1740–1795)
“Women are denied masturbation even more severely than men and that’s another method of control—they’re not taught to please themselves.... Most women—it takes them a while to warm up to the “situation” but once they get into it, I’m sure they’re going to get just as hooked as—well, everyone I know is!”
—Lydia Lunch (b. 1959)