Method
- Divide the available data into training and validation sets.
- Use a large number of hidden units.
- Use very small random initial values.
- Use a slow learning rate.
- Compute the validation error rate periodically during training.
- Stop training when the validation error rate "starts to go up".
It is crucial to realize that the validation error is not a good estimate of the generalization error. One method for getting an unbiased estimate of the generalization error is to run the net on a third set of data, the test set, that is not used at all during the training process. The error on the test set gives estimate on generalization; to have the outputs of the net approximate target values given inputs that are not in the training set.
Read more about this topic: Early Stopping
Famous quotes containing the word method:
“You that do search for every purling spring
Which from the ribs of old Parnassus flows,
And every flower, not sweet perhaps, which grows
Near thereabouts into your poesy wring;
You that do dictionary’s method bring
Into your rhymes, running in rattling rows;”
—Sir Philip Sidney (1554–1586)
“We have not given science too big a place in our education, but we have made a perilous mistake in giving it too great a preponderance in method in every other branch of study.”
—Woodrow Wilson (1856–1924)
“I know no method to secure the repeal of bad or obnoxious laws so effective as their stringent execution.”
—Ulysses S. Grant (1822–1885)