Supervised Learning - How Supervised Learning Algorithms Work

How Supervised Learning Algorithms Work

Given a set of training examples of the form, a learning algorithm seeks a function, where is the input space and is the output space. The function is an element of some space of possible functions, usually called the hypothesis space. It is sometimes convenient to represent using a scoring function such that is defined as returning the value that gives the highest score: . Let denote the space of scoring functions.

Although and can be any space of functions, many learning algorithms are probabilistic models where takes the form of a conditional probability model g(x) = P(y|x), or takes the form of a joint probability model . For example, naive Bayes and linear discriminant analysis are joint probability models, whereas logistic regression is a conditional probability model.

There are two basic approaches to choosing or : empirical risk minimization and structural risk minimization. Empirical risk minimization seeks the function that best fits the training data. Structural risk minimize includes a penalty function that controls the bias/variance tradeoff.

In both cases, it is assumed that the training set consists of a sample of independent and identically distributed pairs, . In order to measure how well a function fits the training data, a loss function L: Y \times Y \to \Bbb{R}^{\ge 0} is defined. For training example, the loss of predicting the value is .

The risk of function is defined as the expected loss of . This can be estimated from the training data as

.

Read more about this topic:  Supervised Learning

Famous quotes containing the words supervised, learning and/or work:

    “It is ultimately in employers’ best interests to have their employees’ families functioning smoothly. In the long run, children who misbehave because they are inadequately supervised or marital partners who disapprove of their spouse’s work situation are productivity problems. Just as work affects parents and children, parents and children affect the workplace by influencing the employed parents’ morale, absenteeism, and productivity.”
    —Ann C. Crouter (20th century)

    “It is no small mischief to a boy, that many of the best years of his life should be devoted to the learning of what can never be of any real use to any human being. His mind is necessarily rendered frivolous and superficial by the long habit of attaching importance to words instead of things; to sound instead of sense.”
    —William Cobbett (1762–1835)

    “Don’t you know there are 200 temperance women in this county who control 200 votes. Why does a woman work for temperance? Because she’s tired of liftin’ that besotted mate of hers off the floor every Saturday night and puttin’ him on the sofa so he won’t catch cold. Tonight we’re for temperance. Help yourself to them cloves and chew them, chew them hard. We’re goin’ to that festival tonight smelling like a hot mince pie.”
    —Laurence Stallings (1894–1968)