Selecting A Loss Function
Sound statistical practice requires selecting an estimator consistent with the actual loss experienced in the context of a particular applied problem. Thus, in the applied use of loss functions, selecting which statistical method to use to model an applied problem depends on knowing the losses that will be experienced from being wrong under the problem's particular circumstances, which results in the introduction of an element of teleology into problems of scientific decision-making.
A common example involves estimating "location." Under typical statistical assumptions, the mean or average is the statistic for estimating location that minimizes the expected loss experienced under the Taguchi or squared-error loss function, while the median is the estimator that minimizes expected loss experienced under the absolute-difference loss function. Still different estimators would be optimal under other, less common circumstances.
In economics, when an agent is risk neutral, the loss function is simply expressed in monetary terms, such as profit, income, or end-of-period wealth.
But for risk averse (or risk-loving) agents, loss is measured as the negative of a utility function, which represents satisfaction and is usually interpreted in ordinal terms rather than in cardinal (absolute) terms.
Other measures of cost are possible, for example mortality or morbidity in the field of public health or safety engineering.
For most optimization algorithms, it is desirable to have a loss function that is globally continuous and differentiable.
Two very commonly-used loss functions are the squared loss, and the absolute loss, . However the absolute loss has the disadvantage that it is not differentiable at . The squared loss has the disadvantage that it has the tendency to be dominated by outliers---when summing over a set of 's (as in ), the final sum tends to be the result of a few particularly-large a-values, rather than an expression of the average a-value.
Read more about this topic: Loss Function
Famous quotes containing the words selecting a, selecting, loss and/or function:
“But compared with the task of selecting a piece of French pastry held by an impatient waiter a move in chess is like reaching for a salary check in its demand on the contemplative faculties.”
—Robert Benchley (1889–1945)
“A man is a method, a progressive arrangement; a selecting principle, gathering his like to him; wherever he goes.”
—Ralph Waldo Emerson (1803–1882)
“War is bestowed like electroshock on the depressive nation; thousands of volts jolting the system, an artificial galvanizing, one effect of which is loss of memory. War comes at the end of the twentieth century as absolute failure of imagination, scientific and political. That a war can be represented as helping a people to “feel good” about themselves, their country, is a measure of that failure.”
—Adrienne Rich (b. 1929)
“Think of the tools in a tool-box: there is a hammer, pliers, a saw, a screwdriver, a rule, a glue-pot, nails and screws.—The function of words are as diverse as the functions of these objects.”
—Ludwig Wittgenstein (1889–1951)