Solution of A Linear System
Gradient descent can be used to solve a system of linear equations, reformulated as a quadratic minimization problem, e.g., using linear least squares. Solution of
in the sense of linear least squares is defined as minimizing the function
In traditional linear least squares for real and the Euclidean norm is used, in which case
In this case, the line search minimization, finding the locally optimal step size on every iteration, can be performed analytically, and explicit formulas for the locally optimal are known.
For solving linear equations, gradient descent is rarely used, with the conjugate gradient method being one of the most popular alternatives. The speed of convergence of gradient descent depends on the maximal and minimal eigenvalues of, while the speed of convergence of conjugate gradients has a more complex dependence on the eigenvalues, and can benefit from preconditioning. Gradient descent also benefits from preconditioning, but this is not done as commonly.
Read more about this topic: Gradient Descent
Famous quotes containing the words solution of, solution and/or system:
“The Settlement ... is an experimental effort to aid in the solution of the social and industrial problems which are engendered by the modern conditions of life in a great city. It insists that these problems are not confined to any one portion of the city. It is an attempt to relieve, at the same time, the overaccumulation at one end of society and the destitution at the other ...”
—Jane Addams (1860–1935)
“I can’t quite define my aversion to asking questions of strangers. From snatches of family battles which I have heard drifting up from railway stations and street corners, I gather that there are a great many men who share my dislike for it, as well as an equal number of women who ... believe it to be the solution to most of this world’s problems.”
—Robert Benchley (1889–1945)
“I have no concern with any economic criticisms of the communist system; I cannot enquire into whether the abolition of private property is expedient or advantageous. But I am able to recognize that the psychological premises on which the system is based are an untenable illusion. In abolishing private property we deprive the human love of aggression of one of its instruments ... but we have in no way altered the differences in power and influence which are misused by aggressiveness.”
—Sigmund Freud (1856–1939)