Solution of A Linear System
Gradient descent can be used to solve a system of linear equations, reformulated as a quadratic minimization problem, e.g., using linear least squares. Solution of
in the sense of linear least squares is defined as minimizing the function
In traditional linear least squares for real and the Euclidean norm is used, in which case
In this case, the line search minimization, finding the locally optimal step size on every iteration, can be performed analytically, and explicit formulas for the locally optimal are known.
For solving linear equations, gradient descent is rarely used, with the conjugate gradient method being one of the most popular alternatives. The speed of convergence of gradient descent depends on the maximal and minimal eigenvalues of, while the speed of convergence of conjugate gradients has a more complex dependence on the eigenvalues, and can benefit from preconditioning. Gradient descent also benefits from preconditioning, but this is not done as commonly.
Read more about this topic: Gradient Descent
Famous quotes containing the words solution of, solution and/or system:
“I herewith commission you to carry out all preparations with regard to ... a total solution of the Jewish question in those territories of Europe which are under German influence.... I furthermore charge you to submit to me as soon as possible a draft showing the ... measures already taken for the execution of the intended final solution of the Jewish question.”
—Hermann Goering (1893–1946)
“The truth of the thoughts that are here set forth seems to me unassailable and definitive. I therefore believe myself to have found, on all essential points, the final solution of the problems. And if I am not mistaken in this belief, then the second thing in which the value of this work consists is that it shows how little is achieved when these problems are solved.”
—Ludwig Wittgenstein (1889–1951)
“There are obvious places in which government can narrow the chasm between haves and have-nots. One is the public schools, which have been seen as the great leveler, the authentic melting pot. That, today, is nonsense. In his scathing study of the nation’s public school system entitled “Savage Inequalities,” Jonathan Kozol made manifest the truth: that we have a system that discriminates against the poor in everything from class size to curriculum.”
—Anna Quindlen (b. 1952)