Reproducing Kernel Hilbert Space - Moore-Aronszajn Theorem

Moore-Aronszajn Theorem

In the previous section, we defined a kernel function in terms of a reproducing kernel Hilbert space. It follows from the definition of an inner product that the kernel we defined is (conjugate) symmetric and positive definite. The Moore-Aronszajn theorem goes in the other direction; it says that every symmetric, positive definite kernel defines a unique reproducing kernel Hilbert space. The theorem first appeared in Aronszajn's Theory of Reproducing Kernels, although he attributes it to E. H. Moore.

Theorem. Suppose K is a symmetric, positive definite kernel on a set E. Then there is a unique Hilbert space of functions on E for which K is a reproducing kernel.

Proof. Define, for all x in E, . Let H0 be the linear span of . Define an inner product on H0 by

\left \langle \sum_{j=1}^n b_j K_{y_j}, \sum_{i=1}^m a_i K_{x_i} \right \rangle = \sum_{i=1}^m \sum_{j=1}^n \overline{a_i} b_j K(y_j, x_i).

The symmetry of this inner product follows from the symmetry of K and the non-degeneracy follows from the fact that K is positive definite.

Let H be the completion of H0 with respect to this inner product. Then H consists of functions of the form

f(x) = \sum_{i=1}^\infty a_i K_{x_i} (x)

where . The fact that the above sum converges for every x follows from the Cauchy-Schwarz inequality.

Now we can check the RKHS property, (*):

\langle f, K_x \rangle = \left \langle \sum_{i=1}^\infty a_i K_{x_i}, K_x \right \rangle = \sum_{i=1}^\infty a_i K (x_i, x) = f(x).

To prove uniqueness, let G be another Hilbert space of functions for which K is a reproducing kernel. For any x and y in E, (*) implies that

By linearity, on the span of . Then G = H by the uniqueness of the completion.

Read more about this topic:  Reproducing Kernel Hilbert Space

Famous quotes containing the word theorem:

    “To insure the adoration of a theorem for any length of time, faith is not enough, a police force is needed as well.”
    —Albert Camus (1913–1960)