Information Bottleneck Method - Defining Decision Contours

Defining Decision Contours

To categorize a new sample external to the training set, apply the previous distance metric to find the transition probabilities between and all samples in, with a normalisation. Secondly apply the last two lines of the 3-line algorithm to get cluster, and conditional category probabilities.

\begin{align}
& \tilde p(c_i ) = p(c_i | x' ) = \sum_j p(c_i | x_j)p(x_j | x') =\sum_j p(c_i | x_j) \tilde p(x_j)\\
& p(y_i | c_j) = \sum_k p(y_i | x_k) p(c_j | x_k)p(x_k | x') / p(c_j | x' )
= \sum_k p(y_i | x_k) p(c_j | x_k) \tilde p(x_k) / \tilde p(c_j) \\
\end{align}

Finally we have

Parameter must be kept under close supervision since, as it is increased from zero, increasing numbers of features, in the category probability space, snap into focus at certain critical thresholds.

Read more about this topic:  Information Bottleneck Method

Famous quotes containing the words defining, decision and/or contours:

    The industrial world would be a more peaceful place if workers were called in as collaborators in the process of establishing standards and defining shop practices, matters which surely affect their interests and well-being fully as much as they affect those of employers and consumers.
    Mary Barnett Gilson (1877–?)

    Concision in style, precision in thought, decision in life.
    Victor Hugo (1802–1885)

    Pregnant women! They had that weird frisson, an aura of magic that combined awkwardly with an earthy sense of duty. Mundane, because they were nothing unique on the suburban streets; ethereal because their attention was ever somewhere else. Whatever you said was trivial. And they had that preciousness which they imposed wherever they went, compelling attention, constantly reminding you that they carried the future inside, its contours already drawn, but veiled, private, an inner secret.
    Ruth Morgan (1920–1978)