Information Bottleneck Method - Defining Decision Contours

Defining Decision Contours

To categorize a new sample external to the training set, apply the previous distance metric to find the transition probabilities between and all samples in, with a normalisation. Secondly apply the last two lines of the 3-line algorithm to get cluster, and conditional category probabilities.

\begin{align}
& \tilde p(c_i ) = p(c_i | x' ) = \sum_j p(c_i | x_j)p(x_j | x') =\sum_j p(c_i | x_j) \tilde p(x_j)\\
& p(y_i | c_j) = \sum_k p(y_i | x_k) p(c_j | x_k)p(x_k | x') / p(c_j | x' )
= \sum_k p(y_i | x_k) p(c_j | x_k) \tilde p(x_k) / \tilde p(c_j) \\
\end{align}

Finally we have

Parameter must be kept under close supervision since, as it is increased from zero, increasing numbers of features, in the category probability space, snap into focus at certain critical thresholds.

Read more about this topic:  Information Bottleneck Method

Famous quotes containing the words defining, decision and/or contours:

    The industrial world would be a more peaceful place if workers were called in as collaborators in the process of establishing standards and defining shop practices, matters which surely affect their interests and well-being fully as much as they affect those of employers and consumers.
    Mary Barnett Gilson (1877–?)

    The women of my mother’s generation had, in the main, only one decision to make about their lives: who they would marry. From that, so much else followed: where they would live, in what sort of conditions, whether they would be happy or sad or, so often, a bit of both. There were roles and there were rules.
    Anna Quindlen (20th century)

    Pregnant women! They had that weird frisson, an aura of magic that combined awkwardly with an earthy sense of duty. Mundane, because they were nothing unique on the suburban streets; ethereal because their attention was ever somewhere else. Whatever you said was trivial. And they had that preciousness which they imposed wherever they went, compelling attention, constantly reminding you that they carried the future inside, its contours already drawn, but veiled, private, an inner secret.
    Ruth Morgan (1920–1978)