Likelihood Lower Bound
Given some set of hidden variables and observed variables, the goal of approximate inference is to lower-bound the probability that a graphical model is in the configuration . Over some probability distribution (to be defined later),
- .
So, if we define our lower bound to be
- ,
then the likelihood is simply this bound plus the relative entropy between and . Because the relative entropy is non-negative, the function defined above is indeed a lower bound of the log likelihood of our observation . The distribution will have a simpler character than that of because marginalizing over is intractable for all but the simplest of graphical models. In particular, VMP uses a factorized distribution :
where is a disjoint part of the graphical model.
Read more about this topic: Variational Message Passing
Famous quotes containing the words likelihood and/or bound:
“Sustained unemployment not only denies parents the opportunity to meet the food, clothing, and shelter needs of their children but also denies them the sense of adequacy, belonging, and worth which being able to do so provides. This increases the likelihood of family problems and decreases the chances of many children to be adequately prepared for school.”
—James P. Comer (20th century)
“Affection, indulgence, and humor alike are powerless against the instinct of children to rebel. It is essential to their minds and their wills as exercise is to their bodies. If they have no reasons, they will invent them, like nations bound on war. It is hard to imagine families limp enough always to be at peace. Wherever there is character there will be conflict. The best that children and parents can hope for is that the wounds of their conflict may not be too deep or too lasting.”
—New York State Division of Youth Newsletter (20th century)