Likelihood Lower Bound
Given some set of hidden variables and observed variables, the goal of approximate inference is to lower-bound the probability that a graphical model is in the configuration . Over some probability distribution (to be defined later),
- .
So, if we define our lower bound to be
- ,
then the likelihood is simply this bound plus the relative entropy between and . Because the relative entropy is non-negative, the function defined above is indeed a lower bound of the log likelihood of our observation . The distribution will have a simpler character than that of because marginalizing over is intractable for all but the simplest of graphical models. In particular, VMP uses a factorized distribution :
where is a disjoint part of the graphical model.
Read more about this topic: Variational Message Passing
Famous quotes containing the words likelihood and/or bound:
“What likelihood is there of corrupting a man who has no ambition?”
—Samuel Richardson (1689–1761)
“I don’t know but a book in a man’s brain is better off than a book bound in calf—at any rate it is safer from criticism. And taking a book off the brain, is akin to the ticklish & dangerous business of taking an old painting off a panel—you have to scrape off the whole brain in order to get at it with due safety—& even then, the painting may not be worth the trouble.”
—Herman Melville (1819–1891)