Likelihood Lower Bound
Given some set of hidden variables and observed variables, the goal of approximate inference is to lower-bound the probability that a graphical model is in the configuration . Over some probability distribution (to be defined later),
- .
So, if we define our lower bound to be
- ,
then the likelihood is simply this bound plus the relative entropy between and . Because the relative entropy is non-negative, the function defined above is indeed a lower bound of the log likelihood of our observation . The distribution will have a simpler character than that of because marginalizing over is intractable for all but the simplest of graphical models. In particular, VMP uses a factorized distribution :
where is a disjoint part of the graphical model.
Read more about this topic: Variational Message Passing
Famous quotes containing the words likelihood and/or bound:
“What likelihood is there of corrupting a man who has no ambition?”
—Samuel Richardson (1689–1761)
“I’the commonwealth I would by contraries
Execute all things; for no kind of traffic
Would I admit; no name of magistrate;
Letters should not be known; riches, poverty,
And use of service, none; contract, succession,
Bourn, bound of land, tilth, vineyard, none;
No use of metal, corn, or wine, or oil;
No occupation; all men idle, all,
And women too, but innocent and pure.”
—William Shakespeare (1564–1616)