Variable Elimination Walkthrough
Book Reference: Artificial Intelligence: A Modern Approach, 14.5
Slide Reference: Day 17, Slides 15, 19-26 ; Day 14, Slides 14-16

For this problem, I will use the same Alarm Bayes' Net which we have used in class. This Bayes' net is seen below. The question I will solve is finding p(e|a) using variable elimination, since we didn't answer this for any of the homeworks.

The first step is determining what factors you need in order to get the result of your query. To figure this out, there are several ways to decompose p(e|a).

• Normalization Trick
  Day 14, Slide 14 shows a normalizing trick which could prove useful. If you select joint probabilities out of a probability table and normalize the result (make it sum to 1), you will get the conditional distribution. For this problem, if we can find p(E,a), we can normalize it to get p(E|a).

• The Product Rule
  The product rule on Day 14, Slide 15 shows a second approach for simplifying the problem -- p(x|y) = p(x,y) / p(y). For this problem, p(e|a) could be simplified to p(e,a) / p(a).

• Bayes' Rule
  Finally, there is Bayes' Rule shown on Day 14, Slide 16 -- p(x|y) = p(y|x)p(x) / p(y). For this problem, p(e|a) could be converted to p(a|e)p(e) / p(a).

There are two basic operations that are repeatedly executed on the variable elimination factors. Factors are the objects that you use for each step in the variable elimination operations. The initial factors are the conditional probability tables of the given Bayes' net.

• Join
  It is like a join in a database. For each entry to be joined, the value is the point-wise product of the two factors. For example, p(a|b) x p(b|c) = p(a,b|c). The variables on the left and right of the conditional "|" operator are combined together. The value b on the right side of the conditional is dropped, since it appears on the left side now.

• Eliminate
  This operation is marginalization. It takes a factor and sums out a variable, to make the factor smaller. The equation is as follows, and an example can be seen below.
  

The initial factors just come from the conditional probability tables in the Bayes' net.

It is now a good time to analyze the problem, to determine the factors needed to solve the problem. This isn't required, you can just join and eliminate all of the hidden variables, but having a plan might prove to reduce the work required. (Hidden variables are those that are not in the original problem statement, but are in your factors. In this case, B, J, and M are hidden variables.) Using any of the three formulas above, we would need the following factors in order to solve for p(e|a).

• Using Normalization Trick: p(E,a) --> p(E|a) --> p(e|a)
• Using the Product Rule: p(e,a) and p(a)
• Using Baye's Rule: p(a|e), p(e), and p(a).

It seems like the easiest way to solve the problem is using the normalization trick. One can get P(E,a) with p(B) x p(A|B,E) = p(A,B|E), summing out B to get p(A|E), and then joining that factor with p(E) to get p(A,E). From this result, one can easily get to p(e|a) using normalization.

Starting off, we can substitute a in for A in all of our factors, since we don't need the !a case. We will keep E instead of substituting it, since we need it to use the normalization trick. The new factors are as follows:

Following our plan, pick B and join the factors. p(B) x p(a|B,E) = p(a,B|E).

Next, eliminate B by summing over B being true and B being false.

Our factors are now:

Next, pick E and join the factors. p(E) x p(a|E) = p(E,a).

Our factors are now:

Using the Normalization Trick discussed above, we normalize p(E,a) in order to get p(E|a).

Finally, we can pick out the value of p(e|a) from the above table: 0.231009

This equals the result obtained from summing over all the probabilities of the hidden variables, but we didn't actually have to calculate all of the 24 sums, as shown below.