Content
Summary
Influence diagrams extend Bayesian networks by adding decision nodes (rectangles) and utility nodes (diamonds) to model multi-stage decision problems under uncertainty. The optimal strategy is determined by maximizing expected utility, which involves computing posterior probabilities after observing evidence and selecting actions that yield the highest expected value.
Intuition
Think of an influence diagram as a roadmap for making tough choices when you don't have all the facts. It starts with a standard Bayesian network to handle uncertainty, but then adds two crucial elements: rectangles for decisions you can make and diamonds for the value or cost of those decisions. The core idea is simple but powerful: you want to pick the action that gives you the best average outcome. If you can gather more information, like a test result, you update your beliefs using Bayes' rule. This updated belief changes the expected utility of each action, potentially leading to a different, better decision. The value of this new information is simply how much it improves your maximum expected utility compared to making the decision without the test.
Key equations
Prior expected utility of action d=1, summing over states o weighted by prior probability p(o) and utility U(d,o)
Maximum Expected Utility, the highest expected utility among all possible actions
Expected utility of action d given observation s=0, using posterior probability p(o|s=0)
Overall Maximum Expected Utility when a test s is performed, summing over test outcomes s weighted by marginal probability p(s) and the expected utility of the optimal action d*(s) given s
Marginal probability of test outcome s, computed by summing over states o
Derivation
- Start with a Bayesian network representing uncertainty via chance nodes and their conditional probability distributions.
- Add decision nodes (rectangles) to represent available actions and utility nodes (diamonds) to quantify the value or cost of outcomes.
- Calculate the expected utility (EU) for each action by summing the product of the probability of each state of nature and the utility of that action in that state.
- If evidence (like a test result) is observed, update the probability distribution of states using Bayes' rule to get posterior probabilities.
- Recalculate the expected utility using these posterior probabilities to determine the optimal action given the new information.
- Determine the Maximum Expected Utility (MEU) by selecting the action with the highest expected utility.
- Compute the value of information as the difference between the MEU with the information and the MEU without it.
Worked example
An oil wildcatter must decide whether to drill (d=1) or not drill (d=0). The prior probability of oil (o=1) is 0.3, and dry (o=0) is 0.7. The utility of drilling with oil is 100, drilling with dry is -50, and not drilling is 0. A test (s) can be performed. If the test is positive (s=1), the posterior probability of oil is 0.8. If negative (s=0), it is 0.1. Calculate the MEU without testing and the MEU with testing.
- Calculate prior EU for drilling: EU(d=1) = p(o=1)U(d=1,o=1) + p(o=0)U(d=1,o=0) = 0.3*100 + 0.7*(-50) = 30 - 35 = -5.
- Calculate prior EU for not drilling: EU(d=0) = 0.
- Determine MEU without testing: MEU = max(-5, 0) = 0. Optimal action is not to drill.
- Calculate posterior EU for drilling if test is positive (s=1): EU(d=1|s=1) = 0.8*100 + 0.2*(-50) = 80 - 10 = 70.
- Calculate posterior EU for not drilling if test is positive: EU(d=0|s=1) = 0.
- Optimal action if s=1 is drill (EU=70).
- Calculate posterior EU for drilling if test is negative (s=0): EU(d=1|s=0) = 0.1*100 + 0.9*(-50) = 10 - 45 = -35.
- Calculate posterior EU for not drilling if test is negative: EU(d=0|s=0) = 0.
- Optimal action if s=0 is not drill (EU=0).
- Assume marginal probabilities p(s=1)=0.4 and p(s=0)=0.6. Calculate MEU with testing: MEU = p(s=1)*EU(d*(s=1)|s=1) + p(s=0)*EU(d*(s=0)|s=0) = 0.4*70 + 0.6*0 = 28.
- Value of information = MEU(with test) - MEU(without test) = 28 - 0 = 28.
Answer MEU without testing is 0. MEU with testing is 28. The value of information is 28.
Common mistakes
- Confusing the prior expected utility with the expected utility after observing evidence.
- Failing to update the probability distribution of states of nature (oil) after observing the test result (sound).
- Ignoring the cost of the test when calculating the net benefit of information.
- Assuming the optimal action is always the same regardless of the test outcome.