Inference in Bayesian Networks: From Naive Bayes to Gibbs Sampling
In our previous post, we looked at how to represent uncertainty. Now, we look at the engine of the system: Inference. Inference allows us to take observed data (evidence) and calculate the probability of hidden states.
Naive Bayes Inference Exact
Naive Bayes assumes that features are independent given the class. Despite its "naive" name, it is a workhorse in modern applications.
$$P(C \mid o_1, \dots, o_m) \propto P(C) \prod_{i=1}^m P(o_i \mid C)$$
Real-World Application: Spam Filtering
When you receive an email containing the words "Offer" and "Free," a Naive Bayes classifier looks at the probability of those words appearing in known Spam vs. Not Spam emails. It assumes the word "Offer" appearing is independent of "Free" being there, allowing for lightning-fast classification of millions of emails per second.
Sum-Product Variable Elimination Exact
Variable elimination is the process of simplifying a complex joint distribution by "summing out" hidden variables that we don't care about.
Real-World Application: Aircraft Radar Tracking
An automated air-traffic control system receives a fuzzy radar blip. To decide if the object is an Airplane or a Bird, it must account for hidden variables like Wind Speed and Altitude. Variable Elimination allows the system to remove these "nuisance" variables mathematically to focus solely on the identity of the aircraft.
Belief Propagation Exact (Trees)
This algorithm works by nodes passing "messages" to their neighbors. Each node updates its belief based on what its neighbors tell it.
Real-World Application: Power Grid Fault Diagnosis
In a smart power grid, sensors are spread across a wide geography. When a transformer fails, nearby sensors pass messages to their neighbors about the surge they detected. This "belief" propagates through the network until the central system can pinpoint the exact location of the fault without needing to check every single wire.
Direct Sampling Approximate
When a network is too large for exact math, we simulate it. We "roll the dice" based on the network's probabilities and count how often the query outcome happens.
Real-World Application: Chemical Leak Detection
In a large industrial plant, engineers simulate thousands of leak scenarios. By sampling these random scenarios, they can estimate the probability that a specific sensor alert indicates a dangerous gas leak vs. a simple sensor malfunction.
Likelihood Weighted Sampling Approximate
Direct sampling fails if we have evidence that is very rare. Likelihood weighting ensures we only generate samples that match our evidence, then "weights" them to stay mathematically honest.
Real-World Application: Medical Diagnosis of Rare Diseases
If a patient has a rare symptom that only occurs in 0.01% of the population, direct sampling would require millions of simulations to find even one match. Likelihood weighting "forces" the symptom to be present in every simulation, making the diagnostic algorithm drastically more efficient.
Gibbs Sampling Approximate (MCMC)
Gibbs sampling is used when variables are highly interconnected. It updates one variable at a time, keeping the others fixed.
Real-World Application: Recommender Systems (Netflix/Amazon)
Your user preferences (hidden) and movie ratings (observed) influence each other. Gibbs sampling is used to iteratively update the model's "guess" of your taste by holding movie categories constant while updating your profile, then holding your profile constant while updating the suggested movies.
The Decision Trap
Because exact inference is NP-hard, AI designers must always choose: do I want a 100% correct answer that takes forever, or a 98% correct answer right now? In algorithmic decision making, speed is often the most important factor.
Comments
Post a Comment