Probability Foundations

Probability starts with a defined set of outcomes. The task here is to move between counts, joint probabilities, and conditional probabilities without changing the population silently. You will implement five small functions, then explain when their formulas apply.

Build the joint table before applying Bayes

A fictional message classifier flags suspicious messages. In a population of 1,000 messages, 100 are spam. It flags 80 of those 100 and also flags 180 of the 900 ordinary messages. These are stipulated counts for learning, not measured classifier performance.

Actual class	Flagged	Not flagged	Total
Spam	80	20	100
Ordinary	180	720	900
Total	260	740	1,000

Let A mean spam and B mean flagged. A joint event requires both properties: P(A and B)=80/1000=0.08. A marginal ignores the other property: P(B)=260/1000=0.26. Conditioning changes the denominator to the selected group: P(A|B)=80/260=4/13, whereas P(B|A)=80/100=0.8.

The two conditional probabilities answer different questions. A detector that flags most spam does not thereby make most flags spam. The larger ordinary-message population contributes many false flags.

Derive the denominator instead of guessing it

Partition all messages into A and not-A. Every flagged message belongs to exactly one side, so:

P(B) = P(B|A)P(A) + P(B|not A)P(not A)
     = 0.8 × 0.1 + 0.2 × 0.9
     = 0.08 + 0.18 = 0.26

P(A|B) = P(B|A)P(A) / P(B)
       = 0.08 / 0.26 = 4/13

This is the chain rule followed by total probability and Bayes' theorem. Each multiplication produces a joint probability; addition combines disjoint parts; division conditions on B. Dropping the ordinary-message term would incorrectly make every flag spam.

In probability-basics, compute total_probability(0.8, 0.1, 0.2) first, then pass that result to bayes_theorem(0.8, 0.1, evidence). bayesian_update(prior, likelihood, evidence) computes the same product-and-division with a different argument order. It does not calculate evidence for you.

All inputs represent consistent probabilities in [0,1]. The exercise returns 0.0 for a zero conditioning/evidence denominator. That is an explicit API convention: conditioning on a zero-probability event is undefined by this elementary ratio, not mathematically proven to be zero. No validation of inconsistent inputs is required.

Independence is an assumption to check

The function joint_probability(p_a, p_b) assumes independence and multiplies the two marginals. It is not a universal formula for joint events. Our table has P(A)P(B)=0.1×0.26=0.026, which differs from 0.08; the flag and spam class are dependent.

For two independent fair coin flips, HH, HT, TH, TT each have probability 1/4. Given at least one head, remove TT and renormalize the three remaining outcomes: the probability of HH is 1/3. Conditioning can create dependence even when the original trials were independent.

Disjoint and independent also differ. Distinct faces on one die cannot occur together, so their joint probability is zero. Their positive marginal probabilities have a positive product; they are not independent. For overlapping events, P(A or B)=P(A)+P(B)-P(A and B) subtracts the intersection counted twice.

Connect events to a random variable

A random variable maps outcomes to numbers. Let X be the number of heads in two independent fair flips. Its distribution is P(X=0)=1/4, P(X=1)=1/2, P(X=2)=1/4. Probabilities are nonnegative and sum to one.

E[X]   = 0×1/4 + 1×1/2 + 2×1/4 = 1
E[X²]  = 0×1/4 + 1×1/2 + 4×1/4 = 1.5
Var(X) = E[X²] - E[X]² = 0.5

A Bernoulli variable represents one yes/no trial. A binomial count requires a fixed number of independent trials with the same success probability. For continuous variables, a density at a point is not itself the probability of that exact point; probability comes from area over an interval. These ideas prepare the statistics and classification modules, but the current exercise implements event probabilities only.

Finish a changed population

Keep P(B|A)=0.8 and P(B|not A)=0.2, but let spam prevalence become 0.25. Before revealing the calculation, find both joint contributions, total flag probability, and the posterior.

Reveal the calculation

The contributions are 0.8×0.25=0.20 and 0.2×0.75=0.15. Evidence is 0.35 and the posterior is 0.20/0.35=4/7. The detector's conditional rates stayed fixed; the composition of flagged messages changed because the population changed.

Implement and transfer

Build the five functions from their argument meanings. Check a certain event, an impossible numerator, and the declared zero-denominator policy separately. Use the count table as an independent oracle, rather than checking one formula with another copy of itself.

Then construct a new table of 200 messages with 40 spam, 30 flagged spam, and 20 flagged ordinary messages. Derive every marginal and both conditional directions. Explain which calculation would fail if you silently assumed independence. Revisit this table in model evaluation: the posterior among flagged messages becomes precision, while the flagged fraction among spam becomes recall.