Probability Foundations

Basic Rules

• 0 ≤ P(A) ≤ 1
• P(A or B) = P(A) + P(B) - P(A and B)
• P(not A) = 1 - P(A)

Conditional Probability

• P(A|B) = P(A and B) / P(B)
• Restricts sample space to B
• Updates beliefs with evidence

Bayes' Theorem

• P(A|B) = P(B|A) * P(A) / P(B)
• Posterior = Likelihood × Prior / Evidence
• Foundation of probabilistic ML

Independence

• P(A and B) = P(A) * P(B)
• Knowing A tells nothing about B
• Naive Bayes assumes feature independence

Distributions

• Bernoulli: binary outcome
• Binomial: count of successes
• Gaussian: N(μ, σ²), the bell curve

Expected Value & Variance

• E[X] = weighted average
• Var[X] = E[(X - μ)²]
• Std = √Variance

MLE

• Find θ that maximizes P(data|θ)
• Usually maximize log-likelihood
• θ_MLE = argmax log P(data|θ)