Entropy Estimator

Implement basic information-theory primitives used to reason about compression.

Functions

func Entropy(freqs []int) float64
func ExpectedLength(freqs []int, lengths []int) float64

Behavior

• freqs[i] is the count of symbol i in the sample.
• Entropy is computed in bits: H = -sum p_i * log2(p_i) for p_i > 0.
• If sum(freqs) == 0, return 0.
• ExpectedLength computes L = sum p_i * lengths[i].
• Ignore symbols with zero frequency.

Notes

• Use math.Log2 (or math.Log(x)/math.Log(2) if needed).
• Floating error tolerance in tests is 1e-9.