Consumer Rebalance Plan

medium · distributed-systems, streaming, partitioning

Consumer Rebalance Plan

A consumer group uses round-robin partition assignment. When the set of consumers changes, some partitions move.

Given the current assignment, the new consumer list, and the partition list, compute the moves required to reach the new round-robin assignment.

Round-robin rules:

  • Consumers are sorted lexicographically.
  • Partitions are sorted ascending.
  • Assign partitions in order, cycling through consumers.

Return a list of moves (partition, from, to) for any partition whose owner changes. The result should be sorted by partition ascending.

Types

type Assignment struct {
    Consumer   string
    Partitions []int
}

type Move struct {
    Partition int
    From      string
    To        string
}

Function signature

func RebalancePlan(consumers []string, partitions []int, current []Assignment) []Move

Example

consumers = ["c1","c2"]
partitions = [0,1,2,3]
current = {c1:[0,2], c2:[1,3]}

new consumers = ["c1","c2","c3"]
new assignment = c1:[0,3], c2:[1], c3:[2]

moves = [(2, c1->c3), (3, c2->c1)]

Constraints

  • 0 <= len(consumers), len(partitions) <= 100000
  • current may omit consumers with no partitions

Notes

  • If consumers is empty, all partitions become unassigned (use empty To).
Run tests to see results
No issues detected