Go Concurrency Patterns (Deep Dive)
Work through examples, try the reasoning yourself, then put it into practice.
View SlidesBefore you start
- Trace channel send/receive/close and synchronization from the memory-model module.
What you will practice
- Design bounded worker ownership and join every worker before returning.
- Distinguish indexed result order from execution order and cooperative cancellation.
In this lesson
Lesson
5 min readBounded Work Needs a Complete Lifetime
BoundMap(inputs, limit, fn) must produce one result per input position while at most limit callbacks are running. Completion order can vary. The caller receives the output only after every callback has returned. This exercise's signature has no context or error channel; cancellation is a separate design extension below.
Start with ownership and states
The calling goroutine owns the output slice, job production, channel closure, and final join. Each job contains an index and value. A worker owns exactly the output slot for the job it received. Jobs move through not submitted → handed to a worker → executing → result stored. A worker moves through waiting → executing → waiting, then exits when the jobs channel closes.
Choose workers = min(limit, len(inputs)); reject empty input, a nonpositive limit, or nil callback by returning nil. A worker pool avoids creating one goroutine for every item. An unbuffered job channel makes the producer wait for a worker; a bounded buffer allows a specified amount of queued work. Neither choice makes an overloaded service's latency automatically stable. Admission limits bound work, while deadlines or rejection policies decide what happens when arrivals exceed capacity.
type job struct { index, value int }
// After validating arguments and allocating out:
jobs := make(chan job)
var wg sync.WaitGroup
wg.Add(workers)
for w := 0; w < workers; w++ {
go func() {
defer wg.Done()
for j := range jobs {
out[j.index] = fn(j.value)
}
}()
}
for i, v := range inputs { jobs <- job{i, v} }
close(jobs) // the only sender has finished
wg.Wait() // all writes finish before out is returned
There is no need for a results channel in this design. Concurrent writes to distinct slice elements are safe when the slice header is fixed, indices are unique, and readers wait for the join. Concurrent append would instead race on the shared slice header and may reallocate its storage. WaitGroup.Done synchronizes with the Wait it unblocks, making completed writes visible to the caller. Register workers before starting them and before waiting. sync.WaitGroup documentation
Trace completion separately from order
Use inputs [6,2,9,4], two workers, and fn(x)=x*x. Worker A receives index 0 and pauses. Worker B receives index 1, stores 4 in out[1], then handles index 2 and stores 81. A later stores 36 in out[0]. Either worker handles index 3. The final result is [36,4,81,16] regardless of which worker finishes first.
At most two callbacks execute because each worker executes one callback at a time. Closing jobs means no more sends; it does not mean workers have exited. Waiting before closing deadlocks: idle workers are waiting for jobs, and the caller is waiting for those workers. A worker closing the shared channel could instead panic a still-active producer. The owner closes after the last send, then joins.
A semaphore can also bound callback concurrency: acquire a token before starting a goroutine and release after the callback. Acquiring inside each new goroutine bounds callbacks but leaves potentially one blocked goroutine per item. Both designs need a join. The worker pool gives a direct bound of min(limit,n) worker goroutines.
Cancellation is cooperative, not a goroutine kill switch
For a different API such as MapContext(ctx, inputs, limit, fn(ctx, value)), decide whether cancellation returns partial output or an error with no output. Suppose the contract discards partial output and joins every owned worker before returning. Then cancellation must reach the producer, waiting workers, and callbacks. The callback must return when its context is canceled; the runtime cannot forcibly stop an arbitrary callback.
A producer's blocking send needs an escape:
select {
case jobs <- j:
case <-ctx.Done():
// stop producing, close jobs once, and join workers
}
Workers select between receiving a job and ctx.Done(), check the receive's ok flag, and invoke the context-aware callback. If results are sent on a separate channel, that send also needs cancellation. A ready cancellation case has no priority over a ready job: select chooses among ready communications. A check immediately before invoking fn narrows the race, but cannot promise that no work starts at the exact instant of cancellation. Go select specification
Trace a cancellation: two workers are executing jobs 0 and 1; the producer is blocked sending job 2. The context is canceled. The producer stops its send, closes jobs, and waits. Both callbacks observe cancellation and return. Workers exit; the join completes; the parent returns the cancellation error and discards partial results. If callback 0 blocks forever without observing context, the join also blocks forever. Returning early would leave that owned work running. Go pipelines and cancellation
The supplied BoundMap cannot provide this cancellation contract: fn(int) int has no context, and the helper returns no error. Assume callbacks terminate normally and synchronize their own shared mutable state. Panics and permanently blocked callbacks are outside the exercise contract.
Test mechanisms, not elapsed-time guesses
Force out-of-order completion using channels: hold index 0, allow index 1 to signal completion, then release index 0. Verify indexed output and no mutation of inputs. Count active callbacks atomically to test the limit, and require zero active callbacks on return. Use unique values and per-value counters to catch dropped or repeated jobs; verify invalid inputs never call the callback. Run the race detector too. A timeout in a test is a deadlock guard, not evidence that a specific schedule occurred.
Reconstruct the design
- Why not append results as workers finish? That changes order and requires synchronization of the shared slice. Fixed indexed writes solve both issues.
- The producer has closed jobs, but one callback is still running. Can the map return? No: closure announces the end of input, whereas joining establishes completion.
- Does a context alone stop a callback blocked on an unrelated channel? No. Its blocking operation needs a cancellation path or another guaranteed way to unblock.
Practice sequence
Use the order below for a guided path, or choose an item to revisit. A pass records a successful attempt; try a fresh example later to check what you can reconstruct.