Errors in parallel processing — “first” vs “all”

1. Why errors in parallel processing are a separate problem

When you are just starting to write concurrent code, it is very tempting to think like this: “well, I got an err in the worker function, so now I’ll return err and that’s it.” And this is where Go gently but firmly reminds you: return returns a value only to the calling code, and for a goroutine the calling code has already “moved on.” So the error must be delivered outward via a shared agreement: a results channel, an error channel, a shared result struct, and so on.

In real programs this problem shows up quickly. For example, you import a batch of tasks (todo items): some lines are valid, some are not. Or you validate several inputs in parallel. Or you process a batch of requests to a service. Everywhere the same question arises: what should you do if one task fails? Stop everything immediately? Continue and collect a full report? And if you do collect a report, how do you return it without hitting a race and without deadlocking?

Let’s define two strategies that we will compare:

Important: “first error” does not always mean “we instantly stopped the whole world.” Early stopping is a separate protocol (usually via context.CancelFunc and ctx.Done()), and we will cover it separately. Here we focus specifically on the result contract: return one error or all errors.

2. Delivering errors from workers: a shared Result

To avoid drowning in variations later, let’s start with the most practical and readable approach: a worker always sends outward a result struct that includes both success and error. This keeps all “task information” in one place, and leaves the decision “what to do with errors” to the collector.

Imagine that in our training app we have a task entity (todo). We want to “normalize” tasks: trim spaces, validate that the title is not empty, and return the normalized string.

package main

type Task struct {
	ID    int
	Title string
}

type Result struct {
	TaskID int
	Title  string
	Err    error
}

Now we need a function that does the work “for a single task.” The key point: the work is synchronous, and it returns errors in the usual way. We will build concurrency around it.

package main

import (
	"errors"
	"strings"
)

var ErrEmptyTitle = errors.New("empty title")

func normalizeTitle(s string) (string, error) {
	t := strings.TrimSpace(s)
	if t == "" {
		return "", ErrEmptyTitle
	}
	return t, nil
}

And a worker that reads a Task, processes it, and sends a Result. Note: the worker does not decide whether it is “first error” or “all errors.” It simply reports outward, honestly, how the work ended.

package main

import "sync"

func worker(jobs <-chan Task, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()

	for t := range jobs {
		title, err := normalizeTitle(t.Title)
		results <- Result{TaskID: t.ID, Title: title, Err: err}
	}
}

This Result contract is one of the most convenient ways to “bring an error out of a goroutine”: it is simple, extensible (you can add time, metrics, any fields), and easy to aggregate.

3. The “all errors” strategy: report and errors.Join

When you are doing an import or validation, it is usually frustrating for the user to learn only the first problem. They fix it, run again, and see the second. Then the third. Then they start suspecting the program is trolling them. In such scenarios, it is better to collect a complete list.

In Go, to “combine many errors into one,” there is errors.Join. It creates one error that contains multiple causes inside — convenient for an “overall result,” while preserving the meaning of “errors as values” (we do not print errors immediately wherever we feel like; we return them as the result of computation).

A collector in the all-errors strategy typically looks like this: it reads results, stores successes separately, errors separately, and at the end does errors.Join(errs...).

package main

import "errors"

func collectAll(results <-chan Result) ([]Task, error) {
	var okTasks []Task
	var errs []error

	for r := range results {
		if r.Err != nil {
			errs = append(errs, r.Err)
			continue
		}
		okTasks = append(okTasks, Task{ID: r.TaskID, Title: r.Title})
	}

	return okTasks, errors.Join(errs...)
}

Here is an important point many people miss at first: errs = append(errs, …) is safe because collectAll runs in a single goroutine. If you were to do append from multiple workers into a shared slice, that would be a data race.

One more nuance: it is better to return errors not “as-is,” but with context: which task failed. Otherwise your logs will say “empty title,” and you will be guessing where it came from.

Add wrapping:

package main

import "fmt"

func withTaskContext(id int, err error) error {
	if err == nil {
		return nil
	}
	return fmt.Errorf("task %d: %w", id, err)
}

And apply it in the worker:

package main

import "sync"

func worker(jobs <-chan Task, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()

	for t := range jobs {
		title, err := normalizeTitle(t.Title)
		err = withTaskContext(t.ID, err)
		results <- Result{TaskID: t.ID, Title: title, Err: err}
	}
}

Wrapping via %w preserves the error “cause” inside the chain. This matters because later you can call errors.Is(err, ErrEmptyTitle) and understand that it is specifically an empty title, not something else. This approach (wrapping while preserving the original error) is the standard Go mechanism since the introduction of wrapping support and errors.Is/errors.As.

4. The “first error” strategy: record it and keep draining results

Sometimes all-errors is overkill. For example, you have parallel processing where any error means the final result is meaningless: “failed to decrypt a critical file,” “no access,” “input data format is broken.” In such cases, the first error is enough, because you will have to abort the scenario anyway.

But there is a nasty concurrency trap here: if you “see an error” and immediately exit the collector, stopping reading from results, workers may get stuck on results <- … This is the most common reason for “everything hung, but CPU is quiet.”

Therefore, the basic rule for first-error without early cancellation is: we can record the first error, but we must continue reading the results channel until it is closed so that nobody is left blocked. (Early cancellation is a separate topic; that is where context and select with ctx.Done() come into play.)

An example collector: “first error, but read to the end”:

package main

func collectFirst(results <-chan Result) ([]Task, error) {
	var okTasks []Task
	var firstErr error

	for r := range results {
		if r.Err != nil && firstErr == nil {
			firstErr = r.Err
			continue
		}
		if r.Err == nil {
			okTasks = append(okTasks, Task{ID: r.TaskID, Title: r.Title})
		}
	}

	return okTasks, firstErr
}

This option gives you the “first error” as the outcome, but does not break the channel protocol. Yes, it does not stop workers instantly. But it is correct.

If you need “as soon as there is an error — stop everything,” that is a cancel-on-error protocol. That is where context.WithCancel, checking ctx.Done() in the producer and in the worker, and also during result sending appear. Done() as a cancellation signal channel is a fundamental element of context, but the details of that protocol are a separate lecture.

5. Useful nuances and patterns

A separate error channel: buffer 1 and non-blocking send

Sometimes you do not want to carry Err inside Result (for example, results are large, and you want to separate the streams). Then you make a separate errCh. And here it is also easy to hang the system: a worker attempts to send an error, while the receiver is busy with something else — and the worker blocks.

A classic Go technique: an error channel with a buffer of 1 + a non-blocking send via select { case errCh <- err: default: }. This lets you “record the first error” and ignore subsequent ones without blocking workers. The exact same principle is shown in concurrency patterns when you need to send the “first result” and avoid leaving goroutines hanging.

Mini-helper:

package main

func reportFirst(errCh chan<- error, err error) {
	if err == nil {
		return
	}
	select {
	case errCh <- err:
	default:
		// the first error is already recorded; do not block
	}
}

Channel creation:

package main

func makeErrCh() chan error {
	return make(chan error, 1)
}

The key meaning of buffer 1: the first error is guaranteed to “fit” even if the reader has not read yet, and the worker will not block. As in other concurrency patterns, the buffer is often needed precisely so that the sending goroutine does not hang “forever” if the receiver is no longer interested or not ready yet.

But be careful: if you make a separate errCh, you still need to think through who reads it and when, and what happens to results. Very often, people still end up with “one results stream with an embedded error,” because it is simpler.

Why early exit is dangerous: a mini diagram

Before moving on to typical mistakes, it helps to see once why “I’ll just return the error and exit” is dangerous in a concurrent architecture.

flowchart LR
    P[producer] -->|jobs <- Task| J[(jobs)]
    J --> W1[worker 1]
    J --> W2[worker 2]
    W1 -->|results <- Result| R[(results)]
    W2 -->|results <- Result| R
    R --> C[collector]

    C -. "exited too early" .-> X((workers block
on send))

If the collector stopped reading results, workers can get stuck on the send operation. Practical takeaway: first-error without cancel is almost always “remember the error, but keep draining the channel until it is closed.” And “stop everything” requires a shared cancellation signal (context) and select in potentially blocking places.

6. Typical mistakes

In parallel processing, errors often break not “code elegance,” but the program’s viability: it either deadlocks, loses errors, or turns diagnosis into a quest. Below are the most common pitfalls specifically in the “first vs all” topic.

Mistake #1: calling append(errs, err) from multiple workers into a shared slice.
It seems logical: “I have []error, I add to it.” But if multiple goroutines do this without synchronization, it is a data race and potential memory corruption. A reliable approach is to collect errors in one place (the collector), reading from the results channel, so that writes to []error happen strictly in a single goroutine.

Mistake #2: early return from the collector on first-error without a stop protocol.
The nastiest bug: you found the first error and “returned it upward,” but you stopped reading results. Workers continue working and block on results <- … From the outside it looks like “the program hung,” but in reality it is honestly waiting for someone to read from the channel. If you do not have a cancel-on-error protocol, it is better to make the collector remember firstErr but keep draining results until it is closed.

Mistake #3: errCh without a buffer and without select default.
If errCh is unbuffered, a worker sending an error must wait for a reader “right now.” If the reader has not started yet or has already left, the worker blocks. For the “first error” strategy, you almost always need errCh := make(chan error, 1) and a non-blocking send, so the second error does not block the system. This technique is a classic way to avoid leaving goroutines hanging on send.

Mistake #4: returning errors without task context.
An error like empty title without specifying the task ID in parallel processing is almost useless: tasks are not processed in order, and you cannot infer which one failed “by time.” Wrapping with fmt.Errorf("task %d: %w", id, err) fixes this, while preserving the original cause for errors.Is.

Mistake #5: mixing the “error strategy” and “printing errors” right in the worker.
It is very tempting to do fmt.Println("error:", err) right where the error occurs. But then in multi-threaded output everything gets mixed up, some errors are duplicated, and some are lost on early stop. It is much more robust when the worker forms the error as a value and sends it outward, and the decision “how many errors to print and how” is made by a single layer (collector or higher-level code), following the idea that “errors are values.”