1. When retry is acceptable: idempotency and request intent
Retries pop into a developer’s head exactly when the network behaves like a network: sometimes it “sneezes,” sometimes it “lags,” sometimes the server is overloaded for a second. And you want to do “well, let’s retry — maybe it’ll work.” It sounds reasonable, but retries have a dark side: they easily turn a small hiccup into an avalanche of requests, and a neat business operation into a duplicate — or a triple duplicate.
Simplified, a retry is an attempt to automatically “fix” a transient failure by making the request again. But “transient” is the key word. If the problem is not transient (for example, you are calling the wrong URL, forgot a token, or are sending invalid JSON), retries only stretch out the wait time and make diagnostics harder: the user waits longer, and you get a batch of identical errors.
Even more important: retry is a change in client behavior. It’s as if you’re saying: “I will decide when it’s OK to throw one more request at the server.” And the server might not love that.
Idempotency: the main rule
The word “idempotent” sounds like a spell from a wizard academy, but the meaning is simple. An idempotent operation is one you can perform multiple times in a row, and the resulting state does not get “multiplied.”
HTTP offers a traditional hint: methods GET, HEAD, PUT, DELETE are considered idempotent by the protocol’s semantics, while POST usually is not. But it’s not a law of nature — it’s a convention: a specific API may have its own quirks.
A small table “how a beginner should think”:
| Method | Usually idempotent? | Can you retry “by default”? | Why be careful |
|---|---|---|---|
|
yes | sometimes yes | you can hammer the server and cause overload yourself |
|
yes | sometimes yes | if you update a resource, retry is usually safe, but you still need to understand the contract |
|
yes | sometimes yes | a repeated DELETE often yields 404/204 — depends on the API |
|
no | almost never | risk of creating “two identical orders” / “two tasks” |
|
depends | very rarely | often a “partial change”; repeating it may be dangerous |
And here’s an important “psychological trap”: a beginner thinks retry means “improve reliability.” In reality, retry is always a trade-off of one risk for another: you reduce the chance of failure during a brief network glitch, but increase the risk of duplicates, overload, and hard debugging.
3. Retry limiters: budget, limit, delays
Retry budget: an overall timeout matters more than “smart backoff”
There is a whole zoo of backoff strategies: fixed delay, exponential, with jitter, with a max cap, with separate phase budgets (DNS/TLS/…).
But at our level right now, what matters more than “the smartest strategy” is a time boundary. A good retry always has a budget: “we are willing to spend no more than N seconds on this operation.” The easiest way to express that is context.WithTimeout, because then any code inside the request gets a single cancellation signal.
This pairs nicely with the basic net/http idea: you create a request with a context (NewRequestWithContext), and Client.Do honors that context for the lifetime of the request.
Mini example “one budget for everything, including retries”:
package main
import (
"context"
"time"
)
func withRetryBudget(parent context.Context) (context.Context, context.CancelFunc) {
return context.WithTimeout(parent, 1200*time.Millisecond)
}
The idea is simple: no matter how many times you try to repeat, once the budget is over — that’s it, “close up shop.”
Attempt limit: no infinite retries
If a retry has no attempt limit, you get an infinite loop that “sometimes” ends. That’s very romantic, but in production it usually ends with someone running out of memory, patience, or API gateway request quota.
We need a minimum: a fixed number of attempts. For a beginner, that is usually 2–3. Not 10 and not “until it works.”
Short skeleton:
package main
func attempts() int {
return 3
}
Yes, it looks funny, but that’s exactly how you should think: retries are a small number, not “let’s keep hammering until the server gives up.”
4. What to retry: transport vs non-2xx
It’s very important to distinguish two classes of problems.
Transport errors are when the request was not actually completed: connection not established, connection dropped, DNS, timeout, etc. It’s like “couldn’t get through on the phone.” In Go, this usually shows up as err != nil from client.Do(req).
Contract errors are when you received a response, but it says: “no,” “validation error,” “not found,” “internal server error.” That is not transport. That’s already a semantic branch: resp != nil, err == nil, but resp.StatusCode is not 2xx.
In previous topics we mentioned that some network errors in Go can have additional signals (for example, Timeout()/Temporary() on net.Error), and clients sometimes decide to “sleep and retry” if the error is temporary. That’s exactly about the transport level.
But with non-2xx, retries should only be enabled if you have a clear agreement with the API. For example, a hypothetical 503 sometimes means “retry later,” but that is a policy of a specific service. If you just start retrying all 500s, you risk “hammering” the server even harder.
The most expensive mistake: forgetting to close resp.Body
This section is like the reminder “buckle your seat belt,” but in the HTTP world.
If client.Do(req) returned resp and err == nil, then resp.Body must be closed always, even if StatusCode is bad. And it’s especially important to close Body if you decide to retry based on status, because otherwise you create connection leaks and mysterious “everything hung after 5 minutes.”
The correct mindset: each attempt is a separate resp, and its body must be closed if you are not returning that resp to the caller.
Mini template:
package main
import "net/http"
func closeBody(resp *http.Response) {
if resp != nil && resp.Body != nil {
resp.Body.Close()
}
}
5. Retry-loop diagram
When code starts “jumping” between errors, pauses, and context exits, it’s easy to get lost. So it helps to keep one simple algorithm in mind:
flowchart TD
A[Start] --> B[Check ctx.Err]
B -->|canceled/deadline| Z[Return ctx error]
B --> C[Build request]
C --> D[Do request]
D -->|err != nil| E[Retry? transport]
D -->|err == nil| F[Check StatusCode]
F -->|2xx| Y[Success: return resp]
F -->|non-2xx| G[Retry? contract]
E -->|no| X[Return error]
G -->|no| X2[Return status error]
E -->|yes| H[Sleep/backoff]
G -->|yes| H
H --> B
This is not “the only correct way,” but as a reference map for a beginner — it’s great.
6. Example: a retry helper for GET in a toy task client
Imagine we have a toy app: a CLI/client that reads tasks from the server via GET /api/v1/tasks. We want retries only for GET, and only on transport errors (no retry on statuses, so we don’t hide server problems).
Mini client: base URL and http.Client
First, let’s make a small client with a base URL and an http.Client.
package main
import "net/http"
type TaskClient struct {
BaseURL string
HTTP *http.Client
}
One request attempt
Now a function that performs a single request attempt. We keep it intentionally small: it creates a request with a context, does Do, and returns resp as-is (the outer code will decide what to do next, including closing).
package main
import (
"context"
"net/http"
)
func (c *TaskClient) doGet(ctx context.Context, url string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
return c.HTTP.Do(req)
}
Retry parameters: attempts and delay
Let’s add the retry loop itself. Notice the discipline: we always check ctx.Err() before a new attempt, always limit the number of attempts, and add a small delay.
package main
import (
"context"
"time"
)
func retryDelay(attempt int) time.Duration {
return 200 * time.Millisecond // fixed, no “fancy” logic
}
func retryAttempts() int {
return 3
}
Main loop
It’s a bit longer than 10 lines, but we’ll try to keep it as “straight-line” as possible (so it reads like a story rather than a puzzle).
package main
import (
"context"
"fmt"
"net/http"
"time"
)
func (c *TaskClient) GetWithRetry(ctx context.Context, url string) (*http.Response, error) {
var lastErr error
for attempt := 1; attempt <= retryAttempts(); attempt++ {
if err := ctx.Err(); err != nil {
return nil, err
}
resp, err := c.doGet(ctx, url)
if err == nil {
return resp, nil // caller must Close()
}
lastErr = err
time.Sleep(retryDelay(attempt))
}
return nil, fmt.Errorf("get failed after retries: %w", lastErr)
}
Here we wrapped the error with %w to preserve the original cause (for example, context.DeadlineExceeded or a network error) and give the caller a way to check it via errors.Is. The %w trick is the standard way to wrap errors in Go.
Important: we do not retry POST /api/v1/tasks (creating a task). Why? Because creation is almost always non-idempotent: you risk creating two identical tasks. Yes, sometimes this is handled with idempotency keys, but that’s a separate topic and a separate API contract, not something you do “by default in the client.”
7. How to decide whether an error is retryable
Retrying everything is a bad idea. But banning retries entirely is also often too much. So we need a simple filter. At our level we’ll do it conservatively: retry only transport errors, and only those that look “temporary.”
In Go there is a historically important idea: some network errors implement the net.Error interface with Timeout() and Temporary(), and the client can use a type assertion to determine whether the error is temporary.
Mini check:
package main
import "net"
func isTemporary(err error) bool {
nerr, ok := err.(net.Error)
return ok && (nerr.Timeout() || nerr.Temporary())
}
And this is where real engineering begins: not all errors will be net.Error, and in real life the word Temporary() does not always mean “it’s safe to hammer again.” So the practical rule for a beginner: if you’re not sure — don’t retry automatically. Better return the error to the user/upstream than cause a request storm.
8. Retry and timeout: who wins
When we have both time.Sleep between attempts and an overall context.WithTimeout, a situation is possible where we’re “sleeping” but the context has already expired. In our code above, we check ctx.Err() at the start of the loop — that’s good, but if time.Sleep is long, we’ll still “sleep through” some time after cancellation.
In practice, for neatness, instead of time.Sleep people often wait in a context-aware way (via time.NewTimer and select), but we don’t want to go deep into concurrency here. So for now we lock in a simple discipline: delays should be short, attempts should be few, and an overall timeout is mandatory.
If you want to do “a bit better” but still keep it simple, you can at least cap the delay by the remaining budget. That already looks more complex, so for a basic course the rules above are enough.
9. Common mistakes
Mistake #1: retrying POST “because the network is unreliable.”
In practice this is one of the most dangerous bugs: you can create duplicate entities (two tasks, two orders, two payments). If you cannot formally prove idempotency of the operation by the API contract, it’s better not to do automatic retry for POST at all.
Mistake #2: retries without an overall timeout.
This creates a situation where “each attempt waits 2 seconds,” with 5 attempts, and the user ends up waiting 10 seconds. It’s much clearer to have a single context.WithTimeout for the whole operation so the behavior is predictable.
Mistake #3: infinite attempts or too large a limit.
If your code tries to “squeeze through” the network at any cost, you risk hanging forever or DoS’ing yourself. For a beginner, a safe default is 2–3 attempts. Anything more should be justified by measurements and an understanding of load.
Mistake #4: not closing resp.Body before a new attempt.
If you decide to retry by status (or even just by error, but resp happens to be non-nil) and you don’t close the body, you’ll get resource leaks and weird hangs due to running out of connections. As soon as resp is obtained, Body.Close() should either be in a defer (when you return resp outward) or called manually before retry.
Mistake #5: retrying contract errors that retries won’t fix.
401/403/400 won’t get better from retries. That’s not a “temporary network issue,” it’s a request/auth error. Retrying only increases latency and pollutes logs.
GO TO FULL VERSION