Auth middleware: API key. 401 vs 403 and fail fast

1. API key and correct responses to the client

If you have ever written several HTTP endpoints, you have probably caught yourself thinking: “Okay, in every handler I need to check that the user is allowed…”. Then the classic scenario happens: in one handler you forgot the check, in the second you did the check but returned a different error format, in the third you accidentally logged a secret—and now security is held together by good intentions and luck.

Middleware solves this problem elegantly: it runs before the business logic and decides “should we let the request go further or not”. This approach gives you a single place where authorization is done the same way for all routes, and it also simplifies handler code—handlers do not need to remember headers, statuses, and error messages.

From an architecture point of view, this is like a metro turnstile: the ticket is checked at the entrance, not in every train car. The car does not have to know who paid—the car’s job is to get you there.

What an API key is—and what it does not guarantee

An API key is a simple secret (a string) that the client must send along with the request. Typically it is passed in a header, for example X-API-Key. The server compares “what was sent” with “what we expect” and either lets the request through or responds with an error.

It is important to be honest about the level: an API key is not magic, not encryption, and not “now it’s like a bank”. It is a basic access filter that is useful in services:

• for an internal API,
• for a simple prototype,
• for admin endpoints,
• to protect against random “anyone at all” requests.

But even in such a simple mechanism, there are two fundamental things you can do right: correct statuses (401/403) and a single error format.

401 vs 403: how not to mix them up

Statuses 401 and 403 get mixed up all the time—it’s almost a tradition, like “confusing len and cap” in slices, only more painful.

The idea is:

• 401 Unauthorized — the client did not provide credentials for authorization. In our case: the X-API-Key header is missing (or empty). Meaning: “you did not identify yourself”.
• 403 Forbidden — the client provided credentials, but they are incorrect. In our case: the key is present, but it does not match. Meaning: “you identified yourself, but you are not the right one”.

To reinforce it, here is a small table:

A single error envelope: why auth errors should not respond with “just text”

In simple examples, people often do this:

http.Error(w, "forbidden", http.StatusForbidden)

This works… until you start building an API that is used by scripts, tests, frontend apps, or other services. Then you suddenly discover that some errors are JSON, some are plain text, and some have no body at all.

So we keep the line: any refusal (including auth) should be in a single JSON format.

We will use an envelope like:

{
  "error": {
    "code": "forbidden",
    "message": "invalid api key"
  }
}

The code (code) is needed so the client can distinguish error types programmatically, and the message (message) is for a human to understand what happened.

Code: a minimal writer for the error envelope

Now we will write a small utility that will be used by the auth middleware and other parts of the server.

package httpapi

import (
	"encoding/json"
	"net/http"
)

type apiError struct {
	Code    string            `json:"code"`
	Message string            `json:"message"`
	Fields  map[string]string `json:"fields,omitempty"`
}

type errorEnvelope struct {
	Error apiError `json:"error"`
}

func writeAPIError(w http.ResponseWriter, status int, code, msg string) {
	w.Header().Set("Content-Type", "application/json; charset=utf-8")
	w.WriteHeader(status)
	_ = json.NewEncoder(w).Encode(errorEnvelope{
		Error: apiError{Code: code, Message: msg},
	})
}

Note: we are not overcomplicating it. We ignore encoder errors (_ = …), because if writing to the ResponseWriter failed, the client most likely will not receive anything reasonable anyway. For a learning project, this is enough.

3. Fail fast and wiring middleware at startup

Critical rule: if the expected key is not set, that is a startup error

This is where many beginners make a logical substitution. It feels like: “If my config does not have the expected key, then I should respond with 401/403 to every request.” But that is not the client’s problem—it is our configuration problem.

If the server starts without the expected key, it does not have a correct access policy. It is like putting a guard at the entrance but not telling them what the passes look like: the guard will let people in or block them randomly. In the real world, this is called an “incident”.

So the rule is: no secret in config → the server does not start. That is fail fast: it is better to crash immediately and loudly than to quietly run in a half-working mode.

It is convenient to formalize it like this: the function that creates auth middleware returns (Middleware, error). If error != nil, we abort startup.

Auth middleware as a factory: inject the dependency at startup

Middleware usually needs to be configurable: the key must not be “hardcoded” into the code (otherwise it will be found even by those who were not looking). So we make a factory function: it accepts expectedKey (from env/config) and returns Middleware.

Let’s introduce the Middleware type (if you already have one—great, just use yours):

package httpapi

import "net/http"

type Middleware func(http.Handler) http.Handler

Now we create the auth middleware. And here is the important part: fail fast.

package httpapi

import (
	"fmt"
	"net/http"
)

func newAPIKeyAuth(expectedKey string) (Middleware, error) {
	if expectedKey == "" {
		return nil, fmt.Errorf("APP_API_KEY is not set")
	}

	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			got := r.Header.Get("X-API-Key")

			if got == "" {
				writeAPIError(w, http.StatusUnauthorized, "unauthorized", "missing api key")
				return
			}
			if got != expectedKey {
				writeAPIError(w, http.StatusForbidden, "forbidden", "invalid api key")
				return
			}

			next.ServeHTTP(w, r)
		})
	}, nil
}

Note three details that usually “slip past your eyes”:

• First: expectedKey == "" is not request handling. It is a configuration error, so we return an error instead of responding to the client.
• Second: after writeAPIError we do return, otherwise the request will “leak” further, and the handler may manage to do something (or even write a successful response—and you will get a strange hybrid).
• Third: middleware does not try to “fix” the situation. It simply applies the access policy.

Code: Chain and an auth-middleware constructor

Building a middleware chain should be readable. We want to see the order in one place. Let’s have a Chain function.

package httpapi

import "net/http"

func Chain(h http.Handler, mws ...Middleware) http.Handler {
	for i := len(mws) - 1; i >= 0; i-- {
		h = mws[i](h)
	}
	return h
}

Now we will make a neat public constructor for auth that takes the key as an argument. This is cleaner and more testable than hiding os.Getenv inside the package.

package httpapi

func NewAuth(expectedKey string) (Middleware, error) {
	return newAPIKeyAuth(expectedKey)
}

How to wire auth middleware into server startup

Let’s show how this is wired in main: read the key from an environment variable and, if the key is not set, print an error and exit.

package main

import (
	"log"
	"net/http"
	"os"

	"example.com/taskapp/httpapi"
)

func main() {
	mux := http.NewServeMux()

	authMW, err := httpapi.NewAuth(os.Getenv("APP_API_KEY"))
	if err != nil {
		log.Println("startup error:", err)
		return
	}

	h := httpapi.Chain(mux, authMW)
	_ = http.ListenAndServe(":8080", h)
}

The key idea: a configuration error must surface at startup. We do not “translate” it into 401/403 for every request—we simply do not run the server in an incorrect mode.

4. Middleware order, secrets, and behavior checks

Where auth middleware should be in the chain

It is very tempting to put auth “as the outermost one” to “cut off immediately”. But there is a nuance: if you have a request_id middleware, request_id should be outside auth, otherwise requests that were rejected will be left without a request_id and will correlate worse in logs.

The ideal picture looks like this:

1. request_id middleware sets X-Request-ID and stores request_id in context,
2. logging writes method/path/status/duration/request_id,
3. recover catches panics from below,
4. auth either lets the request through or responds with 401/403,
5. then come the business handlers.

The idea of context as a transport for request-scoped data implies that lower layers (including auth and handlers) read data from r.Context().

Secrets and logs: why you must not log X-API-Key

When you start logging requests, there is a temptation: “Let’s log all headers so it’s easier to debug.” It is especially tempting when authorization “doesn’t work”.

But X-API-Key is a secret. If you logged it even once, assume you potentially:

• leaked it into a log file,
• sent it to a log aggregation system,
• showed it to colleagues,
• left it in CI/CD history,
• accidentally attached it to a bug report.

And now your key “lives” a much more eventful life than planned.

So the rule is simple and boring: never log the key. At most, log the fact “key is missing” or “key is invalid”, plus request_id.

If you want to make key comparisons a bit “safer” in terms of timing, there is crypto/subtle.ConstantTimeCompare, but for the current learning version, a regular got != expectedKey is enough. The main thing is not to mix levels: here we are learning middleware architecture and error contracts, not building a cryptographic masterpiece.

A small sanity check: how the server will behave

Let’s confirm at the scenario level that the behavior is predictable:

• If the client sends a request without X-API-Key, auth middleware will respond with 401 and a JSON error. The handler will not even run.
• If the client sends X-API-Key: wrong, middleware will respond with 403.
• If the client sends the correct key, the request will continue—and the handler will produce the normal response.

And most importantly: if the environment variable APP_API_KEY is not set, the server should not even start listening on the port. It will exit at startup and print startup error. That is fail fast.

5. Common mistakes

Mistake #1: “the expected key is empty—so everyone gets 401/403”.
This is one of the most dangerous logical mistakes because it turns a configuration problem into a “behavioral feature” of the server. As a result, the server can run in an unpredictable mode. The correct approach: an empty expected key is a startup error; the server must not start.

Mistake #2: mixing up 401 and 403.
If you send 403 when the key was not provided at all, clients and humans will be confused: 403 is usually perceived as “you definitely can’t”, while 401 is “identify yourself first”. Even worse is always returning 401, even when the key is invalid: then the client cannot distinguish “forgot the header” from “wrong key”.

Mistake #3: JSON response in one place, text in another.
It is very easy to start with http.Error “just for a minute”, and then forget. As a result, the API becomes unpredictable: tests are harder, clients are harder, and users get different error formats on neighboring endpoints. Auth middleware must respond with the same error envelope as other errors.

Mistake #4: forgetting to return after rejecting.
A classic middleware bug: we already wrote WriteHeader(401), but forgot to exit, and the request continued, where the handler tried to write 200 and a body. Result: warnings in logs, strange responses, and sometimes “it works intermittently”. After any response in middleware, you almost always need return.

Mistake #5: logging a secret.
Even if you logged the key “just once while debugging”, it usually stays in logs forever. In a learning project it seems like a minor detail, but this is exactly how bad habits form. Do not log secrets: no keys, no passwords, no tokens.