Simple rate limit: a minimal in-memory implementation

1. Why you need rate limiting

In plain terms, rate limit is a turnstile at the entrance to your API. Not to “torture an honest user”, but to keep one overly active client (or a bot) from turning your server into a sad pumpkin that does nothing but crash, log errors, and regret its career choices.

In the HTTP world, rate limit usually answers questions like: “How many requests per minute are we willing to serve from one client?” and “What do we do if the client overdoes it?”. The answer to the second question is standardized: HTTP 429 Too Many Requests. The key point is that 429 is not an “internal server error”, but an expected policy. It’s like a sign that says “admission by ticket” — nobody is surprised they can’t enter without a ticket.

Another reason: rate limit helps make the system more predictable. Even if everything runs fast, a sudden burst of requests (for example, someone runs a script in a loop) can ruin latency for everyone. rate limit reduces the chance of such “life celebrations”.

Fixed window: a simple model and its parameters

The phrase “rate limiting” sounds like we’re about to get a 150-page dissertation, graphs, formulas, and a professor who suspiciously loves the word “token”. But here we’ll use the simplest, most practical learning model: fixed window.

The idea is simple. You choose:

window — the window duration, for example 1 minute
limit  — how many requests are allowed per window, for example 30

Then for each client you store two numbers: how many requests they have already made in the current window, and when that window ends. As soon as the window expires, the counter resets and a new window starts.

A mini table so you don’t get confused:

window

1 minute

limit

30

count

0…limit

until

time.Time

Yes, fixed window isn’t perfect (at the window boundary you can “stuff in” many requests), but it’s great for understanding the principle, getting hands-on with an implementation, and not drowning in details.

Who is a “client”, and how to choose the key for the limit

Rate limit is almost always done “per client”. That means we need to decide what counts as “one client”.

In an ideal world, we’d have a stable client identifier: a user id from auth, an API key, a token — something persistent. But for a learning server we’ll do a simple, clear thing: use r.RemoteAddr.

RemoteAddr is a string in the form IP:port, for example 203.0.113.10:54321. It’s not ideal (ports change, proxies and load balancers exist), but it honestly demonstrates the mechanics: there’s an incoming request, it has an “address”, and we count by it.

To be a bit cleaner, we’ll separate the IP from the port. This is useful so the same client doesn’t look like “a hundred different clients” just because the port changed.

import (
	"net"
	"net/http"
)

func clientKey(r *http.Request) string {
	host, _, err := net.SplitHostPort(r.RemoteAddr)
	if err != nil {
		return r.RemoteAddr
	}
	return host
}

If SplitHostPort can’t parse the address, we keep it as-is. For learning code, this is a fine strategy: don’t complicate your life.

2. Parallel requests and why you need sync.Mutex

A point that often breaks beginners’ brains (and that’s normal): you can come up with perfect rate limit logic on paper, but in a real HTTP server it will start behaving strangely if you forget one detail — the server handles requests in parallel.

In net/http, each request may be handled in its own goroutine. That means two requests can enter your limiter at the same time and simultaneously try to update shared data (usually a map).

And a map in Go does not like concurrent writes: it’s not “sometimes a wrong result”, it’s a very real scenario that can produce a runtime panic. So we need to protect shared state.

We’ll have a shared map where the window state is stored by client key. To make the “read → update → write” operations atomic from the logic point of view, we’ll protect them with a sync.Mutex.

Yes, Go has sync.Map, which is safe for concurrent access. But for learning and for a simple limiter, sync.Mutex + map is simpler and clearer, and it’s also a fundamental skill you’ll need constantly. It’s still good to remember sync.Map as a specialized tool for certain scenarios.

Data structures

We’ll make two structs: one for the “window”, and one for the limiter itself.

windowHits stores the window boundary and the counter, and fixedWindowLimiter stores settings + a map + a Mutex.

import "time"

type windowHits struct {
	until time.Time
	count int
}

import (
	"sync"
	"time"
)

type fixedWindowLimiter struct {
	window time.Duration
	limit  int

	mu   sync.Mutex
	hits map[string]windowHits
}

Note: hits can be created immediately via make, or lazily (on the first request). Lazy init is convenient if the limiter is created as &fixedWindowLimiter{...}.

Allow method: the heart of the limiter

We need a function that decides “allow or not”. It’s usually called Allow.

The key idea: Allow should be as predictable as possible. You give it the client key and the current time now, and it returns true/false.

Why do we pass now in, instead of calling time.Now() inside? So the code is more controllable and potentially easier to test. Even if you’re not writing tests right now, the habit of “pulling time.Now() out” makes the code cleaner.

import "time"

func (l *fixedWindowLimiter) Allow(key string, now time.Time) bool {
	l.mu.Lock()
	defer l.mu.Unlock()

	if l.hits == nil {
		l.hits = make(map[string]windowHits)
	}

	h := l.hits[key]
	if now.After(h.until) {
		h = windowHits{until: now.Add(l.window), count: 0}
	}

	if h.count >= l.limit {
		l.hits[key] = h
		return false
	}

	h.count++
	l.hits[key] = h
	return true
}

What matters here:

• Grab Lock() immediately and immediately set defer Unlock(). This is your “seat belt”: even if you later add an early return, Unlock() will still run.
• Initialize the map if it’s nil.
• Read the current state: h := l.hits[key]. If the key doesn’t exist, you’ll get the zero value (until = time.Time{}, count = 0) — that’s fine for us.
• If the current time is “after the end of the window”, reset the window: set a new until and set count = 0.
• If count >= limit, deny.
• Otherwise increment count and allow.

4. Middleware: turning the limiter into HTTP behavior

Middleware is a function that takes next http.Handler and returns a new http.Handler. Inside, it can do something “before”, “after”, or it can avoid calling next at all and finish the request itself (short-circuit).

Rate limit is classic short-circuit: if the limit is exceeded, we immediately respond with 429 and exit.

Assume we already have a writeAPIError function that writes an error envelope in JSON (a unified error format).

import (
	"net/http"
	"time"
)

func rateLimitMW(l *fixedWindowLimiter) Middleware {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			key := clientKey(r)

			if !l.Allow(key, time.Now()) {
				writeAPIError(w, http.StatusTooManyRequests,
					"rate_limited", "rate limit exceeded")
				return
			}

			next.ServeHTTP(w, r)
		})
	}
}

Here we take the client key, call Allow, and if it’s not allowed, we respond with 429.

It’s important that we respond using the same error envelope contract as the other errors. This makes the API predictable: the client doesn’t need to guess whether the error will be plain text, HTML, or JSON.

Bonus: Retry-After without extra math

You can add a small touch that improves client UX: the Retry-After header. It hints how many seconds to wait before retrying.

For fixed window this is especially simple: if we know until, we can roughly say ceil(until-now).

To avoid complicating Allow, we’ll make a separate method RetryAfterSeconds that reads the current window under the same mutex.

import (
	"math"
	"time"
)

func (l *fixedWindowLimiter) RetryAfterSeconds(key string, now time.Time) int {
	l.mu.Lock()
	defer l.mu.Unlock()

	h := l.hits[key]
	if now.After(h.until) {
		return 0
	}

	sec := h.until.Sub(now).Seconds()
	return int(math.Ceil(sec))
}

And use it in middleware:

ra := l.RetryAfterSeconds(key, time.Now())
w.Header().Set("Retry-After", fmt.Sprintf("%d", ra))
writeAPIError(w, http.StatusTooManyRequests, "rate_limited", "rate limit exceeded")

If you want to keep everything truly minimal, you can skip this bonus. But as a concept it’s useful: the server becomes “more polite”.

5. Wiring into the application and middleware order

Assume we already have:

• mux with routes /api/v1/tasks...
• chain(...) to assemble middleware
• requestIDMW, logMW, authMW — written earlier

We’ll create the limiter on startup and insert it into the chain.

import (
	"net/http"
	"time"
)

func buildHandler(mux *http.ServeMux, auth Middleware) http.Handler {
	limiter := &fixedWindowLimiter{
		window: time.Minute,
		limit:  30,
	}

	return chain(mux,
		requestIDMW,
		logMW,
		auth,
		rateLimitMW(limiter),
	)
}

Order is part of the logic, not just “prettiness”.

If you put rate limit before auth, the limiter will restrict even those who don’t have a key at all. Sometimes that’s fine (for example, to prevent hammering 401 with thousands of requests), but often it’s more reasonable to limit already-authorized clients.

If your rate limit is based on an API key (which is usually very sensible), auth should come first so you can take the client key from X-API-Key (not from IP). Here we use RemoteAddr, so it’s not critical — we’re just documenting a sound scheme.

A small diagram so the order feels physical:

flowchart TD
    A[Request arrived] --> B[request_idMW]
    B --> C[logMW - start timer]
    C --> D[authMW]
    D --> E[rateLimitMW]
    E --> F[handler / mux]
    F --> C2[logMW - log after next]

6. Limitations of the in-memory approach

In-memory rate limiter is a limiter that lives in the memory of a single process.

This means that if you run two server instances behind a load balancer, the limit will apply separately on each one. A client will be able to “bypass” the limit if their requests are spread across different instances. That’s not a bug in the code — it’s a property of the chosen design.

Another nuance: hits can grow over time. For example, if a bunch of unique IP addresses hit you (or you choose a key that changes often), the map will keep entries. In this learning version we don’t implement complex cleanup of “old clients”. The most we do is reuse the entry and reset the window when it expires.

If we wanted a minimal cleanup without extra complexity, we could delete the entry when the window expires and the client comes again. That is, instead of “reset”, do “delete and create again”. But that still doesn’t fully clean up: a client who came once and never came back will remain in the map. For a learning implementation, that’s acceptable.

And one more nuance: yes, you can try sync/atomic for counters, but it quickly turns into more complex mechanics. An “atomic int” doesn’t save you, because you still need to update multiple fields consistently (both the window time and the counter). So here we use a simple and reliable Mutex.

7. Typical mistakes when implementing rate limiting

Mistake #1: mutating a map without a Mutex (“it seems to work”).
This is the most common and the most insidious story. Locally you may have few requests and everything is “apparently fine”. Then under load (or just due to timing coincidence) you get a data race and a possible runtime panic. In an HTTP server, requests are parallel by definition, so the shared map must be protected.

Mistake #2: calling Lock() and forgetting Unlock() on one branch.
That’s how “beginner deadlocks” are born: the server stops responding because someone holds the lock forever. The easiest way to almost completely eliminate this is to put defer l.mu.Unlock() immediately after Lock().

Mistake #3: not short-circuiting and calling next after 429.
Sometimes code writes 429, and then by inertia calls next.ServeHTTP(w, r). As a result the downstream handler also tries to write a response, and you get weird effects: “headers already sent”, “double response body”, and other fun things. If the limit is exceeded — respond and return.

Mistake #4: returning 500 instead of 429.
Rate limit is not “oops, we broke”, it’s “you’re too frequent”. 500 means “server-side problem”, while 429 means “access/load policy”. On 429, the client can correctly slow down and retry later; on 500, it usually assumes the server is dead.

Mistake #5: choosing an unstable client key and being surprised that the limit “doesn’t work”.
If the key changes constantly (for example, you use RemoteAddr together with the port), the limiter will think every request is from a “new client” and will never exceed. So either stabilize the key (for example, by IP), or use a more meaningful identifier (API key / user id) when you have it.

1

Task

Go SELF, level 62, lesson 3

Locked

Key without port

Key without port

Open

1

Task

Go SELF, level 62, lesson 3

Locked

Fixed Window Rate Limiter

Fixed Window Rate Limiter

Open