1. Why two processes are a separate problem
When we say “two processes,” it sounds a bit abstract — as if we were talking about servers with millions of users. But in practice it can literally be you and your coworker running the same CLI command at the same time, or two terminals on one laptop. The tricky part is that the file almost always stays “nice and intact,” so the bug can live for a long time.
Let’s imagine our training app — a simple task manager tasker that stores tasks in the file tasks.json. The command tasker add "buy milk" works roughly like this: read the file → parse JSON → add a task → write the file.
Now we run two commands almost at the same time.
sequenceDiagram
participant A as Process A (tasker add)
participant F as tasks.json
participant B as Process B (tasker done)
A->>F: Read old version (v1)
B->>F: Read old version (v1)
A->>A: Compute v2 (added a task)
B->>B: Compute v3 (marked done)
A->>F: Publish v2 (temp+rename)
B->>F: Publish v3 (temp+rename)
Both processes acted honestly and carefully. And both used reliable writing. But the ending is this: the file will contain v3, and A’s changes (the added task) may disappear, because B read the old version (v1), didn’t know about the new task, and wrote “its” v3 as a full replacement. This is called a lost update.
The key idea: temp+rename protects you from a “half-file,” but it does not protect you from “half-logic” , when two independent computations compete for the right to be last.
2. The lock‑file idea: an agreement between processes
Lock-file is a very “human” solution. Instead of trying to do OS-level magic (that’s where the nuances start: different platforms, different lock types), we introduce a clear rule: if there is a lock file next to it, a write is already in progress — don’t touch it.
It’s important to treat lock-file correctly right away: it’s an agreement, not an absolute “protects-from-everything” guarantee. If one process crashes, the lock-file may remain (we’ll discuss this later). But even such a simple agreement drastically reduces the probability of data loss in typical CLI scenarios.
The policy looks like this:
- for tasks.json we create tasks.json.lock;
- a process that wants to modify tasks.json first tries to create tasks.json.lock exclusively;
- if creating the lock fails because it already exists, then someone else is writing right now, and we exit gracefully with a “storage is busy” error;
- if the lock is created, we do the work: read, compute, write (via temp+rename/backup);
- at the end we delete the lock‑file.
The key technical trick: exclusive creation is done via os.OpenFile with flags os.O_CREATE|os.O_EXCL. The idea is that the OS must perform the operation “create the file, but only if it doesn’t exist yet” atomically, without a window where two processes can “agree differently.”
3. os.O_CREATE|os.O_EXCL: what this flag combination means
Flags in os.OpenFile often look like spells: O_CREATE, O_EXCL, O_WRONLY… But here the logic is pretty down-to-earth, and it’s useful to say it out loud.
os.O_CREATE says: “if the file doesn’t exist, create it.” But by itself it doesn’t prevent two processes from almost simultaneously going down the same path.
os.O_EXCL adds strictness: “I want exclusivity.” And here’s the important nuance: O_EXCL only makes sense together with O_CREATE, because “exclusivity” in this context means exclusively create a new file.
In other words, we don’t care about “exclusive open,” we care about “exclusive create,” so that the second process cannot quietly create the same lock.
Add os.O_WRONLY to open the file for writing. This is useful at least because we can write some helpful debugging info into the lock‑file (for example, a PID). But even if we write nothing, the very fact that the file got created already matters.
4. Implementation: AcquireLock and Release
When writing lock‑file code, it’s especially easy to produce “almost correct code” that later leaves junk behind or doesn’t make it clear why everything broke. So we’ll write it the way we’re used to in Go: small functions, early returns, and clear error text.
Separately: releasing the lock is cleanup. defer is perfect here, because it guarantees execution on any exit path from the function. This is exactly the case where defer makes code safer and easier on the brain.
Mini-struct Lock
To make it convenient to store both the path and the opened file, we’ll make a small type.
package storage
import (
"os"
)
type Lock struct {
f *os.File
path string
}
There’s no “OOP magic” here. It’s just a container with two fields.
Acquiring the lock‑file: AcquireLock
Now the most important part: acquiring. We distinguish two types of errors: “the lock already exists” and “something actually went wrong (permissions, path, disk).”
package storage
import (
"fmt"
"os"
)
func AcquireLock(lockPath string) (*Lock, error) {
f, err := os.OpenFile(lockPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
if os.IsExist(err) {
return nil, fmt.Errorf("storage is busy: lock exists")
}
return nil, fmt.Errorf("create lock %s: %w", lockPath, err)
}
return &Lock{f: f, path: lockPath}, nil
}
Notice: we return an error with context via fmt.Errorf(... %w ...), so higher up you can see “at which step” it failed, while still preserving the original cause. This is a basic and very important Go practice.
Releasing the lock‑file: Release
Release means “close and delete.” Typically, release errors should not overshadow the main error of the business operation, so Release is often best-effort: close, delete, don’t return errors (or log them at the application boundary).
package storage
import "os"
func (l *Lock) Release() {
_ = l.f.Close()
_ = os.Remove(l.path)
}
Yes, we ignore errors here. In an ideal world you could collect and return them, but for the educational pattern “lock-file as an agreement” this is enough. The key is to always call Release via defer.
5. Where to place the lock: around the write or around the whole operation
At this point many people make an intuitive mistake: “well, the lock is needed for writing, so we’ll wrap only the write.” It sounds reasonable, but you can still lose updates.
Lost updates happen because both processes read the same old version. So if your goal is “only one writer,” the lock must cover the entire critical section, including the read.
So the correct form is:
AcquireLock
Read file
Compute new state
Write (temp+rename + backup)
ReleaseLock
And this is not enough:
Read file
Compute new state
AcquireLock
Write
ReleaseLock
Because by the time you call AcquireLock, you have already read old data.
6. Wiring lock‑file into the tasker storage
Let’s assume we have a SaveTasks function that already knows how to write the file reliably (temp+rename, backup). We’ll add the lock in a way that reads top-to-bottom and looks like a protocol.
Let the tasks file be named tasks.json, and the lock be tasks.json.lock.
package storage
import (
"fmt"
)
func SaveTasks(path string, data []byte) error {
lock, err := AcquireLock(path + ".lock")
if err != nil {
return err
}
defer lock.Release()
if err := WriteWithBackup(path, data, 0644); err != nil {
return fmt.Errorf("save tasks: %w", err)
}
return nil
}
Two things matter here.
First: defer lock.Release() must be placed immediately after a successful Acquire. Then you won’t forget to add it, and your future bug will be somewhere else (which, of course, is also unpleasant, but at least it’s variety).
Second: we’re not inventing a new way to write. A lock‑file is not a replacement for temp+rename, it’s an additional “turnstile” at the entrance: only one process gets inside, and inside we execute the reliable write protocol.
7. Practical nuances and limitations of lock‑file
What to tell a CLI user if the lock already exists
When your user sees an error, they usually don’t care about which flags O_EXCL uses or what atomicity is. They want to understand: “Is this my fault? Is it a bug? What should I do?”
So for a “lock exists” message it’s usually helpful to be friendly and brief: “Storage is busy, try again later,” or “Another instance of the program is currently writing.”
At the storage layer we returned "storage is busy: lock exists". At the CLI boundary (in main) you can decide whether to print it as-is or turn it into a more human sentence.
And there’s a subtle trade-off here: if you always print the same thing, that’s great for UX, but sometimes you want diagnostics. A good compromise is a short message for the user plus a wrapped error for logs/debugging (we do this throughout the course).
A “stuck” lock: why it happens
Lock‑file is an agreement, and it has a weak point: if the process died (panic, kill -9, power loss), defer won’t run, and the lock‑file may remain.
As a result, the next run will say “storage is busy,” even though in reality “nobody is writing, we just forgot the lock on the door.”
What can you do about it? Within this lecture we lock in an important idea: a lock‑file solves a common problem, but it is not a perfect locking system. In mature systems people add a “lease” (a lock TTL), write the PID and a timestamp into the lock file, check whether the process is still alive, or use specialized OS mechanisms. But that’s a different depth of the topic.
For our training CLI, it’s usually enough to agree on this: if you’re sure no write is in progress, you can delete tasks.json.lock manually. And to make it easier for users, sometimes people add a tasker doctor or tasker unlock command — but those are product decisions, not required to understand the mechanics of O_EXCL.
Table: what each layer protects
To avoid mixing things up, it helps to keep a simple responsibility map in mind:
| Mechanism | Protects against | Does not protect against |
|---|---|---|
|
a “partial file version” if a failure happens during writing | lost updates with two writers |
|
a “logically bad new version” (you can roll back) | a partial file (that’s handled by temp+rename) |
|
concurrent execution of read→compute→write by two processes | an “eternal lock” after a crash |
This table matters specifically as an engineering model: each layer covers its own hole, but does not replace the others.
8. Typical mistakes with lock‑file
Mistake #1: locking only around Rename, not around the read.
This code looks “correct” because you really do protect the publication moment. But lost updates happen earlier — when two processes read the same old version. If the goal is not to lose updates, the lock must cover the entire read→compute→write cycle.
Mistake #2: using os.O_EXCL without os.O_CREATE and expecting exclusivity.
Intuitively, it feels like O_EXCL means “I won’t let anyone in.” But in this context it is “exclusive creation.” If you forget O_CREATE, the logic breaks and you can get behavior very different from what you expect.
Mistake #3: forgetting defer Release() immediately after successfully acquiring the lock.
Classic: you acquired the lock, then you kept coding, added two more returns, and forgot to release in one of them. Result: the lock remains even on a normal exit, and the program starts “blocking itself.” defer exists specifically to make such mistakes happen less often.
Mistake #4: returning the “raw” system error text to the user without context.
If you just pass through open tasks.json.lock: file exists, the user may not understand what’s going on. In Go we usually wrap the error with the operation context (create lock ... %w), and at the application boundary convert it into a short and clear message.
Mistake #5: treating lock‑file as a rock-solid guarantee of correctness for every situation.
A lock‑file is a good practical agreement, but it doesn’t fix crashes, doesn’t answer “how to wait for the lock,” and doesn’t replace a reliable write protocol. If you remember the boundaries of its responsibility, it works great and makes CLI behavior much more predictable.
GO TO FULL VERSION