Why a simple file overwrite is dangerous

1. A file is not a variable: why writing is harder than it seems

When you first write os.WriteFile("data.txt", data, 0644), you get a pleasant feeling: “that’s it, I’ve conquered data storage.” And that’s true… right up until the first time your program crashes in the middle of a write, the disk suddenly runs out of space, or you forget Flush(). Then it turns out a file is not an in-memory variable, but an object surrounded by the OS, drivers, buffers, and unexpected surprises.

Imagine you are rewriting important notes with a pen. If you first erase the old text and then start writing the new text, and at that moment you run out of ink—you don’t have the “old notes” and you don’t have the “new notes.” You have “half new, half empty.” That’s roughly how a “simple file overwrite” works: it often starts with truncation (truncate) of the old contents, and after that it’s luck of the draw.

And this is where the key engineering question appears: what does it mean to “write a file reliably”? Before choosing specific functions, you need to define the requirements for the outcome.

2. Reliable write invariants: what we want to guarantee

Reliability doesn’t start with os.CreateTemp, not with Rename, and not even with “let’s add one more defer.” Reliability starts with an honest answer: what result do we consider correct, and which failures we must survive without catastrophe. It’s convenient to express such requirements as invariants—properties that must always hold (within our protocol).

Let’s define a basic set of invariants for “write a new version of data to a file”:

You can think of this as a simple state diagram. What matters is that after a failure we do not end up in the “corrupted file” state.

stateDiagram-v2
    [*] --> Old
    Old: old file version

    Old --> Writing: started update
    Writing --> New: write completed successfully
    Writing --> Old: failure, but old version remained
    Writing --> New: failure, but the new version became fully visible

    Writing --> Corrupted: failure left a half-version (NOT ALLOWED)
    Corrupted --> [*]

The Corrupted state is exactly what you can very easily get with a “simple overwrite.” And later in the lecture we’ll break down why.

3. Where a simple overwrite breaks: truncate as the disaster point

The worst part of a “simple overwrite” is that it often starts by destroying the old content. Even if you then write the new content perfectly, the mere fact that you first made the file empty already violates our second invariant (about failures: “either the old version or the new one”).

Take a look at this code:

package main

import (
	"fmt"
	"os"
)

func main() {
	err := os.WriteFile("notes.txt", []byte("new version\n"), 0644)
	if err != nil {
		fmt.Println("write failed:", err)
	}
}

It looks harmless. But if notes.txt already existed, we are effectively saying: “overwrite it completely.” Now mentally insert a failure: the program managed to start the update, the old file has already been truncated, and the new write didn’t finish. Now you have notes.txt that may be empty or contain only part of the data.

A slightly more “manual” variant highlights the problem even more clearly:

package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.Create("notes.txt") // creates OR truncates an existing file
	if err != nil {
		fmt.Println("create failed:", err)
		return
	}
	defer f.Close()

	_, _ = f.WriteString("new version\n")
}

The key point: os.Create is not “open a file for writing.” It is “create the file, and if it exists—set its length to zero (truncate) and return a descriptor.” In other words, the risk point occurs right at the beginning, even before any data is written.

Sometimes a developer thinks: “well, if I write quickly, the chance of failure is small.” This is exactly the kind of case where a “small chance” turns into a “very big one” in production—because production is the place where everything eventually breaks, out of sheer principle.

4. The write protocol: Write, Flush, and Close

Partial writes: why Write returns (n, err)

Many beginners look at Write and see “some weird pair”—the number of bytes written and an error. You may want to think: “I’m checking err anyway, why do I need n?” And then you get a situation where err == nil, but fewer bytes were written than you expected. This is not the most common scenario on a typical disk, but the API contract says: it happens, and that means you must account for it in critical places.

Let’s write a small “building block” for reliable writing: a function that verifies that exactly the number of bytes we planned were written.

package main

import (
	"fmt"
	"os"
)

func writeAll(f *os.File, data []byte) error {
	n, err := f.Write(data)
	if err != nil {
		return fmt.Errorf("write: %w", err)
	}
	if n != len(data) {
		return fmt.Errorf("write: partial write %d/%d", n, len(data))
	}
	return nil
}

Notice there’s no magic here: we simply treat Write as an honest contract. If the function says “I might not write everything,” then we are obligated to check that in places where “not everything” breaks the meaning of the operation.

Now imagine writing “the naive way”: you call Write, ignore n, the program exits “successfully,” and later loading fails because the file is truncated at a random point. The worst part is that the bug looks like “sometimes the file is corrupted,” which is the perfect debugging trap.

Buffering: “I wrote” ≠ “the data is in the file”

Buffering is like food delivery: you can “place an order” (call WriteString), but that doesn’t mean the food is on your table yet (the data is on disk). bufio.Writer accumulates data in memory and flushes it to the file in chunks. This speeds things up, but adds a required step: finalization via Flush(), otherwise the data may remain only in RAM.

The most typical failure scenario looks like this: the code “writes,” there are no errors, but the file is empty or incomplete.

package main

import (
	"bufio"
	"fmt"
	"os"
)

func main() {
	f, err := os.Create("out.txt")
	if err != nil {
		fmt.Println("create:", err)
		return
	}
	defer f.Close()

	w := bufio.NewWriter(f)
	_, _ = w.WriteString("line 1\n")
	_, _ = w.WriteString("line 2\n")

	// Bug: forgot Flush()
}

This example is intentionally “harmful”: it shows that “no errors” does not yet mean “the result is correct.” If you want the data to reliably pass through the buffer into the file, Flush() must be part of the protocol:

package main

import (
	"bufio"
	"fmt"
	"os"
)

func main() {
	f, err := os.Create("out.txt")
	if err != nil {
		fmt.Println("create:", err)
		return
	}
	defer f.Close()

	w := bufio.NewWriter(f)
	_, _ = w.WriteString("ok\n")

	if err := w.Flush(); err != nil {
		fmt.Println("flush:", err)
	}
}

And one more important point: Flush() can also return an error. This is not a “formality,” but a real point where a write problem can surface (for example, if the disk is full). So “reliable writing” always means “check errors at every step of the protocol,” not only on the first line.

Close() is not a formality: an error can arrive on close

It’s very tempting to think that Close() is like “putting the stool back”: it always works and can’t be a source of problems. But in write scenarios, Close() is sometimes the moment when the system tries to “finish the remaining work”: flush internal buffers, complete I/O, commit state. That’s why in serious code, Close() is part of the protocol, and you need to account for its error.

In Go, people often write:

f, err := os.Create(path)
if err != nil {
	return err
}
defer f.Close()

For reading this is usually fine: if we’re reading and Close() suddenly returns an error, most of the time it’s not critical to the result (though it depends on the situation). But for writing, people sometimes do it differently: close the file explicitly and check the error. It looks a bit longer, but it’s more honest.

package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.Create("data.txt")
	if err != nil {
		fmt.Println("create:", err)
		return
	}

	if _, err := f.WriteString("payload\n"); err != nil {
		_ = f.Close()
		fmt.Println("write:", err)
		return
	}

	if err := f.Close(); err != nil {
		fmt.Println("close:", err)
		return
	}
}

Yes, this isn’t the “prettiest” code in the world. But it reinforces the lecture’s point: reliability is a protocol, not “one call.” A protocol has steps, and every step can fail.

Here it’s useful to recall Go’s general philosophy: errors should be checked explicitly because they are ordinary values—and that is the language’s strength.

5. The storage contract in an application: Save is a commitment

When we build a learning app (even a very simple local task manager), there is usually a function like “save the task list to a file.” Early on, you want to make it as simple as possible: take the data, call os.WriteFile, done. But as soon as the file becomes the “source of truth” (meaning the user loses data without it), that simplicity starts to cost too much.

Let’s lock in a minimal API for the storage layer: functions Load and Save, which do not print anything and instead return errors. This is an important style: storage should not do UX; it should do correctness.

package storage

import (
	"fmt"
	"os"
)

func SaveText(path string, data string) error {
	if err := os.WriteFile(path, []byte(data), 0644); err != nil {
		return fmt.Errorf("save file %s: %w", path, err)
	}
	return nil
}

The code is intentionally “naive”: it shows the starting point. From the standpoint of convenience—it’s perfect. From the standpoint of our invariants—it’s dangerous, because overwriting does not guarantee “either the old version or the new one.”

And now we can define the requirements for the next version of SaveText (without implementing it here in detail). We want that after a failure, the file does not turn into a half-version. We want a “commit” point, after which the new version is considered published. We want no junk left nearby. And we want an error that says “which step.”

Notice we are not discussing a specific mechanism yet. We are doing something more important: turning “which function to call” into “which contract we must fulfill.” That is mature development: first agree on correctness, then choose the tool.

6. Typical mistakes with a simple file overwrite

Mistake #1: assuming that os.WriteFile is always safe because “it’s one function, so it’s atomic.”
os.WriteFile does have a convenient interface, but convenience is not reliability. In reality, overwriting often means the old file will be replaced/truncated, and on failure you can be left with a “half-version.” If the file matters, first define invariants and verify whether the chosen approach satisfies them.

Mistake #2: forgetting that writing is multiple steps, and an error can show up somewhere you don’t expect.
Beginners check the error on Write, while Flush() and Close() are treated as “always OK.” Then a mysterious bug appears: “sometimes the file is empty” or “sometimes the file is truncated.” The correct habit is to treat finalization as part of the protocol: if there is a buffer, you need Flush(); if there is a file, Close() can be a meaningful step.

Mistake #3: ignoring n from Write, because “the error is separate anyway.”
The Write contract in Go honestly says: a partial write is possible. If you ignore n, you lose the ability to detect data corruption immediately. Sometimes it’s better to return a meaningful error like “partial write 10/100” than to later figure out why the file parser complains at a random spot.

Mistake #4: returning an error “as is,” without context about the operation.
An error like “permission denied” without stating what you were doing and which file turns debugging into guesswork. Wrap errors using fmt.Errorf("save file %s: %w", path, err). In Go, this is a basic practice: an error must carry context, otherwise it’s nearly useless.

Mistake #5: mixing “storage” and “user output” in one place.
If your Save prints to stdout/stderr, it’s hard to test and hard to reuse. Let Save return error, and let the outer layer (CLI/server) decide whether to print/log/crash. Then it’s easier to check invariants, and the behavior becomes predictable.