os.ReadFile/WriteFile: when it’s OK to read and write

1. Introduction

When you first see os.ReadFile, you might get suspicious: “Is this for lazy people? What about ‘proper’ reading via Open and Read?” In reality, ReadFile/WriteFile are not “cheat codes”, but an honest shortcut for a typical case: the file is small, we need to get it all, process it in memory, and maybe write it back.

It’s like cooking: sometimes you need to “simmer broth for 8 hours”, and sometimes you just need a sandwich. A sandwich isn’t shameful if you actually wanted to eat, not win a “Michelin” for home cooking.

os.ReadFile: the contract, the result type, and why it’s []byte

It’s important to set expectations right away: os.ReadFile(path) reads the entire file and returns []byte and error. That means you get a “raw” set of bytes. This is logical: a file can be text, but it can also be an image, a zip archive, or something that doesn’t resemble text at all. Go won’t pretend that every file is a string.

At the mental-model level, it looks like this:

path → ReadFile → bytes (in memory) → your processing

Minimal reading example:

package main

import (
	"fmt"
	"os"
)

func main() {
	b, err := os.ReadFile("data/todo/items.txt")
	if err != nil {
		fmt.Println("readfile:", err)
		return
	}

	fmt.Printf("bytes=%d\n", len(b)) // for example: bytes=42
}

What matters here is that we’re not pretending it’s text yet. We just read the bytes and looked at the size.

2. Converting bytes to text and back

Once you understand that the file really is text (for example, our task list is a plain .txt), you do an explicit conversion: string(b). This is not “magic”, but a clear statement: “yes, I consider these bytes to be text”.

package main

import (
	"fmt"
	"os"
)

func main() {
	b, err := os.ReadFile("data/todo/items.txt")
	if err != nil {
		fmt.Println("readfile:", err)
		return
	}

	text := string(b)
	fmt.Printf("text=%q\n", text) // text="buy milk\nread go\n"
}

Notice %q: it prints the string in quotes and makes \n visible. For diagnostics, this is more convenient than a plain Println.

The reverse operation for writing is []byte(text):

data := []byte("hello\n")

Yes, this looks like “changing outfits” back and forth, but this is exactly how you keep control over what you treat as text and what is just a byte sequence.

4. os.WriteFile: what it does and why it matters

With os.WriteFile(path, data, perm) there’s one detail beginners regularly gloss over: WriteFile writes everything and, if the file already exists, it usually overwrites its contents. In other words, it’s the “burn the old one down and build a new one” mode (figuratively; the disk won’t be offended, but your data might).

Minimal example:

package main

import (
	"fmt"
	"os"
)

func main() {
	err := os.WriteFile("data/todo/items.txt", []byte("one\ntwo\n"), 0644)
	if err != nil {
		fmt.Println("writefile:", err)
		return
	}
	fmt.Println("ok") // ok
}

The third parameter is permissions (perm). For now, it’s enough to remember: it’s important when creating a file. If the file already exists, the permission behavior depends on the OS and details; typically perm is not treated as “set permissions exactly as I said” for an existing file. We won’t dive into permission subtleties right now—what matters more is understanding that you can’t just drop perm “because it’s unclear”.

Comparison: Open/Create vs ReadFile/WriteFile

When different file-handling approaches get mixed up in your head, a simple comparison table helps. There’s no “better/worse” here—only “fits the task/doesn’t fit the task”.

os.ReadFile

os.WriteFile

os.Open + Read

os.OpenFile(...|os.O_APPEND...)

*os.File

Separately, I want to emphasize: os.Open returns *os.File and error, and it’s normal that almost everything in Go works this way.

5. Where ReadFile/WriteFile are genuinely great

It’s tempting to have “one rule for all cases”, but real life is trickier: you’ll write small utilities, services, and data importers. So it’s useful to keep a practical criterion in mind: if the file is small and semantically you work with the entire contents at once, then ReadFile/WriteFile are a perfect fit.

For example, a small text file with a task list that our learning project stores on disk is exactly the “native” situation. We read the whole file, parse lines, add one task, and write the file back in full. Yes, it’s an “overwrite”, but if the file is a couple of kilobytes, it’s fast and simple.

What matters here is not to confuse convenience with irresponsibility. Convenience is when the data size is known and manageable. Irresponsibility is when you do ReadFile on an 8 GB file “just in case”.

6. Where ReadFile/WriteFile start to bite back

ReadFile/WriteFile have limitations, and they’re not “rare theoretical ones”—they’re very real.

The first limitation is memory. ReadFile reads the entire file into a []byte, which means you need memory roughly equal to the file size (plus overhead). If you ever read a huge file all at once, you’ll get either slowdowns or an out-of-memory crash. And yes, this is the moment when the computer starts acting like a person without caffeine: technically alive, but very unhappy.

The second limitation is write semantics. WriteFile is about “write everything”, not “append a bit at the end”. If you need a log or a format like “add new records line by line”, WriteFile quickly turns into a weird ritual: each time, read everything, add one line, and write everything back.

The third limitation is robustness. Any file write is a potential risk of “didn’t finish writing, and the file became empty/corrupted”. We won’t unpack this topic in depth today (otherwise we’ll suddenly end up discussing reliable-write protocols), but keep this in mind: WriteFile is convenient, but “convenient” doesn’t always mean “100% safe”.

7. Integrating ReadFile/WriteFile into a TODO app

Now we’ll do what’s typical in learning projects: add a “store tasks in a file” layer. We’ll store tasks as plain text: one line equals one task. No JSON, no databases, no magic. The goal is to practice file I/O, not reinvent Notion.

Let’s assume the file path is already built (for example, via filepath.Join from the previous lecture). Next, we want two functions: loadTasks and saveTasks.

loadText: read a file and return a string

Let’s start with a super-simple function: read a file entirely as text.

package main

import (
	"fmt"
	"os"
)

func loadText(path string) (string, error) {
	b, err := os.ReadFile(path)
	if err != nil {
		return "", fmt.Errorf("read %q: %w", path, err)
	}
	return string(b), nil
}

Notice %w: we added context but didn’t lose the root cause of the error. This is an important Go habit: errors are “bubbled up” with context, not swept under the rug.

Parse tasks from text: one line equals one task

Now we turn the text into a slice of tasks. Here we use strings.Split and clean up the strings a little.

package main

import (
	"strings"
)

func parseTasks(text string) []string {
	text = strings.TrimSpace(text)
	if text == "" {
		return nil
	}
	return strings.Split(text, "\n")
}

Here we intentionally return nil for an empty result: that’s a normal practice for slices. What matters is that you’re consistent.

saveText: write a string all at once

Writing is also text-based. Notice: the write payload is []byte(text).

package main

import (
	"fmt"
	"os"
)

func saveText(path string, text string) error {
	if err := os.WriteFile(path, []byte(text), 0644); err != nil {
		return fmt.Errorf("write %q: %w", path, err)
	}
	return nil
}

Join tasks back into text

To save tasks, we need to assemble them back into a single string. It’s straightforward.

package main

import (
	"strings"
)

func formatTasks(tasks []string) string {
	if len(tasks) == 0 {
		return ""
	}
	return strings.Join(tasks, "\n") + "\n"
}

Adding "\n" at the end is a small “text culture” detail: many tools and people like it when a text file ends with a newline.

8. Mini-scenario: add a task via read → parse → append → write

Now let’s assemble a small piece of “business logic” (even though it sounds grand): add a task and save the file.

package main

import (
	"fmt"
)

func addTask(path string, title string) error {
	text, err := loadText(path)
	if err != nil {
		return err
	}

	tasks := parseTasks(text)
	tasks = append(tasks, title)

	return saveText(path, formatTasks(tasks))
}

There’s a very important idea here: we treat the file as the “application state”. We loaded the state, changed it, saved it. It’s like a game where you load a save, pick up a new item, and save again.

For now, we’re not discussing “what if the file doesn’t exist” or “what if permissions aren’t enough”—we will handle that, of course, but we’re not turning every function into a 40-page novel full of error drama. At this stage, what matters is practicing the overall approach of reading and writing everything at once.

Data flow diagram

When code starts growing, it helps the brain to see the “pipe” through which data flows. Here’s the diagram of our approach:

flowchart TD
    A[File on disk] -->|os.ReadFile| B["[]byte"]
    B -->|"string(b)"| C[Text]
    C -->|parseTasks| D["[]string tasks"]
    D -->|append/changes| E["[]string tasks"]
    E -->|formatTasks| F[Text]
    F -->|"[]byte(text)"| G["[]byte"]
    G -->|os.WriteFile| H[File on disk]

If you can look at this diagram and understand what’s happening at each step, you’re learning the topic the right way.

9. Common mistakes

Mistake #1: using os.ReadFile “just in case” for large files.
When you’ve just learned ReadFile, it can seem universal: one line and you have the whole file. The problem is that “the whole file” can be too big. As a result, the program starts consuming lots of memory and suddenly becomes very slow or crashes. If you can’t guarantee the size in advance, that’s a signal the “read everything” approach can be dangerous.

Mistake #2: expecting os.WriteFile to “append to the end”.
Beginners often hear WriteFile as “write to a file” in general. But in reality it’s “write everything” with an overwrite of the contents. If you need to “append a line to the end”, WriteFile is not your tool: it will recreate the file and can wipe what was there. Even if you read the file beforehand, that’s extra work and added risk of mistakes.

Mistake #3: forgetting the string ↔ []byte conversions.
ReadFile returns []byte, not string. WriteFile accepts []byte, not string. Because of this, people sometimes write weird constructions or try to “stuff a string in somehow”. The correct way is simple and honest: text := string(b) when reading text and []byte(text) when writing.

Mistake #4: losing error context when returning it upward.
If you return an error as just return err, then at the top level you’ll see “permission denied” without understanding “where exactly and what we were trying to do”. It’s better to add context via fmt.Errorf("read %q: %w", path, err)—that way the error remains recognizable and also becomes clearer to a human. The habit of writing explicit checks like if err != nil and propagating errors with context is a core part of Go style.

Mistake #5: assuming a write always “finished and got saved”.
Sometimes it feels like: “well, WriteFile returned nil, so it’s rock-solid”. In practice, writing to disk can be a source of many problems: the path might be wrong, the directory might not exist, permissions might be insufficient, the disk might be full. We’re not turning this lecture into an encyclopedia of file disasters, but it’s important to remember: writing is an operation that can honestly return an error, and you must handle it rather than ignore it.

1

Task

Go SELF, level 39, lesson 2

Locked

Byte counter

Byte counter

Open

1

Task

Go SELF, level 39, lesson 2

Locked

Text Under the Magnifying Glass

Text Under the Magnifying Glass

Open

1

Task

Go SELF, level 39, lesson 2

Locked

Document framing

Document framing

Open

1

Task

Go SELF, level 39, lesson 2

Locked

File report

File report

Open