filepath.WalkDir: recursive traversal and safety

1. When you need recursive traversal

If you are just starting out, it seems logical: “Well, I already know os.ReadDir, so I’ll make a loop, and if there’s a folder there, I’ll call ReadDir again…”. And yes, you can do that. Exactly once. After that, your code will start growing checks, depth limits, path collection, error handling, and then you will accidentally forget a return in one place—and you’ll get a very creative infinity loop.

Recursive traversal is constantly needed in real tasks. For example, you want to find all files with the .log extension inside ./logs, or count how many “junk” temporary files were left after a crash, or build a list of project files (but carefully—without crawling into .git and vendor). All of these are the same pattern: “there is a root, there is a tree, you need to walk all nodes”.

filepath.WalkDir exists precisely so you don’t turn your main.go into an 800-page novel with a sad ending.

2. filepath.WalkDir: what it does and how it works

filepath.WalkDir(root, fn) walks the file system tree starting at root and calls your function fn for every object it finds: both directories and files, including root itself. The signature looks like this: func WalkDir(root string, fn fs.WalkDirFunc) error.

A useful guarantee: the walk is performed in lexicographic order (by name). This makes output deterministic (reproducible), but it also means that for each directory it needs to read the entire directory contents before it can go deeper. In most learning and practical scenarios, this behavior is a real gift.

And a very important point about safety and predictability: WalkDir does not follow symbolic links. This reduces the risk of infinite traversal loops (when a link points “upwards” or forms a cycle) and often makes the behavior easier to understand.

Callback: “they call you when they find a file”

When you first see WalkDir(".", func(...) error { ... }), your brain sometimes says: “Wait… hold on… who is calling whom?”. That’s a normal reaction. This is the “inversion of control” model: you don’t walk the tree and invoke the handler; the standard library walks the tree and calls your function at each step.

Imagine you hired a courier to visit every apartment in a building. You don’t go yourself—you just left instructions: “When you find a door, call me, tell me the number and what’s written there.” WalkDir is the courier, and fn is your phone call.

As a schematic, it looks like this:

flowchart TD
    A["filepath.WalkDir(root, fn)"] --> B["finds path"]
    B --> C["calls fn(path, d, err)"]
    C --> D["fn returns nil/SkipDir/SkipAll/error"]
    D --> E["WalkDir decides: continue, skip, stop"]

The main idea: your function fn is the set of rules for each discovered object.

Contract of fs.WalkDirFunc: path, d, err

The handler function has the type fs.WalkDirFunc. The docs say it is called for every file/directory, and the return value controls how traversal continues.

The key trio of parameters is:

path string

d fs.DirEntry

err error

The most important rule in practice sounds boring, but it saves your nerves: check err at the beginning of the handler. The docs explicitly say that err reports an access error for path, and the function decides what to do: return the error (stop the walk) or proceed differently.

And this is where Go’s philosophy comes in: “errors are values; you handle them like data, not like a catastrophe.” In WalkDir, you feel that especially clearly: an error does not necessarily mean “everything is lost”; sometimes it just means “this folder couldn’t be opened”.

3. Basic examples: print and filter

The smallest example: print everything you find

It’s best to start with code that does one clear thing: prints all paths. Yes, it looks like the “well-known ls -R”, but it’s perfect muscle training.

package main

import (
	"fmt"
	"io/fs"
	"path/filepath"
)

func main() {
	err := filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return err
		}
		fmt.Println(path)
		return nil
	})
	if err != nil {
		fmt.Println("walk:", err)
	}
}

Here you see the basic form: one call to WalkDir and one handler function. And note: WalkDir returns an error, meaning if you return a “real” error inside the handler, the walk will stop and the error will “bubble out”. This is a direct consequence of the contract: not magic, just a normal return value.

“Files only” and “directories only”: simple filtering

Once you can print everything, the next natural step is to print only what you need. For example, only files (not directories). This is especially useful when you do analysis: “find all .tmp”, “find all .log”, “find all files bigger than N” (size later; today, no overkill).

package main

import (
	"fmt"
	"io/fs"
	"path/filepath"
)

func main() {
	_ = filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return nil // for now, we just skip problematic spots
		}
		if d.IsDir() {
			return nil
		}
		fmt.Println("file:", path) // file: ./main.go
		return nil
	})
}

It’s important to feel the idea here: d.IsDir() is a fast “branch switch”. We don’t need os.Stat for each object, because WalkDir gives you a DirEntry. That’s the whole point of “lightweight metadata”.

4. Controlling the walk and handling problems

SkipDir and SkipAll: not errors, but control signals

In everyday life, the word “error” sounds like “uh-oh”. In WalkDir, there are two special values that formally have type error, but semantically they are control commands.

filepath and io/fs describe SkipDir and SkipAll as values used as a return value from the handler to skip a directory or stop the walk.

It’s convenient to keep it in your head as a small table:

nil

filepath.SkipDir

fs.SkipAll / filepath.SkipAll

any other error

Let’s make a practical example: skip .git, because in a learning project it is usually not needed for analysis, and there are lots of files in it.

package main

import (
	"io/fs"
	"path/filepath"
)

func main() {
	_ = filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return err
		}
		if d.IsDir() && d.Name() == ".git" {
			return filepath.SkipDir
		}
		return nil
	})
}

Notice an important detail: it’s better to filter by d.Name() rather than by a “magic substring” in path. That way, you are less likely to skip the wrong thing by accident.

Errors in err: part of “safe” behavior

In a file system, the word “security” often means not only “protection from an attacker”, but also “protection from chaos”. For a file system, chaos is a normal state: a file might disappear, permissions might change, a directory might become inaccessible, and an antivirus might decide your .tmp looks suspicious (yes, that happens, and it won’t apologize).

WalkDirFunc describes that WalkDir calls the handler with a non-nil err at least when there are problems with Stat for the root or problems reading a directory. And there is a very “teaching-oriented” detail: the handler may be called before reading a directory so you can return SkipDir and avoid the read attempt altogether, and then again—if the read failed—to report the error.

Practical takeaway: if you ignore err, you ignore reality. And then reality ignores your code and sets you up for a panic… well, okay, more often it just produces the wrong result.

The safest beginner-friendly template inside the handler is usually:

if err != nil {
    // either return err (strict)
    // or return nil (lenient, but the result may be incomplete)
}

And you always choose a strategy explicitly.

5. Path safety: symlink and “grounding” user input

Recursive traversal is also connected to another category of “security”: what happens if the path came from the user? For example, your CLI tool accepts a folder to analyze: todo scan <dir>.

There are two different problems here.

The first is loops and unexpected jumps through the tree via symbolic links. This is partially solved by the library itself: filepath.WalkDir does not follow symbolic links. This is not absolute “protection from everything”, but it already reduces the chance that you “wander off” somewhere unexpected.

The second is string-level path traversal: when the user tries to slip in something like ../../.. or an absolute path. One useful “lexical filter” is filepath.IsLocal. It checks that a path is not absolute, not empty, and lexically stays within the base directory when you do Join(base, path). It also emphasizes that the check is purely lexical and does not take symbolic links in the file system into account.

Here is a minimal validation for an argument you want to interpret as “a path inside the working directory”:

package main

import (
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	p := os.Args[1]

	if !filepath.IsLocal(p) {
		fmt.Println("path must be local:", p)
		return
	}

	fmt.Println("ok:", p) // ok: data
}

This does not turn your tool into an “invulnerable fortress”, but it already prevents the user from accidentally (or intentionally) sending you to scan C:\Windows or /etc if you didn’t plan that.

6. Mini-project: scan command for the data folder

To make the examples more than theory, let’s imagine we already have a learning CLI app for tasks (let’s call it todo). It stores data in the ./data directory, and sometimes (after experiments or crashes) temporary .tmp files or backup .bak files may remain there. The scan command will print suspicious files without touching the contents—names only.

Let’s make the command dispatcher as simple as possible (yes, flag will come later—right now we do it by hand, like in the good old days):

package main

import (
	"fmt"
	"os"
)

func main() {
	if len(os.Args) < 2 {
		fmt.Println("usage: todo <cmd>")
		return
	}

	switch os.Args[1] {
	case "scan":
		scanDataDir()
	default:
		fmt.Println("unknown command:", os.Args[1])
	}
}

Now the traversal function itself. Here we use WalkDir, skip directories named .git (in case someone runs the scan from the repository root), and print files with the .tmp or .bak extension.

package main

import (
	"fmt"
	"io/fs"
	"path/filepath"
	"strings"
)

func scanDataDir() {
	root := "data"

	_ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return nil // lenient: skip the problematic spot
		}
		if d.IsDir() && d.Name() == ".git" {
			return filepath.SkipDir
		}
		if d.IsDir() {
			return nil
		}

		if strings.HasSuffix(d.Name(), ".tmp") || strings.HasSuffix(d.Name(), ".bak") {
			fmt.Println("suspicious:", path) // suspicious: data/tasks.bak
		}
		return nil
	})
}

Here you see the “skeleton” of a real traversal: early returns, checking err, checking for a directory, then simple name-based filtering. The trick is that this code is easy to extend: later you can not only print, but also collect results into a slice, sort them, and output a nice table. But today we are not jumping into the next lecture—our goal is to understand the mechanics of WalkDir.

7. Common mistakes when working with filepath.WalkDir

Mistake #1: ignore err in the handler and use d as if it is always valid.
The err parameter is not there “just because”. It means something went wrong for this path: no permissions, the object disappeared, the directory can’t be read. If you keep working as if there is no error, you get either an incorrect result or weird failures in your own code rather than in the standard library.

Mistake #2: return a regular error where you only wanted to “not enter a folder”.
Beginners sometimes write return fmt.Errorf("skip") hoping the walk will simply skip a directory. But by the contract, that means “stop the walk and return an error to the caller”. If you want to skip a directory, you must return filepath.SkipDir (or fs.SkipDir) because it is a special control signal.

Mistake #3: check that a directory is “not needed” using a substring in path rather than the name d.Name().
Checks like strings.Contains(path, "git") almost always lead to surprises: you might accidentally skip my-git-notes.txt, or fail to skip .git due to Windows path peculiarities. Checking d.Name() is usually simpler and more accurate, because the name is the name of the current element, without the “noisy” parts of the path.

Mistake #4: confuse “OS paths” and “paths inside an abstract FS”, then wonder about slashes.
filepath.WalkDir works with paths in the operating system’s format (the separator depends on the platform). This is explicitly noted in the docs: unlike io/fs.WalkDir, where paths always use forward slashes. If you start concatenating paths “by hand” using /, Windows will give you a cultural shock.

Mistake #5: assume WalkDir is a “safe sandbox” for a user-supplied path.
The fact that WalkDir does not follow symlinks is already useful, but it does not mean you can accept any path from the user and mindlessly walk the entire disk. If the path must be “local” relative to your working directory, use a lexical check like filepath.IsLocal, and remember it does not account for symbolic links and works specifically as a lexical filter.

1

Task

Go SELF, level 41, lesson 2

Locked

Traversal log

Traversal log

Open

1

Task

Go SELF, level 41, lesson 2

Locked

Traversal counters

Traversal counters

Open

1

Task

Go SELF, level 41, lesson 2

Locked

Hunting for an extension

Hunting for an extension

Open

1

Task

Go SELF, level 41, lesson 2

Locked

Safe subdirectory

Safe subdirectory

Open