1. Introduction
When you write a program, you almost always run it on your own machine, with your own editor, with your own settings. And at that moment it is easy to believe a nice fairy tale: “Well, if the file opened fine for me, it will be fine for everyone.” Alas, the world is bigger than your laptop: there are other OSes, other locale settings, other tools that create files with unexpected parameters. As a result, you get the classic situation: it works on my machine, but on my colleague’s machine, instead of “Hello” there is an incomprehensible set of characters.
It is important to separate two things here. First, some APIs (especially from the Java world) when reading/writing text may use the system “default encoding”, and it can genuinely differ. Second, even if a particular Kotlin method has a reasonable default, it still remains a “hidden agreement”: when you read the code, you do not see what format the file is in. And a file is an external contract, and it is better if that contract is written directly in the code, not “in the author’s head”.
In short, the formula is: an explicit Charset makes behavior reproducible. It is like val instead of “well, I will try not to change the variable”. Not because you are a bad person, but because people forget, and code does not.
2. Charset and Charsets.*: how to choose an encoding in code
If in previous lectures we discussed encoding as an idea (“the rule for converting bytes ↔ text”), now we need to get down to concrete objects and calls. In Kotlin/JVM, an encoding is most often represented by java.nio.charset.Charset, and standard options are convenient to take from Charsets.
The simplest practical habit: put the application encoding into a single constant and use it everywhere at the I/O boundary. That way you do not get the chaos of “here it is UTF-8, here someone forgot, here someone set UTF-16 because ‘it looks nicer’”.
import java.nio.charset.Charset
private val APP_CHARSET: Charset = Charsets.UTF_8
Why this is convenient:
- you choose the file format once and then stick to it,
- if you ever need to change the format, you change one line (and consciously accept the consequences),
- the code becomes self-documenting: you can see that the file is expected to be UTF-8.
A small note about ByteArray: when we read a file as bytes, we usually put them into a ByteArray—that is a “primitive byte array” in Kotlin, without wrapping into objects. It is the basic container for the raw contents of a file.
3. Reading and writing text with an explicit Charset
When you are just starting out, you want one universal recipe: “Tell me which one function to learn, and I will go live my life.” Good news: at the beginning, you really only need two methods—“read the entire file into a string” and “write a whole string”. Bad news: life is more diverse, and sometimes you need to read line by line or write carefully via a writer. But let’s start simple.
Imagine we are building a small console app NotesApp that stores notes in a file data/notes.txt. In this course we try to keep examples small but connected, so today we will simply stabilize the read/write layer so that tomorrow and the day after, any parsing logic does not depend on “environment magic”.
Read the entire file as a String (specify the encoding)
import java.io.File
private val APP_CHARSET = Charsets.UTF_8
fun main() {
val file = File("data/notes.txt")
val text = file.readText(APP_CHARSET)
println("Characters in file: ${text.length}") // e.g.: Characters in file: 42
}
Notice an important discipline: readText(APP_CHARSET) is not “extra letters”. It is part of the contract. You are explicitly saying: “my file is UTF-8.” And if it is not UTF-8, that is already useful information: it means the problem is not in your note parsing, but in the format of the input data.
Write a String to a file (specify the encoding)
import java.io.File
private val APP_CHARSET = Charsets.UTF_8
fun main() {
val out = File("data/notes.txt")
out.parentFile?.mkdirs()
out.writeText("First note\nSecond note\n", APP_CHARSET)
println("Wrote file notes.txt") // Wrote file notes.txt
}
Same principle again: you did not just write text, you wrote it in a specific encoding, and that means that other code (or you in a week) can read the file by the same rule.
4. Why the default encoding is a risk
In this section it is important not to turn this into a religion. The goal is not to “ban all defaults” and permanently exile brevity from code. The goal is to remove hidden external dependencies where they actually hurt.
The default encoding is dangerous because:
it depends on the runtime environment, and the environment may differ between your machine, CI, a colleague’s computer, and the server. It may also depend on how exactly the file was created (editor, export from Excel, download from the web). If you “guessed” the encoding by accident, you built your program on sand. It did not break only because you have been lucky so far.
You can check which encoding is considered “default” in the current JVM like this:
import java.nio.charset.Charset
fun main() {
val def = Charset.defaultCharset()
println("Default charset = $def") // e.g.: Default charset = UTF-8
}
This output is useful as diagnostics, but as a development strategy it is weak. More reliable: do not rely on defaults where you read/write external files, and pass Charsets.UTF_8 (or another preselected format).
When it is more convenient to read bytes (readBytes) and decode manually
If a file is “normal and clean”, readText(charset) is ideal. But sometimes you need to step back and take control of the very first stage: bytes. Usually this is when you want to:
carefully diagnose the contents, look at the first bytes, understand “what actually arrived”, or clean/normalize the data before turning it into a string.
Here a simple scheme is helpful (also a mental checklist):
flowchart LR
A[File on disk] --> B[ByteArray]
B --> C["Decoding: String(bytes, charset)"]
C --> D[String processing: replace/trim/split]
D --> E[Application logic]
And code that directly reflects these steps:
import java.io.File
private val APP_CHARSET = Charsets.UTF_8
fun main() {
val file = File("data/notes.txt")
val bytes: ByteArray = file.readBytes()
val text: String = String(bytes, APP_CHARSET)
println(text)
}
Notice: we deliberately use ByteArray as a container for raw data. It is the standard “byte” type in Kotlin that directly corresponds to a byte array.
5. Streaming reads and writes with Charset
Reading a whole file at once is convenient while the file is small. But as soon as the file becomes larger, you want to read it gradually: line by line or in chunks. We are not going into “streaming decoding of gigabytes” yet, but it is useful to learn the basic pattern right now: bufferedReader(charset) / bufferedWriter(charset) + use {}.
The idea is simple: you open a reader/writer, guarantee closing via use, and inside you work with familiar methods.
Read the first non-empty note line
import java.io.File
private val APP_CHARSET = Charsets.UTF_8
fun main() {
val file = File("data/notes.txt")
val firstNonEmpty = file.bufferedReader(APP_CHARSET).use { reader ->
reader.lineSequence()
.map { it.trim() }
.firstOrNull { it.isNotEmpty() }
}
println("First non-empty line: $firstNonEmpty")
}
Yes, there is a Sequence here, but treat it as a “lazy list of lines”: we do not load the whole file into memory; we take lines as needed.
Append a note to the end of the file
import java.io.File
private val APP_CHARSET = Charsets.UTF_8
fun main() {
val file = File("data/notes.txt")
file.parentFile?.mkdirs()
file.appendText("New note\n", APP_CHARSET)
println("Appended a line to notes.txt") // Appended a line to notes.txt
}
Key point: the encoding must match for all operations on the same file. If you wrote the file in UTF-8 once and later appended in a different encoding, you will create a “layer cake” that will be very “fun” to debug when parsing breaks.
6. Practical patterns: transcoding and a mini API
Transcoding a file: two steps, no magic
Sometimes you run into a task like: “I have a file in one encoding, and I need to save it in another.” It is important to understand: transcoding always goes through a string (that is, through the Unicode representation inside the program). It is not copying bytes one-to-one. It is converting text by rules.
The correct scheme:
flowchart LR
A[Bytes in old charset] -->|decode| B[String]
B -->|encode| C[Bytes in new charset]
And a minimal example (without extra complexity):
import java.io.File
fun main() {
val input = File("data/in.txt")
val output = File("data/out.txt")
val text = input.readText(Charsets.UTF_8) // decode
output.writeText(text, Charsets.UTF_16) // encode
println("Done") // Done
}
In real life, you must know what encoding the input file uses; otherwise you are not transcoding—you are “guessing”. We do not cover guessing today: it is a separate hard topic, and it often ends with the program confidently doing the wrong thing.
Mini API: a single source of truth for encoding
When a project grows, the most common beginner problem is “I read it one way here, differently there, and here I forgot entirely.” The best medicine is to create a small utility layer: functions readAppText(...), writeAppText(...) that always use the same encoding and hide it from the rest of the code.
And here a good practical habit helps: group files and functions logically so the code does not turn into a junk drawer. Kotlin recommendations explicitly encourage keeping related things close and giving files meaningful names, instead of “Util.kt” for every case.
Let’s create a file TextFiles.kt (a name that matches the job), and inside it—two functions.
import java.io.File
import java.nio.charset.Charset
private val APP_CHARSET: Charset = Charsets.UTF_8
fun readAppText(file: File): String {
return file.readText(APP_CHARSET)
}
fun writeAppText(file: File, text: String) {
file.parentFile?.mkdirs()
file.writeText(text, APP_CHARSET)
}
Now main in our NotesApp becomes calmer and cleaner:
import java.io.File
fun main() {
val file = File("data/notes.txt")
val current = if (file.exists()) readAppText(file) else ""
val updated = current + "A note added by the program\n"
writeAppText(file, updated)
println("OK, updated notes.txt") // OK, updated notes.txt
}
What we gained:
we separated “the messy external world of files” from the rest of the logic, and also made the encoding part of the infrastructure. As a result, any function that “just works with a string” does not have to know that a file is bytes and that it uses UTF-8. It works with a string—and lives peacefully.
7. Typical mistakes when explicitly specifying Charset
Mistake No. 1: specify the encoding in only one place.
Switching to an explicit Charset often starts with enthusiasm (“I’ll fix everything now!”) and ends with unexpected bugs (“why did it get worse?”). The most common slip is to set the encoding for reading but leave writing “as it was”, or vice versa. Then half the file is processed by one rule, half by another, and you get problems that look random. In practice, the encoding must be consistent at all “bytes ↔ string” boundaries.
Mistake No. 2: try to “re-encode a string” without bytes.
Transcoding is always a pair of actions: decode → encode. If you already have a String, then inside the program it is normalized text in memory, and it has no “built-in encoding”. A string does not store information about which encoding it was read with. The encoding exists only at the moment of conversion between bytes and text. Therefore, “changing the encoding of a string” without working with bytes is a logical error.
Mistake No. 3: confuse the number of characters with the number of bytes.
String.length returns the number of characters (more precisely, Char), while the file size is the number of bytes. For UTF-8 and UTF-16, the difference can be significant: visually you see “6 characters”, but in the file they take more space. Attempts to estimate file size via text.length almost always lead to wrong conclusions. If you need the real size, use file.length() or count bytes.size.
Mistake No. 4: use String(bytes) without explicitly specifying the encoding.
The constructor String(bytes) looks convenient, so it is easy to write it on autopilot. But exactly these spots most often become sources of unexpected problems, because it is not obvious by what rule the bytes are converted into text. The behavior may depend on environment or platform settings. It is much safer to write explicitly: String(bytes, Charsets.UTF_8). A bit longer, but unambiguous and predictable.
GO TO FULL VERSION