i/o streams

the picture

you know that line everyone copies from stack overflow, new BufferedReader(new InputStreamReader(new FileInputStream(f)))? three constructors deep, just to read a file. kinda cursed. but every layer is doing a different job, and once you see the jobs it stops looking silly.

think of plumbing. the file is the water main (raw bytes, whatever's in the pipe). FileInputStream is the tap (the only part connected to the outside world, the OS). InputStreamReader is a filter screwed onto the tap (turns bytes into chars using a charset). BufferedReader is a tank on top (fills up in big gulps so you can take a cup at a time). you drink from the outermost piece. it pulls from the one inside it, which pulls from the one inside that.

here's the code that figure is running. it really does print what the stepper says:

import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
 
void main() throws IOException {
    Path file = Files.createTempFile("onion", ".txt");
    Files.write(file, new byte[] { 0x68, 0x69, 0x0a, (byte) 0xc3, (byte) 0xa9 });
 
    try (var in = new BufferedReader(
            new InputStreamReader(
                new FileInputStream(file.toFile()), StandardCharsets.UTF_8))) {
        String line;
        while ((line = in.readLine()) != null) {
            IO.println(line + " (" + line.length() + " chars)");
        }
    }
    Files.delete(file);
}

hi (2 chars)
é (1 chars)

5 bytes in, 4 chars out, 2 lines. é took two bytes and became one char. that gap between bytes and chars gets its own chapter: charsets and encoding.

two families

java.io is two parallel trees. byte streams (InputStream, OutputStream) move raw bytes and don't care what they mean: images, zip files, class files, network packets. char streams (Reader, Writer) move chars and always have a charset somewhere underneath doing the translation.

the rule for picking one is short:

source / sink

a stream that talks to something real: a file, a socket, a byte array. you can't wrap one around another stream.

decorator

a stream whose constructor takes another stream. it forwards every call and adds one behaviour on the way.

bridge

InputStreamReader and OutputStreamWriter: the only places a byte stream becomes a char stream (or back). the charset lives here.

the read loop

the most basic call there is: int read(). one byte, please. it returns an int, not a byte, and that's not an accident.

import java.io.*;
 
void main() throws IOException {
    var in = new ByteArrayInputStream(new byte[] { 65, -1, 10 });
    int b;
    while ((b = in.read()) != -1) {
        IO.print(b + " ");
    }
    IO.println("| then " + in.read());
}

65 255 10 | then -1

see the 255? the byte was -1 (all eight bits set, 0xFF), but read() hands it back as an unsigned value from 0 to 255. that leaves -1 free to mean one thing only: "the stream is over". if read() returned a byte, a file containing 0xFF would look exactly like the end of the file.

byte b;
while ((b = (byte) in.read()) != -1) {
    out.write(b);
}

the bulk version, read(byte[] buf), fills as much of the array as it can and returns how many bytes it actually put there. it can be fewer than you asked for, even mid-file (sockets do this all the time). so the loop always uses the count:

byte[] buf = new byte[8192];
int n;
while ((n = in.read(buf)) != -1) {
    out.write(buf, 0, n);   // only the first n bytes are new
}

you rarely write that loop anymore: in.transferTo(out) java 9+ does exactly this.

why buffering matters

a FileInputStream has no memory. every read() goes down into the OS as a system call, and a system call is expensive: the CPU switches into the kernel, does the work, and switches back. read a file one byte at a time with no buffer and you pay that toll once per byte.

BufferedInputStream fixes it with one array. the first read() asks the stream inside for 8192 bytes in one go, and the next 8191 read()s are just array lookups.

notice the curve flattens fast. going from 512 bytes to 64 KiB barely moves the time, because the syscalls are already rare and what's left is the cost of calling read() a million times. the fix for that is reading in chunks yourself (read(byte[]), readAllBytes()), or dropping down to buffers and channels.

try (var in = new FileInputStream(f)) {
    int b;
    while ((b = in.read()) != -1) count(b);
}

try (var in = new BufferedInputStream(
        new FileInputStream(f))) {
    int b;
    while ((b = in.read()) != -1) count(b);
}

closing and flushing

closing the outermost layer closes everything inside it. that's the other reason the onion works: you hold one reference, you close one thing. use try-with-resources and stop thinking about it.

output buffers have the opposite problem. a BufferedWriter holds your text in memory until the buffer fills, you call flush(), or you close(). until then, the file doesn't have it.

import java.io.*;
import java.nio.file.*;
 
void main() throws IOException {
    Path file = Files.createTempFile("log", ".txt");
 
    var out = new BufferedWriter(new FileWriter(file.toFile()));
    out.write("order 42 shipped\n");
    IO.println("before flush: " + Files.size(file) + " bytes");
 
    out.flush();
    IO.println("after flush:  " + Files.size(file) + " bytes");
 
    out.close();
    Files.delete(file);
}

before flush: 0 bytes
after flush:  17 bytes

the shortcuts you actually use

most code today never builds the onion by hand. the JDK grew one-liners for the common jobs, and they pick sane defaults (UTF-8, a buffer, closing for you):

import java.io.*;
 
void main() throws IOException {
    var in = new ByteArrayInputStream("hello, streams".getBytes());
    var out = new ByteArrayOutputStream();
    long n = in.transferTo(out);
    IO.println(n + " bytes: " + out);
}

14 bytes: hello, streams

the Files side of this (paths, walking directories, watching for changes) is in files and paths. console input and printf are in reading and printing.

import java.io.*;
 
void main() throws IOException {
    var bytes = new ByteArrayOutputStream();
    try (var out = new DataOutputStream(bytes)) {
        out.writeInt(1000);
        out.writeDouble(0.5);
        out.writeUTF("hé");
    }
    byte[] raw = bytes.toByteArray();
    IO.println(raw.length + " bytes: " + java.util.HexFormat.ofDelimiter(" ").formatHex(raw));
 
    try (var in = new DataInputStream(new ByteArrayInputStream(raw))) {
        IO.println(in.readInt() + " " + in.readDouble() + " " + in.readUTF());
    }
}

17 bytes: 00 00 03 e8 3f e0 00 00 00 00 00 00 00 03 68 c3 a9
1000 0.5 hé

why not one big class?

because the combinations explode. buffered or not, × which charset, × reading numbers or lines, × file or socket or byte array or zip entry. one class per combination would be hundreds of classes. with decorators it's one class per feature, and you compose them like lego. the same BufferedReader works on a file, a socket, a gzip stream or a String, because all it needs is "a Reader inside me".

the price is the onion syntax, and having to know which layer does what. which is this whole chapter :)