Java Programming

Java I/O and NIO.2: Streams, Readers, Path, Files, Channels and Buffers

Intermediate8 min readUpdated

Java has two file APIs that coexist: the original java.io streams and readers, and the NIO.2 Path/Files API that replaced java.io.File. This guide explains the byte and character stream model and decorators, how to read and write files idiomatically with Files, how to walk, watch and copy directories, when channels and buffers or memory-mapped files are worth it, and how encodings and resource handling go wrong. You will be able to pick the right API and use it without leaking handles or corrupting text.

What you will learn

  • Explain the difference between byte streams, character streams and buffering decorators
  • Read, write, copy, move and delete files with java.nio.file.Files and Path
  • Walk directory trees and react to filesystem changes with Files.walk and WatchService
  • Handle character encodings explicitly and avoid platform-default surprises
  • Decide when channels, ByteBuffer and memory-mapped files are worth the complexity

Reading and writing files is where a program meets the operating system, and the JDK exposes that boundary through two generations of API. java.io (1996) gives you streams and readers — composable, simple, still the basis of every network and archive library. java.nio.file, known as NIO.2 (Java 7), gives you Path and Files, a proper model of the filesystem with attributes, links, directory walking and change notification. Modern code uses Files for files and the stream abstractions for everything that flows. This guide covers both, with the encoding and resource rules that separate robust I/O from code that works until it meets a real filesystem. Code targets Java 21.

The stream model in java.io

java.io has two parallel hierarchies. Byte streams (InputStream, OutputStream) move raw bytes and are the right tool for binary data. Character streams (Reader, Writer) move decoded text. The bridge between them is a charset: InputStreamReader decodes bytes into characters, OutputStreamWriter encodes the other way.

Each hierarchy has sources and sinks (FileInputStream, ByteArrayOutputStream, Socket.getInputStream()) and decorators that wrap another stream to add behaviour. This is the Decorator pattern in its natural habitat:

DecoratorAdds
BufferedInputStream / BufferedReaderAn in-memory buffer, so the OS is called once per block instead of once per byte; BufferedReader adds readLine()
DataInputStream / DataOutputStreamPrimitive types in a portable binary layout
ObjectInputStream / ObjectOutputStreamJava serialisation (avoid for anything crossing a trust boundary)
GZIPInputStream / InflaterInputStreamDecompression
PrintWriter / PrintStreamFormatted text output with println and printf
try (InputStream raw = new FileInputStream("archive.gz");
     InputStream gunzip = new GZIPInputStream(raw);
     Reader decoded = new InputStreamReader(gunzip, StandardCharsets.UTF_8);
     BufferedReader lines = new BufferedReader(decoded)) {
    String line;
    while ((line = lines.readLine()) != null) {
        process(line);
    }
}

Reading right to left: a file, decompressed, decoded as UTF-8, buffered, delivered line by line. Closing the outermost stream closes the chain. Unbuffered reads of a FileInputStream one byte at a time are one of the classic ways to make Java look slow; a buffer of 8 KB (the default) fixes it.

The read(byte[]) method returns the number of bytes actually read, which may be fewer than requested, and -1 at end of stream. Code that ignores the return value is wrong; in practice you should use readAllBytes(), readNBytes(n) or transferTo(out) (Java 9), which handle the loop for you.

Path: an immutable filesystem address

A Path is a value describing a location — it need not exist. Create one with Path.of("data", "in.csv") (Java 11) or Paths.get(...), and manipulate it with methods that return new paths:

Path base = Path.of("/var/app");
Path config = base.resolve("conf/app.yml");          // /var/app/conf/app.yml
Path parent = config.getParent();                    // /var/app/conf
Path name = config.getFileName();                    // app.yml
Path rel = base.relativize(config);                  // conf/app.yml
Path clean = Path.of("/var/app/../log/./x").normalize();   // /var/log/x
Path abs = Path.of("x").toAbsolutePath();            // resolved against the working directory
Path real = config.toRealPath();                     // follows symlinks; throws if it does not exist

resolve joins; relativize is its inverse; normalize removes . and .. lexically (without touching the disk); toRealPath asks the filesystem. Path implements Comparable and Iterable<Path> over its name elements. Separators are handled by the FileSystem the path belongs to, so the same code runs on Windows and Linux — Path.of("a", "b") is the portable spelling.

Never build paths by string concatenation from user input. base.resolve(userInput).normalize() followed by a check that the result still startsWith(base) is the minimum defence against ../../etc/passwd traversal.

Files: the operations you use every day

Files is a class of static methods over Path. It throws IOException (usually a specific subtype: NoSuchFileException, AccessDeniedException, FileAlreadyExistsException) instead of returning false, and it covers almost everything the old File could not.

Path file = Path.of("report.txt");

// Whole-file text (Java 11); always UTF-8 unless you pass another charset
String text = Files.readString(file);
Files.writeString(file, "hello\n", StandardOpenOption.CREATE, StandardOpenOption.APPEND);

// Whole-file lines and bytes
List<String> lines = Files.readAllLines(file);
byte[] bytes = Files.readAllBytes(file);
Files.write(file, List.of("a", "b"));

// Streaming, for large files: closes the file only when the stream is closed
try (Stream<String> stream = Files.lines(file)) {
    long errors = stream.filter(l -> l.contains("ERROR")).count();
}

// Buffered reader/writer, when you want the classic loop
try (BufferedWriter w = Files.newBufferedWriter(file, StandardCharsets.UTF_8)) {
    w.write("first line");
    w.newLine();
}

// Metadata and existence
boolean exists = Files.exists(file);
long size = Files.size(file);
Instant modified = Files.getLastModifiedTime(file).toInstant();
boolean dir = Files.isDirectory(file);

// Structure
Files.createDirectories(Path.of("out/reports"));          // mkdir -p, no error if present
Files.copy(file, Path.of("backup.txt"), StandardCopyOption.REPLACE_EXISTING);
Files.move(file, Path.of("archive/report.txt"), StandardCopyOption.ATOMIC_MOVE);
Files.deleteIfExists(Path.of("tmp.txt"));
Path temp = Files.createTempFile("upload-", ".bin");

StandardOpenOption controls how a file is opened: CREATE, CREATE_NEW (fail if it exists), APPEND, TRUNCATE_EXISTING, WRITE. The write methods default to create-or-truncate. ATOMIC_MOVE guarantees the rename is all-or-nothing on the same filesystem, which is how you publish a file safely: write to a temporary name, then move.

Prefer the Files methods over constructing FileInputStreams directly. They pick sensible buffer sizes, handle charsets uniformly, and newInputStream/newOutputStream exist when you need a raw stream for another API.

Files.readAttributes(path, BasicFileAttributes.class) returns size, timestamps and type in one system call, which matters when scanning thousands of files. PosixFileAttributes adds owner, group and permission bits on Unix; Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rw-r-----")) sets them. Symbolic links are followed by default; pass LinkOption.NOFOLLOW_LINKS to inspect the link itself, and use Files.createSymbolicLink and readSymbolicLink to manage them.

Directories: listing, walking, watching

Files.list(dir) streams the direct children; Files.walk(dir) streams the whole tree depth-first; Files.find(dir, depth, matcher) filters with attributes in hand. All three return a Stream<Path> backed by an open directory handle, so wrap them in try-with-resources.

try (Stream<Path> paths = Files.walk(Path.of("src"))) {
    List<Path> javaFiles = paths
            .filter(Files::isRegularFile)
            .filter(p -> p.toString().endsWith(".java"))
            .toList();
}

// Glob matching, the shell way
PathMatcher logs = FileSystems.getDefault().getPathMatcher("glob:**/*.log");

Files.walkFileTree(start, visitor) is the older, more controllable form: a FileVisitor gets callbacks before and after each directory and for each file and failure, can skip subtrees, and is the right tool for recursive delete or copy where errors must be handled per file.

WatchService delivers events when entries in a registered directory are created, modified or deleted. It is the basis of "reload configuration when the file changes" and file-drop integrations:

try (WatchService watcher = FileSystems.getDefault().newWatchService()) {
    Path inbox = Path.of("inbox");
    inbox.register(watcher, StandardWatchEventKinds.ENTRY_CREATE);
    while (true) {
        WatchKey key = watcher.take();                          // blocks until an event
        for (WatchEvent<?> event : key.pollEvents()) {
            Path created = inbox.resolve((Path) event.context());
            handleNewFile(created);
        }
        if (!key.reset()) break;                                // directory no longer accessible
    }
}

Two caveats from the specification: registration is per directory, not recursive (walk the tree and register each subdirectory), and the macOS implementation has historically polled rather than used native events, so latency there is seconds rather than milliseconds. An ENTRY_CREATE event can fire before the writer has finished; wait for a companion ENTRY_MODIFY or for the size to stop changing before processing.

Character encodings

Text is bytes plus a charset, and getting the charset wrong produces mojibake or an exception far from the cause. Until Java 17, every API that omitted the charset used the platform default — UTF-8 on Linux, Windows-1252 on many Windows machines — so the same JAR behaved differently on different servers. JEP 400 made UTF-8 the default in Java 18 for new FileReader, new InputStreamReader(in), String.getBytes(), Scanner and the other charset-less APIs (the Files text methods had always defaulted to UTF-8). Two exceptions remain: System.out/System.err and Console use the terminal's encoding, and code may still be compiled with -Dfile.encoding=COMPAT to restore the old behaviour.

Be explicit anyway. StandardCharsets.UTF_8 costs nothing to write and makes intent visible to the reader. For files that arrive in other encodings, name them: Files.newBufferedReader(path, Charset.forName("windows-1252")). When you decode bytes that are not valid in the charset, the default is to replace them with U+FFFD; configure a CharsetDecoder with CodingErrorAction.REPORT when you would rather fail.

A related detail: String.length() counts UTF-16 code units, not characters, so an emoji has length 2. Use codePointCount when the difference matters, and the Base64 tool or HexFormat (Java 17) when you need to look at raw bytes while debugging.

Channels, buffers and memory-mapped files

java.nio (Java 1.4) introduced channels — bidirectional connections to a file or socket — and buffers, fixed-size containers with a position, limit and capacity. A FileChannel supports what streams cannot: reading and writing at an arbitrary position, locking regions, transferring directly to another channel, and mapping the file into memory.

try (FileChannel channel = FileChannel.open(Path.of("data.bin"), StandardOpenOption.READ)) {
    ByteBuffer header = ByteBuffer.allocate(16);
    channel.read(header, 0);                     // positional read, does not move the channel position
    header.flip();                               // switch from writing into the buffer to reading from it
    int magic = header.getInt();
    long recordCount = header.getLong();

    // Map the rest for random access without copying into the heap
    MappedByteBuffer data = channel.map(FileChannel.MapMode.READ_ONLY, 16, channel.size() - 16);
    byte first = data.get(0);
}

// Zero-copy file to socket or file to file
try (FileChannel in = FileChannel.open(src); FileChannel out = FileChannel.open(dst, StandardOpenOption.WRITE, StandardOpenOption.CREATE)) {
    long position = 0, size = in.size();
    while (position < size) position += in.transferTo(position, size - position, out);
}

The flip()/clear()/compact() dance on ByteBuffer is where bugs live: forgetting flip() after filling a buffer reads garbage from the end. Direct buffers (allocateDirect) live outside the heap and avoid a copy on I/O, at the cost of slower allocation; use them for long-lived buffers, not per-request. Memory-mapped files are excellent for large read-mostly datasets and terrible for small files (the mapping cost dominates), and there is no portable way to unmap one before the buffer is garbage-collected.

Non-blocking sockets with Selector are also part of java.nio, but application code rarely touches them: Netty, the JDK HttpClient and every server framework handle that layer. For files, AsynchronousFileChannel exists but adds little over a thread pool — and with virtual threads, plain blocking file I/O on a virtual thread is the simpler way to get concurrency.

Resource handling and errors

Every open file, channel, directory stream and Files.lines stream holds an operating-system handle. Processes have a limit (often 1,024 or 65,536 on Linux), and a leak of one handle per request turns into IOException: Too many open files a few hours after deployment. The rule is mechanical: anything that implements AutoCloseable goes in a try-with-resources, as the exceptions guide explains. Static analysis tools flag the omission; enable that rule.

IOException is checked, and most callers cannot recover from it. Translate at your boundary into UncheckedIOException with the path in the message — new UncheckedIOException("Cannot read " + path, e) — so the log says which file failed. Files already throws subtypes with the path in the message; keep it when you wrap.

Partial writes are the other failure mode. A crash mid-write leaves a truncated file that the next run happily reads. The pattern for durable writes: write to a temporary file in the same directory, force(true) the channel or fsync, then Files.move with ATOMIC_MOVE. Readers see either the old file or the complete new one.

Common mistakes

Unbuffered byte-at-a-time reads. FileInputStream.read() in a loop makes one system call per byte. Wrap in a buffer or use readAllBytes.

Ignoring the return value of read(byte[]). It may fill part of the array. Loop, or use readNBytes/transferTo.

Files.lines, list, walk without try-with-resources. They keep the directory or file open until closed; in a long-running service that is a handle leak.

Default charsets. On Java 17 and earlier, new FileReader(f) uses whatever the OS says. Pass StandardCharsets.UTF_8 everywhere, and set -Dfile.encoding=UTF-8 on old JVMs.

File.delete() returning false silently. Use Files.delete, which throws with a reason.

String-built paths. dir + "/" + name breaks on Windows and enables traversal. Use resolve and validate against the base.

Reading a whole file that might be huge. readAllBytes on a 4 GB log is an OutOfMemoryError. Stream it.

Java serialisation for persistence or transport. Fragile across versions and a well-known remote-code-execution vector. Use JSON, Protobuf or a database.

Key takeaways

  • Byte streams carry binary data, character streams carry decoded text, and decorators such as BufferedReader or GZIPInputStream compose around them; close the outermost one in a try-with-resources.
  • Use Path and Files for all filesystem work; java.io.File is legacy with uninformative failures.
  • Files.readString, writeString, lines, walk, copy, move and createDirectories cover most needs; ATOMIC_MOVE after writing a temp file gives crash-safe updates.
  • Streams returned by Files.lines, list and walk hold handles — always close them.
  • Name the charset explicitly, even though UTF-8 is the default since Java 18.
  • Channels and ByteBuffer earn their complexity for positional access, locking, memory mapping and zero-copy transfer; for everything else, the Files methods are simpler and fast enough.

Sources and further reading

Frequently asked questions

Should I use java.io.File or java.nio.file.Path?
Path and Files, for anything new. File methods return false on failure with no reason, cannot read symbolic links or file attributes properly, and offer no directory walking or watching. Path is an immutable, filesystem-aware value, and Files throws descriptive IOExceptions. Convert with file.toPath() when a library still hands you a File.
What is the difference between an InputStream and a Reader?
An InputStream delivers raw bytes; a Reader delivers decoded characters. Text must be decoded with a charset, which is what InputStreamReader does between the two. Use streams for binary data such as images, archives and protocol payloads, and readers for text, always naming the charset.
Is Files.lines the same as Files.readAllLines?
No. readAllLines loads the whole file into a List<String>; lines returns a lazy Stream<String> that reads on demand and must be closed with try-with-resources because it holds the file open. Use lines for large files or when you stop early, readAllLines for small files you need in memory anyway.
When do I need channels and ByteBuffer instead of streams?
Rarely. Channels give you random access, file locking, memory mapping, scatter/gather and transferTo, and are the base for non-blocking network I/O. For ordinary file reading and writing, the Files convenience methods and buffered streams are simpler and just as fast. Reach for FileChannel when you need positional reads, mmap of a large file, or zero-copy transfer between channels.