Java Programming
The Java Streams API: Pipelines, Collectors and Pitfalls
Intermediate7 min readUpdated
A stream is a pipeline of operations over a sequence of elements, evaluated lazily and driven by a terminal operation. This guide explains the model, walks through the intermediate and terminal operations you will use daily, shows how collectors turn a stream back into data, and lists the mistakes — reusing streams, side effects, careless parallelism — that cause real bugs.
What you will learn
- Explain the stream model: source, lazy intermediate operations, terminal operation
- Choose the right operation for filtering, mapping, flattening, sorting and reducing
- Use Collectors to group, partition, join and summarise data
- Decide when a stream is clearer than a loop and when it is not
- Recognise the common pitfalls, including when parallel streams make things slower
Before Java 8, transforming a collection meant a loop, a temporary list, and a fair amount of ceremony that obscured what the code was actually doing. Streams replaced that with a pipeline: describe what you want — filter these, transform those, sum the result — and let the library handle the iteration. Used well, stream code reads like a specification. Used carelessly, it becomes slower and harder to debug than the loop it replaced. This guide covers both sides.
The stream model
A stream is not a data structure. It holds no elements of its own; it is a view over a source — a collection, an array, a generator function, a file's lines — through which elements flow one at a time. A stream pipeline has three parts:
- A source that produces elements:
list.stream(),Arrays.stream(array),Stream.of(a, b, c),IntStream.range(0, 10),Files.lines(path). - Zero or more intermediate operations that return a new stream:
filter,map,sorted,distinct,limit. These are lazy — nothing runs when you call them. - One terminal operation that produces a result or a side effect and triggers the whole pipeline:
collect,forEach,count,findFirst,reduce,anyMatch.
List<String> names = List.of("Ada", "Grace", "Linus", "Barbara", "Ken");
List<String> result = names.stream() // source
.filter(n -> n.length() > 3) // intermediate, lazy
.map(String::toUpperCase) // intermediate, lazy
.sorted() // intermediate, lazy
.toList(); // terminal — runs everything
// [BARBARA, GRACE, LINUS]
Laziness matters for two reasons. First, the pipeline can short-circuit: findFirst() after a filter stops as soon as one element passes, without examining the rest. Second, operations are fused: each element flows through filter, then map, then onward, rather than the whole list being filtered into a temporary list and then mapped. Add a peek to see the order:
Stream.of("a", "bb", "ccc")
.peek(s -> System.out.println("filter sees " + s))
.filter(s -> s.length() > 1)
.peek(s -> System.out.println("map sees " + s))
.map(String::length)
.findFirst();
// filter sees a
// filter sees bb
// map sees bb
// (ccc is never examined)
A stream can be consumed once. After the terminal operation runs, the stream is closed; calling another operation on it throws IllegalStateException: stream has already been operated upon or closed. If you need to go over the data twice, create two streams from the source.
Intermediate operations you will actually use
Most day-to-day stream code uses a small vocabulary.
| Operation | What it does | Typical use |
|---|---|---|
filter(Predicate) | Keeps elements matching the predicate | Selecting a subset |
map(Function) | Replaces each element with one new element | Extracting a field, converting type |
flatMap(Function) | Replaces each element with a stream and flattens | Nested collections |
mapMulti(BiConsumer) | Like flatMap without creating an inner stream (Java 16) | Expanding elements cheaply |
distinct() | Removes duplicates by equals | De-duplication |
sorted() / sorted(Comparator) | Sorts; requires buffering every element | Ordering before output |
limit(n) / skip(n) | Truncates or drops a prefix | Pagination, "top N" |
takeWhile / dropWhile | Prefix by predicate (Java 9) | Ordered data |
peek(Consumer) | Runs an action without changing the element | Debugging only |
flatMap is the one people find hard to picture. Suppose each Order has a list of LineItems and you want every line item across all orders:
List<LineItem> allItems = orders.stream()
.flatMap(order -> order.items().stream())
.toList();
map would give you a Stream<List<LineItem>> — a stream of lists. flatMap gives you a Stream<LineItem>.
The primitive specialisations IntStream, LongStream and DoubleStream avoid boxing and add numeric operations such as sum(), average() and summaryStatistics(). Move between them with mapToInt, mapToObj and boxed():
int totalCents = orders.stream()
.flatMap(o -> o.items().stream())
.mapToInt(LineItem::priceCents)
.sum();
Terminal operations and reduction
A terminal operation ends the pipeline. The common ones fall into four groups:
- Collect into something:
collect(Collector),toList()(Java 16),toArray(). - Reduce to one value:
count(),sum(),min(Comparator),max(Comparator),reduce(identity, accumulator). - Search:
findFirst(),findAny(),anyMatch(p),allMatch(p),noneMatch(p). All short-circuit. - Side effects:
forEach(action),forEachOrdered(action).
reduce deserves a closer look because it is the general form of the others. It combines elements pairwise with an associative function, starting from an identity value:
int total = IntStream.of(3, 5, 7).reduce(0, Integer::sum); // 15
Optional<String> longest = words.stream()
.reduce((a, b) -> a.length() >= b.length() ? a : b); // no identity → Optional
The accumulator must be associative and the identity must really be an identity (0 for sum, 1 for product, "" for concatenation). The library relies on this to split the work in parallel streams; get it wrong and a parallel run produces a different answer from a sequential one.
Collectors: turning a stream back into data
Collectors is where streams earn their keep. groupingBy and partitioningBy replace dozens of lines of manual map-building.
record Employee(String name, String department, int salary) {}
// Employees per department
Map<String, List<Employee>> byDept = employees.stream()
.collect(Collectors.groupingBy(Employee::department));
// Average salary per department
Map<String, Double> avgSalary = employees.stream()
.collect(Collectors.groupingBy(Employee::department,
Collectors.averagingInt(Employee::salary)));
// Names per department, as a single comma-separated string
Map<String, String> namesByDept = employees.stream()
.collect(Collectors.groupingBy(Employee::department,
TreeMap::new,
Collectors.mapping(Employee::name, Collectors.joining(", "))));
// Split into high and low earners
Map<Boolean, List<Employee>> highEarners = employees.stream()
.collect(Collectors.partitioningBy(e -> e.salary() > 100_000));
The pattern is always the same: a classifier function chooses the key, and an optional downstream collector decides what happens to the elements that share a key. counting(), summingInt, averagingDouble, mapping, filtering, maxBy, toSet and joining are the downstream collectors you will reach for most. teeing (Java 12) runs two collectors and merges their results, which is handy for computing, say, a minimum and maximum in a single pass.
Two toMap details cause production bugs. First, toMap(keyFn, valueFn) throws IllegalStateException on a duplicate key; pass a merge function as the third argument if duplicates are possible. Second, it rejects null values, so map nulls to a sentinel or filter them out first.
Streams or loops?
A stream is the better choice when the code is a transformation: take this data, filter it, reshape it, aggregate it. The pipeline states the intent and each step is independently readable. A loop is the better choice when:
- You need to break out early on a complex condition that does not fit
takeWhileoranyMatch. - The body has multiple side effects or mutates several variables — streams push you towards
forEachwith captured mutable state, which is the worst of both worlds. - You are handling checked exceptions. The standard functional interfaces cannot throw them, so stream code ends up wrapping every call in a try/catch inside the lambda, which is much noisier than a loop.
- Performance is critical and the collection is small: the per-element overhead of a stream is measurable in tight loops.
A useful rule: if you find yourself writing a stream with a forEach that adds to a list declared outside the stream, rewrite it as collect — or as a loop.
Parallel streams
Calling parallelStream() or .parallel() splits the source into chunks processed on the common ForkJoinPool. It is one line to enable and one line to regret. Parallel streams help when all of these hold:
- The source splits cheaply and evenly (
ArrayList, arrays,IntStream.range— notLinkedListor an iterator). - The work per element is CPU-bound and substantial, not I/O or a trivial arithmetic step.
- There are enough elements — tens of thousands, not dozens.
- Operations are stateless;
sorted,distinctandlimiton an ordered stream force expensive merging.
Otherwise the coordination overhead dominates and the parallel version is slower. There is a second trap: the common pool is shared by the whole JVM. A parallel stream doing blocking I/O starves every other parallel stream and CompletableFuture in the process. Keep blocking work out of parallel streams entirely.
// Reasonable candidate: large array, pure CPU work
long primes = IntStream.rangeClosed(2, 5_000_000).parallel()
.filter(Primes::isPrime)
.count();
// Bad candidate: tiny list, blocking call
customers.parallelStream().forEach(c -> emailService.send(c)); // don't
Common mistakes
Side effects in lambdas. Modifying a shared collection or counter from inside map or filter is a data race under parallel() and confusing even sequentially. Use collectors.
Stateful lambdas. A lambda that depends on what it has seen before (a running total kept in an array cell, for example) breaks the assumption that elements are independent. reduce or a loop is the right tool.
Forgetting the terminal operation. A pipeline of intermediate operations with no terminal call does nothing at all. The code compiles and silently skips the work.
Reusing a stream. Storing a stream in a variable and calling two terminal operations on it throws at runtime.
Sorting when you only need the top element. sorted().findFirst() sorts everything; min(comparator) is a single pass.
Streams over Files.lines without closing. File-backed streams hold an open handle. Wrap them in try-with-resources.
Optional from findFirst handled with get(). Use orElse, orElseThrow or ifPresent; a bare get() on an empty result throws NoSuchElementException.
Where the API is going
Java 22 finalised stream gatherers (JEP 461): Stream.gather(Gatherer) is to intermediate operations what collect is to terminal ones — a way to write your own windowing, folding or scanning steps that plug into the pipeline. The built-in Gatherers.windowFixed, windowSliding, fold and scan cover cases that previously required dropping to a loop. If you are on a current release, learn them before reaching for a hand-rolled iterator.
Key takeaways
- A stream is a lazy pipeline: source → intermediate operations → one terminal operation. Nothing runs until the terminal call.
filter,map,flatMap,sorted,distinctandlimitcover most transformations; primitive streams avoid boxing for numeric work.- Collectors — especially
groupingBywith a downstream collector — replace hand-written map building. Remember the duplicate-key and null rules oftoMap. - Use streams for transformations and loops for early exits, checked exceptions and multi-variable mutation.
- Parallel streams are a performance tool for large, splittable, CPU-bound work only; never put blocking I/O in one.
- Streams are single-use, lambdas must be side-effect-free, and the accumulator you pass to
reducemust be associative.
Sources and further reading
Frequently asked questions
- Are streams faster than for loops?
- Usually not, and often slightly slower for small collections because of the pipeline objects and lambda calls. Streams are about clarity and composability, not raw speed. Parallel streams can be faster for large, CPU-bound, easily splittable work, but they can also be slower; measure before you commit.
- Can I reuse a stream after calling a terminal operation?
- No. A stream can be traversed exactly once. Calling any operation on a stream that has already been consumed throws IllegalStateException. Create a new stream from the source each time, or collect the results into a collection if you need to iterate twice.
- What is the difference between map and flatMap?
- map transforms each element into exactly one new element. flatMap transforms each element into a stream of zero or more elements and concatenates those streams, so it is the tool for flattening nested collections — for example turning a stream of orders into a stream of all their line items.
- When should I use Collectors.toList() versus Stream.toList()?
- Stream.toList(), added in Java 16, returns an unmodifiable list and is the shortest option when you do not need to mutate the result. Collectors.toList() makes no promise about mutability in the specification, although the current implementation returns an ArrayList. Prefer Stream.toList() unless you need a mutable list, in which case use Collectors.toCollection(ArrayList::new) to be explicit.