Spring Boot & Spring Cloud
Spring Batch 5 Fundamentals: Jobs, Steps, Chunks, Restart and Scaling
Intermediate9 min readUpdated
You will be able to design and run a Spring Batch 5 job on Spring Boot 3: model work as jobs and steps, build chunk-oriented steps from readers, processors and writers, pass parameters safely with step scope, make failures skippable or retryable, restart a failed job where it stopped, and choose the right scaling strategy for large volumes.
What you will learn
- Explain the Spring Batch domain model: Job, JobInstance, JobExecution, Step, StepExecution and the JobRepository
- Build a chunk-oriented step with the JobBuilder and StepBuilder APIs of Spring Batch 5
- Read flat files and databases and write with batching using the provided item readers and writers
- Configure skip, retry and restart behaviour and understand what the job repository tables record
- Pass job parameters into step-scoped beans and control flow with conditional transitions
- Select between multi-threaded steps, parallel flows, partitioning and remote chunking
Some work does not fit a request-response model: importing a nightly file with two million rows, recomputing every customer's balance, archiving last quarter's events, sending a monthly statement run. Such jobs need to process large volumes in bounded memory, commit progress incrementally, survive a bad record without discarding the good ones, and resume after a crash instead of starting over. Spring Batch is the framework that encodes those requirements so you do not reinvent them with a for loop and a cron entry. This page covers Spring Batch 5 as auto-configured by Spring Boot 3, using the Java builder API that replaced the older factories and XML.
The domain model
Spring Batch's vocabulary maps directly onto the metadata it stores:
| Concept | Meaning |
|---|---|
| Job | A complete batch process, made of ordered steps; identified by name |
| JobInstance | A logical run of a job for a given set of identifying parameters, e.g. "import-orders for 2026-09-01" |
| JobExecution | One physical attempt to run a JobInstance; a failed instance can have several executions |
| JobParameters | Typed key-value pairs given at launch, part of the instance identity unless marked non-identifying |
| Step | An independent phase of the job with its own transaction handling and status |
| StepExecution | One attempt at a step, with counters: read, written, filtered, skipped, commits, rollbacks |
| ExecutionContext | A persistent map attached to a job or step execution, used to record progress for restart |
| JobRepository | The store for everything above; a set of BATCH_* tables in a relational database |
| JobLauncher / JobOperator | Entry points to start, stop and restart jobs |
The repository is what distinguishes Spring Batch from a scheduler and a loop. Every step execution records how far it got, so a restart of a failed execution begins at the failed step, and a restartable reader picks up at the last committed item.
Boot creates the BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, BATCH_JOB_EXECUTION_PARAMS, BATCH_STEP_EXECUTION and the two *_EXECUTION_CONTEXT tables for embedded databases automatically; for a real database set spring.batch.jdbc.initialize-schema=always once, or better, copy the schema script for your vendor from the spring-batch-core jar into a Flyway migration so the tables are versioned with everything else.
Tasklet steps and chunk-oriented steps
A step is either a tasklet (one unit of work, run once) or chunk-oriented (read items one at a time, optionally process each, write them in groups). Tasklets suit housekeeping: unzip a file, call a stored procedure, delete a directory. Chunks suit everything that processes a stream of records.
@Configuration
class ImportOrdersJobConfig {
@Bean
Job importOrdersJob(JobRepository jobRepository, Step unpackStep, Step importStep) {
return new JobBuilder("importOrders", jobRepository)
.incrementer(new RunIdIncrementer()) // adds run.id: a new instance per launch, unless the last one failed
.start(unpackStep)
.next(importStep)
.build();
}
@Bean
Step unpackStep(JobRepository jobRepository, PlatformTransactionManager tx) {
return new StepBuilder("unpack", jobRepository)
.tasklet((contribution, chunkContext) -> {
Files.createDirectories(Path.of("work"));
return RepeatStatus.FINISHED;
}, tx)
.build();
}
@Bean
Step importStep(JobRepository jobRepository, PlatformTransactionManager tx,
FlatFileItemReader<OrderCsvRow> reader,
ItemProcessor<OrderCsvRow, Order> processor,
JdbcBatchItemWriter<Order> writer) {
return new StepBuilder("import", jobRepository)
.<OrderCsvRow, Order>chunk(500, tx) // 500 items per transaction
.reader(reader)
.processor(processor)
.writer(writer)
.build();
}
}
JobBuilder and StepBuilder take the JobRepository directly; the JobBuilderFactory and StepBuilderFactory from Batch 4 are deprecated since 5.0 and should not appear in new code. The transaction manager is now an explicit argument to chunk() and tasklet(), which makes it obvious which one a step uses when an application has several data sources.
The chunk loop works like this: the reader is called until 500 items have been read (or it returns null, meaning end of input), each item passes through the processor (which may return null to filter it out), the writer receives the resulting Chunk in one call, and the transaction commits. Batch 5 changed ItemWriter.write to accept a Chunk<? extends T> instead of a List; the interface otherwise looks as it always has:
public interface ItemReader<T> { T read() throws Exception; } // null = end of data
public interface ItemProcessor<I, O> { O process(I item) throws Exception; } // null = filter out
public interface ItemWriter<T> { void write(Chunk<? extends T> chunk) throws Exception; }
The chunk size is the central tuning parameter. Larger chunks mean fewer commits and better throughput, but more work is lost on a rollback and more memory is held. Hundreds to a few thousand is typical for database writes; measure rather than guess.
Readers and writers
Spring Batch ships readers and writers for the common sources, all built with fluent builders in Batch 5:
| Source or target | Reader / writer |
|---|---|
| Delimited or fixed-width files | FlatFileItemReader, FlatFileItemWriter |
| JDBC | JdbcCursorItemReader (one open cursor, streaming), JdbcPagingItemReader (page queries, restartable and thread-safe), JdbcBatchItemWriter |
| JPA | JpaPagingItemReader, JpaCursorItemReader, JpaItemWriter |
| Spring Data | RepositoryItemReader, RepositoryItemWriter |
| XML | StaxEventItemReader, StaxEventItemWriter |
| JSON | JsonItemReader, JsonFileItemWriter |
| Messaging | JmsItemReader/Writer, KafkaItemReader/Writer, AmqpItemReader/Writer |
| Composition | CompositeItemProcessor, CompositeItemWriter, ClassifierCompositeItemWriter |
Reading a CSV into a record and writing with JDBC batching:
@Bean
@StepScope // created per step execution so the file name can come from job parameters
FlatFileItemReader<OrderCsvRow> orderReader(@Value("#{jobParameters['inputFile']}") Resource inputFile) {
return new FlatFileItemReaderBuilder<OrderCsvRow>()
.name("orderReader") // required: key for restart state in the execution context
.resource(inputFile)
.linesToSkip(1) // header
.delimited().delimiter(';')
.names("orderNumber", "customerId", "sku", "quantity", "unitPrice")
.targetType(OrderCsvRow.class) // maps columns onto a record's components by name
.build();
}
@Bean
JdbcBatchItemWriter<Order> orderWriter(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Order>()
.dataSource(dataSource)
.sql("""
insert into orders (order_number, customer_id, total_cents, created_at)
values (:orderNumber, :customerId, :totalCents, :createdAt)
""")
.beanMapped() // :name placeholders resolved from the item's JavaBean getters
.build();
}
JdbcBatchItemWriter sends the whole chunk as one JDBC batch, which is where most of the throughput of a batch job comes from. beanMapped() relies on getter-style properties; for records or computed columns, supply an ItemSqlParameterSourceProvider that builds a MapSqlParameterSource from the item instead. When writing through JPA, JpaItemWriter flushes per chunk; enable Hibernate's JDBC batching (hibernate.jdbc.batch_size) and use SEQUENCE identifiers, as explained in the Spring Data JPA guide, or the "batch" will still be one statement per row.
Choose JdbcCursorItemReader for a straightforward single-threaded read of a large table: one query, one open result set, constant memory. Choose JdbcPagingItemReader when the step will be multi-threaded or partitioned, because a cursor cannot be shared, and because a paging reader's position is a sort key that can be persisted for restart. Paging requires a deterministic sort key (a unique column) or pages will overlap.
Job parameters and step scope
Beans are normally created when the context starts, long before anyone knows which file tonight's run should import. @StepScope (and @JobScope) defers creation to the moment the step starts, which is when #{jobParameters['...']}, #{jobExecutionContext['...']} and #{stepExecutionContext['...']} expressions can be resolved. Any bean that reads a job parameter must be step-scoped; a common error is to forget the annotation and receive null.
Parameters are typed and, since Batch 5, can be of any type with a registered converter; the default string form on the command line is name=value,type,identifying, for example inputFile=orders-2026-09-01.csv,java.lang.String,true. Boot passes command-line arguments through to the job when spring.batch.job.enabled is true (the default): java -jar import.jar inputFile=file:/data/orders.csv. With several jobs in one application, spring.batch.job.name selects which one runs at startup, and setting spring.batch.job.enabled=false lets you launch through the JobLauncher from a scheduler or an HTTP endpoint instead.
Use identifying parameters for what defines the run (business date, file name) and non-identifying ones for technical knobs (chunk size override, dry-run flag) so that a retry with a different knob is still the same instance and can restart.
Failures: skip, retry, restart
A batch that stops on the first malformed line is not much use. Fault tolerance is configured on the step:
@Bean
Step importStep(/* ... */) {
return new StepBuilder("import", jobRepository)
.<OrderCsvRow, Order>chunk(500, tx)
.reader(reader).processor(processor).writer(writer)
.faultTolerant()
.skip(FlatFileParseException.class) // bad line: drop it and continue
.skip(UnknownCustomerException.class)
.skipLimit(100) // more than 100 skips fails the step
.listener(rejectedLinesListener) // SkipListener: write rejects to a file
.retry(DeadlockLoserDataAccessException.class)
.retryLimit(3) // transient database error: try again
.noRollback(ValidationWarning.class) // exception that should not roll back the chunk
.build();
}
Skipping during the write phase is more expensive than it looks: because the writer received the whole chunk, Spring Batch cannot tell which item caused the failure, so it rolls back, then re-processes the chunk item by item in transactions of one to isolate the culprit. Keep skip limits modest and make validation happen in the processor where a skip costs nothing.
Restart is what the repository buys you. If an execution fails, launching the job again with the same identifying parameters creates a new JobExecution for the same JobInstance, skips the steps that completed, and restarts the failed step. Readers that implement ItemStream (all the file and paging readers do, when given a name) store their position in the step's execution context at every commit and resume from it. Two flags change this: .allowStartIfComplete(true) on a step re-runs it even after success, and .preventRestart() on the job forbids restarts for jobs that are not idempotent. A step's startLimit caps how many times it may be attempted.
Controlling flow
Steps run in sequence by default. Conditional transitions branch on a step's ExitStatus:
return new JobBuilder("importOrders", jobRepository)
.start(importStep)
.on("FAILED").to(notifyOpsStep) // exit code pattern matching, "*" is a wildcard
.from(importStep).on("*").to(reconcileStep)
.from(reconcileStep)
.on("NOTHING_TO_DO").end() // custom exit status set by a StepExecutionListener
.from(reconcileStep).on("*").to(reportStep)
.end()
.build();
ExitStatus is distinct from BatchStatus: the batch status (COMPLETED, FAILED, STOPPED…) is what the repository records, while the exit status is a string a listener or JobExecutionDecider can set freely to drive routing. A decider is a bean implementing decide(JobExecution, StepExecution) and returning a FlowExecutionStatus; it is the tool when the decision depends on data rather than on a step's outcome. Listeners (JobExecutionListener, StepExecutionListener, ChunkListener, ItemReadListener, ItemProcessListener, ItemWriteListener, SkipListener) hook every level of the lifecycle and are where logging, auditing and reject files belong.
Scaling
When a single-threaded chunk step is not fast enough, Spring Batch offers four options, roughly in order of complexity:
- Multi-threaded step. Give the step a
TaskExecutor; each thread processes its own chunks. Readers and writers must be thread-safe, which rules out cursors andFlatFileItemReader(wrap the latter inSynchronizedItemStreamReader), and restart is no longer reliable because item order is undefined. - Parallel steps. Independent steps run concurrently in a
splitwith aTaskExecutor. No thread-safety concerns inside each step; the parallelism is between steps. - Partitioning. A
Partitionersplits the input into ranges (id ranges, one file per partition, one region per partition), and aPartitionHandlerruns a copy of the worker step per partition with its ownStepExecution, locally on threads or remotely on other JVMs. Each partition is restartable on its own. This is the workhorse for large database tables. - Remote chunking. The manager reads and sends items over messaging; workers process and write. Useful when processing, not reading, is the bottleneck and the input cannot be partitioned.
A local partitioned step looks like this:
@Bean
Step partitionedImport(JobRepository jobRepository, Step importWorkerStep, TaskExecutor executor) {
return new StepBuilder("partitionedImport", jobRepository)
.partitioner("importWorker", new IdRangePartitioner(dataSource)) // yields minId/maxId per partition
.step(importWorkerStep)
.gridSize(8)
.taskExecutor(executor)
.build();
}
The worker's reader is step-scoped and reads #{stepExecutionContext['minId']} and ['maxId'] to build its query. Before any of this, confirm that the database and not the JVM is the bottleneck; a batch writer without JDBC batching, or a processor doing a query per item, is usually slower than any lack of threads.
Running and testing
spring-batch-test provides @SpringBatchTest, which registers a JobLauncherTestUtils and a JobRepositoryTestUtils in a @SpringBootTest context. Launch the whole job or a single step with parameters and assert on the resulting JobExecution:
@SpringBootTest
@SpringBatchTest
class ImportOrdersJobTest {
@Autowired JobLauncherTestUtils jobLauncherTestUtils;
@Test
void importsValidRowsAndSkipsBadOnes() throws Exception {
JobParameters params = new JobParametersBuilder()
.addString("inputFile", "classpath:orders-sample.csv")
.addLong("run.id", System.currentTimeMillis())
.toJobParameters();
JobExecution execution = jobLauncherTestUtils.launchJob(params);
assertThat(execution.getStatus()).isEqualTo(BatchStatus.COMPLETED);
StepExecution step = execution.getStepExecutions().stream()
.filter(s -> s.getStepName().equals("import")).findFirst().orElseThrow();
assertThat(step.getWriteCount()).isEqualTo(998);
assertThat(step.getSkipCount()).isEqualTo(2);
}
}
Set spring.batch.job.enabled=false in the test properties so Boot does not also launch the job when the context starts. For operations, Spring Batch has no built-in web console since Spring Batch Admin was retired; Spring Cloud Data Flow, a small Actuator endpoint over the JobExplorer, or plain SQL against the BATCH_* tables are the usual ways to see what ran.
Common mistakes
Adding @EnableBatchProcessing from an old tutorial. On Boot 3 it switches off auto-configuration; the symptoms are missing JobRepository beans or jobs that never launch.
Forgetting @StepScope on beans that use job parameters. The expression resolves at context startup to null, and the failure appears far from the cause.
Committing per item. A chunk size of 1, or a writer that opens its own transaction per item, throws away the reason to use a batch framework.
Reading with JPA and never clearing the persistence context. JpaPagingItemReader and JpaItemWriter handle it per chunk, but a hand-written reader that loads entities in a loop grows the first-level cache until memory runs out.
Non-deterministic paging. A paging reader sorted on a non-unique column skips or duplicates rows between pages. Sort by the primary key.
Making everything skippable. skip(Exception.class) with a large limit turns data corruption into a warning nobody reads. Skip specific, understood exceptions and route rejects somewhere visible.
Launching the same instance twice. Without an incrementer or a distinguishing parameter, the second launch of a completed job fails; with a run id on everything, restart of a failed instance becomes impossible because every launch is new. Use business parameters for identity and reserve run.id for jobs that genuinely have no natural key.
Key takeaways
- Spring Batch models work as jobs of steps and records every execution in a database-backed
JobRepository, which is what makes restart, auditing and skip counting possible. - On Spring Boot 3 use
JobBuilderandStepBuilderwith an explicitJobRepositoryand transaction manager; leave@EnableBatchProcessingout. - Chunk-oriented steps read items one by one, process them, and write them in transactional chunks; the chunk size is the primary tuning knob and JDBC batching is the primary source of speed.
- Step-scoped beans receive job parameters and execution-context values late enough for them to exist; identifying parameters define the
JobInstance. - Skip and retry make steps tolerant of bad records and transient errors; restart resumes a failed execution from the last commit.
- Scale with partitioning for large tables, multi-threaded steps for thread-safe readers, parallel flows for independent steps, and remote chunking when processing dominates.
Sources and further reading
Frequently asked questions
- Do I still need @EnableBatchProcessing with Spring Boot 3?
- No, and adding it changes behaviour. Spring Boot 3 auto-configures the JobRepository, JobLauncher and related beans from your DataSource; @EnableBatchProcessing tells Boot to back off entirely so that you take over the infrastructure yourself. Leave it out unless you deliberately want to configure Spring Batch by hand, for example to use a dedicated DataSource for the metadata tables.
- Why does my job say 'A job instance already exists and is complete'?
- A JobInstance is identified by the job name plus its identifying parameters. Once an instance has completed successfully, Spring Batch refuses to run it again with the same parameters to protect you from reprocessing. Either pass a parameter that changes per run (a run id from RunIdIncrementer, the input file name, a business date) or mark a parameter as non-identifying if it must not count.
- Can I run Spring Batch without a database?
- Spring Batch 5 removed the old map-based in-memory job repository, so the standard JobRepository needs a relational database for its metadata tables, which is also what makes restart work. For development an embedded H2 database is enough. Recent 5.x releases add a resourceless repository for cases where restartability is not required; read the release notes for the version you use before relying on it.
- What is the difference between commit interval and page size?
- Commit interval (the chunk size) is how many items are processed before the writer is called and the transaction commits. Page size belongs to paging readers such as JdbcPagingItemReader and controls how many rows each database query fetches. They are independent; a common mistake is to set them differently and then wonder why queries do not align with commits. Setting them equal is a sensible default.