Sequential batch after migrating from 5 to 6

[monnetchr]

When upgrading from Spring Batch 5 to 6 (Boot 3.5 to 4) I was surprise that our multithreaded chunk steps became sequential.
I've written simplified examples: https://github.com/monnetchr/springbatch56

The batch step looks like this:

new StepBuilder("myStep", jobRepository)
            .<Long, Long>chunk(1, transactionManager)
            .transactionAttribute(new DefaultTransactionAttribute(Propagation.REQUIRES_NEW.value()))
            .faultTolerant().skipLimit(2).skip(Exception.class)
            .reader(myReader)
            .writer(myWriter)
            .taskExecutor(taskExecutor)
            .build();

Basically only reader and writer, and the chunk size is configurable.

With Spring Batch 5 the data is processed in parallel on several threads:

2026-02-23T14:55:27.289+01:00  INFO 93806 --- [batch5] [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [myStep]
2026-02-23T14:55:27.296+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : doPageRead
2026-02-23T14:55:27.296+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : read 1
2026-02-23T14:55:27.296+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : read 3
2026-02-23T14:55:27.296+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : read 2
2026-02-23T14:55:27.299+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : writing chunk [items=[2], skips=[]] ...
2026-02-23T14:55:27.299+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : writing chunk [items=[3], skips=[]] ...
2026-02-23T14:55:27.299+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : writing chunk [items=[1], skips=[]] ...
2026-02-23T14:55:27.494+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[1], skips=[]]
2026-02-23T14:55:27.496+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : read 4
2026-02-23T14:55:27.496+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : writing chunk [items=[4], skips=[]] ...
2026-02-23T14:55:27.650+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[2], skips=[]]
2026-02-23T14:55:27.655+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[3], skips=[]]
2026-02-23T14:55:27.661+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : read 5
2026-02-23T14:55:27.662+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : writing chunk [items=[5], skips=[]] ...
2026-02-23T14:55:27.664+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : read 6
2026-02-23T14:55:27.665+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : writing chunk [items=[6], skips=[]] ...
2026-02-23T14:55:28.089+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[4], skips=[]]
2026-02-23T14:55:28.095+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : read 7
2026-02-23T14:55:28.095+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : writing chunk [items=[7], skips=[]] ...
2026-02-23T14:55:28.323+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[6], skips=[]]
2026-02-23T14:55:28.325+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : read 8
2026-02-23T14:55:28.326+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : writing chunk [items=[8], skips=[]] ...
2026-02-23T14:55:28.363+01:00  INFO 93806 --- [batch5] [myParallelJob-3] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[5], skips=[]]
2026-02-23T14:55:28.505+01:00  INFO 93806 --- [batch5] [myParallelJob-1] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[8], skips=[]]
2026-02-23T14:55:28.766+01:00  INFO 93806 --- [batch5] [myParallelJob-2] ch.sbb.batch5.Batch5Application          : ... written chunk [items=[7], skips=[]]
2026-02-23T14:55:28.770+01:00  INFO 93806 --- [batch5] [           main] o.s.batch.core.step.AbstractStep         : Step: [myStep] executed in 1s480ms

After migrating to Spring Batch 6, everything happens on the main thread:

2026-02-23T14:56:42.647+01:00  INFO 93852 --- [batch6] [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [myStep]
2026-02-23T14:56:42.650+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : doPageRead
2026-02-23T14:56:42.650+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 1
2026-02-23T14:56:42.652+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[1], skips=[]] ...
2026-02-23T14:56:42.859+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[1], skips=[]]
2026-02-23T14:56:42.860+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 2
2026-02-23T14:56:42.860+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[2], skips=[]] ...
2026-02-23T14:56:43.349+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[2], skips=[]]
2026-02-23T14:56:43.349+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 3
2026-02-23T14:56:43.350+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[3], skips=[]] ...
2026-02-23T14:56:43.505+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[3], skips=[]]
2026-02-23T14:56:43.507+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 4
2026-02-23T14:56:43.507+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[4], skips=[]] ...
2026-02-23T14:56:43.732+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[4], skips=[]]
2026-02-23T14:56:43.734+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 5
2026-02-23T14:56:43.734+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[5], skips=[]] ...
2026-02-23T14:56:44.391+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[5], skips=[]]
2026-02-23T14:56:44.392+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 6
2026-02-23T14:56:44.393+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[6], skips=[]] ...
2026-02-23T14:56:44.721+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[6], skips=[]]
2026-02-23T14:56:44.722+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 7
2026-02-23T14:56:44.722+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[7], skips=[]] ...
2026-02-23T14:56:44.936+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[7], skips=[]]
2026-02-23T14:56:44.936+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : read 8
2026-02-23T14:56:44.936+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : writing chunk [items=[8], skips=[]] ...
2026-02-23T14:56:45.345+01:00  INFO 93852 --- [batch6] [           main] ch.sbb.batch6.Batch6Application          : ... written chunk [items=[8], skips=[]]
2026-02-23T14:56:45.347+01:00  INFO 93852 --- [batch6] [           main] o.s.batch.core.step.AbstractStep         : Step: [myStep] executed in 2s699ms

Comparing the documentations it is indeed clear that things have changed:

• Spring Batch 5 https://docs.spring.io/spring-batch/reference/5.2/scalability.html#multithreadedStep

the Step executes by reading, processing, and writing each chunk of items (each commit interval) in a separate thread of execution

• Spring Batch 6 https://docs.spring.io/spring-batch/reference/scalability.html#multithreadedStep

the ItemProcessor will be called from multiple threads at the same time. [...] The reading and writing of items is still done in serial by the main thread

But what is not clear is how are we supposed to migrate our job step, so that writing happens in parallel in different threads each with its own transaction?

[ctrung]

Configuring a taskExecutor on a stepBuilder only makes the item processor multi-threaded in spring batch 6. If you also want the writing part to be multi-threaded, you must use local chunking that was introduced to this effect.

Be aware that the skipping part doesn't work with local chunking. The whole chunk is skipped instead of the individual items. See discussion #5232

[fmbenhassine]

@monnetchr the concurrency model has changed in v6, and indeed, only the processing phase is concurrent now as documented.

If you want concurrent writes, then you should set up local chunking as mentioned by @ctrung . And I do confirm there is a bug in the implementation as of v6.0.2, but I planned to address that in 6.0.3.

[fmbenhassine]

The issues reported by @ctrung have been addressed and will be shipped in 6.0.3, see #5232 (comment).

I am closing this discussion for now as the answer for concurrent chunk processing is to use local chunking.