Batch Chunk Exception

Run
How to run the sample
The source code for this sample can be found in the javaee7-samples GitHub repository. The first thing we need to do is to get the source by downloading the repository and then go into the samples folder:
git clone git://github.com/javaee-samples/javaee7-samples.git
cd javaee7-samples/batch/chunk-exception/
Now we are ready to start testing. You can run all the tests in this sample by executing:
mvn test
Or you can run individual tests by executing one of the following:
mvn test -Dtest=BatchChunkExceptionTest

Chunk Exception Handling - Retrying and Skipping

BatchChunkExceptionTest

Exceptions are a natural part of Batch Processing, and the batch itself should be prepared to deal with exceptions during processing.

Batch Processing deals with two kinds of exceptions: skippable and retryable. Skippable Exceptions are used to skip elements during reading, processing and writing and continue to the next element. Retryable Exceptions on the other hand when thrown will try to retry the chunk on which the exception occurred.

When the same exception is specified as both retryable and skippable, retryable takes precedence over skippable during regular processing of the chunk. While the chunk is retrying, skippable takes precedence over retryable since the exception is already being retried.

The Reader:

@Named()
public class MyItemReader extends AbstractItemReader {

    public MyItemReader();
    private StringTokenizer tokens;
    private MyInputRecord lastElement;
    private boolean alreadyFailed;

    @Override()
    public void open(Serializable checkpoint);

    @Override()
    public Object readItem();

    @Override()
    public Serializable checkpointInfo() throws Exception;
}

Just reads elements from a list and simulate a retry exception.

The Processor:

@Named()
public class MyItemProcessor implements ItemProcessor {

    public MyItemProcessor();

    @Override()
    public Object processItem(Object t);
}

Process and simulate a skip exception.

The Writer:

@Named()
public class MyItemWriter extends AbstractItemWriter {

    public MyItemWriter();
    private static int retries;

    @Override()
    public void writeItems(List list);
}

The writer will retry an exception and then skip it.

The batch specification also allows you to provide listeners for skipping and retrying for every operation. Have a look into the following classes:

  • MySkipReadListener

  • MySkipProcessorListener

  • MySkipWriteListener

  • MyRetryReadListener

  • MyRetryProcessorListener

  • MyRetryWriteListener

Events can be caught via extending the following classes, for the appropriate batch lifecycle event:

<?xml version="1.0" encoding="UTF-8"?>
<job id="myJob" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0">
    <step id="myStep" >
        <listeners>
            <listener ref="mySkipReadListener"/>
            <listener ref="mySkipProcessorListener"/>
            <listener ref="mySkipWriteListener"/>
            <listener ref="myRetryReadListener"/>
            <listener ref="myRetryProcessorListener"/>
            <listener ref="myRetryWriteListener"/>
        </listeners>

        <chunk checkpoint-policy="item" item-count="3" skip-limit="3" retry-limit="3">
            <reader ref="myItemReader"/>
            <processor ref="myItemProcessor"/>
            <writer ref="myItemWriter"/>
            <skippable-exception-classes>
                <include class="java.lang.RuntimeException"/>
                <include class="java.lang.UnsupportedOperationException"/>
            </skippable-exception-classes>
            <retryable-exception-classes>
                <include class="java.lang.IllegalArgumentException"/>
                <include class="java.lang.UnsupportedOperationException"/>
            </retryable-exception-classes>
        </chunk>
    </step>
</job>

A very simple job is defined in the myJob.xml file. Just a single step with a reader, a processor and a writer. For this sample we are going to process a few records and mix some exceptions during read, processing and write of the chunk. Batch exception handling is achieved by defining the elements skippable-exception-classes and retryable-exception-classes into the chunk. Both elements should indicate the full qualified name of the exceptions that we are trying to catch. The listeners element can be used at the step level to define which listeners to run for each batch processing event.

We’re just going to deploy the application as a web archive. Note the inclusion of the following files:

/META-INF/batch-jobs/myJob.xml

The myJob.xml file is needed for running the batch definition.

@Deployment
public static WebArchive createDeployment() {
    WebArchive war = ShrinkWrap.create(WebArchive.class)
            .addClass(BatchTestHelper.class)
            .addPackage("org.javaee7.batch.chunk.exception")
            .addAsWebInfResource(EmptyAsset.INSTANCE, ArchivePaths.create("beans.xml"))
            .addAsResource("META-INF/batch-jobs/myJob.xml");
    System.out.println(war.toString(true));
    return war;
}

In the test, we’re just going to invoke the batch execution and wait for completion. To validate the test expected behaviour we need to query the Metric object available in the step execution.

@Test
public void testBatchChunkException() throws Exception {
    JobOperator jobOperator = BatchRuntime.getJobOperator();
    Long executionId = jobOperator.start("myJob", new Properties());
    JobExecution jobExecution = jobOperator.getJobExecution(executionId);

    jobExecution = BatchTestHelper.keepTestAlive(jobExecution);

    List<StepExecution> stepExecutions = jobOperator.getStepExecutions(executionId);
    for (StepExecution stepExecution : stepExecutions) {
        if (stepExecution.getStepName().equals("myStep")) {
            Map<Metric.MetricType, Long> metricsMap = BatchTestHelper.getMetricsMap(stepExecution.getMetrics());

            assertEquals(1L, metricsMap.get(Metric.MetricType.PROCESS_SKIP_COUNT).longValue());
            // There are a few differences between Glassfish and Wildfly. Needs investigation.
            //assertEquals(1L, metricsMap.get(Metric.MetricType.WRITE_SKIP_COUNT).longValue());
            assertEquals(1L, ChunkExceptionRecorder.retryReadExecutions);
        }
    }

    assertTrue(ChunkExceptionRecorder.chunkExceptionsCountDownLatch.await(0, TimeUnit.SECONDS));
    assertEquals(BatchStatus.COMPLETED, jobExecution.getBatchStatus());
}

Share the Knowledge

Find this sample useful? Share on

There's a lot more about JavaEE to cover. If you're ready to learn more, check out the other available samples.

Help Improve

Find a bug in the sample? Something missing? You can fix it by editing the source, making the correction and sending a pull request. Or report the problem to the issue tracker

Recent Changelog

  • Dec 14, 2014: Switch from polling on jobexecution (for job completion) to polling with joboperator and executionid by Scott Kurz
  • Jul 05, 2014: Removed header license for batch xml files by Roberto Cortez
  • Jun 22, 2014: Removed header license. the licensing is now referenced in the license file in the root of the project by Roberto Cortez
  • Jun 20, 2014: Added fqn to java ee api references to generate direct links to javadocs by radcortez
  • Jun 19, 2014: Documentation clarifications and typos by radcortez
  • May 27, 2014: Fixed a few javadocs typos by Roberto Cortez
  • May 08, 2014: Fixed chunk exception wildfly test by Roberto Cortez
  • Apr 04, 2014: Fixed chunk exception test by Roberto Cortez
  • Dec 31, 2013: Code style issues by Roberto Cortez
  • Dec 31, 2013: Removed servlets and jsp's by Roberto Cortez
How to help improve this sample
The source code for this sample can be found in the javaee7-samples GitHub repository. The first thing you need to do is to get the source by downloading the repository and then go into the samples folder:
git clone git://github.com/javaee-samples/javaee7-samples.git
cd javaee7-samples/batch/chunk-exception/

Do the changes as you see fit and send a pull request!

Good Luck!