Spring batch with boot provides a light weight framework to make batch processing easier. It provides the components for the batch out-of-the-box. The Spring Boot also provides a variety of features like in-memory database, embedded servers, metrics, etc. that helps developer to set up and configure the project in a short time. A novice programmer can quickly learn, understand and implement a batch job.

Following is a guideline for using Spring Batch with Boot to consume WebService response and POST JSON to a REST-API endpoint.

  1. Add the dependency:

The following Spring Batch starter dependency has to be added to pom.xml. Configuring resources is made easier in Spring Boot and is based on what is defined in the POM.

org.springframework.boot

spring-boot-starter-batch

  1. Configure the Job and Step in Batch Configuration:

The code snippet below shows simple methods that define a Job and a Step. The step includes a reader that consumes a WebService response, a processor that transforms the WebService response to REST API request and a writer that POSTs the request to a REST API endpoint. The incrementer is used to maintain the execution state of the in-memory database.

@Bean

public Job consumeWebserviceJob(JobBuilderFactory jobs, Step s1) {

return jobs.get("consumeWebserviceJob")

.incrementer(new RunIdIncrementer())

.flow(s1)

.end()

.build();

}

@Bean

public Step jobStep(StepBuilderFactory stepBuilderFactory, ItemReader reader, ItemWriter writer, ItemProcessor processor) {,>

return stepBuilderFactory.get("jobStep")

. chunk(1),>

.reader(reader)

.processor(processor)

.writer(writer)

.build();

}

  1. WebService Client:

The WebServiceTemplate uses sendAndReceive method to marshal or unmarshal the WebService message. SoapRequestMessageCallback is a custom class that implements WebServiceMessageCallback. It gets the request ready by adding additional attributes (like namespace, security , etc.). SoapResponseMessageExtractor is a custom class that implements WebServiceMessageExtractor and is used for extracting the response.

public GetDataResponse getDataById(Long param, int index, int batchsize) {

GetDataResponse response = null;

GetDataRequest request = createRequest(param, index, batchsize);

WebServiceTemplate template = getWebServiceTemplate();

JAXBElement jaxbRequest = new ObjectFactory().getData(request);

JAXBElement jaxbElement = (JAXBElement) template.sendAndReceive(template.getDefaultUri(), new SoapRequestMessageCallback(template.getMarshaller(), jaxbRequest), new SoapResponseMessageExtractor(template.getUnmarshaller())

);

if(jaxbElement != null){

response = (GetDataResponse) jaxbElement.getValue();

}

return response;

}

  1. ItemReader:

The following shows the overridden read() method of the ItemReader. The read() method calls the WebService operation to retrieve the data. The retrieved data object is then sent to the ItemProcessor to be processed. The reader is invoked until no results are found and the flag isReaderExhaused is set to true. A null value returned from the read() method will not invoke the ItemProcessor and indicates the end of the process.

@Override

public GetDataResponse read() throws Exception {

final GetDataResponse returnValue = null;

if (!isReaderExhausted) {

GetDataResponse itemResponse = (GetDataResponse) webserviceClient.getDataById(webserviceConfiguration.getId(), index, webserviceConfiguration.getBatchsize());

if (itemResponse != null) {

index = index + webserviceConfiguration.getBatchsize();

return itemResponse;

} else {

isReaderExhausted = true;

}

}

return returnValue;

}

  1. ItemProcessor:

The 'process' method transforms the WebService response to a REST request object which in turn will be used by the ItemWriter to perform a POST to the REST API Service.

@Override

public RestApiRequest process(final GetDataResponse response) throws Exception {

final RestApiRequest request = new RestApiRequest();

if (response.getReturnData() !=null){

transformGetDataResponseToRestApiRequest(request, response.getReturnData());

}

return request;

}

  1. ItemWriter:

The ItemWriter POSTs each chunk of item defined in the batch configuration to the REST API. RestTemplate is used in the ItemWriter to call the REST Service. The exchange method of RestTemplate is used to send a request and get a response from the REST API Service.

@Override

public void write(final List<? extends="" restapirequest=""?> requestObjects) throws Exception {

final String uri = restapiConfiguration.getUri();

final HttpHeaders headers = new HttpHeaders();

headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));

headers.setContentType((MediaType.APPLICATION_JSON));

if (requestObjects!= null && !requestObjects.isEmpty()) {

for(RestApiRequest request : requestObjects) {

try{

HttpEntity entity = new HttpEntity(request, headers);

ResponseEntity response = restTemplate.exchange(

uri, HttpMethod.POST, entity, String.class);

if (response.getStatusCode() == HttpStatus.OK) {

logger.info("\n Post Success :");

}else {

logger.info("\n Post Failed ");

}

}catch(HttpClientErrorException ex){

logger.error("ERROR in ItemWriter : ",ex);

}catch(IllegalArgumentException illegalEx){

logger.error("ERROR in ItemWriter : ",illegalEx);

}

}

}

}

Conclusion:

The Spring Batch architecture defines a structure for developers to use with ease. Developers can leverage the Spring Batch architecture to implement the batch job, this simplifies and mitigates the need to define boilerplate code. Spring Batch is scalable, reliable, robust and concurrent. It has been built to process large data without crashing the application and to interact and integrate with other systems. The Spring Boot makes it easier to set up Spring development. So why not use Spring Batch with Boot for the batch jobs!

References:

http://spring.io/guides/gs/batch-processing/

http://docs.spring.io/spring-batch/reference/html/domain.h