Java Ninja Chronicles By Norris Shelton

Things I learned in the pursuit of code

Spring-Batch provides many types of readers and writers. The previous article, Spring-Batch – Reading and Writing XML provided the configuration needed to read and write XML. In this installment, I will present the configuration needed to read and write CSV files.

Maven Dependencies

You will need the Spring-Batch dependency.

<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>${spring.batch.version}</version>
</dependency>

You do not need a library for the CSV functionality.

The entire pom.xml looks like:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.javaninja</groupId>
    <artifactId>spring-batch-csv</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <!--This is the version of Spring that Spring Batch uses-->
        <spring.version>4.0.5.RELEASE</spring.version>
        <spring.batch.version>3.0.6.RELEASE</spring.batch.version>
        <slf4j.version>1.7.13</slf4j.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-core</artifactId>
            <version>${spring.batch.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>commons-logging</artifactId>
                    <groupId>commons-logging</groupId>
                </exclusion>
            </exclusions>
        </dependency>

        <!--
            Logging
        -->
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.1.3</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>log4j-over-slf4j</artifactId>
            <version>${slf4j.version}</version>
        </dependency>

        <!--
            Testing
        -->
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-test</artifactId>
            <version>${spring.batch.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Spring and Spring-Batch Context

Job Configuration

You will need the standard JobRepository, JobLauncher and TransactionManager.

<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>
<!--This repository is only really intended for use in testing and rapid prototyping-->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"
      p:transactionManager-ref="transactionManager"/>
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"
      p:jobRepository-ref="jobRepository"/>

FlatFileItemReader

The reading of a CSV is handled by a FlatFileItemReader. You will also need the following to configure the ItemReader to read the lines of CSV data into objects:

  • LineMapper – Maps lines in a flat file to objects.
  • LineTokenizer – Splits lines of text by a delimiter.
  • FieldSetMapper – Maps fields into an object.

The Spring XML configuration for the ItemReader looks like:

<bean id="fieldSetMapper" class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper"
      p:targetType="com.javaninja.batch.Car"/>
<bean id="delimitedLineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer
      p:names="make, model, color, doors"/>
<bean id="lineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper"
      p:lineTokenizer-ref="delimitedLineTokenizer"
      p:fieldSetMapper-ref="fieldSetMapper"/>
<!-- ItemReader which reads data from CSV file -->
<bean id="csvItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
      p:resource="classpath:cars-input.xml"
      p:lineMapper-ref="lineMapper"/>

FlatFileItemWriter

The writing of the CSV file is handled by a FlatFileItemWriter. You will also need the following to configure the item writer to write the objects as lines of CSV data:

  • LineAggregator – Maps the object into a string representation.
  • FieldExtractor – Extracts the specified properties of a bean in the specified order.

The Spring XML configuration for the ItemWriter looks like:

<bean id="fieldExtractor" class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"
      p:names="make, model, color, doors"/>
<bean id="lineAggregator" class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"
      p:fieldExtractor-ref="fieldExtractor"/>
<!-- ItemWriter which writes the data in CSV format -->
<bean id="csvItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"
      p:resource="file:csv/cars.xml"
      p:lineAggregator-ref="lineAggregator"/>

The entire Spring context looks like:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:p="http://www.springframework.org/schema/p"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans   http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

    <import resource="classpath:META-INF/spring/batchContext.xml"/>

    <context:component-scan base-package="com.javaninja.batch"/>

    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>

    <!--This repository is only really intended for use in testing and rapid prototyping-->
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"
          p:transactionManager-ref="transactionManager"/>

    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"
          p:jobRepository-ref="jobRepository"/>



    <bean id="fieldSetMapper" class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper"
          p:targetType="com.javaninja.batch.Car"/>

    <bean id="delimitedLineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"
          p:names="make, model, color, doors"/>

    <bean id="lineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper"
          p:lineTokenizer-ref="delimitedLineTokenizer"
          p:fieldSetMapper-ref="fieldSetMapper"/>

    <!-- ItemReader which reads data from CSV file -->
    <bean id="csvItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
          p:resource="classpath:cars-input.xml"
          p:lineMapper-ref="lineMapper"/>



    <bean id="fieldExtractor" class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"
          p:names="make, model, color, doors"/>

    <bean id="lineAggregator" class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"
          p:fieldExtractor-ref="fieldExtractor"/>

    <!-- ItemWriter which writes the data in CSV format -->
    <bean id="csvItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"
          p:resource="file:csv/cars.xml"
          p:lineAggregator-ref="lineAggregator"/>

</beans>

Job configuration

The XML configuration for the CSV batch job is pretty much the same as any other example.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:batch="http://www.springframework.org/schema/batch"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd">

    <batch:job id="carJob">
        <batch:step id="step1">
            <batch:tasklet>
                <batch:chunk reader="csvItemReader" writer="csvItemWriter" commit-interval="1000"/>
            </batch:tasklet>
        </batch:step>
    </batch:job>

</beans>

Running a job is the same way as in the XML example. Spring-Batch – Reading and Writing XML

The entire project used to write this blog is located on GitHub sheltonn / spring-batch-csv

February 18th, 2016

Posted In: Java, java ninja, Javaninja, Spring Batch

Leave a Reply

Your email address will not be published. Required fields are marked *

WP to LinkedIn Auto Publish Powered By : XYZScripts.com