Spring batch 예제

Spring Batch 예제

- spring framework 4.2.5

- spring batch 3.0.6

- cglib 3.2.2

- myBatis 3.3.1

- myBatis spring 1.2.4

- logback 1.1.7

- mariadb 1.4.2

여기선 myBatis 연동과 트랜잭션 처리 예제까지는 안했다.

트랜잭션은 AOP나 @Transactional 사용하면 될 듯.

DB는 MariaDB를 사용했다.

스프링 설정에서 자동으로 스키마를 생성해주는게 있는 듯 한데, 그냥 테이블 생성 스크립트를 수동으로 돌렸다.

spring-batch-core-x.x.x.RELEASE.jar 파일 안에 org.springframework.batch.core 패키지 안에 있다.

살펴보면 mariadb 스크립트는 따로 없다. 그래서 schema-mysql.sql 파일을 수행했다.

jar 파일 안에 batch-mysql.properties 파일을 참고해도 좋다.

언제나 프로젝트의 시작은 로깅 설정부터

- logback.xml

<?xml version="1.0" encoding="UTF-8"?>

<configuration scan="true" scanPeriod="1 second">

<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">

<encoder>

<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread][%-5level] %mdc %logger.%M:%L - %msg%n</Pattern>

</encoder>

</appender>

<logger name="com.my.app" level="debug" />

<logger name="org.springframework" level="info" />

<logger name="org.mybatis" level="info" />

<root>

<level value="info" />

<appender-ref ref="consoleAppender" />

</root>

</configuration>

스프링 배치 루트 xml 설정 파일

- launch-context.xml

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:aop="http://www.springframework.org/schema/aop"

xmlns:batch="http://www.springframework.org/schema/batch"

xmlns:context="http://www.springframework.org/schema/context"

xmlns:tx="http://www.springframework.org/schema/tx"

xmlns:util="http://www.springframework.org/schema/util"

xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd

http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.2.xsd

http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd

http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.2.xsd

http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.2.xsd

http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-4.2.xsd">

<context:annotation-config />

<context:component-scan base-package="com.my.app" />

<bean id="dataSource" class="org.apache.tomcat.dbcp.dbcp2.BasicDataSource" destroy-method="close">

<property name="driverClassName" value="org.mariadb.jdbc.Driver" />

<property name="url" value="jdbc:mariadb://192.168.0.10:3306/test" />

<property name="username" value="root" />

<property name="password" value="admin123" />

</bean>

<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">

   <property name="dataSource" ref="dataSource"/>

</bean>

<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">

<property name="dataSource" ref="dataSource" />

<property name="transactionManager" ref="transactionManager" />

</bean>

<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">

   <property name="jobRepository" ref="jobRepository" />

</bean>

<import resource="classpath:spring/jobs/job*/job*.xml" />

</beans>

잡을 하나만 만들었다.

- job1.xml

reader, processor, writer 빈 태그를 주석처리했는데 이유는 @Component 애노테이션을 썼기 때문이다.

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:aop="http://www.springframework.org/schema/aop"

xmlns:batch="http://www.springframework.org/schema/batch"

xmlns:context="http://www.springframework.org/schema/context"

xmlns:tx="http://www.springframework.org/schema/tx"

xmlns:util="http://www.springframework.org/schema/util"

xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd

http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.2.xsd

http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd

http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.2.xsd

http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.2.xsd

http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-4.2.xsd">

<!--

<bean id="job1Reader" class="com.my.app.jobs.job1.reader.Job1Reader" scope="step">

<property name="id" value="#{jobParameters['id']}" />

</bean>

<bean id="job1Processor" class="com.my.app.jobs.job1.processor.Job1Processor" scope="step" />

<bean id="job1Writer" class="com.my.app.jobs.job1.writer.Job1Writer" scope="step" />

-->

<batch:job id="job1">

<batch:step id="step1">

<batch:tasklet transaction-manager="transactionManager">

<batch:chunk reader="job1Reader" processor="job1Processor" writer="job1Writer" commit-interval="1" />

</batch:tasklet>

</batch:step>

</batch:job>

</beans>

수집된 데이터를 저장할 클래스

package com.my.app.jobs.job1.domain;

import java.io.Serializable;

import java.util.Date;

import org.apache.commons.lang3.builder.ToStringBuilder;

import org.apache.commons.lang3.builder.ToStringStyle;

public class Data1 implements Serializable {

private static final long serialVersionUID = -2760916978465912344L;

private String id;

private String name;

private Date date;

public String getId() {

return id;

}

public void setId(String id) {

this.id = id;

}

public String getName() {

return name;

}

public void setName(String name) {

this.name = name;

}

public Date getDate() {

return date;

}

public void setDate(Date date) {

this.date = date;

}

@Override

public String toString() {

return ToStringBuilder.reflectionToString(this, ToStringStyle.SHORT_PREFIX_STYLE);

}

}

데이터 수집하는 클래스

- Reader

package com.my.app.jobs.job1.reader;

import java.util.Date;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.batch.item.ItemReader;

import org.springframework.batch.item.NonTransientResourceException;

import org.springframework.batch.item.ParseException;

import org.springframework.batch.item.UnexpectedInputException;

import org.springframework.beans.factory.annotation.Value;

import org.springframework.context.annotation.Scope;

import org.springframework.stereotype.Component;

import com.my.app.jobs.job1.domain.Data1;

@Component

@Scope("step")

public class Job1Reader implements ItemReader<Data1> {

private static final Logger logger = LoggerFactory.getLogger(Job1Reader.class);

private String id;

private int count;

@Value("#{jobParameters['id']}")

public void setId(String id) {

this.id = id;

}

@Override

public Data1 read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {

if (count >= 1) return null;

count++;

Data1 item = new Data1();

item.setId(this.id);

item.setName(Thread.currentThread().getName());

item.setDate(new Date());

logger.info("Job1 reader: {}", item.toString());

return item;

}

}

수집된 데이터를 가공 처리하는 클래스

- Processor

package com.my.app.jobs.job1.processor;

import java.util.Date;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.batch.item.ItemProcessor;

import org.springframework.context.annotation.Scope;

import org.springframework.stereotype.Component;

import com.my.app.jobs.job1.domain.Data1;

@Component

@Scope("step")

public class Job1Processor implements ItemProcessor<Data1, Data1> {

private static final Logger logger = LoggerFactory.getLogger(Job1Processor.class);

@Override

public Data1 process(Data1 item) throws Exception {

item.setDate(new Date());

logger.info("Job1 process: {}", item.toString());

return item;

}

}

수집 또는 가공된 데이터를 List로 가져와 기록하는 클래스

- Writer

package com.my.app.jobs.job1.writer;

import java.util.List;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.batch.item.ItemWriter;

import org.springframework.context.annotation.Scope;

import org.springframework.stereotype.Component;

import com.my.app.jobs.job1.domain.Data1;

@Component

@Scope("step")

public class Job1Writer implements ItemWriter<Data1> {

private static final Logger logger = LoggerFactory.getLogger(Job1Writer.class);

@Override

public void write(List<? extends Data1> items) throws Exception {

for (Data1 item : items) {

logger.info("Job1 write: {}", item.toString());

}

}

}

배치를 실행시킬 클래스

- Main

package com.my.app.main;

import java.util.UUID;

import org.springframework.batch.core.Job;

import org.springframework.batch.core.JobParameters;

import org.springframework.batch.core.JobParametersBuilder;

import org.springframework.batch.core.JobParametersInvalidException;

import org.springframework.batch.core.launch.JobLauncher;

import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException;

import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException;

import org.springframework.batch.core.repository.JobRestartException;

import org.springframework.context.support.GenericXmlApplicationContext;

public class Main {

public static void main(String[] args) {

GenericXmlApplicationContext ctx = new GenericXmlApplicationContext();

ctx.load("classpath:spring/launch-context.xml");

ctx.refresh();

JobLauncher jobLauncher = ctx.getBean("jobLauncher", JobLauncher.class);

try {

Job job1 = ctx.getBean("job1", Job.class);

JobParameters jobParameters = new JobParametersBuilder()

.addString("id", UUID.randomUUID().toString())

.toJobParameters();

jobLauncher.run(job1, jobParameters);

} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException

| JobParametersInvalidException e) {

e.printStackTrace();

}

ctx.close();

}

}

결과

2016-04-20 06:29:00.643 [main][INFO ]  org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions:317 - Loading XML bean definitions from class path resource [spring/launch-context.xml]

2016-04-20 06:29:00.859 [main][INFO ]  org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions:317 - Loading XML bean definitions from file [C:\dev\workspace\spring-batch\target\classes\spring\jobs\job1\job1.xml]

2016-04-20 06:29:00.959 [main][INFO ]  org.springframework.context.support.GenericXmlApplicationContext.prepareRefresh:578 - Refreshing org.springframework.context.support.GenericXmlApplicationContext@13fee20c: startup date [Wed Apr 20 06:29:00 KST 2016]; root of context hierarchy

2016-04-20 06:29:01.090 [main][INFO ]  org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition:839 - Overriding bean definition for bean 'job1Processor' with a different definition: replacing [Generic bean: class [com.my.app.jobs.job1.processor.Job1Processor]; scope=step; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\processor\Job1Processor.class]] with [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\processor\Job1Processor.class]]

2016-04-20 06:29:01.090 [main][INFO ]  org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition:839 - Overriding bean definition for bean 'job1Reader' with a different definition: replacing [Generic bean: class [com.my.app.jobs.job1.reader.Job1Reader]; scope=step; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\reader\Job1Reader.class]] with [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\reader\Job1Reader.class]]

2016-04-20 06:29:01.090 [main][INFO ]  org.springframework.beans.factory.support.DefaultListableBeanFactory.registerBeanDefinition:839 - Overriding bean definition for bean 'job1Writer' with a different definition: replacing [Generic bean: class [com.my.app.jobs.job1.writer.Job1Writer]; scope=step; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\writer\Job1Writer.class]] with [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in file [C:\dev\workspace\spring-batch\target\classes\com\my\app\jobs\job1\writer\Job1Writer.class]]

2016-04-20 06:29:01.399 [main][INFO ]  org.springframework.batch.core.repository.support.JobRepositoryFactoryBean.afterPropertiesSet:183 - No database type set, using meta data indicating: MYSQL

2016-04-20 06:29:01.578 [main][INFO ]  org.springframework.batch.core.launch.support.SimpleJobLauncher.afterPropertiesSet:195 - No TaskExecutor has been set, defaulting to synchronous executor.

2016-04-20 06:29:02.108 [main][INFO ]  org.springframework.batch.core.launch.support.SimpleJobLauncher.run:133 - Job: [FlowJob: [name=job1]] launched with the following parameters: [{id=7a3f2560-c837-4f5d-944b-c267aa4f4f45}]

2016-04-20 06:29:02.145 [main][INFO ]  org.springframework.batch.core.job.SimpleStepHandler.handleStep:146 - Executing step: [step1]

2016-04-20 06:29:02.230 [main][INFO ]  com.my.app.jobs.job1.reader.Job1Reader.read:41 - Job1 reader: Data1[id=7a3f2560-c837-4f5d-944b-c267aa4f4f45,name=main,date=Wed Apr 20 06:29:02 KST 2016]

2016-04-20 06:29:02.230 [main][INFO ]  com.my.app.jobs.job1.processor.Job1Processor.process:22 - Job1 process: Data1[id=7a3f2560-c837-4f5d-944b-c267aa4f4f45,name=main,date=Wed Apr 20 06:29:02 KST 2016]

2016-04-20 06:29:02.230 [main][INFO ]  com.my.app.jobs.job1.writer.Job1Writer.write:22 - Job1 write: Data1[id=7a3f2560-c837-4f5d-944b-c267aa4f4f45,name=main,date=Wed Apr 20 06:29:02 KST 2016]

2016-04-20 06:29:02.245 [main][INFO ]  org.springframework.batch.core.launch.support.SimpleJobLauncher.run:136 - Job: [FlowJob: [name=job1]] completed with the following parameters: [{id=7a3f2560-c837-4f5d-944b-c267aa4f4f45}] and the following status: [COMPLETED]

2016-04-20 06:29:02.245 [main][INFO ]  org.springframework.context.support.GenericXmlApplicationContext.doClose:960 - Closing org.springframework.context.support.GenericXmlApplicationContext@13fee20c: startup date [Wed Apr 20 06:29:00 KST 2016]; root of context hierarchy

끝.