Data Format Appendix. Apache Camel

PDF

Dokument

Annonce

Data Format Appendix

DATA FORMAT

Camel supports a pluggable DataFormat to allow messages to be marshalled to and from binary or text formats to support a kind of Message Translator.

The following data formats are currently supported:

• Standard JVM object marshalling

◦ Serialization

◦ String

• Object marshalling

◦

JSON

◦

Protobuf

• Object/XML marshalling

◦ Castor

◦ JAXB

◦ XmlBeans

◦ XStream

• Object/XML/Webservice marshalling

◦ SOAP

• Flat data structure marshalling

◦

Artix Data Services

◦

Bindy

◦ CSV

◦ EDI

◦ Flatpack DataFormat

• Domain specific marshalling

◦ HL7 DataFormat

• Compression

◦ GZip data format

◦ Zip DataFormat

• Security

◦

Crypto

◦ XMLSecurity DataFormat

• Misc.

◦ TidyMarkup

◦ Syslog

And related is the following Type Converters:

▪ Dozer Type Conversion

DATA FORMAT APPENDIX 248

Unmarshalling

If you receive a message from one of the Camel Components such as File, HTTP or JMS you often want to unmarshal the payload into some bean so that you can process it using some

Bean Integration or perform Predicate evaluation and so forth. To do this use the

unmarshal

word in the DSL in Java or the Xml Configuration.

For example

DataFormat jaxb = new JaxbDataFormat( "com.acme.model" ); from( "activemq:My.Queue" ).

unmarshal(jaxb).

to( "mqseries:Another.Queue" );

The above uses a named DataFormat of jaxb which is configured with a number of Java package names. You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file.

You can also use the DSL itself to define the data format as you use it. For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to

ActiveMQ from( "file: //foo/bar" ).

unmarshal().serialization().

to( "activemq:Some.Queue" );

Marshalling

Marshalling is the opposite of unmarshalling, where a bean is marshalled into some binary or textual format for transmission over some transport via a Camel Component. Marshalling is used in the same way as unmarshalling above; in the DSL you can use a DataFormat instance, you can configure the DataFormat dynamically using the DSL or you can refer to a named instance of the format in the Registry.

The following example unmarshals via serialization then marshals using a named JAXB data format to perform a kind of Message Translator from( "file: //foo/bar" ).

unmarshal().serialization().

marshal( "jaxb" ).

to( "activemq:Some.Queue" );

Using Spring XML

This example shows how to configure the data type just once and reuse it on multiple routes

249 DATA FORMAT APPENDIX

</dataFormats>

<route>

</route>

<route>

</route>

</camelContext>

You can also define reusable data formats as Spring beans

</bean>

ARTIX DATA SERVICES

Deprecated, will be removed in Apache Camel 2.1

The Artix DS Data Format supports the Artix Data Services (ADS) product which provides a framework for reading and writing a huge number of vertical message payloads like SWIFT,

SEPA, FpML, TWIST, ISO 20022, CREST and FIX. In addition ADS provides tooling and a framework for reading and writing any legacy binary or text file using any kind of encoding like fixed width, delimited, XML, CSV and so forth.

ADS also provides a transformation framework making it very easy to implement the

Message Translator pattern using the ADS tooling to design the transformation.

Unmarhalling

The first step to using ADS is usually to unmarshal some message from one of the Camel

Components like File, HTTP or JMS etc.

from( "activemq:InputQueue" ).

unmarshal().artixDS(DocumentElement.class).

to( "mqseries:OutputQueue" );

DATA FORMAT APPENDIX 250

The above unmarshals using the Artix DS Data Format for the element

DocumentElement

which is generated by the Artix DS tooling; DocumentElement is the root element of the message structure.

The above will use the default formatting for the data type. However with Artix DS you can switch from the default format to other formats easily. So you could add a specific format if you wish...

from( "activemq:InputQueue" ).

unmarshal().artixDS(DocumentElement.class, ArtixDSContentType.Xml).

to( "mqseries:OutputQueue" );

If you use static imports this can be even more readable...

unmarshal().artixDS(DocumentElement.class, Xml).

Unmarshalling SWIFT messages

If you are working with SWIFT messages then as of camel-artixds version 1.3.6.0 or later there is a handy SwiftFormat helper class which avoids you having to know which Element class you want to use.

So you can unmarshal SWIFT messages using this code from( "activemq:InputQueue" ).

unmarshal( new SwiftFormat()).

to( "mqseries:OutputQueue" );

Marshalling

Marshalling is the reverse of unmarshalling suprise suprise .

Here's an example which unmarshals using one format (XML) and then marshals using a different format (in this case tagged value pairs).

from( "activemq:XmlInput" ).

unmarshal().artixDS(DocumentElement.class, ArtixDSContentType.Xml).

marshal().artixDS(ArtixDSContentType.TagValuePair).

to( "mqseries:TagOutput" );

Type conversions

An alternative to explicit unmarshalling in the DSL you can just use the common

convertBodyTo(Class) method in the DSL to convert using the default content type to a

251 DATA FORMAT APPENDIX

particular ComplexDataObject from Artix DS. This mechanism uses the inbuilt Camel Type

Converter mechanism ot automatically marshal and unmarshal using the default content type for a model.

For example the following...

from( "activemq:InputQueue" ).

convertBodyTo(DocumentElement.class).

to( "mqseries:OutputQueue" );

Is equivalent to this from( "activemq:InputQueue" ).

unmarshal().artixDS(DocumentElement.class).

to( "mqseries:OutputQueue" );

Using Transformations

To use the Message Translator pattern with ADS its a simple matter of using the ADS tooling to create your transformation, then just using the generated transformation class in the Bean

Integration in Camel.

For example image you define a transformation in the ADS IDE to translate SWIFT to FIX format. You will then have a generated SwiftToFix Java class. You can then use the transformation in your Camel DSL via Java or the Xml Configuration as follows from( "activemq:SwiftQueue" ).

bean(SwiftToFix.class).

to( "mqseries:FixQueue" );

Configuring via Spring XML

The following example shows how to use Artix DS using Spring XML; in this case it unmarshals the content of a JMS queue as XML using the Artix DS data model; then marshals it using a tag/ value pair.

<route>

</unmarshal>

</marshal>

DATA FORMAT APPENDIX 252

</route>

</camelContext>

Content Types and auto discovery

You may have spotted in the above that we use the ArtixDSContentType which is an enum in

Java and the Xml Configuration to describe the kind of XML format to use such as binary, XML,

Text etc.

If no content type is specified we always use the default content type of the Artix DS model in question. This is equivalent to the

Default content type.

If you wish to be flexible in what you accept or emit, we also support the

Auto content

type which will look for the

Content-Type header on the input message and use that to try

determine which of the content types to use; if none can be found then

Default is used.

e.g. you could support content posted with a MIME type of

application/xml to indicate

XML or

application/x-java-serialized-object for serialization or text/plain for text

etc.

Using camel-artixds

To use this module you need to use the FUSE Mediation Router distribution. Or you could just add the following to your pom.xml, substituting the version number for the latest & greatest release.

<groupId>org.apache.camel</groupId>

<artifactId>camel-artixds</artifactId>

</dependency>

And ensure you are pointing at the maven repo

<name>FUSESource Open Source Community Release Repository</name>

<url>http: //repo.fusesource.com/maven2/</url>

<enabled> false </enabled>

</snapshots>

</releases>

</repository>

253 DATA FORMAT APPENDIX

SERIALIZATION

Serialization is a Data Format which uses the standard Java Serialization mechanism to unmarshal a binary payload into Java objects or to marshal Java objects into a binary blob.

For example the following uses Java serialization to unmarshal a binary file then send it as an

ObjectMessage to ActiveMQ from( "file: //foo/bar" ).

unmarshal().serialization().

to( "activemq:Some.Queue" );

Dependencies

This data format is provided in

camel-core so no additional dependencies is needed.

JAXB

JAXB is a Data Format which uses the JAXB2 XML marshalling standard which is included in

Java 6 to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

Using the Java DSL

For example the following uses a named DataFormat of jaxb which is configured with a number of Java package names to initialize the JAXBContext.

DataFormat jaxb = new JaxbDataFormat( "com.acme.model" ); from( "activemq:My.Queue" ).

unmarshal(jaxb).

to( "mqseries:Another.Queue" );

You can if you prefer use a named reference to a data format which can then be defined in your

Registry such as via your Spring XML file. e.g.

from( "activemq:My.Queue" ).

unmarshal( "myJaxbDataType" ).

to( "mqseries:Another.Queue" );

Using Spring XML

The following example shows how to use JAXB to unmarshal using Spring configuring the jaxb data type

DATA FORMAT APPENDIX 254

<route>

</unmarshal>

</route>

</camelContext>

This example shows how to configure the data type just once and reuse it on multiple routes.

For Camel versions below 1.5.0 you have to set the <jaxb> element directly in

<camelContext>.

</dataFormats>

<route>

</route>

<route>

</route>

</camelContext>

Partial marshalling/unmarshalling

This feature is new to Camel 2.2.0.

JAXB 2 supports marshalling and unmarshalling XML tree fragments. By default JAXB looks for

@XmlRootElement annotation on given class to operate on whole XML tree. This is useful but not always - sometimes generated code does not have @XmlRootElement annotation, sometimes you need unmarshall only part of tree.

In that case you can use partial unmarshalling. To enable this behaviours you need set property partClass. Camel will pass this class to JAXB's unmarshaler.

<route>

<jaxb prettyPrint= "false" contextPath= "org.apache.camel.example"

255 DATA FORMAT APPENDIX

Multiple context paths

It is possible to use this data format with more than one context path. You can specify context path using : as separator, for example com.mycompany:com.mycompany2. Note that this is handled by JAXB implementation and might change if you use different vendor than RI.

partClass= "org.apache.camel.example.PurchaseOrder" partNamespace= "{http://example.camel.org/apache}po" />

</marshal>

</route>

<route>

</unmarshal>

</route>

</camelContext>

For marshalling you have to add partNamespace attribute with QName of destination namespace. Example of Spring DSL you can find above.

Ignoring the NonXML Character

This feature is new to Camel 2.2.0.

JaxbDataFromat supports to ignore the NonXML Character, you just need to set the filterNonXmlChars property to be true, JaxbDataFromat will replace the NonXML character with " " when it is marshaling or unmarshaling the message. You can also do it by setting the

Exchange property Exchange.FILTER_NON_XML_CHARS.

JDK 1.5

JDK 1.6+

Filtering in use

StAX API and implementation No

Filtering not in use

StAX API only No

This feature has been tested with Woodstox 3.2.9 and Sun JDK 1.6 StAX implementation.

Working with the ObjectFactory

If you use XJC to create the java class from the schema, you will get an ObjectFactory for you

JAXB context. Since the ObjectFactory uses JAXBElement to hold the reference of the schema and element instance value, from Camel 1.5.1 jaxbDataformat will ignore the JAXBElement by

DATA FORMAT APPENDIX 256

default and you will get the element instance value instead of the JAXBElement object form the unmarshaled message body.

If you want to get the JAXBElement object form the unmarshaled message body, you need to set the JaxbDataFormat object's ignoreJAXBElement property to be false.

Setting encoding

In Camel 1.6.1 and newer you can set the

encoding option to use when marshalling. Its the

Marshaller.JAXB_ENCODING encoding property on the JAXB Marshaller.

You can setup which encoding to use when you declare the JAXB data format. You can also provide the encoding in the Exchange property Exchange.CHARSET_NAME. This property will overrule the encoding set on the JAXB data format.

In this Spring DSL we have defined to use iso-8859-1 as the encoding:

<route>

</marshal>

</route>

</camelContext>

Dependencies

To use JAXB in your camel routes you need to add the a dependency on

camel-jaxb which

implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-jaxb</artifactId>

</dependency>

XMLBEANS

XmlBeans is a Data Format which uses the XmlBeans library to unmarshal an XML payload into

Java objects or to marshal Java objects into an XML payload.

257 DATA FORMAT APPENDIX

from( "activemq:My.Queue" ).

unmarshal().xmlBeans().

to( "mqseries:Another.Queue" );

Dependencies

To use XmlBeans in your camel routes you need to add the a dependency on

camel-

xmlbeans which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-xmlbeans</artifactId>

</dependency>

XSTREAM

XStream is a Data Format which uses the XStream library to marshal and unmarshal Java objects to and from XML.

// lets turn Object messages into XML then send to MQSeries from( "activemq:My.Queue" ).

marshal().xstream().

to( "mqseries:Another.Queue" );

XMLInputFactory and XMLOutputFactory

The XStream library uses the javax.xml.stream.XMLInputFactory and javax.xml.stream.XMLOutputFactory, you can control which implementation of this factory should be used.

The Factory is discovered using this algorithm:

1. Use the javax.xml.stream.XMLInputFactory , javax.xml.stream.XMLOutputFactory system property.

2. Use the lib/xml.stream.properties file in the JRE_HOME directory.

3. Use the Services API, if available, to determine the classname by looking in the META-INF/ services/javax.xml.stream.XMLInputFactory, META-INF/services/ javax.xml.stream.XMLOutputFactory files in jars available to the JRE.

4. Use the platform default XMLInputFactory,XMLOutputFactory instance.

DATA FORMAT APPENDIX 258

How to set the XML encoding in Xstream DataFormat?

From Camel 1.6.3 or Camel 2.2.0, you can set the encoding of XML in Xstream DataFormat by setting the Exchange's property with the key Exchange.CHARSET_NAME, or setting the encoding property on Xstream from DSL or Spring config.

from( "activemq:My.Queue" ).

marshal().xstream( "UTF-8" ).

to( "mqseries:Another.Queue" );

</dataFormats>

<route>

</route>

<route>

</route>

</camelContext>

Dependencies

To use XStream in your camel routes you need to add the a dependency on

camel-xstream

which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-xstream</artifactId>

</dependency>

259 DATA FORMAT APPENDIX

CSV

The CSV Data Format uses Apache Commons CSV to handle CSV payloads (Comma Separated

Values) such as those exported/imported by Excel.

Options

Option

config strategy autogenColumn delimiter

Type Description

CSVConfig Can be used to set a custom CSVConfig object.

CSVStrategy

Camel uses by default

CSVStrategy.DEFAULT_STRATEGY.

boolean

String

Camel 1.6.1/2.0: Is default true. By default, columns

are autogenerated in the resulting CSV. Subsequent messages use the previously created columns with new fields being added at the end of the line.

Camel 2.4: Is default ,. Can be used to configure the

delimiter, if it's not the comma.

Marshalling a Map to CSV

The component allows you to marshal a Java Map (or any other message type that can be converted in a Map) into a CSV payload.

An example: if you send a message with this map...

Map< String , Object > body = new HashMap< String , Object >(); body.put( "foo" , "abc" ); body.put( "bar" , 123);

... through this route ...

from( "direct:start" ).

marshal().csv().

to( "mock:result" );

... you will end up with a String containing this CSV message abc,123

Sending the Map below through this route will result in a CSV message that looks like foo,bar

DATA FORMAT APPENDIX 260

Unmarshalling a CSV message into a Java List

Unmarshalling will transform a CSV messsage into a Java List with CSV file lines (containing another List with all the field values).

An example: we have a CSV file with names of persons, their IQ and their current activity.

Jack Dalton, 115, mad at Averell

Joe Dalton, 105, calming Joe

William Dalton, 105, keeping Joe from killing Averell

Averell Dalton, 80, playing with Rantanplan

Lucky Luke, 120, capturing the Daltons

We can now use the CSV component to unmarshal this file: from( "file:src/test/resources/?fileName=daltons.csv&noop= true " ).

unmarshal().csv().

to( "mock:daltons" );

The resulting message will contain a List<List<String>> like...

List<List< String >> data = (List<List< String >>) exchange.getIn().getBody(); for (List< String > line : data) {

LOG.debug( String .format( "%s has an IQ of %s and is currently %s" , line.get(0), line.get(1), line.get(2)));

}

Marshalling a List<Map> to CSV

Available as of Camel 2.1

If you have multiple rows of data you want to be marshalled into CSV format you can now store the message payload as a List<Map<String, Object>> object where the list contains a Map for each row.

File Poller of CSV, then unmarshaling

Given a bean which can handle the incoming data...

// Some comments here public void doHandleCsvData(List<List< String >> csvData)

{

// do magic here

}

... your route then looks as follows

261 DATA FORMAT APPENDIX

<route>

</route>

Marshaling with a pipe as delimiter

Using the Spring/XML DSL:

<route>

</marshal>

</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();

CSVConfig config = new CSVConfig(); config.setDelimiter('|'); csv.setConfig(config); from( "direct:start" )

.marshal(csv)

.convertBodyTo( String .class)

.to( "bean:myCsvHandler?method=doHandleCsv" );

CsvDataFormat csv = new CsvDataFormat(); csv.setDelimiter( "|" ); from( "direct:start" )

.marshal(csv)

.convertBodyTo( String .class)

.to( "bean:myCsvHandler?method=doHandleCsv" );

Unmarshaling with a pipe as delimiter

Using the Spring/XML DSL:

DATA FORMAT APPENDIX 262

<route>

</unmarshal>

</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();

CSVStrategy strategy = CSVStrategy.DEFAULT_STRATEGY; strategy.setDelimiter('|'); csv.setStrategy(strategy); from( "direct:start" )

.unmarshal(csv)

.to( "bean:myCsvHandler?method=doHandleCsv" );

CsvDataFormat csv = new CsvDataFormat(); csv.setDelimiter( "|" ); from( "direct:start" )

.unmarshal(csv)

.to( "bean:myCsvHandler?method=doHandleCsv" );

Dependencies

To use CSV in your camel routes you need to add the a dependency on

camel-csv which

implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-csv</artifactId>

</dependency>

The String Data Format is a textual based format that supports encoding.

Options

Option Default Description

263 DATA FORMAT APPENDIX

charset null

To use a specific charset for encoding. If not provided Camel will use the JVM default charset.

Marshal

In this example we marshal the file content to String object in UTF-8 encoding.

from( "file: //data.csv" ).marshal().string("UTF-8").to("jms://myqueue");

Unmarshal

In this example we unmarshal the payload from the JMS queue to a String object using UTF-8 encoding, before its processed by the newOrder processor.

from( "jms: //queue/order" ).unmarshal().string("UTF-8").processRef("newOrder");

Dependencies

This data format is provided in

camel-core so no additional dependencies is needed.

HL7 DataFormat

The HL7 component ships with a HL7 data format that can be used to format between

String and HL7 model objects.

▪ marshal = from Message to byte stream (can be used when returning as response using the HL7 MLLP codec)

▪ unmarshal = from byte stream to Message (can be used when receiving streamed data from the HL7 MLLP

To use the data format, simply instantiate an instance and invoke the marhsal or unmarshl operation in the route builder:

DataFormat hl7 = new HL7DataFormat();

...

from( "direct:hl7in" ).marshal(hl7).to( "jms:queue:hl7out" );

In the sample above, the HL7 is marshalled from a HAPI Message object to a byte stream and put on a JMS queue.

The next example is the opposite:

DATA FORMAT APPENDIX 264

DataFormat hl7 = new HL7DataFormat();

...

from( "jms:queue:hl7out" ).unmarshal(hl7).to( "patientLookupService" );

Here we unmarshal the byte stream into a HAPI Message object that is passed to our patient lookup service.

Notice there is a shorthand syntax in Camel for well-known data formats that is commonly used.

Then you don't need to create an instance of the HL7DataFormat object: from( "direct:hl7in" ).marshal().hl7().to( "jms:queue:hl7out" ); from( "jms:queue:hl7out" ).unmarshal().hl7().to( "patientLookupService" );

EDI DATAFORMAT

This component ships with a EDI dataformat that can be used to format from a java.io.InputStream to XML as a org.w3c.Document Object.

• marshal = currently not supported by Smooks

• unmarshal = from stream to XML (can be used when reading EDI files)

The EDIDataFormat must be configued with either a:

• setSmooksConfig(configfile) = a fully Smooks configuration file

• setMappingModel(modelfile) = just the mapping model xml file and Camel will use a default Smooks configuration

To use the data format simply instantiate an instance, set the configuration (above) and invoke the unmarshal operation in the route builder:

DataFormat edi = new EDIDataFormat(); edi.setMappingModel( "my-order-mapping.xml" );

...

from( "file: //edi/in" ).

unmarshal(edi).

to( "jms:queue:edi" );

And you can also provide the full Smooks configuration file where you can configure Smooks as you want, in case the default configuration isn't useful:

DataFormat edi = new EDIDataFormat(); edi.setSmooksConfig( "my-smooks-config.xml" );

...

from( "file: //edi/in" ).

unmarshal(edi).

to( "jms:queue:edi" );

265 DATA FORMAT APPENDIX

Dependencies

To use EDI in your camel routes you need to add the a dependency on

camel-smooks

which implements this data format.

This component is hosted at the Camel Extra project since the Smooks library uses a licenses which cant be included directly in an Apache project.

FLATPACK DATAFORMAT

The Flatpack component ships with the Flatpack data format that can be used to format between fixed width or delimited text messages to a List of rows as Map.

▪ marshal = from List<Map<String, Object>> to OutputStream (can be converted to String)

▪ unmarshal = from java.io.InputStream (such as a File or String) to a java.util.List as an one by one you can split the exchange, using Splitter.

Notice: The Flatpack library does currently not support header and trailers for the marshal

operation.

org.apache.camel.component.flatpack.DataSetList instance.

The result of the operation will contain all the data. If you need to process each row

Options

The data format has the following options:

Option Default Description

definition null

The flatpack pzmap configuration file. Can be omitted in simpler situations, but its preferred to use the pzmap.

fixed ignoreFirstRecord false

Delimited or fixed.

true

Whether the first line is ignored for delimited files

(for the column headers).

textQualifier delimiter

If the text is qualified with a char such as ".

The delimiter char (could be ; , or similar) parserFactory null Uses the default Flatpack parser factory.

Usage

To use the data format, simply instantiate an instance and invoke the marhsal or unmarshal operation in the route builder:

DATA FORMAT APPENDIX 266

FlatpackDataFormat fp = new FlatpackDataFormat(); fp.setDefinition( new ClassPathResource( "INVENTORY-Delimited.pzmap.xml" ));

...

from( "file:order/in" ).unmarshal(df).to( "seda:queue:neworder" );

The sample above will read files from the order/in folder and unmarshal the input using the

Flatpack configuration file INVENTORY-Delimited.pzmap.xml that configures the structure of the files. The result is a DataSetList object we store on the SEDA queue.

FlatpackDataFormat df = new FlatpackDataFormat(); df.setDefinition( new ClassPathResource( "PEOPLE-FixedLength.pzmap.xml" )); df.setFixed( true ); df.setIgnoreFirstRecord( false ); from( "seda:people" ).marshal(df).convertBodyTo( String .class).to( "jms:queue:people" );

In the code above we marshal the data from a Object representation as a List of rows as

Maps. The rows as Map contains the column name as the key, and the the corresponding value. This structure can be created in Java code from e.g. a processor. We marshal the data according to the Flatpack format and convert the result as a String object and store it on a

JMS queue.

Dependencies

To use Flatpack in your camel routes you need to add the a dependency on

camel-flatpack

which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-flatpack</artifactId>

</dependency>

JSON

JSON is a Data Format to marshal and unmarshal Java objects to and from JSON.

In Camel 1.6 its only the XStream library that is supported and its default.

In Camel 2.0 we added support for more libraries:

Camel provides integration with two popular JSon libraries:

▪ The XStream library and Jettsion

▪ The Jackson library

267 DATA FORMAT APPENDIX

By default Camel uses the XStream library.

Using JSon data format with the XStream library

// lets turn Object messages into json then send to MQSeries from( "activemq:My.Queue" ).

marshal().json().

to( "mqseries:Another.Queue" );

Using Json data format with the Jackson library

// lets turn Object messages into json then send to MQSeries from( "activemq:My.Queue" ).

marshal().json(JsonLibrary.Jackson).

to( "mqseries:Another.Queue" );

Using Json in Spring DSL

When using Data Format in Spring DSL you need to declare the data formats first. This is done in the

DataFormats XML tag.

<!-- here we define a Json data format with the id jack and that it should use the TestPojo as the class type when doing unmarshal. The unmarshalTypeName is optional, if not provided

Camel will use a Map as the type -->

</dataFormats>

And then you can refer to this id in the route:

<route>

</route>

Dependencies for XStream

To use JSON in your camel routes you need to add the a dependency on

camel-xstream

which implements this data format.

DATA FORMAT APPENDIX 268

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-xstream</artifactId>

</dependency>

Dependencies for Jackson

To use JSON in your camel routes you need to add the a dependency on

camel-jackson

which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-jackson</artifactId>

</dependency>

The Zip Data Format is a message compression and de-compression format. Messages marshalled using Zip compression can be unmarshalled using Zip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads. It facilitates more optimal use of network bandwidth while incurring a small cost in order to compress and decompress payloads at the endpoint.

Options

Option

compressionLevel

Default Description

null

To specify a specific compression Level use java.util.zip.Deflater settings. The possible settings are

- Deflater.BEST_SPEED

- Deflater.BEST_COMPRESSION

- Deflater.DEFAULT_COMPRESSION

If compressionLevel is not explicitly specified the compressionLevel employed is

Deflater.DEFAULT_COMPRESSION

269 DATA FORMAT APPENDIX

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing zip compression Deflater.BEST_COMPRESSION and send it an ActiveMQ queue called

MY_QUEUE.

from( "direct:start" ).marshal().zip(Deflater.BEST_COMPRESSION).to( "activemq:queue:MY_QUEUE" );

Alternatively if you would like to use the default setting you could send it as from( "direct:start" ).marshal().zip().to( "activemq:queue:MY_QUEUE" );

Unmarshal

In this example we unmarshal a zipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnZippedMessageProcessor. Note that the compression Level employed during the marshalling should be identical to the one employed during unmarshalling to avoid errors.

from( "activemq:queue:MY_QUEUE" ).unmarshal().zip().process( new

UnZippedMessageProcessor());¬†

Dependencies

This data format is provided in

camel-core so no additional dependencies is needed.

TIDYMARKUP

TidyMarkup is a Data Format that uses the TagSoup to tidy up HTML. It can be used to parse ugly HTML and return it as pretty wellformed HTML.

TidyMarkup only supports the

unmarshal operation as we really don't want to turn well

formed HTML into ugly HTML

Java DSL Example

An example where the consumer provides some HTML from( "file: //site/inbox" ).unmarshal().tidyMarkup().to("file://site/blogs");

DATA FORMAT APPENDIX 270

Camel eats our own dog food soap

We had some issues in our pdf Manual where we had some strange symbols. So

Jonathan used this data format to tidy up the wiki html pages that are used as base for rendering the pdf manuals. And then the mysterious symbols vanished.

Spring XML Example

The following example shows how to use TidyMarkup to unmarshal using Spring

<route>

</unmarshal>

</route>

</camelContext>

Dependencies

To use TidyMarkup in your camel routes you need to add the a dependency on

camel-

tagsoup which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-tagsoup</artifactId>

</dependency>

BINDY

Available as of Camel 2.0

The idea that the developers has followed to design this component was to allow the parsing/binding of non structured data (or to be more precise non-XML data) to Java Bean using annotations. Using Bindy, you can bind data like :

▪ CSV record,

▪ Fixedlength record,

▪

FIX messages,

▪ or any other non-structured data

271 DATA FORMAT APPENDIX

to one or many Plain Old Java Object (POJO) and to convert the data according to the type of the java property. POJO can be linked together and relation one to many is available in some cases. Moreover, for data type like Date, Double, Float, Integer, Short, Long and BigDecimal, you can provide the pattern to apply during the formatting of the property.

For the BigDecimal number, you can also define the precision and the decimal or grouping separators

Type

Format

Type

Pattern example

Link

Date DateFormat "dd-MM-yyyy" http://java.sun.com/j2se/1.5.0/docs/api/java/ text/SimpleDateFormat.html

Decimal* Decimalformat "##.###.###" http://java.sun.com/j2se/1.5.0/docs/api/java/ text/DecimalFormat.html

Decimal* = Double, Integer, Float, Short, Long

To work with camel-bindy, you must first define your model in a package (e.g.

com.acme.model) and for each model class (e.g. Order, Client, Instrument, ...) associate the required annotations (described hereafter) with Class or property name.

ANNOTATIONS

The annotations created allow to map different concept of your model to the POJO like :

▪ Type of record (csv, key value pair (e.g. FIX message), fixed length ...),

▪ Link (to link object in another object),

▪ DataField and their properties (int, type, ...),

▪ KeyValuePairField (for key = value format like we have in FIX financial messages),

▪ Section (to identify header, body and footer section),

▪ OneToMany

This section will describe them :

1. CsvRecord

The CsvRecord annotation is used to identified the root class of the model. It represents a record = a line of a CSV file and can be linked to several children model classes.

Annotation name Record type Level

CsvRecord

csv Class

Parameter name type Info

DATA FORMAT APPENDIX 272

Format supported

This first release only support comma separated values fields and key value pair fields (e.g. : FIX messages).

separator skipFirstLine crlf string boolean string mandatory - can be ',' or ';' or 'anything'. This value is interpreted as a regular expression. If you want to use a sign which has a special meaning in regular expressions, e.g. the '|' sign, than you have to mask it, like '

|' optional - default value = false - allow to skip the first line of the CSV file optional - default value = WINDOWS - allow to define the carriage return character to use generateHeaderColumns isOrdered boolean boolean optional - default value = false - uses to generate the header columns of the CSV generates optional - default value = false - allow to change the order of the fields when CSV is generated

This annotation is associated to the root class of the model and must be declared one time.

case 1 : separator = ','

The separator used to segregate the fields in the CSV record is ',' :

10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

@CsvRecord( separator = "," ) public Class Order {

...

}

case 2 : separator = ';'

Compare to the previous case, the separator here is ';' instead of ',' :

10; J; Pauline; M; XD12345678; Fortis Dynamic 15/15; 2500; USD; 08-01-2009

@CsvRecord( separator = ";" ) public Class Order {

273 DATA FORMAT APPENDIX

...

}

case 3 : separator = '|'

Compare to the previous case, the separator here is '|' instead of ';' :

10| J| Pauline| M| XD12345678| Fortis Dynamic 15/15| 2500| USD| 08-01-2009

@CsvRecord( separator = "\\|" ) public Class Order {

...

}

case 3 : separator = '\",\"'

When the field to be parsed of the CSV record contains ',' or ';' which is also used as separator, we whould find another strategy to tell camel bindy how to handle this case. To define the field containing the data with a comma, you will use simple or double quotes as delimiter (e.g : '10', 'Street 10, NY', 'USA' or "10", "Street 10, NY", "USA").

Remark : In this case, the first and last character of the line which are a simple or double quotes will removed by bindy

"10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009"

@CsvRecord( separator = "\" ,\"" ) public Class Order {

...

}

case 5 : separator & skipfirstline

The feature is interesting when the client wants to have in the first line of the file, the name of the data fields : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date

To inform bindy that this first line must be skipped during the parsing process, then we use the attribute :

@CsvRecord(separator = "," , skipFirstLine = true ) public Class Order {

...

}

case 6 : generateHeaderColumns

DATA FORMAT APPENDIX 274

To add at the first line of the CSV generated, the attribute generateHeaderColumns must be set to true in the annotation like this :

@CsvRecord( generateHeaderColumns = true ) public Class Order {

...

}

As a result, Bindy during the unmarshaling process will generate CSV like this : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date

10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

case 7 : carriage return

If the platform where camel-bindy will run is not Windows but Macintosh or Unix, than you can change the crlf property like this. Three values are available : WINDOWS, UNIX or MAC

@CsvRecord(separator = "," , crlf= "MAC" ) public Class Order {

...

}

case 8 : isOrdered

Sometimes, the order to follow during the creation of the CSV record from the model is different from the order used during the parsing. Then, in this case, we can use the attribute isOrdered = true to indicate this in combination with attribute 'position' of the DataField annotation.

@CsvRecord(isOrdered = true ) public Class Order {

@DataField(pos = 1, position = 11) private int orderNr;

@DataField(pos = 2, position = 10) private String clientNr;

...

}

Remark : pos is used to parse the file, stream while positions is used to generate the CSV

2. Link

The link annotation will allow to link objects together.

275 DATA FORMAT APPENDIX

Annotation name Record type Level

Link

all Class & Property

Parameter name type Info

linkType LinkType optional - by default the value is LinkType.oneToOne - so you are not obliged to mention it

Only one-to-one relation is allowed.

e.g : If the model Class Client is linked to the Order class, then use annotation Link in the

Order class like this :

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1) private int orderNr;

...

@Link private Client client;

AND for the class Client :

@Link public class Client {

...

}

3. DataField

The DataField annotation defines the property of the field. Each datafield is identified by its position in the record, a type (string, int, date, ...) and optionally of a pattern

Annotation name Record type Level

DataField

all Property

Parameter name

pos

type

int

Info

pattern string mandatory - digit number starting from 1 to ...

optional - default value = "" - will be used to format Decimal,

Date, ...

DATA FORMAT APPENDIX 276

length precision int int optional - represents the length of the field for fixed length format optional - represents the precision to be used when the Decimal number will be formatted/parsed pattern position string int optional - default value = "" - is used by the Java Formater

(SimpleDateFormat by example) to format/validate data optional - must be used when the position of the field in the CSV generated must be different compare to pos boolean optional - default value = "false" required trim boolean optional - default value = "false"

case 1 : pos

This parameter/attribute represents the position of the field in the csv record

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1) private int orderNr;

@DataField(pos = 5) private String isinCode;

...

}

As you can see in this example the position starts at '1' but continues at '5' in the class Order.

The numbers from '2' to '4' are defined in the class Client (see here after).

public class Client {

@DataField(pos = 2) private String clientNr;

@DataField(pos = 3) private String firstName;

...

}

@DataField(pos = 4) private String lastName;

case 2 : pattern

The pattern allows to enrich or validates the format of your data

277 DATA FORMAT APPENDIX

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1) private int orderNr;

@DataField(pos = 5) private String isinCode;

@DataField(name = "Name" , pos = 6) private String instrumentName;

@DataField(pos = 7, precision = 2) private BigDecimal amount;

@DataField(pos = 8) private String currency;

@DataField(pos = 9, pattern = "dd-MM-yyyy" ) -- pattern used during parsing or when the date is created private Date orderDate;

...

}

case 3 : precision

The precision is helpful when you want to define the decimal part of your number

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1) private int orderNr;

@Link private Client client;

@DataField(pos = 5) private String isinCode;

@DataField(name = "Name" , pos = 6) private String instrumentName;

@DataField(pos = 7, precision = 2) -- precision private BigDecimal amount;

@DataField(pos = 8) private String currency;

@DataField(pos = 9, pattern = "dd-MM-yyyy" ) private Date orderDate;

DATA FORMAT APPENDIX 278

...

}

case 4 : Position is different in output

The position attribute will inform bindy how to place the field in the CSV record generated.

By default, the position used corresponds to the position defined with the attribute 'pos'. If the position is different (that means that we have an asymetric processus comparing marshaling from unmarshaling) than we can use 'position' to indicate this.

Here is an example

@CsvRecord(separator = "," ) public class Order {

@CsvRecord(separator = "," , isOrdered = true ) public class Order {

// Positions of the fields start from 1 and not from 0

@DataField(pos = 1, position = 11) private int orderNr;

@DataField(pos = 2, position = 10) private String clientNr;

@DataField(pos = 3, position = 9) private String firstName;

@DataField(pos = 4, position = 8) private String lastName;

@DataField(pos = 5, position = 7) private String instrumentCode;

...

}

@DataField(pos = 6, position = 6) private String instrumentNumber;

case 5 : required

If a field is mandatory, simply use the attribute 'required' setted to true

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1) private int orderNr;

@DataField(pos = 2, required = true )

279 DATA FORMAT APPENDIX

This attribute of the annotation @DataField must be used in combination with attribute isOrdered = true of the annotation @CsvRecord private String clientNr;

@DataField(pos = 3, required = true ) private String firstName;

...

}

@DataField(pos = 4, required = true ) private String lastName;

If this field is not present in the record, than an error will be raised by the parser with the following information :

Some fields are missing (optional or mandatory), line :

case 6 : trim

If a field has leading and/or trailing spaces which should be removed before they are processed, simply use the attribute 'trim' setted to true

@CsvRecord(separator = "," ) public class Order {

@DataField(pos = 1, trim = true ) private int orderNr;

@DataField(pos = 2, trim = true ) private Integer clientNr;

@DataField(pos = 3, required = true ) private String firstName;

...

}

@DataField(pos = 4) private String lastName;

4. FixedLengthRecord

The FixedLengthRecord annotation is used to identified the root class of the model. It represents a record = a line of a file/message containing data fixed length formatted and can be linked to several children model classes. This format is a bit particular beause data of a field can be aligned to the right or to the left.

DATA FORMAT APPENDIX 280

When the size of the data does not fill completely the length of the field, then we add 'padd' characters.

Annotation name Record type Level

FixedLengthRecord fixed

Class

Parameter name type Info

crlf string paddingChar char length int hasHeader hasFooter boolean boolean optional - default value = WINDOWS - allow to define the carriage return character to use mandatory - default value = ' ' mandatory = size of the fixed length record optional - NOT YET IMPLEMENTED optional - NOT YET IMPLEMENTED

This annotation is associated to the root class of the model and must be declared one time.

case 1 : Simple fixed length record

This simple example shows how to design the model to parse/format a fixed message

10A9PaulineMISINXD12345678BUYShare2500.45USD01-08-2009

@FixedLengthRecord(length=54, paddingChar=' ') public static class Order {

@DataField(pos = 1, length=2) private int orderNr;

@DataField(pos = 3, length=2) private String clientNr;

@DataField(pos = 5, length=7) private String firstName;

@DataField(pos = 12, length=1, align= "L" ) private String lastName;

@DataField(pos = 13, length=4) private String instrumentCode;

@DataField(pos = 17, length=10) private String instrumentNumber;

@DataField(pos = 27, length=3) private String orderType;

281 DATA FORMAT APPENDIX

@DataField(pos = 30, length=5) private String instrumentType;

@DataField(pos = 35, precision = 2, length=7) private BigDecimal amount;

@DataField(pos = 42, length=3) private String currency;

@DataField(pos = 45, length=10, pattern = "dd-MM-yyyy" ) private Date orderDate;

...

case 2 : Fixed length record with alignment and padding

This more elaborated example show how to define the alignment for a field and how to assign a padding character which is ' ' here''

10A9 PaulineM ISINXD12345678BUYShare2500.45USD01-08-2009

@FixedLengthRecord(length=60, paddingChar=' ') public static class Order {

@DataField(pos = 1, length=2) private int orderNr;

@DataField(pos = 3, length=2) private String clientNr;

@DataField(pos = 5, length=9) private String firstName;

@DataField(pos = 14, length=5, align= "L" ) the block private String lastName;

@DataField(pos = 19, length=4) private String instrumentCode;

@DataField(pos = 23, length=10) private String instrumentNumber;

@DataField(pos = 33, length=3) private String orderType;

// align text to the LEFT zone of

@DataField(pos = 36, length=5) private String instrumentType;

@DataField(pos = 41, precision = 2, length=7) private BigDecimal amount;

@DataField(pos = 48, length=3)

DATA FORMAT APPENDIX 282

private String currency;

@DataField(pos = 51, length=10, pattern = "dd-MM-yyyy" ) private Date orderDate;

...

case 3 : Field padding

Sometimes, the default padding defined for record cannnot be applied to the field as we have a number format where we would like to padd with '0' instead of ' '. In this case, you can use in the model the attribute paddingField to set this value.

10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009

@FixedLengthRecord(length = 65, paddingChar = ' ') public static class Order {

@DataField(pos = 1, length = 2) private int orderNr;

@DataField(pos = 3, length = 2) private String clientNr;

@DataField(pos = 5, length = 9) private String firstName;

@DataField(pos = 14, length = 5, align = "L" ) private String lastName;

@DataField(pos = 19, length = 4) private String instrumentCode;

@DataField(pos = 23, length = 10) private String instrumentNumber;

@DataField(pos = 33, length = 3) private String orderType;

@DataField(pos = 36, length = 5) private String instrumentType;

@DataField(pos = 41, precision = 2, length = 12, paddingChar = '0') private BigDecimal amount;

@DataField(pos = 53, length = 3) private String currency;

@DataField(pos = 56, length = 10, pattern = "dd-MM-yyyy" ) private Date orderDate;

...

283 DATA FORMAT APPENDIX

5. Message

The Message annotation is used to identified the class of your model who will contain key value pairs fields. This kind of format is used mainly in Financial Exchange Protocol Messages (FIX).

Nevertheless, this annotation can be used for any other format where data are identified by keys. The key pair values are separated each other by a separator which can be a special character like a tab delimitor (unicode representation : \u0009) or a start of heading (unicode representation : \u0001)

Annotation name Record type Level

Message

key value pair Class

Parameter name

pairSeparator keyValuePairSeparair crlf

type

string string string

Info

mandatory - can be '=' or ';' or 'anything' mandatory - can be '\u0001', '\u0009', '#' or 'anything' optional - default value = WINDOWS - allow to define the carriage return character to use type version string string optional - define the type of message (e.g. FIX, EMX, ...) optional - version of the message (e.g. 4.1) isOrdered boolean optional - default value = false - allow to change the order of the fields when FIX message is generated

This annotation is associated to the message class of the model and must be declared one time.

case 1 : separator = 'u0001'

The separator used to segregate the key value pair fields in a FIX message is the ASCII '01' character or in unicode format '\u0001'. This character must be escaped a second time to avoid a java runtime error. Here is an example :

8=FIX.4.1 9=20 34=1 35=0 49=INVMGR 56=BRKR 1=BE.CHM.001 11=CHM0001-01 22=4

...

and how to use the annotation

@Message(keyValuePairSeparator = "=" , pairSeparator = "\u0001" , type= "FIX" , version= "4.1" ) public class Order {

...

}

DATA FORMAT APPENDIX 284

"FIX information"

More information about FIX can be found on this web site : http://www.fixprotocol.org/. To work with FIX messages, the model must contain a

Header and Trailer classes linked to the root message class which could be a Order class. This is not mandatory but will be very helpful when you will use camel-bindy in combination with camel-fix which is a Fix gateway based on quickFix project http://www.quickfixj.org/.

Look at test cases

The ASCII character like tab, ... cannot be displayed in WIKI page. So, have a look to the test case of camel-bindy to see exactly how the FIX message looks like

(src\test\data\fix\fix.txt) and the Order, Trailer, Header classes

(src\test\java\org\apache\camel\dataformat\bindy\model\fix\simple\Order.java)

6. KeyValuePairField

The KeyValuePairField annotation defines the property of a key value pair field. Each

KeyValuePairField is identified by a tag (= key) and its value associated, a type (string, int, date,

...), optionaly a pattern and if the field is required

Annotation name Record type Level

KeyValuePairField Key Value Pair - FIX

Property

Parameter name type Info

tag pattern precision position int string int int mandatory - digit number identifying the field in the message must be unique optional - default value = "" - will be used to format Decimal,

Date, ...

optional - digit number - represents the precision to be used when the Decimal number will be formatted/parsed optional - must be used when the position of the key/tag in the

FIX message must be different required boolean optional - default value = "false"

case 1 : tag

This parameter represents the key of the field in the message

285 DATA FORMAT APPENDIX

@Message(keyValuePairSeparator = "=" , pairSeparator = "\u0001" , type= "FIX" , version= "4.1" ) public class Order {

@Link Header header;

@Link Trailer trailer;

@KeyValuePairField(tag = 1) // Client reference private String Account;

@KeyValuePairField(tag = 11) // Order reference private String ClOrdId;

@KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String IDSource;

@KeyValuePairField(tag = 48) // Fund code private String SecurityId;

@KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String Side;

@KeyValuePairField(tag = 58) // Free text private String Text;

...

}

case 2 : Different position in output

If the tags/keys that we will put in the FIX message must be sorted according to a predefine order, then use the attribute 'position' of the annotation @KeyValuePairField

@Message(keyValuePairSeparator = "=" , pairSeparator = "\\u0001" , type = "FIX" , version

= "4.1" , isOrdered = true ) public class Order {

@Link Header header;

@Link Trailer trailer;

@KeyValuePairField(tag = 1, position = 1) // Client reference private String account;

@KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId;

...

}

DATA FORMAT APPENDIX 286

7. Section

In FIX message of fixed length records, it is common to have different sections in the representation of the information : header, body and section. The purpose of the annotation

@Section is to inform bindy about which class of the model represents the header (= section

1), body (= section 2) and footer (= section 3)

Only one attribute/parameter exists for this annotation.

Annotation name Record type Level

Section

FIX Class

Parameter name type Info

number int digit number identifying the section position

case 1 : Section

A. Definition of the header section

@Section(number = 1) public class Header {

@KeyValuePairField(tag = 8, position = 1) // Message Header private String beginString;

...

}

@KeyValuePairField(tag = 9, position = 2) // Checksum private int bodyLength;

B. Definition of the body section

@Section(number = 2)

@Message(keyValuePairSeparator = "=" , pairSeparator = "\\u0001" , type = "FIX" , version

= "4.1" , isOrdered = true ) public class Order {

@Link Header header;

@Link Trailer trailer;

@KeyValuePairField(tag = 1, position = 1) // Client reference private String account;

@KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId;

C. Definition of the footer section

287 DATA FORMAT APPENDIX

@Section(number = 3) public class Trailer {

@KeyValuePairField(tag = 10, position = 1)

// CheckSum private int checkSum; public int getCheckSum() { return checkSum;

}

8. OneToMany

The purpose of the annotation @OneToMany is to allow to work with a List<?> field defined a

POJO class or from a record containing repetitive groups.

The relation OneToMany ONLY WORKS in the following cases :

▪

Reading a FIX message containing repetitive groups (= group of tags/keys)

▪

Generating a CSV with repetitive data

Annotation name Record type Level

OneToMany

all property

Parameter name type Info

mappedTo string optional - string - class name associated to the type of the

List<Type of the Class>

case 1 : Generating CSV with repetitive data

Here is the CSV output that we want :

Claus,Ibsen,Camel in Action 1,2010,35

Claus,Ibsen,Camel in Action 2,2012,35

Claus,Ibsen,Camel in Action 3,2013,35

Claus,Ibsen,Camel in Action 4,2014,35

Remark : the repetitive data concern the title of the book and its publication date while first, last name and age are common and the classes used to modeling this. The Author class contains a List of Book.

@CsvRecord(separator= "," ) public class Author {

@DataField(pos = 1) private String firstName;

DATA FORMAT APPENDIX 288

Restrictions OneToMany

Be careful, the one to many of bindy does not allow to handle repetitions defined on several levels of the hierarchy

@DataField(pos = 2) private String lastName;

@OneToMany private List<Book> books;

@DataField(pos = 5) private String Age;

...

public class Book {

@DataField(pos = 3) private String title;

@DataField(pos = 4) private String year;

Very simple isn't it !!!

case 2 : Reading FIX message containing group of tags/keys

Here is the message that we would like to process in our model :

"8=FIX 4.19=2034=135=049=INVMGR56=BRKR"

"1=BE.CHM.00111=CHM0001-0158=this is a camel - bindy test"

"22=448=BE000124567854=1"

"22=548=BE000987654354=2"

"22=648=BE000999999954=3"

"10=220" tags 22, 48 and 54 are repeated and the code public class Order {

@Link Header header;

@Link Trailer trailer;

@KeyValuePairField(tag = 1) // Client reference private String account;

289 DATA FORMAT APPENDIX

@KeyValuePairField(tag = 11) // Order reference private String clOrdId;

@KeyValuePairField(tag = 58) // Free text private String text;

@OneToMany(mappedTo =

"org.apache.camel.dataformat.bindy.model.fix.complex.onetomany.Security" )

List<Security> securities;

...

public class Security {

@KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String idSource;

@KeyValuePairField(tag = 48) // Fund code private String securityCode;

@KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String side;

Using the Java DSL

The next step consists in instantiating the DataFormat bindy class associated with this record type and providing Java package name(s) as parameter.

For example the following uses the class CsvBindyFormat (who correspond to the class associated with the CSV record type) which is configured with "com.acme.model" package name to initialize the model objects configured in this package.

DataFormat bindy = new CsvBindyDataFormat( "com.acme.model" ); from( "file: //inbox" ).

unmarshal(bindy).

to( "bean:handleOrder" );

The Camel route will pick-up files in the inbox directory, unmarshall CSV records in a collection of model objects and send the collection to the bean referenced by 'handleOrder'.

The collection is a list of Map. Each Map of the list contains the objects of the model. Each object can be retrieve using its class name.

int count = 0;

List<Map< String , Object >> models = new ArrayList<Map< String , Object >>();

DATA FORMAT APPENDIX 290

Map< String , Object > model = new HashMap< String , Object >(); models = (List<Map< String , Object >>) exchange.getIn().getBody();

Iterator<Map< String , Object >> it = models.iterator(); while (it.hasNext()){ model = it.next(); for ( String key : model.keySet()) {

Object obj = model.get(key);

LOG.info( "Count : " + count + ", " + obj.toString());

} count++;

}

LOG.info( "Nber of CSV records received by the csv bean : " + count);

To generate CSV records from a collection of model objects, you create the following route : from( "bean:handleOrder" ) marshal(bindy) to( "file: //outbox" )

You can if you prefer use a named reference to a data format which can then be defined in your

Registry such as via your Spring XML file. e.g.

from( "file: //inbox" ).

unmarshal( "myBindyDataFormat" ).

to( "bean:handleOrder" );

Unit test

Here is two examples showing how to marshall or unmarshall a CSV file with Camel package org.apache.camel.dataformat.bindy.csv; import java.math.BigDecimal; import java.util.ArrayList; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.HashMap; import java.util.List; import java.util.Map;

291 DATA FORMAT APPENDIX

import org.apache.camel.EndpointInject; import org.apache.camel.Produce; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Client; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Order; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests;

@ContextConfiguration(locations =

"org.apache.camel.dataformat.bindy.csv.BindyComplexCsvMarshallTest$ContextConfig" , loader = JavaConfigContextLoader.class) public class BindyComplexCsvMarshallTest extends AbstractJUnit4SpringContextTests { private List<Map< String , Object >> models = new ArrayList<Map< String , Object >>(); private String result = "10,A1,Julia,Roberts,BE123456789,Belgium Ventage 10/

12,150,USD,14-01-2009" ;

@Produce(uri = "direct:start" ) private ProducerTemplate template;

@EndpointInject(uri = "mock:result" ) private MockEndpoint resultEndpoint;

@Test public void testMarshallMessage() throws Exception { resultEndpoint.expectedBodiesReceived(result); template.sendBody(generateModel()); resultEndpoint.assertIsSatisfied();

} private List<Map< String , Object >> generateModel() {

Map< String , Object > model = new HashMap< String , Object >();

Order order = new Order(); order.setOrderNr(10); order.setAmount( new BigDecimal( "150" )); order.setIsinCode( "BE123456789" ); order.setInstrumentName( "Belgium Ventage 10/12" ); order.setCurrency( "USD" );

Calendar calendar = new GregorianCalendar(); calendar.set(2009, 0, 14); order.setOrderDate(calendar.getTime());

Client client = new Client();

DATA FORMAT APPENDIX 292

client.setClientNr( "A1" ); client.setFirstName( "Julia" ); client.setLastName( "Roberts" ); order.setClient(client); model.put(order.getClass().getName(), order); model.put(client.getClass().getName(), client); models.add(0, model); return models;

}

@Configuration public static class ContextConfig extends SingleRouteCamelConfiguration {

BindyCsvDataFormat camelDataFormat = new

BindyCsvDataFormat( "org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink" );

@Override

@Bean public RouteBuilder route() { return new RouteBuilder() {

@Override public void configure() { from( "direct:start" ).marshal(camelDataFormat).to( "mock:result" );

}

};

}

} package org.apache.camel.dataformat.bindy.csv; import org.apache.camel.EndpointInject; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests;

@ContextConfiguration(locations =

"org.apache.camel.dataformat.bindy.csv.BindyComplexCsvUnmarshallTest$ContextConfig" , loader = JavaConfigContextLoader.class) public class BindyComplexCsvUnmarshallTest extends AbstractJUnit4SpringContextTests {

293 DATA FORMAT APPENDIX

@EndpointInject(uri = "mock:result" ) private MockEndpoint resultEndpoint;

@Test public void testUnMarshallMessage() throws Exception { resultEndpoint.expectedMessageCount(1); resultEndpoint.assertIsSatisfied();

}

@Configuration public static class ContextConfig extends SingleRouteCamelConfiguration {

BindyCsvDataFormat csvBindyDataFormat = new

BindyCsvDataFormat( "org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink" );

@Override

@Bean public RouteBuilder route() { return new RouteBuilder() {

@Override data?noop= true " ).unmarshal(csvBindyDataFormat).to("mock:result");

}

};

} public void configure() { from( "file: //src/test/

}

In this example, BindyCsvDataFormat class has been instantiated in a traditional way but it is also possible to provide information directly to the function (un)marshal like this where

BindyType corresponds to the Bindy DataFormat class to instantiate and the parameter contains the list of package names.

public static class ContextConfig extends SingleRouteCamelConfiguration {

@Override

@Bean public RouteBuilder route() { return new RouteBuilder() {

"org.apache.camel.dataformat.bindy.model.simple.oneclass" )

.to( "mock:result" );

}

};

}

@Override public void configure() { from( "direct:start" )

.marshal().bindy(BindyType.Csv,

}

DATA FORMAT APPENDIX 294

Using Spring XML

This is really easy to use Spring as your favorite DSL language to declare the routes to be used for camel-bindy. The following example shows two routes where the first will pick-up records from files, unmarshal the content and bind it to their model. The result is then send to a pojo

(doing nothing special) and place them into a queue.

The second route will extract the pojos from the queue and marshal the content to generate a file containing the csv record

<?xml version= "1.0" encoding= "UTF-8" ?>

<beans xmlns= "http: //www.springframework.org/schema/beans" xmlns:xsi= "http: //www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http: //www.springframework.org/schema/beans http: //www.springframework.org/schema/beans/spring-beans.xsd

http: //camel.apache.org/schema/spring http: //camel.apache.org/schema/spring/camel-spring.xsd">

<constructor-arg value= "org.apache.camel.bindy.model" />

</bean>

</bean>

<route>

</route>

<route>

</route>

</beans>

</camelContext>

295 DATA FORMAT APPENDIX

Be careful

Please verify that your model classes implements serializable otherwise the queue manager will raise an error

Dependencies

To use Bindy in your camel routes you need to add the a dependency on

camel-bindy which

implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<groupId>org.apache.camel</groupId>

<artifactId>camel-bindy</artifactId>

</dependency>

XMLSECURITY DATA FORMAT

Available as of Camel 2.0

The XMLSecurity DataFormat facilitates encryption and decryption of XML payloads at the

Document, Element and Element Content levels (including simultaneous multi-node encryption/ decryption using XPATH).

The encrytion capability is based on formats supported using the Apache XML Security

(Santaurio) project. Encryption/Decryption is "currently" supported using Triple-DES and AES

(128, 192 and 256) encryption formats. Additional formats can be easily added later as needed.

(Note: The support currently offered is for symmetric encryption. This means the same keyset is needed at both ends of the communication to encrypt/decrypt payloads).

The capability allows Camel users to encrypt/decrypt payloads while being dispatched or received along a route.

Options

Option

secureTag

Default

null

Description

The XPATH reference to the XML Element selected for encryption/decryption. If no tag is specified, the entire payload is encrypted/decrypted.

DATA FORMAT APPENDIX 296

secureTagContents passPhrase false null xmlCipherAlgorithm TRIPLEDES

A boolean value to specify whether the XML Element is to be encrypted or the contents of the XML

Element

• false = Element Level

• true = Element Content Level

A String used as passPhrase to encrypt/decrypt content. The passPhrase has to be provided. If no passPhrase is specified, a default passPhrase is used.

The passPhrase needs to be put together in conjunction with the appropriate encryption algorithm.

For example using TRIPLEDES the passPhase can be a "Only another 24 Byte key"

The cipher algorithm to be used for encryption/ decryption. The available choices are:

• XMLCipher.TRIPLEDES

• XMLCipher.AES_128

• XMLCipher.AES_192

• XMLCipher.AES_256

Marshal

In order to encrypt the payload, the marshal processor needs to be applied on the route followed by the

secureXML() tag.

Unmarshal

In order to decrypt the payload, the unmarshal processor needs to be applied on the route followed by the

secureXML() tag.

Examples

Given below are several examples of how marshalling could be performaed at the Document,

Element and Content levels.

Full Payload encryption/decryption

from( "direct:start" ).

marshal().secureXML().

unmarshal().secureXML().

to( "direct:end" );

297 DATA FORMAT APPENDIX

Partial Payload Content Only encryption/decryption*

String tagXPATH = " //cheesesites/italy/cheese" ; boolean secureTagContent = true ;

...

from( "direct:start" ).

marshal().secureXML(tagXPATH , secureTagContent ).

unmarshal().secureXML(tagXPATH , secureTagContent).

to( "direct:end" );

Partial Multi Node Payload Content Only encryption/ decryption*

String tagXPATH = " //cheesesites/*/cheese" ; boolean secureTagContent = true ;

....

from( "direct:start" ).

marshal().secureXML(tagXPATH , secureTagContent ).

unmarshal().secureXML(tagXPATH , secureTagContent).

to( "direct:end" );

Partial Payload Content Only encryption/decryption with choice of passPhrase(password)*

String tagXPATH = " //cheesesites/italy/cheese" ; boolean secureTagContent = true ;

....

String passPhrase = "Just another 24 Byte key" ; from( "direct:start" ).

marshal().secureXML(tagXPATH , secureTagContent , passPhrase).

unmarshal().secureXML(tagXPATH , secureTagContent, passPhrase).

to( "direct:end" );

Partial Payload Content Only encryption/decryption with passPhrase(password) and Algorithm*

import org.apache.xml.security.encryption.XMLCipher;

....

String tagXPATH = " //cheesesites/italy/cheese" ; boolean secureTagContent = true ;

DATA FORMAT APPENDIX 298

String passPhrase = "Just another 24 Byte key" ;

String algorithm= XMLCipher.TRIPLEDES; from( "direct:start" ).

marshal().secureXML(tagXPATH , secureTagContent , passPhrase, algorithm).

unmarshal().secureXML(tagXPATH , secureTagContent, passPhrase, algorithm).

to( "direct:end" );

Dependencies

This data format is provided in the

camel-xmlsecurity component.

The GZip Data Format is a message compression and de-compression format. It uses the same deflate algorithm that is used in Zip DataFormat, although some additional headers are provided. This format is produced by popular gzip/gunzip tool. Messages marshalled using

GZip compression can be unmarshalled using GZip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large

XML and Text based payloads or when you read messages previously comressed using gzip tool.

Options

There are no options provided for this data format.

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing gzip compression format and send it an ActiveMQ queue called MY_QUEUE.

from( "direct:start" ).marshal().gzip().to( "activemq:queue:MY_QUEUE" );

Unmarshal

In this example we unmarshal a gzipped payload from an ActiveMQ queue called

MY_QUEUE to its original format, and forward it for processing to the

UnGZippedMessageProcessor.

from( "activemq:queue:MY_QUEUE" ).unmarshal().gzip().process( new

UnGZippedMessageProcessor());

Dependencies

This data format is provided in

camel-core so no additional dependencies is needed.

299 DATA FORMAT APPENDIX

CASTOR

Available as of Camel 2.1

Castor is a Data Format which uses the Castor XML library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

As usually you can use either Java DSL or Spring XML to work with Castor Data Format.