Lessons learned from using Apache Camel, MTOM and JMS

Recently, I did a project that involved sending messages from a database or file system to a SOAP service. Because it solves a lot of integration problems, I chose Apache Camel to handle the integration challenges with the various components. During the implementation of the project I learned a few things that I would now like to share with you. Hopefully it will be useful to you.

The basic workflow for this system is as follows:

1. A snippet of XML-like text is placed in the datastore (database or filesystem)

2. The XML is parsed, which in turn is basically a two step phase:

3.  

   1. 1. The XML itself is made valid e.g. namespaces and XML start tags are added

   2. If the XML contains an attachment, the attachment is read from the datastore and included as an MTOM attachment

4. A SOAP request is made to an endpoint using two-way SSL.

5. An acknowledgement is written back (again, parsed) to the datastore

The system is also capable of receiving requests. For that the workflow is pretty much in reverse, save the acknowledgement.

In order to support loose coupling between the various components and to promote extensibility, the components in the system communicate with each other through messages: component A sends a message to component B which then does some processing and forwards the message to another component. The underlying transport for these messages is JMS.

Overall the technologies used are: SOAP with MTOM, XML, JMS and JDBC (database). The frameworks and tools I chose to support me are: Apache Camel for the integration; Apache CXF for communication with the SOAP endpoints; ActiveMQ as message broker.

Camel, JMS and attachments

Originally, I had a couple of components: one that polled the datastore for new messages that needed to be sent; one that parsed the message; one that handled the attachments and one that would send the message to the SOAP endpoint. As mentioned before, each of these components communicate with each other through JMS. The attachment component would add a file as attachment to the Camel exchange and to my surprise the component that would receive the message would not have access to the attachment on the exchange. The reason for that is stated here... but the bottom line is: if you use JMS, you cannot add attachments to the exchange.

There's a couple of ways to cope with this: either you choose to only add the attachment at the end of the chain or – if possible – you send the attachment as a separate message.

Testing

I've found it particularly helpful to test the individual components. By being able to test each of the components separately it is easy to isolate issues should they arise.

An example test could be as follows:

@RunWith(SpringJUnit4ClassRunner.class)

@ContextConfiguration(locations = { "classpath*:pipeline-acknowledge.xml" })

public class AcknowledgeComponentTest {

@Produce
private ProducerTemplate producerTemplate;

@EndpointInject(uri = "mock:result")
private MockEndpoint resultEndpoint;

@Resource
private CamelContext camelContext;

@Before
public void before() throws Exception {
if (camelContext.getRoute("acknowledge") == null) {
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() throws Exception {
from("activemq:acknowledgement-output").routeId("acknowledge")
.convertBodyTo(String.class)
.to("mock:result");
}
});
}
}

@Test
public void testValidateOk() throws Exception {
String acknowledgeResponse = FileUtils.readFileToString(new ClassPathResource("/soap/acknowledgeResponse.xml").getFile());
String acknowledgeXml = FileUtils.readFileToString(new ClassPathResource("/berichten/acknowledge.xml").getFile());

producerTemplate.sendBody("activemq:acknowledgement-input", acknowledgeResponse);

long wait = 0L;
while (resultEndpoint.getReceivedExchanges().size() < 1 && wait < 2000) {
Thread.sleep(500L);
}

String actualBody = resultEndpoint.getReceivedExchanges().get(0).getIn().getBody().toString();
Diff diff = new Diff(acknowledgeXml, actualBody);
XMLUnit.setIgnoreComments(true);
XMLUnit.setIgnoreWhitespace(true);
Assert.assertTrue("XML Should be similar", diff.similar());
}
}

This is a simple test that basically sends a message to the acknowledge component and then verifies that the components produces an XML file that is similar to expected XML. Lets take a quick look at some interesting parts.

Firstly, the test is being executed with a Spring context. This context simply starts Camel and defines the acknowledge component's route, also ActiveMQ is started.

The method annotated with @Before adds another route if it does not already exist. It simply registers a route that listens to the queue that the acknowledge component will write to, we will use this to verify the response.

Then in the method annotated with @Test, we create a String that holds some XML and sends it to the acknowledge component by using Camel's ProducerTemplate. Then we will wait a short while for the message to be processed, and finally we verify that the response is similar to the expected outcome. Hopefully, someone will be able to use this in their work!