Some time ago I did some CXF performance measurements. See How fast is CXF ? - Measuring CXF performance on http, https and jms.
For cxf 3.0.0 I did some massive changes on the JMS transport. So I thought it is a good time to compare cxf 2.x and 3 in JMS performance. My goal was to reach at least the original performance. As my test system is different now I am also measuring the cxf 2.x performance again to have a good comparison.
Dell Precision with Intel Core i7, 16 GB Ram, 256 GB SSD running ubuntu Linux 13.10.
I am using a new version of my performance-tests project on github.
The test runs on one machine using one activemq Server, one test server and one test client.
The test calls the example cxf CustomerService.
The following call types are supported:
Asynchronous one way call. Sends one soap message to server
List<Customer> customers = customerService.getCustomersByName("test2");
Synchronous request reply. Sends one soap message to server and waits for the reply
Future<GetCustomersByNameResponse> resp = customerService.getCustomersByNameAsync("test2");
Sends an asynchronous request reply. Sends one soap message to server and returns without waiting.
The requests above are sent using an executor with a fixed number of threads.
For the test you can specify the total number of messages, the number of threads and the call type.
First the number of requests are sent for warmup and then for the real measured test.
To run the test with cxf 3.0.0-SNAPSHOT you have to compile cxf from source.
1. Run a standalone activemq 5.9.0 server with the activemq.xml from the github sources above.
2. Start the jms server in a new console from the project source using:
mvn -Pserver test
3. Start the jms client using:
mvn -Pclient test -Dmessages=40000 -Dthreads=20 -DcallType=oneway
The test is executed with several combinations of the parameters. Using the pom property cxf.version we also switch between cxf 2.7.10 and cxf 3.0.0-SNAPSHOT.
The first interesting fact here is that one way messaging does not profit from the number of threads. One thread already seems to achieve the same performance like 40 threads. This is quit intuitive as activemq needs to synchronize the calls on the one thread holding the jms connection. On the other hand using more processes also does not seem to improve the performance so we seem to be quite at the limit of activemq here which is good.
For request reply the performance seems to scale with the number threads. This can be explained as we have to wait for the response and can use this time to send some more requests.
One really astonishing thing here is that CXF 2.7.10 seems to be really bad when using synchronous request reply. This is because it uses consumer.receive in this case while it uses a jms message listener for async calls. So the jms message listener seems to perform much better than the consumer.receive case. For CXF 2.7.10 this means we can speed up our calls if we use the asynchronous interface even if it is more inconvenient.
The most important observation here is that CXF 3 performs a lot better for the synchronous request reply case. It is as fast as for the asynchronous case. The reason is that we now also use a message listener for synchronous calls as long as our correlation id is based on the conduit id prefix. This is the default so this case is vastly improved. CXF 3 is up to 5 times faster than CXF 2.7.10.
There is one down side still. If you use message id as correlation id or a user correlation id set on the cxf message then cxf 3 will switch back to consumer.receive and will be as slow as CXF 2 again.