Blog from June, 2015

This tutorial shows how to use Declarative Services together with the new Aries JPA 2.0.

You can find the full source code on github Karaf-Tutorial/tasklist-ds

Declarative Services

Declarative Services (DS) is the biggest contender to blueprint. It is a slim service injection framework that is completely focused on OSGi. DS allows you to offer and consume OSGi services and to work with configurations.

At the core DS works with xml files to define scr components and their dependencies. They typically live in the OSGI-INF directory and are announced in the Manifest using the header "Service-Component" with the path to the component descriptor file.  Luckily it is not necessary to directly work with this xml as there is also support for DS annotations. These are processed by the maven-bundle-plugin. The only prerequisite is that they have to be enabled by a setting in the configuration instructions of the plugin.

For more details see http://www.aqute.biz/Bnd/Components

DS vs Blueprint

Let us look into DS by comparing it to the already better known blueprint. There are some important differences:

  1. Blueprint always works on a complete blueprint context. So the context will be started when all mandatory service deps are present. It then publishes all offered services. As a consequence a blueprint context can not depend on services it offers itself. DS works on Components. A component is a class that offers a service and can depend on other services and configuration. In DS you can manage each component separately like start and stop it. It is also possible that a bundle offers two components but only one is started as the dependencies of the other are not yet there.
  2. DS supports the OSGi service dynamics better than blueprint. Lets look into a simple example:
    You have a DS and blueprint module component that offers a service A and depends on a mandatory service B. Blueprint will wait on the first start for the mandatory service to be available. If it does not come up it will fail after a timeout and will not be able to recover from this. Once the blueprint context is up it stays up even if the mandatory service goes away. This is called service damping and has the goal to avoid restarting blueprint contexts too often. Services are injected into blueprint beans as dynamic proxies. Internally the proxy handles the replacement and unavailability of services. One problem this causes is that calls to a non available service will block the thread until a timeout and then throw a RuntimeException.
    In DS on the other hand a component lifecycle is directly bound to dependent services. So a component will only be activated when all mandatory services are present and deactivated as soon as one goes away. The advantage is that the service injected into the component does not have to be proxied and calls to it should always work.
  3. Every DS component must be a service. While blueprint can have internal beans that are just there to wire internal classes to each other this is not possible in DS. So DS is not a complete dependency injection framework and lacks many of the features blueprint offers in this regard.
  4. DS does not support extension namespaces. Aries blueprint has support for quite a few other Apache projects using extension namespaces. Examples are: Aries jpa, Aries transactions, Aries authz, CXF, Camel. So using these technologies in DS can be a bit more difficult.
  5. DS does not support support interceptors. In blueprint an extension namespace can introduce and interceptor that is always called before or after a bean. This is for example used for security as well as transation handling. For this reason DS did not support JPA very well as normal usage mandates to have interceptors. See below how jpa can work on DS.

So if DS is a good match for your project depends on how much you need the service dynamics and how well you can integrate DS with other projects.

JEE and JPA

The JPA spec is based on JEE which has a very special thread and interceptor model. In JEE you use session beans with a container managed EntityManger
to manipulate JPA Entities. It looks like this:

JPA

In JEE calling getTask will by default participate in or start a transaction. If the method call succeeds the transaction will be committed, if there is an exception it will be rolled back.
The calls go to a pool of TaskServiceImpl instances. Each of these instances will only be used by one thread at a time. As a result of this the EntityManager interface is not thread safe!

So the advantage of this model is that it looks simple and allows pretty small code. On the other hand it is a bit difficult to test such code outside a container as you have to mimic the way the container works with this class. It is also difficult to access e.g. em
 as it is private and there is not setter.

Blueprint supports a coding style similar to the JEE example and implements this using a special jpa and tx namespace and
interceptors that handle the transaction / em management.

DS and JPA

In DS each component is a singleton. So there is only one instance of it that needs to cope with multi threaded access. So working with the plain JEE concepts for JPA is not possible in DS.

Of course it would be possible to inject an EntityManagerFactory and handle the EntityManager lifecycle and transactions by hand but this results in quite verbose and error prone code.

Aries JPA 2.0.0 is the first version that offers special support for frameworks like DS that do not offer interceptors. The solution here is the concept of a JPATemplate together with support for closures in Java 8. To see how the code looks like peek below at chapter persistence.

Instead of the EntityManager we inject a thread safe JpaTemplate into our code. We need to put the jpa code inside a closure and run it with jpa.txEpr() or jpa.tx(). The JPATemplate will then guarantee the same environment like JEE inside the closure. As each closure runs as its own
instance there is one em per thread. The code will also participate/create a transaction and the transaction  commit/rollback also works like in JEE.

So this requires a little more code but the advantage is that there is no need for a special framework integration.
The code can also be tested much easier. See TaskServiceImplTest in the example.

Structure

  • features
  • model
  • persistence
  • ui

Features

Defines the karaf features to install the example as well as all necessary dependencies.

Model

This module defines the Task JPA entity, a TaskService interface and the persistence.xml. For a detailed description of model see the tasklist-blueprint example. The model is exactly the same here.

Persistence

TaskServiceImpl

We define that we need an OSGi service with interface TaskService and a property "osgi.unit.name" with the value "tasklist".

InitHelper

The class InitHelper creates and persists a first task so the UI has something to show. It is also an example how business code that works with the task service can look like.
@Reference TaskService taskService injects the TaskService into the field taskService.
@Activate makes sure that addDemoTasks() is called after injection of this component.

Another interesting point in the module is the test TaskServiceImplTest. It runs outside OSGi and uses a special
persistence.xml for testing to create the EntityManagerFactory. It also shows how to instantiate a ResourceLocalJpaTemplate
to avoid having to install a JTA transaction manager for the test. The test code shows that indeed the TaskServiceImpl can
be used as plain java code without any special tricks.

UI

The tasklist-ui module uses the TaskService as an OSGi service and publishes a Servlet as an OSGi service. The Pax-web whiteboard bundle will then pick up the exported servlet and publish it using the HttpService so it is available on http.

TaskListServlet

The above snippet shows how to specify which interface to use when exporting a service as well as how to define service properties.

The TaskListServlet is exported with the interface javax.servlet.Servlet with the service property alias="/tasklist".
So it is available on the url http://localhost:8181/tasklist.

Build

Make sure you use JDK 8 and run:

Installation

Make sure you use JDK 8.
Download and extract Karaf 4.0.0.
Start karaf and execute the commands below

Create DataSource config and Install Example

Validate Installation

First we check that the JpaTemplate service is present for our persistence unit.

Aries JPA should have created this service for us from our model bundle. If this did not work then check the log for messages from Aries JPA. It should print what it tried and what it is waiting for. You can also check for the presence of an EntityManagerFactory and EmSupplier service which are used by JpaTemplate.

A likely problem would be that the DataSource is missing so lets also check it:

This is like it should look like. Pax-jdbc-config created the DataSource out of the configuration in "etc/org.ops4j.datasource-tasklist.cfg".  By using a DataSourceFactory wit the property "osgi.jdbc.driver.name=H2-pool-xa". So the resulting DataSource should be pooled and fully ready for XA transactions.

Next we check that the DS components started:

If any of the components is not active you can inspect it in detail like this:

Test

Open the url below in your browser.
http://localhost:8181/tasklist

You should see a list of one task

 http://localhost:8181/tasklist?add&taskId=2&title=Another Task

 

You may already know the old CXF LoggingFeature (org.apache.cxf.feature.LoggingFeature). You added it to a JAXWS endpoint to enable logging for a CXF endpoint at compile time.

While this already helped a lot it was not really enterprise ready. The logging could not be controlled much at runtime and contained too few details. This all changes with the new CXF logging support and the up coming Karaf Decanter.

Logging feature in CXF 3.1.0

In CXF 3.1 this code was moved into a separate module and gathered some new features.

  • Auto logging for existing CXF endpoints
  • Uses slf4j MDC to log meta data separately
  • Adds meta data for Rest calls
  • Adds MD5 message id and exchange id for correlation
  • Simple interface for writing your own appenders
  • Karaf decanter support to log into elastic search

Manual Usage

CXF LoggingFeature

Auto logging for existing CXF endpoints in Apache Karaf

Simply install and enable the new logging feature:

Logging feature in karaf

Then install CXF endpoints like always. For example install the PersonService from the Karaf Tutorial Part 4 - CXF Services in OSGi. The client and endpoint in the example are not equipped with the LoggingFeature. Still the new logging feature will enhance the clients and endpoints and log all SOAP and Rest calls using slf4j. So the logging data will be processed by pax logging and by default end up in your karaf log.

A log entry looks like this:

Sample Log entry

This does not look very informative. You only see that it is an incoming request (REQ_IN) and the SOAP message in the log message. The logging feature provides a lot more information though. You just need to configure the pax logging config to show it.

Slf4j MDC values for meta data

This is the raw logging information you get for a SOAP call:

FieldValue
@timestamp2015-06-08T14:43:27,097Z
MDC.addresshttp://localhost:8181/cxf/personService
MDC.bundle.id90
MDC.bundle.nameorg.apache.cxf.cxf-rt-features-logging
MDC.bundle.version3.1.0
MDC.content-typetext/xml; charset=UTF-8
MDC.encodingUTF-8
MDC.exchangeId56b037e3-d254-4fe5-8723-f442835fa128
MDC.headers{content-type=text/xml; charset=UTF-8, connection=keep-alive, Host=localhost:8181, Content-Length=251, SOAPAction="", User-Agent=Apache CXF 3.1.0, Accept=*/*, Pragma=no-cache, Cache-Control=no-cache}
MDC.httpMethodPOST
MDC.messageIda46eebd2-60af-4975-ba42-8b8205ac884c
MDC.portNamePersonServiceImplPort
MDC.portTypeNamePersonService
MDC.serviceNamePersonServiceImplService
MDC.typeREQ_IN
levelINFO
loc.classorg.apache.cxf.ext.logging.slf4j.Slf4jEventSender
loc.fileSlf4jEventSender.java
loc.line55
loc.methodsend
loggerClassorg.ops4j.pax.logging.slf4j.Slf4jLogger
loggerNameorg.apache.cxf.services.PersonService.REQ_IN
message<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><ns2:getAll xmlns:ns2="http://model.personservice.cxf.karaf.tutorial.lr.net/"; xmlns:ns3="http://person.jms2rest.camel.karaf.tutorial.lr.net"/></soap:Body></soap:Envelope>;
threadNameqtp80604361-78
timeStamp1433774607097

Some things to note:

  • The logger name is <service namespace>.<ServiceName>.<type> karaf by default only cuts it to just the type.
  • A lot of the details are in the MDC values

You need to change your pax logging config to make these visible.

You can use the logger name to fine tune which services you want to log this way. For example set the debug level to WARN for noisy services to avoid that they are logged or log some services to another file.

Message id and exhange id

The messageId allows to uniquely identify messages even if you collect them from several servers. It is also transported over the wire so you can correlate a request sent on one machine with the request received on another machine.

The exchangeId will be the same for an incoming request and the response sent out or on the other side for an outgoing request and the response for it. This allows to correlate request and responses and so follow the conversations.

Simple interface to write your own appenders

Write your own LogSender and set it on the LoggingFeature to do custom logging. You have access to all meta data from the class LogEvent.

So for example you could write your logs to one file per message or to JMS.

Karaf decanter support to write into elastic search

Many people use elastic search for their logging. Fortunately you do not have to write a special LogSender for this purpose. The standard CXF logging feature will already work.

It works like this:

  • CXF sends the messages as slf4j events which are processed by pax logging
  • Karaf Decanter LogCollector attaches to pax logging and sends all log events into the karaf message bus (EventAdmin topics)
  • Karaf Decanter ElasticSearchAppender sends the log events to a configurable elastic search instance

As Decanter also provides features for a local elastic search and kibana instance you are ready to go in just minutes.

Installing Decanter for CXF Logging


After that open a browser at http://localhost:8181/kibana. When decanter is released kibana will be fully set up. At the moment you have to add the logstash dashboard and change the index name to [karaf-]YYYY.MM.DD.

Then you should see your cxf messages like this:

Kibana easily allows to filter for specific services and correlate requests and responses.

This is just a preview of decanter. I will do a more detailed post when the first release is out.