WS-Security woes in ServiceMix 3.2

Previously I had to get WS-Security UsernameToken based authentication working in ServiceMix 3.2 for a customer. All the customer wanted was to have an external SOAP client connecting securely to a service deployed in ServiceMix and have the client being authenticated and authorized by ServiceMix. Sounds like a straight forward use-case and the fact that there already is a ws-security demo in ServiceMix made me hope this would be a simple project.
As Murphy's Law dictates, it turned out to be the opposite.

My initial hope was to simply define a CXF-BC consumer that is WS-Security enabled and that authenticates the client and passes on the request to a CXF-SE service. So overall use-case is
External client -> CXF-BC consumer -> CXF-SE service.
This can be set up using Maven in less than 10 minutes. The tricky part was to configure security. The ws-security demo of ServiceMix already shows the configuration for using XML-Signature and XML-Encryption, so I could simply copy that part. For using WS-Security functions ServiceMix simply leverages WSS4J, so it becomes a matter of correctly configuring WSS4J interceptors for the incoming request and outgoing reply.
Adding WS-Security UsernameToken based authentication configuration to the WSS4J interceptor is relatively simple too:

<bean class="org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor"
  id="TimestampSignEncrypt_Request">
  <constructor-arg>
    <map>
      <entry key="action" value="Timestamp Signature Encrypt UsernameToken"/>
      <entry key="signaturePropFile" value="alice.properties"/>
      <entry key="decryptionPropFile" value="bob.properties"/>
      <entry key="passwordCallbackClass" 
     value="org.apache.servicemix.samples.cxf_ws_security_usernametoken.KeystorePasswordCallback"/>
    </map>
  </constructor-arg>
</bean>

I simply had to add UsernameToken to the <action> element in the above WSS4JInInterceptor configuration (config for XML-Signature and XML-Encryption was directly taken from ws-security demo). I won't care about the client configuration here, as this could be any SOAP client but need to add that my client was configured to transmit username "alice" with password "password".
For authorization to work, I had to add these credentials to conf/users-passwords.properties

# users-passwords.properties
smx=smx
alice=password

and make alice a member of the administrators role in conf/groups.properties (for a real use-case I would have defined a new role):

# groups.properties
admin=smx,alice

Finally to restrict only users of the administration role to invoke my service, I had to change conf/security.xml

<sm:authorizationMap id="authorizationMap">
    <sm:authorizationEntries>
      <sm:authorizationEntry service="*: SOAPServiceWSSecurity" 
         roles="admin" />
    </sm:authorizationEntries>
  </sm:authorizationMap>

Easy I thought and went to testing it. But it did not work. There was simply no authentication happening. Only when debugging through the code I realized the CXF-BC component did not delegate these credentials to JAAS for authentication! So there was no authentication possible in CXF-BC! I raised a bug report and took a deep breath...
The good news is that by now this bug got resolved and things work as expected. However you need a very recent version of ServiceMix to have this fix included. Now if a client sends a WS-Security header containing username and password, these will be authenticated correctly by JAAS inside ServiceMix.
Still at the time of discovering the bug I had to get things working for the customer and got to know that the HTTB-BC component supports username/password based authentication. So I started to replace my CXF-BC consumer with a HTTP-BC consumer. And this is where the real pain started.
The configuration again is straight forward and does not look much different from the CXF-BC consumer configuration. However, for authentication to work in HTTP-BC, I must set soap="true" in my bc consumer config:

<beans xmlns:http="http://servicemix.apache.org/http/1.0"
       xmlns:greeter="http://apache.org/hello_world_soap_http"
       xmlns:soap="http://servicemix.apache.org/soap/1.0">
  <http:endpoint service="greeter:SOAPServiceWSSecurityHTTP"
    endpoint="endpoint"
    role="consumer"
    targetService="greeter:SOAPServiceWSSecurity"
    targetEndpoint="endpoint"
    locationURI="http://localhost:9000/"
    defaultMep="http://www.w3.org/2004/08/wsdl/in-out"
    soap="true">
    <http:policies>
      <soap:ws-addressing />
      <soap:ws-security receiveAction="UsernameToken" keystore="default" />
    </http:policies>
  </http:endpoint>
</beans>

This is because the WSSecurityHandler that is called by the HTTP-BC consumer expects a DOM representation of the SOAP header, which won't get created unless soap="true" is set. With that I got the HTTP-BC component to authenticate the client, but the request failed to be un-marshaled by the CXF-SE runtime. Setting soap="true" in my http consumer had the side effect of stripping off the SOAP header from the request before sending it on to the next component inside ServiceMix. The CXF-SE runtime however requires a valid SOAP request, which it now did not get. Aargh...

So what to do with this dilemma? The easiest option I could think of was to insert a Camel component between the HTTP-BC consumer and the CXF-SE service that adds a default SOAP envelope to the In message using xslt transformation and removes the SOAP envelope from the Out message produced by the CXF-SE component again (as the http-bc component will add an envelope when soap="true" is set.) Not an ideal solution and rather a hack than a proper workaround. Another option would have been to use a custom CXF interceptor that takes care of adding/stripping the SOAP envelope but I felt a Camel XSLT component will be simpler to use.
I inserted the Camel component that did add/remove the SOAP envelope. Still I had a problem with the reply. The reply message had to be signed and encrypted but that was not the case, despite my configuration being correct. After another few hours of debugging (I am new to the codebase) I realized this part had yet not been implemented and I had hit another bug. Because of that bug outgoing SOAP reply message cannot be encrypted nor signed when using the HTTP-BC component.

I still found some other issues on the way that I raised accordingly. Password digests as defined by the WS-Security UsernameToken spec are currently not supported (passwords need to be send in clear text in the SOAP header - but the header can still be encrypted, so this is no security problem). And the call to authorization in ServiceMix does not produce any logging. Finally I raised a few documentation issues.

Lessons learned:
As you can see I hit a number of issues on the way. But the good news is that these will now be addressed and fixed in future versions of ServiceMix, making the product more stable and easier to use. The most important bug is already resolved so there is no need to use a HTTP-BC consumer anymore when doing WS-Security (and I highly recommend against it).

The demo that I worked on is already incorporated into the cxf-ws-security demo of ServiceMix in the current release.

The most difficult part for me was to hit very low level error messages that did not tell me much about the real cause of the problem. This can be quite frustrating and the only way I got to understand the error was to debug through the source code. So error reporting needs a hell lot of improvement in ServiceMix.

Secondly, the low-level XML configuration is highly error-prone. ServiceMix really needs proper tooling that free the developer from configuring xml files. It is not only easy to make mistakes in the config, there are also so many configuration options that have implications unknown to the regular developer (like the soap="true" story).