subbu.org

Several months ago, I blogged
about versioning of web services. Between then and now, I realized that versioning of web services is a very nebulous topic, and that the problem of versioning could test the motivation of even the most committed web services architect. In this post, I would like to take the next step and discuss some possible ways of versioning web services. In this process, I would like to answer two questions:

• How to version the public view of a web serice, i.e. the WSDL?
• How to make web service implementations support versioning?

Before I get too far in this discussion, let me clarify that in some situations it may be perfectly acceptable to not support previous versions of a web service, i.e. break backwards-compatibility. For instance, between close-knit applications, it is possible to make changes to both the service and its clients at the sime time, and avoid the question of compatibility altogher. On the otherhand, there are cases where the web service and its clients cannot be upgraded at the same time for a variety of reasons. I would argue that the more popular and useful your web services are, the more difficult it would be to make simultaeous upgrades to clients. In a loosely-coupled enterprise, such a big-bang migration should not even be required. For a certain period of time (if not indefitely), the service implementation may be required to support multiple versions of its interface at the same time, just because some of the clients have not chosen to upgrade.

Since web services carry XML messages, it would seem logical that recognizing the version of an incoming message is adequate for building a web service that can support multiple versions of the services it offers. For instance, a web service could accept two kinds of messages, with each message described by a different schema. The problem is then to version those
schemas, e.g. as discussed here.

|-------------|
----->{Message m1/Schema v1}----->|             |
| Web Service |
----->{Message m2/Schema v2}----->|             |
|-------------|

The web service could, at least in thoery, examine the message, and depending on the version, or the presence/absence of certain elements/attributes in the message, could support multiple versions of the service using the same implementation. For those bothered with versioning, this is the most ideal solution. However, in reality, current web services related standards and tools make it difficult to support such a simplified solution. Firstly, SOAP and WSDL’s view of web services is more complex than the simplified view above. Secondly, web services programming standards like JAX-RPC (or JAX-WS as JAX-RPC is called now as) and JWS do not consider versioning either.

WSDL Versioning

Since WSDL is the de-facto standard for specifying web service interfaces, the first problem to tackle is versioning the WSDL. WSDLs have service bindings, port types, messages and types. Messages are constructed using elements of types, ports specify operations with input/output messages, bindings bind operations to a protocol, and finally services associate bindings to a network address. Therefore, you must start the versioning exercise from the types used in the WSDL, and then extend it to the WSDL level.

Let me start with a simple web service that creates a new employee.

<definitions targetNamespace="empl:v1"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:v1="empl:v1"
xmlns:s1="http://schemas.xmlsoap.org/wsdl/soap/"">
<import location="v1_bindings.wsdl" namespace="empl:v1"/>
<service name="EmplManagerService">
<port binding="v1:EmplManagerBinding" name="EmplManagerPort">
<v1:address location="http://noname.org/emplManager/v1"/>
</port>
</service>
</definitions>

where the imported file contains the following.

<wsdl:definitions targetNamespace="empl:v1"
xmlns:v1="empl:v1"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
<wsdl:types>
<schema targetNamespace="empl:v1"
xmlns:v1="empl:v1"
xmlns="http://www.w3.org/2001/XMLSchema"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified">
<xs:complexType name="NameType">
<xs:sequence>
<xs:element name="fName" type="xs:string"/>
<xs:element name="lName" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="AddressType">
<xs:sequence>
<xs:element name="street" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
<xs:element name="zip" type="xs:string"/>
<xs:element name="state" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="EmployeeType">
<xs:sequence>
<xs:element name="name" type="v1:NameType"/>
<xs:element name="id" type="xs:string"/>
<xs:element name="homeAddress" type="v1:AddressType"/>
</xs:sequence>
</xs:complexType>
</schema>
</wsdl:types>
<wsdl:message name="createEmployee">
<wsdl:part name="createEmployee" type="v1:EmployeeType"/>
</wsdl:message>
<wsdl:message name="createEmployeeResponse">
<wsdl:part name="result" type="xs:string"/>
</wsdl:message>
<wsdl:portType name="EmplManagerPort">
<wsdl:operation name="createEmployee">
<wsdl:input message="v1:createEmployee"/>
<wsdl:output message="v1:createEmployeeResponse"/>
</wsdl:operation>
</wsdl:portType>
<wsdl:binding name="EmplManagerBinding" type="v1:EmplManagerPort">
<wsdl:operation name="createEmployee">
<wsdl:input>
<soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
use="literal"/>
</wsdl:input>
<wsdl:output>
<soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
use="literal"/>
</wsdl:output>
<soap:operation soapAction=""/>
</wsdl:operation>
<soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
</wsdl:binding>
</wsdl:definitions>

Given employee details, this web service creates an employee. In this WSDL, I defined the bindings, messages, and types in another WSDL file. Except for the “hello world” kind of web services, I find it convenient to split the WSDL into smaller chunks and wire them up together by using imports.

Extending the example used in my schema versioning post, let us add a country field to the address of the employee. As I discussed in that post, adding a country to the AddressType requires creating a new EmployeeType, and we cannot use schema extensions (i.e. via xs:extension) to add this element. We need to create a new schema and add the country to the AddressType in the new schema with a new namespace URI that reflects the new version. Here is my new schema.

<xs:schema targetNamespace="empl:v2"
xmlns:v2="empl:v2"
xmlns="http://www.w3.org/2001/XMLSchema"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified">
<xs:complexType name="NameType">
<xs:sequence>
<xs:element name="fName" type="xs:string"/>
<xs:element name="lName" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="AddressType">
<xs:sequence>
<xs:element name="street" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
<xs:element name="zip" type="xs:string"/>
<xs:element name="state" type="xs:string"/>
<xs:element name="country" type="xs:string" default="US" minOccurs="0" maxOccurs="1"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="EmployeeType">
<xs:sequence>
<xs:element name="name" type="v2:NameType"/>
<xs:element name="id" type="xs:string"/>
<xs:element name="homeAddress" type="v2:AddressType"/>
<xs:element name="ssn" type="xs:string" minOccurs="0" maxOccurs="1"/>
</xs:sequence>
</xs:complexType>
<xs:element name="Employee" type="v2:EmployeeType"/>
</xs:schema>

From the WSDL’s point of view, we also need to define new messages, port types, and bindings, since the same set of messages/port types/bindings cannot be used to carry messages from two different schemas.

Here is the new WSDL.

<wsdl:definitions targetNamespace="empl:v2"
xmlns:v1="empl:v1"
xmlns:v2="empl:v2"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">

<import namespace="empl:v1" location="v1_bindings.wsdl"/>
<import namespace="empl:v2" location="v2_bindings.wsdl"/>

<wsdl:service name="EmplManagerService">
<wsdl:port name="EmplManagerPort" binding="v1:EmplManagerBinding">
<soap:address location="http://localhost:7001/emplManager/v1"/>
</wsdl:port>
<wsdl:port name="EmplManagerPort_v2" binding="v2:EmplManagerBinding">
<soap:address location="http://localhost:7001/emplManager/v2"/>
</wsdl:port>
</wsdl:service>
</wsdl:definitions>

Here is the imported vs_bindings.wsdl file.

<definitions targetNamespace="empl:v2"
xmlns:v2="empl:v2"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
<types>
<!-- snip -->
</types>
<message name="createEmployee">
<part name="createEmployee" type="v2:EmployeeType"/>
</message>
<message name="createEmployeeResponse">
<part name="result" type="xs:string"/>
</message>
<portType name="EmplManagerPort">
<operation name="createEmployee">
<input message="v2:createEmployee"/>
<output message="v2:createEmployeeResponse"/>
</operation>
</portType>
<binding name="EmplManagerBinding" type="v2:EmplManagerPort">
<operation name="createEmployee">
<input>
<soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
use="literal"/>
</input>
<output>
<soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
use="literal"/>
</output>
<soap:operation soapAction=""/>
</operation>
<soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
</binding>
</definitions>

By choosing to define a new schema for the types for versioning sake, we, in essence, defined a new set of messages, port types, bindings, and a new WSDL port for the versioned operation. At the WSDL level, this is as far as we can get for introducing a new version of a web service. To summarize, the following are the steps for versioning a WSDL.

• Define new or extended types for changes in types in a new namespace.
• Define new messages.
• Define new port types and bindings.
• Add a new service or new ports to an existing service in the WSDL.

For the client, this approach means that, it must choose the WSDL port based on the version it can support.

|-------------|
----------|             |
----->{Message m1/Schema v1}----->| Port v1 |             |
----------|             |
| Web Service |
----------|             |
----->{Message m2/Schema v2}----->| Port v2 |             |
----------|             |
|-------------|

Clients supporting the new version will therefore use new ports, and clients using the old version(s) will continue with old ports.

Implementing Versioned Web Services

I consider versioning the WSDL the easier part of the problem. Once we decide how to version schma types, the rest of the process is simple to deal with. For the web service provider as well as the client, a more difficult problem is supporting multiple versions at the same type. Moreover, web services programming standards like JAX-RPC and JWS have not matured enough to support versioning of web services yet.

Let me now start with the above versioned WSDL. When you generate code from thus WSDL using various WSDL to Java tools, you will essentially get two sets of artifacts, one for each version. For instance, here is a the Java interface generated by Axis 1.2’s wsdl2java task.

public interface EmplManagerService extends javax.xml.rpc.Service {
public java.lang.String getEmplManagerPort_v2Address();

public v2.EmplManagerPort getEmplManagerPort_v2() throws javax.xml.rpc.ServiceException;

public v2.EmplManagerPort getEmplManagerPort_v2(java.net.URL portAddress) throws javax.xml.rpc.ServiceException;
public java.lang.String getEmplManagerPortAddress();

public v1.EmplManagerPort getEmplManagerPort() throws javax.xml.rpc.ServiceException;

public v1.EmplManagerPort getEmplManagerPort(java.net.URL portAddress) throws javax.xml.rpc.ServiceException;
}

This interface has two different sets of methods one for each version. From the WSDL-to-Code translation point of view, this interface is valid, and is to be expected. However, from versioning point of view, the developer implementing this interface will have to deal with supporting two seperate code paths, one for each version, and that makes it supporting versioning more challenging.

For example, take the implementation of v1.EmplManagerPort.

public class EmplManagerPortImpl implements v1.EmplManagerPort
{
public java.lang.String createEmployee(v1.EmployeeType createEmployee) throws java.rmi.RemoteException
{
// Validate the employee object

// Create the employee record in a DB

// Return the employee ID
}
}

In order to support the new version of the WSDL, you will need to create an implementation of v2.EmplManagerPort.

public class EmplManagerPortImpl implements v2.EmplManagerPort
{
public java.lang.String createEmployee(v2.EmployeeType createEmployee) throws java.rmi.RemoteException
{
// How to implement this?
}
}

The question is how best to create this implementation. One of the most obvious answers is to simply duplicate the code. You may even be able to share some code at a lower level, but code dealing with schema types will have to be duplicated. So, the downside of this approach is maintenance of two versions of the software with different source code.

|-------------|
----------|             |
----->{Message m1/Schema v1}----->| Port v1 | Web Service |
----------|             |
--------------
----------|             |
----->{Message m2/Schema v2}----->| Port v2 | Web Service |
----------|             |
|-------------|

Subsequent versioning only worsenss this duplication, and leads to more code maintenance nightmares.

Here are some choices that, though take considerable effort to build, would address versioning given the current state of support in web services standards.

• Message transformation and Routing
• Common layer for version-indepent types and business logic

Let me elaborate on these approaches.

Message Transformation and Routing

I have explored the idea of transformation of XML documents from one version to another in one of my previous posts on processing versioned XML documents.

|-------------|
-------------    |             |
----->{Message m1/Schema v1}----->| Transform |    |             |
-------------    |             |
|          | Web Service |
|          |             |
----->{Message m2/Schema v2}---------------------->|             |
|             |
|-------------|

The core idea is to somehow transform v1 messages to conform to the v2 schema, and then route messages targeted to v1 port(s) to corresponding v2 port(s), such that the same code path can be used for both versions. For such a transformation and routing, you may need some extra plumbing not often found in most web service stacks. In these web service stacks, such message transformation can only happen before a message reaches the server, but not afterwards. For example, in JAX-RPC based web service stacks, the first entry point for programmatic manipulation of an incoming SOAP message is a JAX-RPC handler. But in JAX-RPC, handlers are associated with specific ports, and you cannot change the message after the web services runtime already bound the message to a given port. Doing so would most likely cause subsequent message processing to fail.

One possibility is to build a proxy server (or use a SOAP intermediary) to transform SOAP messages to conform to the latest version, and route the messages targeted to v1 ports to corresponding v2 ports. Most products offering web services management frameworks/tools provide some degree of transformation and routing capabilities. It would be interesting to see if such products can be put to use to solve web services versioning.

Common Data Abstraction Layer

Another possibility is do the transformation within the application implementing the web service.

|-------------------------------------------|
----------|     -------------                         |
----->{Message m1/Schema v1}----->| Port v1 |---> | Transform |                         |
----------|     -------------     ------------------  |
|            |--------> | Business Logic |  |
----------|     -------------     ------------------  |
----->{Message m2/Schema v2}----->| Port v2 |---> | Transform |                         |
----------|     -------------                         |
|-------------------------------------------|

In this case, the core business logic of the web service needs to be implemented in a version independent manner. Preferrably, this core layer should not also depend on any code generated using the WSDL of a particular version. For each version, you need to translate the incoming request into API calls of this core layer. In this case, versioning related changes can be limited to the layer that does this transformation.

In the case of the above sample web service, the first step is to build (or refactor from the existing version) an EmplManager component that can create an employ.

public class EmplManager
{
public String createEmployee(Employee employee) throws EmplManagerException
{
// Validate the employee object

// Create the employee record in a DB

// Return the employee ID
}
}

Here, the Employee object must semantically match the latest version of the EmployeeType used in the schema, so that Employee objects can be created from both v1.EmployeeType and v2.EmployeeType objects.

public class EmplManagerPortImpl implements v2.EmplManagerPort
{
public java.lang.String createEmployee(v2.EmployeeType createEmployee) throws java.rmi.RemoteException
{
// Create EmployeeType
Employee empl = new Employee();
// Set data
empl.setFirstName(createEmployee.getFName());
...

return emplManager.createEmployee(empl);
}
}

Similar code could be used to implement the v1.EmplManagerPort interface. The only difference between these two would be the new optional elements added in the v2 schema.

Although cumbersome to implement, the advantage of this approach is that you can maintain compatible versions of a web service without duplicating core business logic.

Yet another possibility is to implement the business logic using low-level APIs like SAAJ and DOM. Since SAAJ and DOM are not typed, you can use the same processing code to process documents of compatible versions. Unfortunately, most web service stacks don’t make it easy to generate SOAP response messages at this low level. For example, in the case of JAX-RPC, handlers can be used to process request and response messages but can replace the need for port components that use schema-generated types.

Similar approaches could be used on the client-side, and so I will skip repeating these for the client implementation.

Lessons Learned

As I remarked at the beginning of this post, versioning is a very nebulous topic. One of the basic issue is specifying what a version is. As in my previous posts on this topic, in this post, I assumed that only compatible changes are allowed in the schema as well as WSDL. But what is a compatible version? If you change the name of an element or attribute, is it still a compatible change? I think the answer varies on a case by case basis. A change considered compatible on one case may turn to be incomatible in some other case. I think the key point is semantics. Even when an element or an attribute is changed in the new version of a schema, the change may not alter the sematics. The change may enhance an existing semantic, but not alter it. Such a change is compatible.

Here are some guidelines that may help minimize the impact of versioning in an implementation.

• Keep changes compatible: This is the basic versioning rule. You may have reasons for incompatible versions, but the consequences of incompatible changes are usually painful.
• Avoid using generated types: Avoid using WSDL/schema generated classes/interfaces in your business logic. Instead design your core business logic to use simple Java objects (a.k.a. POJOs). This would keep the core part of your implementation independent of version-specific code generated by web services toolkits.
• WSDL first: Although most toolkits encourage Java-first approach to designing web services, I find it useful to start with WSDL and schema first, mainly because it allows me to focus on the external view of the web service without worrying about how it is implemented.