Is choreography necessary for the enterprise? – this was the question I raised several weeks ago. In response to that post, there were several mixed reactions. Some were positive while some were critical. Over the past week or so, I spent some time reading more background material, particularly a couple of papers by Robert Milner, and on process calculus with the hopes of finding the origins and the rationale behind choreography.
The concept of choreography is based on Robin Milner’s work on Pi-Calculus. Thanks to Stu Charlton for referring me to one of Milner’s very lucid papers on some of the ideas. This paper, titled Turing, Computing and Communication, compares and contrasts van Nuemann style of procedural computing to today’s interactive systems, and makes the following observations:
"Software no longer just prescribes behaviour to take place inside a computer; instead, it describes information flow in wider systems."
"We can no longer confine ourselves to systems which are neatly organised …. Consider the internet; it is a linkage of autonomous agents, more of an informatic rabble than an army."
"we have to take non-determinism as elementary, not just temporary laziness which we can amend later by supplying values for all the hidden variables."
"… the meaning (of a computer systems) surely lies in the whole conversation (between systems), not just the end result"
These are important observations. Even though we are trying to build software systems that are truly describable by their interfaces and loosely coupled with other systems, the net result is always based on how these systems interact with, often concurrently, each other, and how each of these systems transfer or synchronize state between them. That brings us to Pi-Calculus.
Since the description of Pi-Calculus at Wikipedia.org is mostly mathematical, let me quote the essence of Pi-Calculus from http://www.fairdene.com/picalculus/pi-c-tutorial.pdf:
(Pi-Calculus is a) formal study of concurrency (that) enables:
- understanding the essential nature of concurrency
- reasoning about the behavior of concurrent systems
- developing tools to aid in producing correct systems
That is, using the concepts of Pi-Calculus, one can describe interactions between concurrent processes, understand their behavior, and also verify that a given set of interactions between processes adhere to their descriptions. Pi-Calculus provides a formal notation for describing interactions between processes. WS-Choreography is a similar attempt to formalize interactions between loosely-coupled WSDL-described processes. According to WS-CDL,
The Web Services Choreography Description Language (WS-CDL) is an XML-based language that describes peer-to-peer collaborations of participants by defining, from a global viewpoint, their common and complementary observable behavior; where ordered message exchanges result in accomplishing a common business goal.
This is essentially a language to describe interaction between web services. WSDLs and other interface description languages don’t meet this goal since a WSDL describes the behavior of a given process/system, and not a series of interactions between a set of processes/systems. So, WS-CDL attempts to fill this gap by providing a formal language to describe interactions between systems from a neutral point of view, something like, if process A, process B, and process C interact in such and such manner, such and such are the possible outcomes.
What choreography calls "observable behavior" is what we, developers, call as "interfaces". What choreography calls as "ordered message exchanges" is what we, developers, routinely describe using UML or some ad hoc notations to communicate how a set of systems interact with each other and what happens as a net result of their interactions (i.e. meeting some use case).
Here is the first surprising part. Most systems built today, whether described formally via interface description languages or not, are not silos. They interact with other systems to come up with a meaningful result. At the programming language level, most software is still sequential and still follow van Neumann’s principles. However, the days of silos is long gone. As users, we are used to interacting with software systems, which in turn interact with other software systems to get anything done. Most software developers and architects build these systems without even having a clue of what Choreography is, let alone the mathematical nuances of Pi-Calculus. While some folks may complain that software built without such description languages is too wild and needs to be curtailed, you can’t deny the fact that software is also getting more fun to build and use.
The second surprise is that, although WS-CDL is just a description language, there are still attempts to use it as an execution language. See this latest post at InfoQ on A Second WS-CDL Tool-Suite is Born, which refers two tools – one from Pi4 Technologies Foundation, and the other (more recent) is from a team from Southeast University, China. I must really be missing the point here. When did WS-CDL become an execution language?
My takeaway from this exercise is this. There are more interesting things that I can do other than debating whether we need choreography or not. If you happen to do some fun things with choreography, please drop me a line – I would love to know.
{ 2 comments… read them below or add one }
My understanding of it is limited, but the question is: how is a CDL tool or framework able to actually monitor and/or enforce a choreography description?
A number of ways for monitor and/or enforce
- An application-layer network intermediary , particularly for CDL-enabling legacy services, like an ESB.
- A framework that augments intercepts calls at the framework level (i.e. inside Axis)
- Some code generation, taking it a bit further than WSDL already does
- A framework that will help pick “next possible steps” in the choreography, track variables, channels, etc. for you
- A framework that hooks into your WS stack to watch what you’re doing, perform correlation, etc.
- One also needs to ensure the services are “CDL-aware”, in that they need to pass channel references around, as a URI, or WS-Addressing, EPR, etc., and if they’re not already doing so with some kind of unique Message ID, this could be automated by a CDL framework.
I think most of the toolkits opt for the “Framework-level” proxy instead of the “network-level” proxy to insert CDL policy. Or, in other words, they’re not building an ESB, they’re building an Axis-wrapper.
During the standards process, I recall there was contention about the necessity of BPEL vs. CDL because it is hard to see the difference at a high level when “enforcing” a CDL: they’re both supported by implementation-level frameworks, and they’re both talking about describing business processes. People didn’t really get CDL’s nuance, hence why BPEL has taken off.
The difference, as I see it, is subtle:
- BPEL is really an *execution language* on a single node. The endpoints that BPEL talks to don’t need to know that the parent was implemented in Java, .NET, Ruby, or BPEL.
- Whereas CDL is not something executed, it’s more of a spec of control flow that is shared across all of the systems that interact with one another; while it can also indicate what “next steps” are required, like BPEL, it also has a declarative way to specify the detection of errors & throwing of exceptions to who isn’t playing by the rules *without* requiring a centralized orchestrator.
So you could think of CDL as being *both* an “interaction variance checker” and a “distributed process execution language”. The latter part is somewhat more controversial, obviously.
The Pi4Tech stuff, by the way, was started by Steve Ross-Talbot, who was probably the #1 public champion for WS-CDL (that I’ve seen), and was the chair of the W3C working group. I recall he has some very good reasons for popularizing this technology.
Anyway, Pi4SOA isn’t documented (at all), so you’d have to browse CVS to figure it out, along with the Sourceforge discussions.
Some discussions I found a bit enlightening:
http://sourceforge.net/forum/forum.php?thread_id=1515687&forum_id=419139
One area that interests me in CDL vs. REST: “Hypermedia as the engine of application state”, in the RESTful web, is how we construct interaction machines. To me, hypermedia already exhibits much of what we want in a choreography-driven SOA, in that it specifies the possible interactions at runtime, with ordering constraints and whether they’ll have side-effects or not (GET, PUT or POST) defined by the media type. The problem is that these are usually defined in a human-readable RFC or specification.
So, it seems a RESTful CDL would
- describe the ordering constraints of a particular media type’s anchors
- provide a language to help a user agent track long-lived client-side variables as it crawls across representations.
- handle more than XML-based interaction
“I think most of the toolkits opt for the “Framework-level” proxy instead of the “network-level” proxy to insert CDL policy. Or, in other words, they’re not building an ESB, they’re building an Axis-wrapper.”
Right. Of course, some ESBs may be able to do well in this area, possibly tying this with governance.
Just like there is no single distributed programming model/protocol, I would argue that there won’t/can’t be a single formal description language to describe interactions between systems. WS-CDL can only do so much. Just like UML did not fix all the modeling problems!
“So, it seems a RESTful CDL would
- describe the ordering constraints of a particular media type’s anchors
- provide a language to help a user agent track long-lived client-side variables as it crawls across representations.
- handle more than XML-based interaction”
Possibly. If someone claims to have a universal description/orchestration language, they better include resource-oriented systems, or whatever we come up with in future as well. IMO, that is going to be a limitation with most service composition-level specifications. They can only formalize certain aspects of certain distributed systems, but unfortunately claims don’t stop there. Instead of saying that “okay, we understand what you are doing with REST, but sorry our technology does not deal with your systems”, they say “your approach is incomplete, and so use our approach for everything” (quotes mine).
Thanks for your thoughtful comments.
Subbu