Protocols vs Styles

My RESTful Web Services Cookbook has been out for nearly 26 months. From Feb 2010 till the end of 2011, O’Reilly sold 13,052 copies (both print and eBook format combined, per the royalty statements received to date) of this book. Given that O’Reilly’s eBooks are DRM free, I would like to think that several thousands more copies were downloaded for free on torrent sites, but there is no way to know.

Writing this book was a painful exercise. I spent most of 2009 writing the drafts – at that time Yahoo allowed me to spend about 20% of my time for nearly 4 months to write the first draft (for which they own the copyright of the book). It was painful because REST was about style. Not having seen all aspects of the style work in reality, I was not fully convinced about parts of the style. Separating style from protocols was a time consuming yet learning exercise. How could I say that following such and such Roy Fielding’s constraint leads to evolvability without dissecting evolvability into concrete details? The last thing I wanted was to write a book that paints a nice picture with the benefits one would get out of REST without discussing the specifics. I wanted to avoid handwaving at all cost.

In these two years, I changed quite a bit.

• I no longer actively engage myself in discussions about REST.
• I unsubscribed out of rest-discuss in 2011.
• I don’t even subscribe to Roy’s idea that REST APIs must be hypertext-driven or that “A REST API should spend almost all of its descriptive effort in defining the media types”.

What I learned in these two years were the following:

• Don’t confuse styles with protocols.
• Understand what you are going to interoperate with and make tradeoffs as necessary.
• Interoperating between style deviations is more important than arguing for style adherence.
• Interoperability is partly a protocol problem and partly a software problem.

Architecture styles are guides, but protocols specify rules. Roy’s REST codifies (to some extent reverse-engineeres) the Web into a set of constraints. Constaints are what they are – guides sans context. They are subject to tradeoffs. They postulate benefits when applied. But protocols are different. They set ground rules for different systems to work together. Just like you don’t start driving on the left side of the road (in the US), you don’t break TCP or HTTP’s messaging formats. When you break such protocols, you prevent networked systems from working together.

But we tend to mistake the style (i.e, REST) for the protocols. I made that mistake several times (shame on me!) in the past, and I still see it being made. I took care to avoid this while writing the book, but I may not have succeeded fully. Strong belief in a style often leads to zealotry. This is no different from the zealotry we see in socio-political debates.

Yet not all Protocols are the same. I tend to view protocols making up a pyramid. Where a protocol stays in this pyramid shows how widely it helps interoperability.

    /\
   /  \
  /    \
 /______\

Towards the bottom are protocols that strive for the widest range of interoperability possible. TCP is an example. HTTP is another example. As you go up the pyramid, the amount of relevance and influence a protocol carries becomes narower. For instance, I would keep RFC 2616 towards the bottom of the pyramid, but an RFC like 5023 (AtomPub) towards the top. AtomPub’s reign of influence was limited then and is even now. I would similarly place a media type like application/json towards the bottom, and a type like application/my-grand-fathers-facebook-profile+json at the top. A perspective of where a protocol belongs in this pyramid certainly helps ration energy.

So, would I write this book differently now? I certainly would.

• I would change the title to something along the lines of “Recipes to Build Client Server Apps Using Web Protocols”, though the editor might ask me to change to include some current trending words.
• I would emphasize even more on protocols.
• I would cut down or clearly demarcate the parts of the book that suggested design styles (such as designing formats).
• I would spend a big chunk on how to interopere without necessarily agreeing on the style.

If you’ve read my book, and built some interoperable Web based systems, you might come to the same conclusion as I did. If that happens, I count my book as a success.