subbu.org » Uncategorized

BigPipe Done in Node.js

Subbu Allamaraju — Mon, 12 Jul 2010 21:53:32 +0000

I’ve implemented a proof of concept of BigPipe in Java (should run as-is in every servlet container):

See his blog post for the Java servlet class.

Here is the same (or more?) written in Node.js.

var http = require('http');
var sys = require('sys');
var url = require("url");

http.createServer(function(request, response) {
    // Write the document
    response.writeHead(200, {"Content-Type" : "text/html"});
    response.write("");
    response.write("");
    response.write("Progressive Loading");
    for(var i = 0; i < 6; i++) {
        response.write("&;lt;/div>");
    }
    response.write("");

    // Now the snippets
    var down = 6;
    for (i = 0; i < 6; i++) {
        var proxy = http.createClient(2000, "localhost");
        var proxyRequest = proxy.request("GET", "/?id=" + i,
                                         {"host" : "localhost"});
        proxyRequest.addListener('response', function (proxyResponse) {
            --down;
            proxyResponse.addListener('data', function(chunk) {
                response.write(chunk, 'binary');
            });
            proxyResponse.addListener('end', function() {
                if(down == 0) {
                    response.end();
                }
            });
        });
        proxyRequest.close();
    }
    response.write("");

}).listen(8080);

http.createServer(function(request, response) {
    // Some delay upto upto 2 seconds
    var delay = Math.round(Math.random() * 2000);

    setTimeout(function() {
        var params = url.parse(request.url, true);
        var id = params.query.id;
        response.writeHead(200, {"Content-Type" : "text/html"});
        var content = "Content of Module " + id + "";
        response.write("");
        response.close();
	}, delay);
}).listen(2000);

I've no comments on the technique itself. The basic idea may have been implemented a number of times in different languages. The mileage varies based on the language/environment used.

New Co-Motion Tandem

Subbu Allamaraju — Sun, 30 May 2010 03:32:10 +0000

Co-Motion Periscope Torpedo Tandem – Shimano Ultegra – FSA Gossamer 52-39-30 – 12×28 – S+S couplers – Custom built – Fit by Counter Balance Bicycles (University Village, Seattle)

Still learning to maneuver – 103 flat miles in the first week

Uncomplicated Hypermedia – Facebook’s Graph API

Subbu Allamaraju — Wed, 21 Apr 2010 20:01:27 +0000

It is a pleasure to read the Facebook Graph API. It avoided many of the traps that web services offered by the other major players in the industry suffer from. Facebook’s API is simple, consistent and inter-connected. It is true to the spirit of the web.

It is simple, because as a developer, I do not have to learn about 10 different web services for each of users, pages, events, groups, applications etc. Having a separate web service for each is like having "end points" – resources that are not connected, each with a different set of conventions. In the Facebook Graph API, there are no end-points.

It is consistent, because all the representations look similar. Each representation has an identifier, a self-link, timestamps, and links to related resources. Apart from these being the ideal characteristics of representations on the web, for a developer, such representations are a pleasure to work with. Facebook just proves that you don’t need to focus heavily on formats like Google Data Protocol or Open Data Protocol to design representations that are consistent.

It is interconnected as every representation is linked to related resources. For instance, an album representation is linked to the profile that posted the album. Each user representation is linked to a collection of groups that the user is member of. This is hypermedia in action. Hypermedia does not have to be complicated – and Facebook just proves it.

I am sure it is possible to nitpick on minor details, but the underlying philosophy cannot be ignored. Finally, support for OAuth 2.0 is the icing on the cake.

Using PATCH

Subbu Allamaraju — Sat, 03 Apr 2010 04:24:14 +0000

Now that HTTP PATCH is official, would lack of standard media types and patch formats to describe partial updates to resource hamper its adoption? Most likely not.

The problem with any write request is maintaining integrity of the data in the backend. When a client submits POST or PUT requests, the server has to ensure that the representation in the request is consistent with the business rules that govern what is valid and what is not. The media types used for representations along with documentation can describe client developers how to make such write requests that honor server's business rules.

The introduction of PATCH complicates this a bit more. In addition to describing valid resource representations for creating, updating, or doing other kinds of things with POST and DELETE, the server now needs to tell clients what combinations of partial updates are valid for use with the PATCH method. General diff formats would leave the door open for clients to submit invalid combinations of partial updates that violate the business rules on the server side. To avoid this, it is better to come up with application specific representations for partial updates. See How to Use PATCH for an example. Lack of standard media type for diffs should not hold you from using the PATCH method.

RESTful Web Services Cookbook Released

Subbu Allamaraju — Thu, 11 Mar 2010 18:23:36 +0000

I am happy to announce that the RESTful Web Services Cookbook is now available in stores.

Many thanks to Mark Nottingham, Eben Hewitt, Colin Jack, Stefan Tilkov, Norbert Lindenberg, Chris Westin, Dan Theurer, Shaunak Kashyap, Larry Cable, Alan Dean, Surya Suravarapu, Jim D’Ambrosia, Randolph Kahle, Dhananjay Nene and Brian Sletten for their valuable and critical feedback on this book.

If you are interested reviewing the book, please email me directly. The Yahoo! Developer Network may be interested in publishing a review.

Writing for O’Reilly

Subbu Allamaraju — Fri, 12 Feb 2010 03:08:43 +0000

A source control system (Subversion) to manage the drafts from day 1
DocBook as the preferred format – which let me use IntelliJ with DocBook DTDs attached for validation
An auto-triggered build that generates a PDF version that looks more or less like the printed book
An editor that provides the right dose of nudging on occasion
Tool staff that are ready to help with tools and provide guidance on preferred use of DocBook
Copy editors that remind how badly I write, and painstakingly provide corrections
Production staff that follow through the process meticulously

Keechelus Ridge Snowshoeing

Subbu Allamaraju — Sun, 31 Jan 2010 00:38:08 +0000

We hiked up the Keechelus Ridge today. This is a moderately steep climb from the Price Creek Sno-Park. In just about three miles, this trail climbs over 1600 ft.

Envelope for Signatures

Subbu Allamaraju — Thu, 21 Jan 2010 05:49:11 +0000

In Magic Signatures for Salmon, John Panzer describes a way to pass signed XML data without involving XML canonicalization (c14n). Anyone who has dealt with WS-Security specs knows that canonicalization can be fragile (back in my days at BEA, getting signatures to work on the WebLogic stack turned out to be hard due to bugs in the c14n implementation) and slow. John Panzer’s approach is quite simple, but it requires introducing an XML based envelope format.

In John’s approach, to send signed XML, the sender does the following.

Serialize the XML using into UTF-8 encoded text.
Generate a signature of the BASE64 encoded data.
Use an XML envelope to send the BASE64 encoded data and the signature.

Before decoding BASE64 data, the receiver first verifies the signature of the BASE64 encoded data. If the signature matches, the receiver BASE64 decodes the data, and then parses the decoded text into XML.The most elegant part of this process is that canonicalization is not needed.

The envelope in his example looks like the following:


  
PD94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGluZz0nVVRGLTgnPz4KPGVudHJ5IHhtbG5
zPSdodHRwOi8vd3d3LnczLm9yZy8yMDA1L0F0b20nPgogIDxpZD50YWc6ZXhhbXBsZ
S5jb20sMjAwOTpjbXQtMC40NDc3NTcxODwvaWQ-ICAKICA8YXV0aG9yPjxuYW1lPnRlc3RAZXhhbXBsZS5jb208L25hbWU-PHVyaT5hY2N0OmpwYW56ZXJAZ29vZ2xlLmNvbTwvdXJpPjwvYXV0aG9yPgogIDx0
aHI6aW4tcmVwbHktdG8geG1sbnM6dGhyPSdodHRwOi8vcHVybC5vcmcvc3luZGljYX
Rpb24vdGhyZWFkLzEuMCcKICAgICAgcmVmPSd0YWc6YmxvZ2dlci5jb20sMTk5OTp
ibG9nLTg5MzU5MTM3NDMxMzMxMjczNy5wb3N0LTM4NjE2NjMyNTg1Mzg4NTc5
NTQnPnRhZzpibG9nZ2VyLmNvbSwxOTk5OmJsb2ctODkzNTkxMzc0MzEzMzEyNzM
3LnBvc3QtMzg2MTY2MzI1ODUzODg1Nzk1NAogIDwvdGhyOmluLXJlcGx5LXRvPgog
IDxjb250ZW50PlNhbG1vbiBzd2ltIHVwc3RyZWFtITwvY29udGVudD4KICA8dGl0bGU-U2FsbW9uIHN3aW0gdXBzdHJlYW0hPC90aXRsZT4KICA8dXBkYXRlZD4yMDA5LTEyLT
E4VDIwOjA0OjAzWjwvdXBkYXRlZD4KPC9lbnRyeT4KICAgIA==
  RSA-SHA1
  EvGSD2vi8qYcveHnb-rrlok07qnCXjn8YSeCDDXlbhILSabgvNsPpbe
76up8w63i2fWHvLKJzeGLKfyHg8ZomQ==>/me:sig>

In this XML envelope, the me:data element contains BASE64 encoded data. The type attribute tells that the data is an Atom document. The me:sig element contains the signature.

But does sending a signature require an envelope? Here is an alternative that does not use an envelope.

Content-Encoding: base64
Content-Type: application/atom+xml;chatset=UTF-8
Authorization: salmon EvGSD2vi8qYcveHnb-rrlok07qnCXjn8YSeCDDXlbhILSabgvNsPpbe76up8w63i2fWHvLKJzeGLKfyHg8ZomQ==

D94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGluZz0nVVRGLTgnPz4KPGVudHJ5IHhtbG5
zPSdodHRwOi8vd3d3LnczLm9yZy8yMDA1L0F0b20nPgogIDxpZD50YWc6ZXhhbXBsZ
S5jb20sMjAwOTpjbXQtMC40NDc3NTcxODwvaWQ-ICAKICA8YXV0aG9yPjxuYW1lPnRlc3RAZXhhbXBsZS5jb208L25hbWU-PHVyaT5hY2N0OmpwYW56ZXJAZ29vZ2xlLmNvbTwvdXJpPjwvYXV0aG9yPgogIDx0
aHI6aW4tcmVwbHktdG8geG1sbnM6dGhyPSdodHRwOi8vcHVybC5vcmcvc3luZGljYX
Rpb24vdGhyZWFkLzEuMCcKICAgICAgcmVmPSd0YWc6YmxvZ2dlci5jb20sMTk5OTp
ibG9nLTg5MzU5MTM3NDMxMzMxMjczNy5wb3N0LTM4NjE2NjMyNTg1Mzg4NTc5
NTQnPnRhZzpibG9nZ2VyLmNvbSwxOTk5OmJsb2ctODkzNTkxMzc0MzEzMzEyNzM
3LnBvc3QtMzg2MTY2MzI1ODUzODg1Nzk1NAogIDwvdGhyOmluLXJlcGx5LXRvPgog
IDxjb250ZW50PlNhbG1vbiBzd2ltIHVwc3RyZWFtITwvY29udGVudD4KICA8dGl0bGU-U2FsbW9uIHN3aW0gdXBzdHJlYW0hPC90aXRsZT4KICA8dXBkYXRlZD4yMDA5LTEyLT
E4VDIwOjA0OjAzWjwvdXBkYXRlZD4KPC9lbnRyeT4KICAgIA==

In this case, the Authorization header contains the signature, and the body of HTTP message contains base64 encoded bytes. I extended Content-Encoding to specify base64. I left out details like the signature method name, and the Authorization header can be extended to include such data, or such details can be specified as part of the authorization scheme (the way basic auth, digest auth, or OAuth do).

The magic envelope is HTTP (thanks to MIME).

Cache Invalidation

Subbu Allamaraju — Tue, 19 Jan 2010 04:58:18 +0000

In the ideal world, what we see is the current. In the distributed software world, what we see may be stale. We can't tell. Would not it be nice to specify a cache invalidation API such that the source of the change can notify everyone that it changed? That is what an OpenSocial 1.0 draft aims to do.

Containers MUST support the invalidation endpoint even if they do not perform any caching and MUST provide an entry for it in their XRDS. To invalidate content a developer’s backend notifies the container of the content it wishes to invalidate by making a 2-legged OAuth call to the Cache service with one or many keys to be invalidated. The consumer key in the 2-legged OAuth call is used by the container to identify the calling application.

This text is followed by an example.

POST /api/rest/cache/invalidate HTTP/1.1
Host: api.example.org
Authorization: hh5s93j4hdidpola
Content-Type: application/json

{
  "invalidationKeys" : [
      "http://www.myapp.com/gadgetspec.xml",
      "http://www.myapp.com/messagebundle.xml"
  ]
}

Excellent.

WS-REST 2010 Call for Papers

Subbu Allamaraju — Fri, 15 Jan 2010 16:54:35 +0000

The due date (February 8, 2010) for WS-REST 2010 First International Workshop on RESTful Design is fast approaching. There is still time to submit papers for this workshop. Topics include:

Applications of the REST architectural style to novel domains
Design Patterns and Anti-Patterns for RESTful services
RESTful service composition
Inverted REST (REST for push events)
Integration of Pub/Sub with REST
Performance and QoS Evaluations of RESTful services
REST compliant transaction models
Mashups
Frameworks and toolkits for RESTful service implementations
Frameworks and toolkits for RESTful service consumption
Modeling RESTful services
Resource Design and Granularity
Evolution of RESTful services
Versioning and Extension of REST APIs
HTTP extensions and replacements
REST compliant protocols beyond HTTP
Multi-Protocol REST (REST architectures across protocols)

This list is exhaustive. In particular, I would love to see contributions presenting on performance and QoS, tradeoffs, HTTP extensions and client/server side frameworks. See the call for papers for more details.