A lot has been written about caching issues with XMLHttpRequest based apps. There is certainly some amount of frustration and confusion about this topic. I had the same kind of experience when I updated my blog recently to show a paginated blog entry list via an XMLHttpRequest. I noticed that mainstream browsers behave inconsistently when it comes to caching.
Current releases of Firefox (1.1.7) and Internet Explorer (6.0) handle caching of content downloaded via XMLHttpRequest quite differently.
Firefox 1.1.x does not allow the content download via an XMLHttpRequest to be cached. Although this gives the user a chance to to see the latest content on every request, the downside is increased traffic and bandwidth usage. For example, with Firefox, request for every page in the paginated entry listing on my blog causes an extra network roundtrip to fetch that page, followed by some extra cycles on the server to generate the content.
I looked at the request and response headers that Firefox was sending and receiving. The request included the following headers.
These headers imply that every intermediary (like a caching proxy) and the web server should ignore all caching headers, and MUST send the complete response. For example, this header prevents the web server to responsd with a
304 Not Modified response header and skip sending the content. In effect, these headers force the server to return fresh content on every request. Why would a HTTP client want to send such headers? I can only think of two use cases. Either the client cannot cache the response, or it has lost previously cached response. But servers cannot return a
304 Not Modified response header when there is no conditional request header (such as an If-Modified-Since header).
I checked the Rico script that I used to generate entry list pages for any caching headers that the script could be sending. There were none. So, these request headers were not added by the script, but by Firefox. After some searching, I found that this is a known bug in Firefox. This has been fixed in the latest Firefox 1.5 Beta 2 release. I am not aware of any work-arounds for this bug.
On the other extreme, Internet Explorer 6.0 always shows stale content. Once Internet Explorer downloads the content via an XMLHttpRequest, it caches it, and never bothers to check for updates. I came across a number of solutions that developers were using to get around this problem. One often mentioned solution was to add some random query parameter to the URL so that it will force the browser to treat each request differently disregarding all caching issues. Solutions like this are inefficient as they bypass caching completely.
The source of the problem turned out be an inconsistency with the heuristic algorithm that Internet Explorer uses to decide its caching behavior for resources that do not specify response caching headers.
Internet Explorer uses four browser-specific settings to influence browser caching. These settings read like "check for new versions of stored pages on every visit to the page, every time you start Internet Explorer, automatically, or never". Of these settings, "automatically" is the default setting. As discussed in this MSDN article, this default setting causes Internet Explorer to use a heuristic algorithm to decide its behavior for caching responses that do not contain caching headers. I could not find any reference to known issues with regards to this algorithm, and my guess is that this algorithm does not treat XMLHttpRequest responses the same as regular http responses. The result is that Internet Explorer caches the response forever (subject to its flush policies) for XMLHttpRequests. When I added the script to generate the XML for entry list, I did not bother to add any caching headers to the response (I fixed this later). This caused Internet Explorer to use this algorightm. So, if you are using XMLHttpRequest and find that the browser is not refreshing the page, this might be the reason.
It is easy to fix this problem. To get Internet Explorer not use this heuristic algorithm for caching, add whatever caching headers are appropriate to the response on the server side. When caching headers are present in a response, Internet Explorer uses those headers to decide whether to cache a response, and when cached, how long to cache it, and when to invalidate it.