In my previous post on HTTP Caching, I left out two topics. The first is caching dynamically generated files (e.g. pdf). The second is choosing headers like Expires, Last-Modified etc.
Caching Dynamically Generated Files
Let’s say, you build a feature in your web app to generate pdf reports dynamically based on some user-specified criteria. You can use one of the techniques discussed on my previous post to let browsers cache the report. But the way browser plug-ins or download managers handle the download can complicate this. For instance, the report may be large, and the user may be using a download manager to download the report. These download managers can download the file in several chunks and then combine the chunks once all the chunks are downloaded. Can your web app support such download managers?
This issue is related to caching as well. When you enable caching for a response, clients and caching proxy servers may cache only the first several bytes of the response, and request the rest of the response on demand.
The trick to downloading a single file partially or in chunks is based on a few HTTP 1.1 headers that lets clients download specific ranges of bytes of the response. The first header is the "Accept-Ranges" response header. A value of "bytes" tells clients that this response supports byte ranges, and value of "none" tells that the server does not support byte ranges for the current response. But this response header is optional, and if this header is absent, the clients can still assume that the server supports byte ranges for the response. The client may receive the following response headers from a server supporting byte ranges for the response:
Content-Length: 1234567890 Accept-Ranges: bytes Last-Modified: Mon, 10 Jan 2005 00:37:53 GMT Expires: Tue, 11 Jan 2005 00:37:53 GMT
Since the response is large, a caching proxy server in the middle may decide to cache only the first several bytes of the response body. Let’s say, the proxy stored 2048000 bytes in its cache. On subsequent requests for the same resource, when the browser starts reading the same response, the proxy can serve the first 2048000 from the cache. The proxy must then get the rest of the response from the original server. It can do so by sending a request for a range of bytes by including the following request headers:
Range: bytes=2048000-
This request header indicates the server that the client would like to have the response from 2048000th byte till the end of the response body. The server can then respond with the following response status and headers, and the response body from 2048000th byte of the resource.
HTTP/1.x 206 Partial Content Content-Range: bytes 2048000-1234567890
If the server is not capable of server byte range requests, it can return the full response again with the following response status and headers:
HTTP/1.x 200 OK Content-Length: 1234567890
In case of download managers, once the first response with the Content-Length is received, the download managers typically abort reading the response body, and instead send several requests to download ranges of bytes.
How to build support for byte ranges in your apps? It is possible to implement this support, but the easiest option is to store the generated file to some user-accessible location on the web server or app server, and let the web server or app server serve the file statically. Most web servers and app servers support byte ranges for static files.
Open Questions
The second issue relates to suggestions on choosing various caching-related response headers. Here are the list of questions:
- How to choose the Last-Modified header?
- How to choose the Expires header?
- How to force browsers see my modified JSP files and not use cached versions?
- How to determine whether to return a 304 (Not Modified) status for a given request or not?
In the example code snippet of my previous post, I avoided answering these questions by making the code absurdly simple. The correct answer to these questions is "It Depends" (a.k.a. "I don’t know).
Strictly speaking, the Last-Modified header should reflect the last time the data/content used to generate the response was last modified. In some cases, it may be possible to use a timestamp in the database to determine whether the data was modified since the response was generated last time. But this may turn out to be difficult (at least without incurring heavy database queries) sometimes. In such cases, the best solution is to take a reasonable guess depending on the nature of changes made to the data.
The same argument goes for the Expires header as well. Make a reasonable guess. Whenever caching is involved, there is a possibility that the content that the end user is receiving is stale. Is this acceptable or not? It depends.
