Comments on: Obvious Choices http://www.subbu.org/blog/2008/06/obvious-choices HTTP, REST and some Cycling Wed, 07 Jan 2009 04:24:47 +0000 http://wordpress.org/?v=2.7 hourly 1 By: Anand http://www.subbu.org/blog/2008/06/obvious-choices/comment-page-1#comment-273 Anand Fri, 20 Jun 2008 23:02:07 +0000 http://wp.subbu.org/2008/06/obvious-choices/#comment-273 Great post Subbu. Scalability is a factor of the architecture, not the framework or language - that is the single biggest thing I have learnt. In my current job [for one more week], we use Java and all the nice stuff only to stumble at the O-R layer for large datasets doing silly things very similar to what twitter guys did initially, badly formed queries for M-2-M relationships and breaking the tiering in a few places. I think choosing a language / framework is a factor of 4 things. - ease of development - raw performance - maintainability - hardware requirements Rails, to me, is the best in 2 out of 4 of those categories. With ruby 1.9 [and 2.0 roadmap], raw performance will at least be comparable to Java, which leaves us with hardware requirements. Unless you are google and have 10s of millions of servers [even they use python], what difference does it make between using 4 vs 6 servers? Great post Subbu.

Scalability is a factor of the architecture, not the framework or language - that is the single biggest thing I have learnt.

In my current job [for one more week], we use Java and all the nice stuff only to stumble at the O-R layer for large datasets doing silly things very similar to what twitter guys did initially, badly formed queries for M-2-M relationships and breaking the tiering in a few places.

I think choosing a language / framework is a factor of 4 things.

- ease of development
- raw performance
- maintainability
- hardware requirements

Rails, to me, is the best in 2 out of 4 of those categories. With ruby 1.9 [and 2.0 roadmap], raw performance will at least be comparable to Java, which leaves us with hardware requirements.

Unless you are google and have 10s of millions of servers [even they use python], what difference does it make between using 4 vs 6 servers?

]]> By: Subbu Allamaraju http://www.subbu.org/blog/2008/06/obvious-choices/comment-page-1#comment-272 Subbu Allamaraju Thu, 19 Jun 2008 08:37:11 +0000 http://wp.subbu.org/2008/06/obvious-choices/#comment-272 To add to my previous reply, for use cases needing bulk GETs, proper use of caching can avoid the need for such GETs without losing on performance. To add to my previous reply, for use cases needing bulk GETs, proper use of caching can avoid the need for such GETs without losing on performance.

]]> By: Subbu Allamaraju http://www.subbu.org/blog/2008/06/obvious-choices/comment-page-1#comment-271 Subbu Allamaraju Thu, 19 Jun 2008 08:00:06 +0000 http://wp.subbu.org/2008/06/obvious-choices/#comment-271 It is not necessarily slower. More importantly, it is necessary to consider a few questions: a. Optimistic concurrency: How will optimistic concurrency work when you want to batch a number of changes into a single request? b. How will clients be notified of partial success or failures? c. What is the client expected to do on partial failures? How will the client know what parts of the request can be resubmitted without worrying about duplicate processing, i.e., is the bulk request idempotent as a whole, or each update within the bulk idempotent? Some of these may be easy to address in certain specific use cases, but I am not sure if the solution can be generalized. If these question can be answered well, and if it is proven that the client/server cannot pipeline connections, batch processing over HTTP can be justified. It is not necessarily slower. More importantly, it is necessary to consider a few questions:

a. Optimistic concurrency: How will optimistic concurrency work when you want to batch a number of changes into a single request?

b. How will clients be notified of partial success or failures?

c. What is the client expected to do on partial failures? How will the client know what parts of the request can be resubmitted without worrying about duplicate processing, i.e., is the bulk request idempotent as a whole, or each update within the bulk idempotent?

Some of these may be easy to address in certain specific use cases, but I am not sure if the solution can be generalized.

If these question can be answered well, and if it is proven that the client/server cannot pipeline connections, batch processing over HTTP can be justified.

]]> By: Will Sargent http://www.subbu.org/blog/2008/06/obvious-choices/comment-page-1#comment-270 Will Sargent Wed, 18 Jun 2008 17:22:04 +0000 http://wp.subbu.org/2008/06/obvious-choices/#comment-270 "In the REST community, there is a similar pattern. Every now and then someone comes along with a need to do batch processing asserting that making "n" connections is slower than POSTing all requests in a single batch. When told of the problems with this approach, that person would immediately conclude that REST and/or HTTP is broken and that he/she "needs" to do batch processing." So it isn't slower? Or the slowness isn't important? I'm confused. “In the REST community, there is a similar pattern. Every now and then someone comes along with a need to do batch processing asserting that making “n” connections is slower than POSTing all requests in a single batch. When told of the problems with this approach, that person would immediately conclude that REST and/or HTTP is broken and that he/she “needs” to do batch processing.”

So it isn’t slower? Or the slowness isn’t important? I’m confused.

]]> By: Mike Amundsen http://www.subbu.org/blog/2008/06/obvious-choices/comment-page-1#comment-269 Mike Amundsen Wed, 18 Jun 2008 11:33:46 +0000 http://wp.subbu.org/2008/06/obvious-choices/#comment-269 Good Post. Good Post.

]]>