Article
Cost-Effective Website Acceleration
Dynamic Pages: Build Them Now or Build Them Later?
Interestingly, many developers dynamically build many or all of their sites' pages at visit time. For example, the URL http://www.domain.com/article.php?id=5 suggests that the page is being built from a database query or template fill-in. The problem with this common practice is that, in many cases, building such a page at request time is pointless, because most of the time it is primarily static. So-called static dynamic pages, or scripted pages, whose contents don't change for long periods of time, obviously do nothing to improve page delivery time. In fact, on a heavily trafficked site they can needlessly bog down your server.
One approach to addressing unnecessary dynamic page generation would be to pre-build the page into a static content page complete with a .html extension every time it changes. By preference, any such generated pages would not only be static HTML files, but would also be code optimized versions (as discussed in Part I). Not only does this make it easier for the server to deliver the page quickly, because it doesn't have to do any work before it returns it, but the technique actually makes the site more search engine friendly.
Unfortunately, in many cases, simply generating pages to static HTML is not possible because the pages do indeed have dynamic content that must be executed at page view time. In this case, your best bet is to "bake" the page into a faster-to-execute form. In the case of ASP.NET, this would utilize the byte code format, which is much faster for the server to execute. Unfortunately, you need to force the byte code by executing pages first, before you let your users access them. Fortunately, upcoming versions of ASP.NET 2.0 are going to help mitigate the tiresome task some developers currently undertake of "touching" all of their pages once to ensure quick first downloads for users. In the case of PHP, you may find a product like the Zend optimization suite to be a wise investment.
Given the differing requirements of serving static and dynamic pages, or HTML and images, it seems wise to revisit the physical server and other hardware for additional acceleration ideas. One of the more intelligent impulses that can lead people to add or beef up hardware for the sake of Web acceleration is the idea of specialization -- that different elements are required to do different jobs with maximum efficiency. Even though, in the name of cost-efficiency, our main focus through this series has been on the source code and the Web server (and related) software, let's now take a quick look at these other elements.
Turbo Charging Your Web Server
A good place to turn in order to speed up your site is the server software and hardware itself. Starting with software, it is fairly unlikely that Web administrators are going to quickly dump IIS for Apache or Apache for IIS, even when security, ease of use, or performance are cited as reasons to leave one camp for another. Put simply, the Web server and its associated operating system are often so intertwined with the Website(s) that migration becomes an onerous and even risky task. If you do start thinking about dumping one platform for another, you should also seriously consider alternate platforms, such as Zeus, particularly if speed is a primary concern.
When it comes to hardware, carefully consider the main tasks that the server(s) performs. In the case of a static Website, it is primarily to marshal network connections and to copy files from disk to network. To accelerate such a site, you want to focus on building a Web server with very fast disk and network subsystems, as well as enough memory to handle simultaneous requests. You may, in fact, choose to get very aggressive in adding memory to your system and try to cache all heavily used objects in memory, in order to avoid disk access altogether. Interestingly, processor speed is not nearly as important as you might think in a site serving static files; it does help, but the disk will often prove to be a bigger bottleneck.
When the site is serving dynamic as well as static pages, obviously, processor speed becomes more important, but even in this case, having a fast drive or dual NICs can still be more important. On the other hand, the combination of a lot of dynamic page generation with other processor-intensive responsibilities like SSL and/or HTTP compression makes it more likely that the CPU is the resource in need of enhancing. In other words, what the server does will determine how it can most effectively be beefed up.
Even if you don't have the budget to customize your server hardware or software, there might be some changes you can make fairly cheaply. For example, you might consider tuning your operating system's TCP/IP settings for the most efficient use of those underlying TCP/IP resources on which HTTP is dependent. This might be a matter of adjusting the TCP receive window to a size best suited to your application and your server's network connection. You might make sure that TCP features that can be toggled, such as delayed ACKs or TCP_NODELAY, are used or not used, depending again on the characteristics of the application and the network environment. Or it might simply be a matter of making sure your box is not suffering from port exhaustion due to excessive TIME_WAIT times or other causes.
However, before twisting the knobs on your network characteristics, both in the Web server and on the operating system, make sure to set up realistic load tests to verify that your improvements don't actually slow users down or cause excessive retransmits on your network. In other words, don't try to implement these types of fixes unless you understand how to measure their value.
Acceleration by Division of Labor
Another consideration when addressing site acceleration is that not every form of site content has the same delivery characteristics. Given that different content has different characteristics, we may find that dividing the specific serving duties of a site between several servers is better than simply duplicating site duties evenly across a server farm.
A simple example of acceleration via division of labor occurs when a site uses SSL encryption for a shopping cart or extranet. In the case of SSL, you might find that your server bogs down very rapidly when you have multiple users in the https sections of your site, due to the significant overhead introduced by the encryption. In this situation, it makes sense to offload this traffic to another server. For example, you might keep your main site on www.domain.com and, for the checkout portion of the site, link to shop.domain.com. The shop.domain.com server could be a special box equipped to handle the SSL traffic, potentially using an SSL acceleration card.
Segmentation allows you to focus the necessary resources on those users in checkout without bogging down those who are just browsing. You might even find that serving your images or other heavy binaries like PDF files or .EXE downloads from one server might make sense, since these connections take much longer to shut down than ordinary connections and, in the meantime, hold valuable TCP/IP resources hostage. Furthermore, you would not bother with HTTP content encoding on the media server, but would, of course, apply it to the primary text server for HTML, CSS, and JavaScript. In this way, you maximize the use of that server's otherwise underutilized CPU for the sake of bandwidth optimization, while leaving the CPU of the media server free to do its job.
Segmentation can be further applied to generated pages. You might consider serving generated pages from a box built for that purpose, offloading your static content to a different machine. Already, many sites employ just such a scenario, using a reverse proxy cache such as Squid. In this setup, the proxy serves static content quickly, while the back-end server builds the content that truly must be generated at page view time. The cache control policies and rules discussed in Part II of this series become quite important in this type of setup; you will want to make sure that the proxy server's cache stores everything that it is safe to store in a shared cache, and that it doesn't retain anything that is intended only for a particular user.
Speed for Sale
We have focused primarily on low cost acceleration techniques in this series, but as we get towards the end, costs appear in the form of new software and new hardware. It is possible to spend quite a bit of money on Web acceleration by purchasing exotic acceleration appliances that perform connection offloading, compression, caching, and a variety of other functions for your site. If you have an extremely high bandwidth bill, these solutions can be very effective, but most sites can stick with the cheaper approaches we've discussed here, such as code optimization, caching, and HTTP encoding.
Yet, even if you did have a lot of money and were able to build out a specialized server farm and add exotic acceleration devices, you would eventually reach a limit to the benefits gained from compressing and caching content. To improve your speed now you'd have only one last option -- to move the content closer to the user (and, if possible, apply all the same techniques again). You've probably noticed that sites that offer downloads often mirror content so that users can download from multiple locations. However, it is possible to provide content in a more geographically-sensitive manner, and to do so transparently.
Content distribution networks (CDNs) like Akamai allow us to move heavy content such as images and other binary downloads closer to users by taking advantage of thousands of edge caches located around the world. While this technique provides significant performance improvements and is widely used by the world's largest sites, it certainly does not fit in the category of cost-effective, and thus we mention it only to follow our acceleration options to their logical conclusion.
Before you resort to these drastic and usually unnecessary measures, we conclude our series with a brief example of how the low cost ideas we have presented over the last three weeks can improve page load times, bandwidth usage, and server load for a real site, in this case, looking at the home page of SitePoint.com itself.
Applied Acceleration: SitePoint.com
Let's grab the SitePoint.com homepage using Internet Explorer 6.0. When we request SitePoint.com for the first time, we see that the homepage is actually made up of about 37 distinct objects. But, we also notice right away several interesting things that SitePoint's developers are doing to accelerate their home page:
- On the main or containing part of the page, the HTML content is compressed using HTTP content encoding (specifically, gzip). Without gzip, the original file size would be 28,218 bytes. With it, the transferred size is only 7,774 bytes -- a saving of about 72%. SitePoint's dial-up users will definitely thank them for that. A closer look at the response headers tells us that this same page is also sent using chunked transfer-encoding, which can help mitigate the time to first byte penalty associated with HTTP compression. This means that even broadband users might experience a faster page load.
- Something else we learn from the response headers is that, while the home page is dynamically built using PHP, the developers have minimized the performance impact of this by using a third-party tool that pre-interprets the PHP script and caches the interpreted instructions in memory (the giveaway is the response header X-Accelerated-By: PHPA/1.3.3r2). With a script cache like this one, the page is still dynamic (in the sense that its HTML output is not cached), but the server-side overhead of loading up the script and interpreting it into executable instructions every time it is requested is avoided -- a nice compromise between full pre-generation and a purely dynamic page.
Is there anything else that SitePoint.com could do, on the cheap, to optimize things even further? There are several possibilities, which we have summarized in the following table:
This table (click here for the larger version) shows the effects of file size reduction in two stages -- first, the application of source optimization described in the first installment of this series (the byte count in the "Optimized" column), then the application of HTTP compression, which we covered in this article (the byte count in the "Gzipped" column). We've given the compressed (gzipped) size for the main page (index.php) just as we got it in IE, but we haven't applied source code optimization to it, since it is dynamic.
In addition, however, there are eight static external text files (5 CSS and 3 JavaScript) that are not being compressed or code optimized at all, but that could definitely benefit from both. We've shown the results here, and as you can see in the Percentage Saved columns, the resulting overall savings are substantial -- a 13% saving for the whole home page (all dependencies included) just with source code optimization, and a 50% saving with source code optimization plus HTTP compression. These percentages could go even higher if the index.php were source code-optimized as well (which would probably yield an additional 1000 bytes or so).
The last two columns focus on the topic of the second article in the series: cache control. Expiration-based cache control isn't being used on the SitePoint.com home page, except for a few externally hosted or highly specialized resources. The "Reval?" column shows which of the files, as a consequence, need to be revalidated upon a return visit, even though they are present in the browser's cache. SitePoint could consider making much wider use of explicit expiration times, especially for the plethora of relatively invariant images that make up the home page, as well as for the static external text files that contain CSS and JavaScript. Authoring and implementing a good set of cache control policies for such objects would help to avoid some or all of the 31 separate client-server round trips required to revalidate already cached objects when a user returns to the home page in a new browser session. That's 31 fewer 304 Not Modified messages in the server logs for every return page view, but more importantly, would also reduce return visitor page load times.
The last column shows that these revalidation round trips take between one and seven tenths of a second each. Even though a number of them occur simultaneously when a user returns to the home page (and take up a bunch of TCP resources on the server-side in the process!), that's still a substantial wait compared to essentially instantaneous serving of those same objects from the browser's cache -- which is where they wound up coming from in any case.
Conclusion
Our simple case study clearly demonstrates that, when used together, the techniques presented in this series can significantly improve the server load, bandwidth usage, and page load times of almost any Website. All are relatively simple to implement and none require large expenditures on new hardware or software.
Granted, no acceleration technique is without its downsides; code optimization requires a time investment from developers, caching requires increased coordination between development and administration, and compression can tax server resources in order to save bandwidth.
In conclusion, we hope we have provided you with a better understanding of the principles behind Website performance and a number of specific enhancement techniques that will help you to carefully put your foot on the gas.
About the Authors
Thomas Powell is the founder of PINT, Inc., an instructor at the UCSD Computer Science department, and the author of Web development books including HTML & XHTML: The Complete Reference, and JavaScript: The Complete Reference.
Joe Lima is a lead architect at Port80 Software, which makes IIS server modules such as CacheRight, httpZip and ZipEnable and Web developer tools such as the w3compiler. Joe also teaches Web Server technology for UCSD Extension.