Related Articles

1. Successful Web Development Methodologies
   Rated: 9/10

About the Author

Shirley Kaiser

Shirley Kaiser owns SKDesigns, a company focused on creating visually appealing web sites that are based on web standards and universal accessibility, and are optimized for speed and search engines. Aside from her web work, Shirley is a professional musician with Bachelor of Arts (B.A.) and Master of Arts (M.A.) degrees in Piano Performance.

View all articles by Shirley Kaiser...

The Ultimate Testing Checklist

Testing plays a critical role in the development of your web site and its long-term maintenance. While smaller web sites—especially those with more limited budgets—may not need to follow the formal testing procedures that are required for large-scale, commercial web sites, every site needs to be thoroughly tested to ensure that it's error-free, user-friendly, accessible, and standards compliant.

Testing should be completed during each phase of a site's development. Two of the most costly web project mistakes are to delay testing until just before launch, or not to test at all. Testing during production makes it easier to locate and resolve errors, and minimizes the chance of existing bugs being replicated throughout later stages of development. Early and continued testing can eliminate the need for the costly redesigns and other major fixes that can result from overlooked errors.

This chapter's checklists, extracted from SitePoint's new release, Deliver First Class Web Sites: 101 Essential Checklists, will help you test your site both during development, and after. Download this checklist, along with others covering SEO and content management -- you'll also receive .pdf versions of the documents for immediate use in your web projects.

Getting Started

Document your baseline web site testing requirements.

Make use of the preliminary data you collected in Chapter 2 to help determine and document your baseline site testing requirements. For instance, the information you collected on the browsers and connection speeds that your visitors will use, and those visitors' skills and age groups, can be used as a basis for your testing plan.

Source and install all necessary tools.

You can conduct tests very cheaply using free and low-cost testing tools. Most browsers are free or offer conditionally free versions, and you can download and install multiple browsers—including several versions of Internet Explorer for Windows—on one computer. Alternative device simulators and emulators, such as those for cell phones and PDAs, can also be downloaded and used for free, and most shareware and commercial software creators allow a 30-day trial of their products for testing purposes. Some web design-related discussion lists, as well as the SitePoint Forums, welcome testing requests and posts that solicit bug reports and feedback.

Provide acceptable testing protocols.

Ultimately, the design and layout for each page of the site must be both aesthetically pleasing and functionalacross multiple platforms and browsers.

One of the difficulties associated with testing design elements on different platforms is that the test results can be subjective. For example, a designer may create a CSS-driven site that works perfectly on newer browsers, but doesn't look that hot in older browsers such as Netscape 4. The tester may interpret this as a failure point rather than acceptable "design degradation."

To avoid the time and effort involved in re-submitting pages for testing, spell out acceptable cross-browser, cross-platform differences in advance—including protocols for supporting old browsers.

Use Annotated Screenshots to Determine Display Protocols
At the end of the final prototyping stage, ask the designer to provide a series of screenshots that have been annotated to identify the differences in the ways the site displays on different browsers and platforms. These screenshots can be used in later testing to identify the displays that are deemed acceptable on each platform. Testers can then compare the protocols and screenshots with their findings.

Set up a staging server.

A staging or test server that's identical to your web site's live server environment should be used for testing and development purposes prior to your site's launch.

Automated Programs for Larger Sites
Larger web sites typically require more robust testing procedures that test large sets of records, values, and content. You might consider using an automated program, such as Badboy, eValid, or TestComplete for such purposes.

Good Testing Practice

Systematically test individual pages.

Thoroughly test your site one page at a time, making sure each page is solid before proceeding. Carry out a comprehensive test of the final version of the site in the same way.

Track bugs and confirm fixes.
Document the details of discovered bugs and their resolutions.

Regression test, especially when fixing bugs.

Regression testing, which is also known as verification testing, is the process of retesting pages or sections of a site to make sure that a recent bug fix hasn't broken some other aspect of the site, or reinstated bugs that were fixed previously. Conducting and documenting regression tests will allow you to identify any parts of the site that break, and document the causes of those errors and how you resolved them. If a bug resurfaces at some point, it will be spotted immediately and resolved quickly. (See Regression Testing, Webopedia Computer Dictionary).

Validate the markup for each individual web page.

Validate the markup for all your web pages to ensure that each uses structural, compliant markup. This will allow you to confirm that you've used the correct markup for the specified DOCTYPE, fix any typos you may have made, and resolve any syntax errors. Validating your markup as you build a page, rather than waiting until you've created an entire page (or worse, an entire web site), makes it easier to isolate, find, and correct markup problems.

Using Markup Validators
Some HTML editors integrate validation tools such as W3C's HTML and CSS validators, the WDG HTML Validator, and the CSE HTML Validator (an HTML/XHTML syntax checker and validator), which allow you to validate your markup conveniently while you work.

Ensure that the validation tool you use is based on W3C Recommendations.

Validate all CSS.

Validate all the CSS you use on the site, including each external style sheet, plus all embedded and inline CSS.

Conduct load testing to stress-test programming technologies and server hardware capacities.

You need to make sure that the programming technologies you've used, and the capacities of your server hardware, can cater to higher levels of traffic than you expect the site to receive. Taking this approach will help you ensure that your site remains online and fully functional at all times. Load testing procedures should include a trial run—tests that usually involve the use of automated scripts that emulate multiple simultaneous user sessions. The trial run will reveal how the programming technologies and server hardware you've used will cope with traffic spikes.

Matching Hosting with Projected Server Loads
Your ISP should match your hosting package with your projected server loads if you're using a web hosting service, so you should let them know what your loads will be. Your web host might recommend a different web hosting package that offers greater capacity if your load testing reveals that your existing setup has the potential for problems as traffic levels increase.

If you're managing your own server(s), check whether your server software provider offers free load testing software—many do. You might also consider using dedicated load-testing software products, such as those offered by WebPerformance and NetMechanic. In addition, numerous open-source performance analysis software tools are available to help you assess, monitor, and manage your web site's performance. (See also Stress Tools to Test your Web Server, Microsoft.com.)

General Testing

Test your web site on multiple browsers and platforms.

The fact that your web site looks good and works well in one browser doesn't mean it will look as good or function as well on other browsers and/or platforms. Even if you're developing an intranet web site for an organization whose employees use the same browser and operating system, the hardware and software used by those employees can and will vary. In the long run, it's better to develop your web site on the basis of the W3C recommendations, then to follow up with extensive cross-browser, cross-platform testing. Stay informed of the release of new or updated browsers and operating systems so that you can keep your cross-browser, cross-platform tests current.

Which Browsers and Platforms to Test?
Check your server logs—as well as other on- and offline resources—that cover browser statistics, operating systems, and emerging Web use trends, so that you know which browsers, versions, and operating systems your web site visitors use now, and are likely to use in future. As part of your approach to site maintenance in the long term, plan to check your server logs weekly (monthly checks might be fine if you run a smaller web site with low traffic volumes) to follow changes and trends in your browser statistics. Take this data into account as your hone you your cross-browser, cross-platform testing procedures, to ensure that you're testing all of the browsers and operating systems that your visitors use. Here's a list of the popular operating systems and browsers on which you might test your site.

Test page optimization with every update.

Optimizing your pages right from the start can help ensure that your site design and images support fast page load times, and that the site's development is smooth and efficient. Whenever you add anything to your web site—new content, new images, or new web pages—check that content's optimization to help keep the site streamlined. If you're not yet familiar with web site optimization, see Chapter 11, Web Site Optimization.

Document Weight and Load Time Tools
Some HTML editors include features that tell you a document's weight, or a page's load time at various connection speeds, and most image editing software products indicate file size load times at various connection speeds.

There are also free online tools that calculate document weight, composition, and page load times, and even offer recommendations for optimizing web documents. Siteoptimization.com's helpful Web Page Analyzer is one such tool.

View pages on a variety of displays.

You may find that your beautiful design, which is wonderfully contrasted on an LCD monitor, is impossible to read on a CRT monitor, and completely unusable on alternative devices. Different displays will not always interpret your site's design and colors consistently, so it's worth testing your initial design, as well as the site's ongoing development, in a variety of displays.

View pages on different screen resolutions and with various color settings.

Some HTML editors provide a feature that allows you to check your pages at various screen resolutions and with a range of color settings. Similar tools can also be downloaded as plug-ins for some browsers. For example, the highly recommended Web Developer Toolbar extension for Firefox includes a customizable window resizing tool, among many other useful features. (For more on Firefox's tools for web developers, see Chris Pederick's article, Web Developer Extensions for FireFox and Mozilla.)

Consider Nonstandard Resolutions
Screen display sizes may or may not reflect the actual dimensions of the viewable web page. Many users don't expand browser windows to fill the entire screen (especially on larger monitors), and various toolbars and side panels can take up portions of the screen width and height. For example, on an 800x600 display, a browser that has a side panel and a couple of custom toolbars across its top would display only 500x400 pixels of a web page—less if the browser window isn't maximized. Be sure to conduct testing that accounts for this wide range of variability—not just the standard maximum screen resolutions. Table 14.1, which shows screen resolutions of various devices, and Table 14.2, which shows the color depth settings of different displays, should help your testing.

Table 14.1. Typical screen resolutions and displays (in pixels)

[a] Lisa Gade, palmOne Treo 650 Palm OS Smartphone, Mobile Tech Review (December 10, 2004).

Table 14.2. Typical display color depth settings

[a] Browser Statistics: What Is the Trend in Browser Usage, Operating Systems and Screen Resolution? W3Schools (January 2005).
[b] What is a Pocket PC (PPC)? What models are out there? Mobile Tech Review (no date).

In addition to the displays mentioned in Table 14.1 and Table 14.2, there are standalone tools—such as BrowserCam, an online subscription service—which provide screenshots depicting the way your content will appear in a wide range of screen resolutions and browsers.

Check for adequate color contrast.

A good way to check color contrast is to change your browser display to grayscale, or print the web page in black and white. A tool that can be very helpful for checking your design's color contrasts is Vischeck, which simulates colorblind vision, allowing you to see your pages as a colorblind user might.

Test the functionality of external and embedded scripts and functions.

Some scripts and functions might work fine in one browser, though they may not work correctly, if at all, in other browsers or devices. Minor tweaking can resolve many cross-browser, cross-platform functionality issues. If older browsers or alternative and adaptive technologies don't support some functions, you may wish to provide alternative methods to allow those users to access those capabilities some other way.

Test all links, including navigation.

Ensure that all links work properly, and point to the correct location. Link checking software can check for valid, broken, and redirected links. Some HTML editors, such as Macromedia Dreamweaver and HomeSite, offer link checking capabilities. Free tools are also available—try Xenu's LinkSleuth and W3C Link Checker. See WebsiteTips.com's section, "Link Checkers, Maintenance" for more. Note, though, that link checkers aren't intuitive enough to check whether links direct to the correct location—be sure to check for this manually.

Check error pages.

Intentionally enter incorrect URLs into the browser's address bar to check that the appropriate error pages (404 errors, etc.) are in place.

Make sure that the error pages include helpful information and links that help users find what they seek. Keep in mind that, sometimes, visitors will incorrectly guess a domain name or slightly misspell a URL. Help those visitors by accepting common misspellings, typos, incorrect case sensitivity, different terminology for the same items, and predictable domain name errors: provide suggestions to redirect users to the proper page. (Matthew Linderman with Jason Fried, Defensive Design for the Web: How to Improve Error Messages, Help, Forms, and Other Crisis Points (Indianapolis: New Riders, 2004), 99, 224.)

Test all downloads.

Check to ensure that all your download links point to the correct files, and that the download files exist.

Test the search feature

Ensure that the search feature functions correctly, and that search results are accurate and helpful.

Solid Security

Clear your Cache!
Be sure to clear the browser cache, including cookies, before each test.

Check that digital certificates and SSL URLs work correctly.

Your SSL URLs should begin with https://. Also, when a visitor accesses a web page protected by SSL, most browsers will show a "lock" icon at the bottom of the browser, like that being shown by IE 6 for Windows in Figure 14.1.

Figure 14.1. The "lock" icon displaying at the bottom of the browser window

SSL is commonly used to encrypt online transactions and other sensitive data, as well as intranets and extranets. Verisign and Thawte are two popular companies that issue SSL certificates.

Check that all pages requiring SSL access are accessible only via SSL.

Test the security of restricted areas.

Users might share or try to guess the URLs of content and downloads in protected areas of your site. Restricted content URLs might appear within publicly available referrer logs. People might even share usernames and passwords without authorization or permission. As such, it's critical that you anticipate the kinds of security breaches that might take place, and test methods for their prevention.

Hack your Site: the Ultimate Security Test
Download popular hacking tools to use in testing, to see if your protected areas can easily be comprised. Those managing larger-scale web sites might even consider outsourcing or hiring a hacking expert for testing purposes.

Ensuring security and data integrity—especially in terms of your confidential data, including customer credit card information—is critical for promoting and maintaining trust among your site's visitors. Don't assume that your web site's security is always okay. Test regularly to make sure it remains secure.

Test forms and form controls.

Check to ensure that forms are submitted correctly, and that they're only submitted when the correct information is entered and required fields have been completed. Review form error messages to ensure that they are helpful and informative within the context of the form itself.

Test online shopping facilities.

If your web site includes a shopping cart or similar functionality, thoroughly test back-end operations to ensure that all transactions are secure, and everything runs smoothly.