Professional Website Usability

Part 2: Planning A Study

So, you've learned from earlier mistakes (or read Part 1 and avoided mistakes!) and decided that adding a step up-front in your web development process is a wise and prudent way to spend your resources, secure the confidence of your users and keep your brand image strong. The next step is to actually sit down with the Web Usability Strategy you completed in Part 1 of this series and decide how you want to administer the study.

I can't stress enough that planning is the key to a successful study. With enough time to understand your approach, your surroundings, your users, and designing a quality evaluation tool, you'll have all the necessary ingredients you need.

Decision-Making Time

Along with the information gathered in the strategy, the approach involves making decisions early about the following:

• Where you'll conduct the study. You may have a training facility you can use -- a conference room where you can hook up PCs or be at an off-site location or have the resources to utilize the services of a usability lab. Early in the planning process, you'll want to decide where you will conduct the study and secure that space. Many studies fail because the facilities weren't secured.
• How you'll conduct the study. Testing on the Web may mean that you'll need flash cards that users will sort for logical order or navigation or it may mean having computers with browsers or modems connected. Don't underestimate how you will administer the study! The level of effort involved and the logistics involved in administering the study depend on what props and equipment you need.
• How you'll select your participants, seek their willingness to participate, communicate the logistics and what incentive they will receive. You could study the demographics of your users ahead of time and select users that meet the criteria for who you want to test (new versus experienced users, technology-savvy versus not, certain age ranges, etc.) Once you have their names or email addresses, you'd simply send them an invitation. The invitation could include what you're trying to achieve with the study (e.g., We've tried to anticipate your needs, now we need you to tell us if we've done our jobs well.), a description of the incentive and how they will receive the incentive (It's not unusual to offer a user $100 for a one-hour study or to have food at the session)

With these foundation pieces in place and key decisions made, you're ready to begin to develop your plan document.

Your Usability Plan

As with any plan, you'll want to bring all the pieces of your plan together in one place. I prefer to put my plans in table format simply because it's easier to read and forces me to be brief and to the point when I describe my activities. The headings for this plan could be:

• Sorted by phases e.g. planning phase, logistics phase, user selection and communication phase, evaluation tool development, study implementation, results interpretation, reporting
• Activities e.g. finalize location, send invitation using distribution list, confirm attendance, send payment for food or room, etc.
• Roles and responsibilities e.g., identifying who will carry out each activity
• Planned and actual dates e.g. the date by which the activity should be complete along with the actual date the activity was complete; if you're running behind, this will help you get back on track
• Expected results tied to each activity e.g. these are the deliverables that would be done in each phase such as completing the strategy, securing the location, completing the plan, completing the evaluation tool, finalizing the invitation, etc.
• Planned and actual costs e.g. identify where charges will be incurred to carry out the study which may include how much the location costs, how much travel or hotel costs, how much for food, how much to copy any materials, how much to rent or secure any cameras, recorders, computers, etc. Compare your planned cost to your actual so you can work to stay within your budget.
• Issues do arise, so add a column or separate area to keep track of issues, how they will be resolved, by whom, date opened, date closed, etc. Not all issues are show-stoppers, but you want to be proactive and stay on top of them and their timely solution because one or two significant ones could bring your study to a halt.

Design the plan document in whatever way works best for you and in whatever software package you choose. This plan document will be shared with the members of your team as well as your management, so choose a software package that is eMailable and accessible. Now that you have secured management approval for your strategy document, you need to secure their approval for your plan document. This helps bring management along in the process and educates them on the benefits of performing the study, the returns and benefits that will be gained, and the level of effort it takes to conduct a Web usability study.