Professional Website Usability

Part 5: Interpreting Results and Producing a Report

Quite a lot of work has taken place to bring you to this point in the website usability testing process. You've produced a strategy and plan, evaluation criteria and an evaluation tool, and conducted your website usability study. The final step is to package all the feedback and scoring together into a meaningful report that can be used to change and improve the website. How you package this information and the level of formality used depends on whether you are the sponsor yourself (it's your website) or whether you were the facilitator who conducted the study for a business sponsor (it's their website).

Of course, if you're the sponsor, the report can be more informal, but you'll still want it to be complete and to-the-point for your own records and for your management. For the business sponsor and their management, though, who committed human and budgetary resources, you'll want to produce a more formal report. Regardless of how you package the report, the information should be easily understood, free of technical jargon and provide a complete picture of how the study was conducted, what users experienced and what conclusions have been formed from the results.

Report Categories

Regardless of whether you were the sponsor or if you facilitated the study for someone else, you could describe results in these high-level categories:

• Overview: State the goals of the study and what you hoped to achieve.
• Background: Describe when, where and how the study was conducted, how participants were selected, how many people participated in the study and how they were incented or rewarded; describe the timeframes for the study and how they were or were not met and why.
• Study Plan: Provide a synopsis of the evaluation criteria, what you tested against and why you tested those particular features.
• Study Results: Break down the five categories according to the evaluation tool you used. For example, in this series of articles, I recommended 1) a General Survey, 2) Treasure Hunt, 3) Anticipation/Intuition, 4) Terms and Language and 5) Look and Feel. These would be the headings for the results report.
• Budget Expenditures: Describe the costs associated with administering the study and provide an analysis of the forecasted and actual costs; if you were over or under budget, describe why. Include costs for communications/mailings, transportation, lodging, facilities, equipment rentals, food, incentives, premiums, etc.
• Conclusion: Reconsidering the goals you or your sponsor set out to achieve; state how you did or did not achieve them here. Communicate how this study benefited your project, product, process, organization, industry, customer, corporation, etc. and if you felt that the time and energy spent was worth it. Note that conclusions have been formed as they relate to the evaluation criteria and described in further detail in this report.

Interpret and Summarize Your Findings

For the purposes of this part of the article, we're going to focus specifically on the study results and how to interpret and package them for the final report. I am not a statistician, so these are simply ideas and ways that I've compiled my reports in the past; if you have access to a statistician, certainly see if they have a more proven methodology for tabulating these types of results-if there's an easier way to sift through all the raw information to uncover the proverbial 'golden nuggets,' I'd love to hear about it.

You've collected raw data and now you need to summarize the findings, form conclusions and recommendations and offer any other feedback that's been provided to you. Most of your information will come from the facts that have been provided by the participants and their experience. It's important to use the facts that have been provided by your participants and not read too much into the information -- interpret what's comfortable for you, collaborate with others on your team if something proves difficult to understand or interpret, and try not to embellish, over-simplify or exaggerate. Be honest in reporting what the data is telling you. If your study produced significant issues, remember that's what you came here for. You did this study to improve the quality of the website, and whatever the findings are, you tested your users and achieved your goal. Your management bought into the study and agreed that it was a worthwhile activity to undertake; now, if the results reveal that there are problems-that's what you were looking for and what you can now begin to digest.