By: Gerry Gaffney

December 10th, 2004

Reader Rating: 9

Designers who don't understand their users frequently develop products that are difficult to use and understand, do not meet real-world requirements, or provide irrelevant functionality.

The best way to get to know users is to spend time with them, in their own environments, watching them do the things that your Website is going to support or enable.

Of course, you can just go out and visit a few users informally, and that in itself will provide valuable information. To get the most out of such visits, however, you need to take a more formal approach. Contextual enquiry provides a framework for doing so. It consists of the following steps:

1. Identify appropriate users


2. Schedule visits


3. Conduct site visits and gather information


4. Analyze what you have found so that you can use it effectively during design

In this guide, we'll look at each of these steps in turn. We'll talk about how you can implement each, and what you'll get out of it. Finally, we'll see how the outcomes of a contextual enquiry can influence the Web development process.

Contextual enquiry is one of the most valuable activities you can undertake, because it helps you understand users and their requirements with relatively little effort. What happens if you don't understand your users? Let's talk about that now.

'See no Evil' -- Designing in the Dark

The world of users is messy. Once we start to ask questions like, 'How do our customers normally carry out this transaction?' or, 'Why don't our customers use the online order entry form?' we enter a murky world with multiple answers that are not easy to obtain.

Add to this the perceived difficulty of gaining access to site users, and it becomes tempting to design without having an understanding of the site's audience.

Often, developers simply extrapolate from their own experience. Given the huge gap in technical expertise between developers and users, and the lack of domain knowledge on the part of developers, the resulting product is likely to be less than ideal. This is a pity, because getting good quality, useful information about users is not terribly difficult.

Designing in the dark rarely (if ever) allows developers to meet the needs of users of a given site or application. To do that, you have to get to know your users.

Getting to Know You

To design effectively, we need to know about:

• our users (what sort of people are they?)
• their tasks (what exactly will thy do with our Website?)
• their values, concerns and issues

This information isn't just academic -– it feeds directly into the design process.

For example, some years ago I was involved in a project to redesign a Web-based application that managed incoming telephone calls (the sorts of system typically used by ticketing agents and large service departments). According to management, the users of the existing system were technically sophisticated, tertiary-educated, and used the system frequently -– probably daily. Based on this understanding, the new application would provide sophisticated tools to allow users access call routing tables, time-of-day and day-of-week settings, and so on.

When we got to spend time with a few users, however, I found that they were not tertiary-educated or technically sophisticated. They were primarily in clerical and secretarial roles. We also discovered that most of them could not actually use the existing application -– instead, they called technical support whenever they needed to make any but the most basic changes.

Finding information like this -– at the right time -– can save you enormous amounts of wasted effort. Let's take a moment to delve more deeply into each of these areas of information.

Who Are Your Users?

The information you need varies from project to project, but typically it's useful to know things like the age, gender, computer and Internet experience, education, occupation and language of you target users. An educational Website for 15-year-old schoolgirls will be very different to a travel booking service for retirees.

Some of the information may already be available -– marketing departments are often a good source. However, make sure you treat all information with a degree of skepticism unless you can verify its accuracy.

You also need to know about the kinds of technology that people will use to access your site -– platform, browser versions, screen resolution and Internet connection. Log files may be able to provide some of these details; population-wide statistics are also available from a variety of sources.

What Are Their Tasks?

It's important to know what users do, or will do, with your Website -- not in a theoretical fashion, but in real life.

We often have an ideal model of the kinds of tasks we're trying to support, and in the absence of other data, we design to meet that ideal. This is risky because we may be unaware of important aspects of the task.

This is particularly true in a corporate environment (for example, when designing an intranet application). In such cases, there will frequently be a significant gap between the official description of a task and the way it's actually conducted by users.

One client told us that their customers logged on to the extranet to get production reports. When we visited customers, they told us that the reports weren't up-to-date, so they had developed individual contacts within the organization who would fax them reports on demand, completely bypassing one of the major reasons why the extranet was developed in the first place.

The best way to get a grip on these kinds of user objectives (and hurdles) is to spend time watching users carry them out -- a task that lies at the heart of contextual enquiry.

For an intranet project, this will mean spending time with users in relevant departments. For an Internet site, you may need to visit call centers, stores, sales departments, and other locations from which people conduct the tasks you need to support through your site or application.

What Are Their Values, Concerns and Issues?

If you know what your users value, you'll be able to cater to those values in your design. If you know users' concerns, you can address them. Some of the values and concerns are generic (e.g. privacy, security, speed), but others will be specific to your project. For example, we found that one client's customers particularly valued information at certain times of the month; by setting up an email list, the client was able to meet that need simply and efficiently.

Issues provide opportunities if you can address them in your design, but they become pitfalls if you're unaware of them, and fail to allow for their effects on users.

Once you've identified what it is you need to know about your users, it's time to work out how to get that information.

Strategy

Understanding that you need to get to know your users is one thing. Actually doing something about it requires some strategic thinking. Taking a strategic approach to usability research (of any sort -- not just contextual enquiry) will help you minimize effort and maximize effectiveness.

Build User Research into your Project

User research should be a part of every Web project. And it should occur early in the project. Why? Because getting information early allows you to shape the project to meet the real-world needs of users. Good data will help you focus the design effort -- you'll be better equipped to address actual problems, deliver real value, and build genuinely useful functionality. And early information is most valuable when the design is fluid and new options can be explored.

Uninformed decisions will lead to an inefficient development process, and increase the likelihood that you'll need to complete unnecessarily extensive post-implementation rework. A sound strategy is essential to ensure that user research -- and the project itself – can be completed successfully.

Planning the Enquiry

The next step in a contextual enquiry involves planning the actual research effort.

The logistics of even a relatively small-scale contextual enquiry can be daunting, as can any activity that involves interacting with users.

Good planning is vital! Let's take a look at each of the steps in the planning phase.

1. Know your Limits
   How much time and money you have will dictate to a large extent the nature of your contextual enquiry. As a general rule, you can expect a contextual enquiry to take 3 weeks of elapsed time:
   • one week planning and preparing
   • one week conducting site visits
   • one week of analysis

   It's possible to conduct contextual enquiry in less time, but you will probably be cutting corners. Try to spend any time you can with users -- even if it's only an informal, half-hour visit to one individual -- but understand that a full contextual enquiry involves more time than that.

   In terms of budget, contextual enquiry is labour-intensive, but requires little additional expenditure unless you have to travel extensively.

   Once you've identified how much time you can devote to the contextual enquiry, it's time to think about who you'll visit.

2. Identify Appropriate Users

   To get the most out of contextual enquiry, you need to make sure you visit the right people.

   The most important people are those who will actually use your application, or are currently completing the tasks that your application will support or replace. This means people on the factory floor, or answering the phones, or dealing face-to-face with customers. Try to avoid supervisors and management -- except for background and policy information -- as they will generally have an idealized view of the work being conducted.

   For consumer-based Websites, you may need to visit users in their homes, and watch them carry out the activities you need to support in your design.

   Avoid non-representative users. For example, beta-testers tend to be dedicated customers who are very different from the majority of users. Similarly, subject-matter experts may be unrepresentative.

   Try to visit at least 5 or 6 individuals within each major user group. If you have a single user group (for example, call centre users of an intranet application), then 5 or 6 people in total may be sufficient. If you have more than one user group, and the groups are sufficiently different, then you will need to increase the number of visits, and perhaps compromise by visiting fewer users from each group.

   Even two or three users will give you valuable data. You are unlikely to be able to visit enough users to get 'statistically significant' data, but that does not negate the value of doing a small number of visits.

3. Scheduling

   It's often tempting to try to visit as many people as possible in a given timeframe, but remember: you'll be gathering a lot of information on each visit. Site visits require intensive concentration and can be very tiring.

   While the duration and format of visits will of course change from project to project, a good rule of thumb is to allow two hours for any visit, and to do two visits per day -- three if timing is really tight. This means that in a 5-day working week you would expect to see 10 or 12 people.

   You can conduct site visits on your own, although it is very common to use two people, with one focused on note-taking. Having two people also makes data analysis easier.

   If you can assign an assistant to look after the scheduling, great -- it tends to be a time-consuming task in itself. It's often easiest to use a standard message to communicate with the people you intend to visit and their managers (if relevant), which explains the purpose of your visit and offers available time-slots. If you're booking visits by phone, have a script that you can read from. Otherwise, use a standard email.

   People will tend to try to get you to visit on quiet days. This may be appropriate sometimes, but if your application will be used on busy days, try to schedule the visits at those times. You may need to compromise by agreeing not to interrupt or ask questions, or by having a shorter visit. As a common courtesy (and to avoid wasted time!), re-confirm appointments the day before each visit.

   As you conduct site visits to workplaces, you may find that a manager or other supervisor wants to brief you, or that the person you want to meet has booked a meeting room. In such cases, explain politely that you want to observe the actual work in progress.

4. Workplace and Legal Issues

   As you conduct the meetings, you may have to work around a range of location-related issues, depending on the locations you're visiting. Here are some of the major considerations.

   If you're visiting a workplace, make sure you have approval from management. It may also be necessary get union approval. If you're visiting a hazardous area, you may need to attend an induction session, and to wear protective clothing. In any case, check with the people you want to visit prior to arriving on site.

   If you'll be listening in on telephone calls (for example, in a call centre) you will probably be legally obliged to let callers know that their calls are being monitored. Many companies already cater for this in their IVR scripts, by announcing that calls may be monitored for quality and training purposes.

   If you are visiting minors, you will need to get permission from parents or guardians. For example, when we did some work for a primary school application, we had to ensure that we had written permission to conduct the analysis, and that a teacher was present at all times.

   Don't be deterred by these requirements -- they're not difficult to meet. It's really just a matter of being organized and methodical.

Preparing for Site Visits

Before you race out to meet site users, you must prepare! Think about the materials you'll require, the people who will conduct the visits, and plan to brief yourself and others.

Materials

The minimum requirements are a pen and some paper, but consider which of the following you might also need:

• A script. This is a short, formal statement that you read to participants when you first meet them. It explains the purpose of the visit and provides assurance about the anonymity of the information you'll collect (if you are, in fact, guaranteeing this).
• A demographic questionnaire. This gives you the opportunity to begin building a picture of the people who will use the product, but it the data you obtain will also be useful in you reports back to the project team about the people who will use the product. Demographic information will also help if you're creating personas to represent users during the design phase. The demographic questionnaire is often filled in at the start of the session. Don't ask for personal information that you don't need.
• Consent form. This document outlines the information you'll gather and how it will be used. The person you are visiting signs this to grant you permission to conduct the visit (and to record, if you're doing so). If a subject is reluctant to allow recording, conduct the visit without the recording devices.
• An 'Additional Information' questionnaire. At the end of the site visit, you may want to ask for specific feedback about current applications the person uses, or issues that you may not have the opportunity to observe during the visit.
• Payment. You may need to pay participants, in which case you'll need them to sign a receipt. In some case, such as workplace visits to employees within an organization, it may be appropriate to provide a 'thank-you' gift such as movie tokens, rather than money.
• Video and audio equipment. Some practitioners advocate the use of video, but I avoid it as it involves additional set-up and management, consideration of additional legal and ethical issues, and may make participants ill-at-ease.

Personnel

Conducting site visits doesn't require specialized skills, but the person who conducts the visits should be a good communicator. The interviewer should be capable of putting participants at ease, and be observant and attentive to detail.

It's a good idea to work in pairs for several reasons, which we'll discuss in this section. This approach also provides the opportunity to train a junior staff member, or to have different developers get a taste for "user land".

Conducting the Visits

Each site visit is unique. By structuring all visits in the same way, you can make sure that you get the most out of your contextual enquiry.

Put Users at Ease

Having people watch you work can be an unsettling experience, and participants are likely to feel uncomfortable. Reassure them that you're not there to evaluate their performance, but to learn about their workplace, tasks and issues, and to improve the products they use. Be relaxed and friendly, and honest about what you're trying to achieve.

In my experience, people have generally been very happy to provide information that can help improve their workplace, or the products and tools that they use.

The 'Master-Apprentice' Model

Sometimes it's tempting to show people better ways of doing their work, or using their applications. However, this is not the purpose of a site visit; you are there to learn, not teach.

Think of yourself as an apprentice who is visiting a master to learn about what they do. This model (suggested by Beyer and Holtzblatt) is very effective, and is an accurate representation of the actual relationship between you and the participant.

Ask Questions

To learn about the tasks and issues the user faces, you will need to ask questions. Don't be afraid of appearing foolish or asking 'stupid' questions. Avoid any suggestion of criticism in your questions. For example, rather than "Why didn't you know that would happen?" ask, "Is that what you expected to happen?"

Take Notes and Recordings

After you've completed a few sessions, it can be hard to remember what happened at each one. Good note-taking is vital. If you're recording your visits, consider the recording as a no more than a backup for emergency purposes, because reviewing a recording will add significantly to the amount of time you need to spend in the analysis phase.

If you take hand-written notes, it is important to get them into electronic format as quickly as possible after the visits. If you can afford it, and if your hand-writing is reasonably good, consider paying someone to transcribe your written notes.

It may be tempting to take a notepad computer and make electronic notes. However, this makes it difficult to establish and maintain a comfortable relationship with the people you are visiting. Taking notes on paper is the way to go.

What to Watch Out For

You need to be alert for anything that can help you design a better product. In particular, take note of the following:

• Environment and facilities. Is it busy? What sort of computers do users have? What level of Internet access? Do they use headsets or standard telephones? Do they have printers, fax machines and photocopiers? Do they have to share these? Do they have to walk to these machines? It's often helpful to sketch a rough map of the work area, so that you don't have to try to remember these details.
• Interruptions. Can people concentrate on their work, or are the frequently interrupted by other applications, people or events? What is the impact of these interruptions on their workflow? Do current systems support these interruptions?
• Applications. What applications, systems and Websites are currently being used? Exactly which screens and fields are used, and in what sequence? Get screen dumps if possible.
• Issues. What problems do users currently have? You should be interested not only in application or system issues, but also workplace issues -- particularly if it is possible to address them in your design.
• Artifacts. Do users have 'cheat sheets' and notes on their computers? Do they have tables or charts or phone lists that they rely on? It may be possible to include some of this information within your application. Get copies of any artifacts, if possible.
• Double-handling. For example, are they entering account numbers in two different systems?
• Workarounds. Are users compensating for poor product design or poor work design by applying workarounds? Will these have an impact on your design or can you accommodate or eliminate them?
• Triggers. What is the event that triggers users to take a particular course of action or begin a particular task? Triggers are often events like phone calls, arrival of emails for faxes, or work queues.
• Hand-over points. When do people receive or hand over tasks?
• Variations. Remember to ask users about whether the work you see is typical and, if not, why not. Ask about what it's like when things get particularly busy (or particularly quiet).

The Day After...

After site visits, you will have an enormous amount of information, and analyzing it can be a challenge.

Contextual analysis has four key outcomes, all of which aid the design and development of the project in various ways. Let's look at each in turn.

Task Analysis

One of the first things to do is to identify the tasks that were carried out. Usually, there will be a relatively small set of tasks, but there may be multiple variations.

For example, imagine you're building a Web application to allow people to book DVDs online. If you spent some time in a video store, you might discover a small set of tasks, such as 'find a movie', 'hire a movie', 'return a movie' and so on. However, 'find a movie' might have several variants: some people would just browse new releases; others might look for a movie by name, or by the name of a director or actor, and so on.

It can be a good idea to draw flowcharts to represent the tasks. The flowcharts should identify triggers (what starts the task?), systems usage, interactions with people or departments, what marks the conclusion of the task, and discontinuities. Although there may be many variations to the flowchart, it can be very useful to have a consolidated view.

A clear, systematic task analysis will help you develop effective information architecture and aid user interface design.

User Needs Analysis

Make a list of the things that users explicitly requested, or that you noticed would have helped them.

Consider whether these needs can be met by your application or Website.

Consider also whether there are proposed elements of a new design that will not be valued by users; perhaps effort can be shifted from them and into the areas that are perceived to be more important.

Through one contextual enquiry, we found that people were reliant on an old application that was expensive to maintain, but they only used it to retrieve one piece of information that was not available anywhere else; by incorporating that information into the new design, it was possible to retire the old application.

Personas

Based on the people you meet on site visits, and on any other demographic or segmentation information available, you may be able to produce personas -- fictitious but representative user profiles -- which can be used as decision aids by design teams.

Reporting

The information you gather during contextual enquiry is often extremely valuable not only to the design team but to other parts of the client organization (for example, marketing).

Take the time to write a report that conveys your findings, or commission a technical writer to write the report.

No Time or Money?

If you really can't get an appropriate level of commitment and resources to carry out a contextual enquiry, consider starting small.

Visit one or two users informally, and take any opportunity you can to visit more. Keep notes, and use these to help make informed design decisions.

Recommended Reading

• Contextual Design: A Customer-Centered Approach to Systems Designs [1] by Hugh Beyer, Karen Holtzblatt (Paperback)
• Observing the User Experience: A Practitioner's Guide to User Research [2] (Morgan Kaufmann Series in Interactive Technologies) by Mike Kuniavsky

[1] http://www.amazon.com/exec/obidos/tg/detail/-/1558604111/qid=1102490941/
[2] http://www.amazon.com/exec/obidos/ASIN/1558609237/qid%3D1102490989