User Research

Preliminary User Research

My approach to user research was influenced initially by working with a team of ethnographers affiliated with IRIS, the research institute where I worked at Brown University. I was excited by the way their observational techniques lead to an in-depth understanding of problems faced by the groups and organizations they studied. At the same time, I was immensely frustrated by the lack of proscriptive advice that resulted from the research. My orientation has always been to solve problems. Understanding those problems is necessary, but not at all sufficient to coming up with solutions.

When I met members of the Apple Macintosh design team during several retreats I was invited to attend, I was introduced to their brainstorming, sketching, and rapid prototype techniques. This gave me an appreciation for how to use the rich observational data gathered from the ethnographic studies as input into the design process. The job of the designer is to invent solutions to problems, inspired by an understanding of users’ pain points, but not bound by the current technological constraints that often hampers people from coming up with effective solutions.

Years later, while working at Sun, the company embraced the Six Sigma quality system for a period of time. One of the major tenants of this approach is understanding the “voice of the customer.” As one of the few people in the company with any substantial user research experience, I was pulled into a number of corporate-level Six Sigma teams – one to study the effectiveness of distributed meetings and another to look at issues related to the company’s work-from-home program. While I found the Six Sigma process to be quite cumbersome, I did greatly appreciate the emphasis on identifying problems that had a serious impact on effectiveness. For example, in our distributed meeting study, one of the top problems faced by employees was their inability to find a conference room that was the optimal size for their meeting. The data analysis showed, however, that while this was a common problem, it did not have any impact on a meeting’s effectiveness. This emphasis on effectiveness is vital as it allows designers to focus their time and effort on the problems that matter most.

My philosophy is that given limited resources for user research, it makes the most sense to put the majority of effort into up-front user research that results in understanding and prioritizing users’ problems. If you don’t end up solving significant problems, then no matter how good a job you do on interaction design and implementation, your software is not going to be a success.

Usability Testing and Iterative Design

Assuming that the preliminary software design was based on a deep understanding of the problem space, the design concept should be sound. The next step is to refine the details of the interface and the interaction. I had the pleasure of working with Jacob Nielsen when I first joined Sun Microsystems. His “discount usability” methodology for testing software was enormously appealing to me, particularly the iterative design methodology Nielsen advocated. I never felt comfortable with large-scale usability tests, particularly when designing first-generation software. If there are major flaws in the software that you discover after testing the first few participants, it strikes me as a waste of time to continue testing dozens more people. It’s much better to fix the problems quickly and test a few more people.

All the engineering teams I have managed get the same message: be prepared to change your code. Over the years, I’ve have gotten much better at creating preliminary designs that are usable, but there are always many details that need to be tweaked in terms of layout, language, flow, and interaction. There is simply no substitute for trying out a design on real end users. I also insist that engineers watch the users, preferably in real time, or at least via video recording. I find engineers are quite motivated to fix problems if they observe people struggling with their software.