By Jeffrey Bergin, Ph.D.
The Power of Surprises
Years ago, as part of an R&D effort to design a learning experience for the first generation of mobile-enabled learners, I entered the project with a number of hypotheses, assumptions, and biases about everything from the nature of mobile learning to what kinds of things learners would want to do on their mobile devices. Then, we began engaging with learners: we interviewed a few users, commissioned a study, and began testing designs. With each engagement, we found ourselves asking: “Are you sure?” We were, in short, surprised—surprised by the way students thought about their devices; by what they wanted in a mobile learning experience; by how they wanted to read, quiz themselves, listen to lectures, and communicate. These surprises fascinated us and fundamentally changed not only how we designed the experience, but also how we thought about mobile learning.
Surprises come in many forms in the design process—as unexpected insights, “aha” moments, and seemingly anomalous findings. They are new ways of thinking about the people with whom we interact—the people we call “users,” “customers,” or “segments.” Surprises help us identify problems, ideate solutions, and optimize experiences. Surprises facilitate setting aside our preconceptions to look at things from a fresh, often more accurate, perspective. Surprises enable us to rethink what we as researchers, experience designers, and technology users believe about each other, about an experience, and sometimes about the wider world. Most importantly, though, surprises change our minds—sometimes in profound ways.
Yet, surprises are far from guaranteed. Poor human-centered design (HCD) practices may not create an environment that reveals surprises; instead, we may design research that confirms our own biases, validates our own hypotheses, or privileges our own opinions. We may ask leading questions, create invalid research protocols, or use body language that influences research participants. Or we may attempt objectivity but overlook opportunities to unearth those surprises that could yield new ideas and innovations. Indeed, HCD work has been criticized for not adequately addressing or reducing cognitive biases (see Cordes, 2001; Liedtka, 2015).
Surfacing surprises and eliciting the rich insights that they provide requires skilled research, thoughtful methodologies, and focused facilitation. We need to actively cultivate HCD methodologies through ethnographic research, participatory design, and early formative evaluation. Most importantly, HCD requires an inquisitive, investigative mindset—a true desire to go beyond what we currently know, what we may have been trained to believe—that embraces surprises.
In the following section, I outline six types of surprises that good human-centered design processes can surface along with the HCD methods that may yield these surprises. I conclude by providing a few guidelines for catalyzing surprises in the design process and exploiting the discoveries for a delightful user experience.
Triggering Six Common Types of Surprises During HCD Processes
1. Surprises that Reveal Novel Behaviors
Some surprises, like the ones I experienced related to mobile learning, reveal how people go about a particular task or use technologies in ways that are novel or unconventional. For example, one novel way to use the selfie feature on a phone camera is as a mirror to see how one looks. These novel behaviors may signal problems or opportunities, or reveal unintended use cases in product, information, or experience design. One of the best ways to elicit these kinds of surprises is simply through true ethnography: unobtrusive observation. The more we observe the people who use our experiences, the more likely we are to find things that surprise us and create rich opportunities to learn more.
2. Surprises that Surface Biases
Other surprises reveal biases, often unconscious or implicit, that people hold. These biases may be against people, contexts, technologies, and behaviors. Not understanding biases can be a major impediment to the design process. For example, I’ve led design work related to concepts that are widely familiar to many laypersons, but often misunderstood. In one instance, as biases began to surface, we were surprised by both how varied and entrenched they were—and we realized we needed to tackle these biases head-on in our participatory design sessions to avoid being encumbered by them. We also realized our process would have been faster had we surfaced these biases sooner. To do this, it can be helpful to ask participants to identify and attempt to suspend their own biases while continuing with other generative or participatory activities. Sometimes it’s not possible to spend time trying to identify biases—indeed, it may be counterproductive to do so because it may inadvertently reinforce them. In these cases, one way to avoid biases is to engage in an activity (such as a think-aloud or usability test) prior to discussing it, to solicit feedback on the experience itself rather than the broader concept (which may have many preconceived notions attached to it).
3. Surprises that Reveal Gaps
Sometimes surprises reveal gaps—gaps in technologies, where innovative new solutions can help; gaps in an experience, where navigation or features can help; or gaps in the research team’s understanding of user types, beliefs, and behaviors, where additional clarity can help. Researchers can elicit these gaps in a number of ways—journey mapping, rapid ideation, and gallery walks, for example—so long as they are always encouraging participants to uncover what is missing. Researchers might also ask participants what they do not do, and why, to identify rich opportunities for problem identification and solutions. For example, when we were researching mobile learning, we might have asked, “What types of learning activities might you never do on your phone and why?” to understand the perceived limitations of mobile learning and identify gaps to be addressed.
4. Surprises that Reveal Affective States
Occasionally surprises reveal affective states that were previously unknown, underappreciated, or unacknowledged. Researchers may be surprised to find that people associate aspects of a user experience with extreme negative feelings, or they may associate certain points in time (such as the holidays, early mornings, or Mondays) with extreme feelings. I was surprised to learn, for example, how students feel at various points throughout the academic term and the complex emotions experienced before and after midterms—emotions that helped us to consider how mobile learning might help during tough times. These surprises can create opportunities to improve experiences based in part on emotional responses. Detailed journey-mapping is one way to get at this, especially if researchers dig deep into the emotional states at each step in the journey. Researchers may also begin routinely asking subjects to include simple emotional indicators—using emoticons, for example—during other participatory design activities such as think-aloud activities, usability testing, or ideation sessions.
5. Surprises in How People Relate to One Another
Sometimes surprises reveal how people perceive their relationships and interactions. For example, in complex systems that represent multiple user types (for instance, buyers, sellers, and brokers; instructors, students, parents, and administrators; doctors, nurses, and patients; and broad circles of social media contacts) with privileges, roles, and interaction patterns, it can be helpful to understand how each of these types, or personas, regard one another and how this may need to be reflected, managed, or mediated in a system. For example, when working with instructors and sub-populations of students (such as traditional and returning students), I have found interesting divides in how they perceive one another and activities that require interaction. Sometimes these relationship issues surface during participatory design activities; other times they surface only after substantial research. Regardless, one way to uncover these dynamics is to ask participants to create illustrations that depict how one or more groups are related. When coded and aggregated, these can tell us many things about interpersonal relationships that can improve designs and potentially also improve perceptions and relationships.
6. Surprises that Surface Emerging Trends
Occasionally, surprises point toward an emerging trend—a sustained change in a cohort’s behavior over time. These are often spotted through longitudinal, quantitative, or analytics research focused on larger sample sizes over time. Some surprising trends, however, can be spotted early using qualitative methods. Years ago, I found that many students used their mobile devices to skim or review content rather than to conduct deeper, more sustained reading, as they might with a print or e-book. This may seem obvious today, but it was a trend when mobile devices were emerging. Some of the best ways to spot trends are to conduct observational research, contextual inquiries, and exploratory data mining; however, these methods benefit from progressively larger sample sizes.
The Perils of Surprises
While surprises can reveal important (and sometimes hidden) realities, they also come with a degree of peril. Sometimes a surprise is simply an anomaly—an outlier to the norm that can be a distraction. Sometimes they represent an uncommon behavior or use case, one that few other users would want to replicate. And sometimes they may actually be detrimental—for example, a surprise that encourages a substandard design or implementation at odds with the goal of the experience. However, more often than not, surprises enable us to advance our understanding in meaningful ways and, ultimately, accommodate our users through a more holistic and accurate understanding of them.
Making the Most of Surprises
Surprises are important to elicit, interrogate, and understand; however it’s often unclear how to “use” them to best effect. Do they represent an anomaly or outlier? What if they are at odds with our own long-held beliefs or run antithetical to other research and evaluation findings? Or what if they fundamentally change an experience, product, or feature in ways that are challenging to accommodate? When you encounter surprises, I recommend the following:
- Ask other researchers, observers, or participants to share their impressions—do your interpretations match? You may even ask them to develop alternate interpretations to document, consider, and interrogate.
- Consider whether the surprise opens up new paths for further research and what the best methodology might be for learning more. Often, surprises are the beginning of new studies wherein findings are validated or invalidated.
- Socialize the surprise (either before or after you have further researched it) with other stakeholders. Surprises can help creative teams think about things in new ways and challenge fixed beliefs.
- Apply the surprise to the design, optimization, or implementation of your experience if, and only if, it seems like an important way to expand your user base or to enhance usability, engagement, and overall delight.
Surprises that emerge from participatory design research are, at a minimum, thought-provoking and, at maximum, groundbreaking. They help us to check our own assumptions, dislodge our biases, and more fully understand both our users and the experiences we design for them. However, it’s not always clear how to elicit surprises, if they are to be trusted, or how to take them forward in meaningful ways. Being thoughtful about surprises is the first step—and it can make a world of difference for researchers, stakeholders, and users alike.
Jeffrey Bergin is Vice President of Learning Research & Design at Macmillan Learning.
Richard E. Cordes (2001) “Task-Selection Bias: A Case for User-Defined Tasks,” International Journal of Human–Computer Interaction. 13:4, 411-419.
Liedtka, Jeanne (2014) “Linking Design Thinking with Innovation Outcomes through Cognitive Bias Reduction,” The Journal of Product Innovation Management. 32:6. 925-938.