Introduction

In today’s competitive market, understanding user needs and providing exceptional experiences are critical for product success. Mastering the art of user research, usability testing, and discovery can save time and money and ensure your product’s success. In this comprehensive guide, we’ll explore the importance of user research, delve into the concepts of usability and discovery, learn how to conduct your own user studies and discover effective strategies for analyzing and applying results. Let’s embark on this journey to create products that truly resonate with your users and stand out from the competition.

Why is user research important?

User research helps us understand how people attempt to perform tasks and achieve goals that are important to them. It gives technology designers, product managers, and startup CEOs context and perspective to build simple, functional, intuitive products.

I’ve been helping teams build technology products for over 30 years – based on a better understanding of their users. In a series of posts, I will explore the “why” and the “how” of running do-it-yourself user studies.

Why is Speed-to-Market Not Always the Best Strategy?

There’s a constant theme that I’ve witnessed over the years. A startup or product team comes up with a goal – there’s a product that they want to build. They want to move as quickly as possible to build and take it to market. They are 100% convinced that this thing “will be awesome.” They know precisely what it is; they’re just going to charge ahead and move as quickly as possible toward that goal. “We’ll launch, then learn and figure out what to do next.”

This is a model that I have witnessed over and over from teams starting on this journey. Much like technology projects, in general, today, it represents an oversimplification of proven processes and a lack of understanding of the work that must be done.

When Speed Trumps Insight: The Cost of Delayed Learning

The problem with this approach – is that by the time you have launched your product and learned your first set of lessons, you could have drifted away from the “best possible” product that “could have been” or “should have been.”

If you don’t catch these things until after your first launch, there’s a risk that you spent a lot of time and a lot of energy moving as fast as you could – but towards the wrong thing.

How Does Early Learning Influence the Success of a Product Launch?

What if you could learn a bit sooner? What if, before you get down there, you can just check-in – in and ask yourself some tough questions? “Are we on the right track?”. “Is this what people want?”. What do people want? What can you learn? You may be able to push yourself back onto a better line – a more successful target.

Thus, if you learn sooner, you can save valuable time and money. It reduces the risk of the overall project. So that, by the time you launch, you can be more confident that you’re closer to the mark that you would otherwise have been.

What is Usability and Discovery in Product Development?

Usability testing is an essential part of product development that focuses on evaluating a product or design by testing it with real users. This process aims to ensure that users can effectively and efficiently interact with a product, ultimately leading to better user experiences and higher customer satisfaction.

There are two primary lines of questioning to approach usability testing:

The first is “usability.”

Fundamentally, what you’re trying to discover is, “can somebody do this?”.

You’ve given a user some product. Can they use it? Can they complete a task you want them to? For example, can they complete a sign-up flow? If they start, do they make it through? Do they understand what this product is for when they land on the marketing or product page? What does it do? Why would it be helpful to them? If there’s some feature, button, or link somewhere, are they finding it? Do they discover it?

The second part is “discovery.” These are more contextual questions. So these are the “who, what, where, when, why, how” questions. It provides a different kind of understanding.

It lets you understand what they need, their problems, and what they do. How would this idea, product, or design fit into how they do things right now?

It also helps you understand why they are having trouble. This provides you with some context to understand their past experiences. What are other similar things that they’ve done? What are their points of reference? “Oh, they’ve never used a marketplace before!” – that may explain why they’re struggling with your product.

So you want to know things like – what are:

  • What are they trying to accomplish?
  • What are their goals and needs?
  • What are the things that they struggle with in their life?
  • The problems? What are the pain points?

The following sections will delve deeper into usability and discovery, providing examples, case studies, and tips on effectively conducting usability tests and discovery sessions. By understanding and implementing these two approaches in your product development process, you’ll be better equipped to create products that genuinely resonate with users and meet their needs.

What is Usability in Product Development?

Definition and Importance

Usability measures how easy it is for users to interact with a product, complete tasks, and achieve their goals. It plays a critical role in ensuring users have a positive experience with your product, ultimately affecting user retention, customer satisfaction, and overall success. By focusing on usability, you can identify potential issues early on and make necessary improvements to enhance the user experience.

Key Questions to Ask in Usability Testing

When conducting usability testing, consider asking the following questions:

  1. Can users complete the tasks they are expected to perform?
  2. How long does it take for users to complete tasks?
  3. Are there any points of confusion or frustration during the process?
  4. Are users able to navigate through the product easily?
  5. Can users recover from errors or mistakes?

Examples and Case Studies

Consider a company that has developed a new app for booking local events. The usability testing process might involve users trying to find and book events, adding events to their calendars, and managing their bookings. By observing how users interact with the app, the company can identify areas that need improvement, such as unclear navigation, confusing booking processes, or difficulties in managing bookings.

Tips for Conducting Usability Tests

  1. Test with a diverse group of users that represent your target audience.
  2. Use realistic scenarios to give users context and make the testing process more authentic.
  3. Encourage users to think aloud during testing to gain insights into their thought processes and decision-making.
  4. Record the testing sessions for future analysis and to identify trends or recurring issues.
  5. Remain objective and avoid leading users towards specific actions or decisions.

What is Discovery in Product Development?

Definition and Importance

Discovery gathers insights and understands users’ needs, goals, and pain points. By focusing on discovery, you can better understand your target audience and ensure your product meets their expectations and requirements. This information is invaluable for informing the design and development process and helping you create products that meet user needs and provide value.

Key Questions to Ask in Discovery Sessions

In discovery sessions, consider asking questions such as:

  1. Who are our target users?
  2. What are their goals and needs?
  3. What are the main problems or pain points they face?
  4. How do they currently address these issues?
  5. How can our product improve or streamline its current processes?

Examples and Case Studies

An online education platform might conduct discovery sessions with potential users to understand their learning needs, preferences, and pain points. The platform might learn through these sessions that users desire more personalized learning experiences, struggle to stay motivated, and have difficulty finding relevant content. Armed with this information, the platform can develop solutions that cater to these needs, such as personalized learning paths, gamification features, and improved content discovery.

Tips for Conducting Discovery Sessions

  1. Use qualitative and quantitative research methods, such as interviews, surveys, and focus groups.
  2. Approach discovery with an open mind and be willing to challenge assumptions.
  3. Encourage users to share their thoughts, experiences, and feedback.
  4. Look for patterns and trends in the data to identify key insights and areas of opportunity.
  5. Document and organize findings to facilitate collaboration and decision-making.

How do you Combining Usability and Discovery?

Why is it important to combine Usability and Discovery?

It is essential to do these things together – they are like two great tastes.

So you should ideally combine the two approaches during an interview. You spend some time talking about the context. How do you do these things now? What do you think about these things now?

And then show them the product. With that initial context, you can better understand how and why they interpret and react to your product in a certain way. So it gives you these multiple insights, and as you start doing more and more interviews, you start building up this body of knowledge about your product.

What are the benefits of Combining Usability?

By integrating both usability and discovery into your product development process, you can:

  • Gain a more comprehensive understanding of user needs and expectations.
  • Identify potential usability issues early on and make informed design decisions.
  • Develop functional products, provide value, and address user pain points.
  • Increase user satisfaction and improve overall product success.
  • Continuously iterate and improve your product based on user feedback and insights.

How do you integrate Usability and Discovery when Conducting Interviews?

To effectively combine usability and discovery in interviews, consider the following strategies:

  1. Begin the interview by discussing the user’s recent experiences, needs, and pain points to set the context.
  2. Transition to usability testing by presenting your product and asking users to complete specific tasks or navigate the product.
  3. Encourage users to think aloud and share their thought processes during testing.
  4. Ask follow-up questions to understand users’ reactions, struggles, and suggestions for improvement.
  5. Conclude the interview by revisiting the initial context and exploring how the product fits into users’ lives and addresses their needs.

How do you do your own User Research?

The ability to do User Research is a very important core discipline for all product teams to acquire. “User testing” or “usability testing” is a key human-centered design practice. It points to a better way to do the “learning” part of building a product. It helps with course correction along the way. I will deal with basic usability testing, how to prepare and conduct user interviews, and how to do basic qualitative data gathering.

What is the goal of user research?

When you’re conducting a user interview – sitting down and talking to that person – your goal is to see and experience their world and your product through their eyes. You already know what you think, and your team already knows what they think – you want to collect new data.

You already know what you think. You want to experience your product through your user’s eyes.

People will do the strangest things. As you start conducting user interviews, you will see people report all kinds of wacky things that make no sense. They will find a completely roundabout way to do something compared to what you had anticipated. But to these people, sometimes it just “makes sense” for them to do things that way. You want to understand why.

“Why is that the way the user did this thing?”

What you want to do, is to understand. You want to observe their behaviors, which aren’t working well for them – their pain points. Understanding this is interesting because it helps clarify the “why” behind specific actions. When you’re using analytics to unpack behavior – you may see “what” people are doing, but you would not understand “why” they are doing those things. Why did this design work better than that? Why are people dropping off at a certain point? What is preventing people from getting through some flow? Once you understand the “why,” you’ll be better able to fix it.

When using data analytics – you may see “what” people are doing, but you would not understand “why” they are doing it.

Note, however, that you’re not asking users to provide answers. You want to observe their existing behaviors. You want to understand why they’re doing these things and determine their pain points. People are pretty bad at predicting their behaviors. They are not great at reporting or accurately remembering what they’ve done.

How to do your User Research

This section will cover the basics of how to do your user research studies. This is not intended to be an exhaustive guide on the topic but rather an introduction with a simple process, tips and tricks, and a couple of templates to get you started.

What do you want to learn from your user research?

As you embark on this journey, you want to start by thinking about – “what you want to learn” with this process. This will guide you toward the discovery and usability questions you must ask.  I’ve included the following two templates to help you unpack this more.

How do you find the participants for your user studies?

Once you have an idea of the kinds of questions you would like to ask, the next thing you want to think about is – how do you find people for your user studies. Who are the correct people, and how do you get them involved?

Finding people can take a fair amount of lead time, and can hold your process up, so this is why you want to start recruiting users as early in the process as possible – as soon as you know what you’re after.

What is a good sample size?

The next question is, “how many people do I want to include in the user study?”. Research has shown that the number of usability problems you find starts leveling off at around 5-6 people – after that, you get diminishing returns on your time and effort invested. You can even do smaller numbers if you want to iterate faster.

You may be surprised that this number isn’t larger, but the key is: this is not about statistical significance. If you recruit the correct people – who represent the actual users you have in mind- they will likely run into the same problems that other similar people will run into.

So, typically, you should start seeing some patterns emerge after two or three interviews. Once you reach four or five users, the issues should start jumping out at you.

Once you have learned enough, it is time to make some changes – because you will know the significant things you need to deal with.

How to create a screener questionnaire for your user studies

For this reason, it is worth investing some effort and being selective about who you want to recruit for your user studies. This is because the quality of the results you will get will be much better if you manage to recruit a representative sample. Otherwise, you will probably waste valuable time and get unhelpful or irrelevant feedback. Especially considering that you’re going to use a small sample size.

By creating a screener questionnaire, you can target a large potential audience, automate data collection, and then quickly and easily select the best, most representative sample of people to include in your study. The template below will guide you through developing your screener questionnaire.

Determining who you want to exclude from the sample is essential. Because the sample size is small, thus it’s easy to get skewed results, and your time is valuable – no need to waste it on collecting irrelevant data.

Should I outsource user studies or do it myself?

To execute the user studies, you can either “insource” or “outsource” the activity. If you use an external vendor, they should take care of most of the process, including recruiting participants, scheduling people, conducting the interviews, collecting the data, and compiling the feedback report.

The other approach is to do it yourself. This means you do your recruiting, own scheduling, conduct user studies, record the data, and draw inferences. With a little effort, it should be relatively easy for you to do much of this work quicker and at a lower cost than outsourcing.

Vendor lead user researchDo it yourself user research
Pros

  • It can save your time
  • You can be blind (in case you do not want your identity to be revealed)

Cons

  • It can take some time (1-2 weeks)
  • Can be expensive
  • You have less control (e.g., filtering candidates)
  • The vendor owns the relationships (you cannot re-use the sample or easily do follow-up sessions)
Pros

  • It can be very quick (2-3 days)
  • Cheap
  • You have more control
  • You own the relationships
  • Can be blind (with a bit of effort)
  • You learn a lot about users just through the recruitment process (e.g., looking at the data collected)

Cons

  • It can take more of your time (e.g., to set up templates and processes)

Where do I recruit people for my user studies?

To recruit users, you have many places to look. For example,

  • There’s an online service called usertesting.com
  • Use “classifieds” sites, online forums, Reddit, etc.
  • Add a link to your product, email footer, or website.
  • Post or advertise on social media – Twitter, Facebook, Google, Pinterest, Instagram, etc.
  • Approach professional associations or clubs within the industry you are targeting

Template – DIY user studies – an invitation to participate

What are legal and ethical considerations when conducting user studies?

  • You need to ensure you will comply with whatever SPAM rules there are.
  • Also, note that you will be collecting a lot of personal information about people, so you must ensure you manage and store this information in compliance with privacy regulations.
  • Remember to sign a non-disclosure agreement – and review all legal templates with your counsel.
    • In addition to the non-disclosure portion of the agreement,
    • it is important to include provisions to protect the ownership of the ideas generated through these user studies.
    • And also to get permission to record the sessions.
  • Be careful to recruit and pay journalists, government workers, analysts, or big customers – as this may trigger procurement regulations or be construed as bribery.
  • Don’t do site visits alone, especially when working after hours or dealing with minors.

To get you started, the template below contains some sample text you can use for your social media posts, email invitations, or online adverts.

Template – DIY user studies – non-disclosure agreement

How do you run a User Study Interview?

The Conversation Arc

The Conversation Arc is a way to structure user study sessions. While planning your interview guide, you are thinking through the whole conversation. You want to consider collecting all the data we’ve identified previously and how you do this smoothly and naturally. You want to think about setting it up, how to transition from one part to the next – how do you organize things so that it flows logically and you can bring the interviewee along in the process?

Conversation arc

The user study interview should form a natural conversational arc, starting when the person arrives at your location or when you meet up until you hand them their gift and they depart.

Part 1: Build Rapport and trust

At the beginning of the conversation, you’re building rapport and trust. You want to invest some time and energy into this process. If you jump too far ahead (for example, jump straight to showing the product prototype) before they know what is going on, what you want from them, and what the ground rules are, you won’t get good data out of the process. It’s the same as any natural conversation when you first meet someone. You don’t start by asking really deep and personal questions.

Rapport and trust-building start from the first contact – when the receptionist answers the door, or they are greeted when entering the building. You start by having a bit of chit-chat about the weather. Was it easy to find the place? How was the traffic? Do you work near here? Have you done this before? Small-talk. You want to make the person feel at ease as quickly as possible, make them feel comfortable, and ensure they know exactly what is happening.

Part 2: Introduction

During the introduction, you set some ground rules to let them know what you are doing, what you expect from them, and what they can expect from you. Refer to the interview guide for an example of an introduction.

Part 3: Discovery

It’s good to start the interview with the discovery part, as this provides context for the next part (usability). It will help you interpret how they respond to the product as you understand more about them and their experience. It also allows you to customize some of the tasks you will ask them to do in the usability part, making the process more relevant to them and seem more natural. The purpose of the discovery part is to open up and broaden the scope of the conversation. You will go outside the specific product you want to ask about and understand the user’s larger life context.

The template includes several examples of both discovery and usability questions. Pay attention not only to what the questions are about but also to how they are posed.

Part 4: Usability

In the usability part of the interview, you want to give the user some tasks to perform. If you refer back to the interview guide, it provides a template for setting these scenarios up. You want to prompt users to travel to points of interest.

Note that you want to send them in a general direction, and you should expect them to “bump into” the functionality but not exactly direct them on what to do, e.g., don’t say “click that button” but rather position it as “now I want you to try and do xxxx”.

You start with a broader, more general task and observe the user as they work their way through your product. Once they get to the end of the task and stop, you can ask them to complete some smaller-level tasks if they have missed anything you wanted them to do on the first go. So, you give them the freedom to explore the product as they wish and then return to complete any steps or additional functionality after the fact.

Remember, you should not say any words that can be seen on the UI. You should never use any words that can be on the screen – for example, use the name of a field label, button, or anything visible to the user. You want the user to find their way naturally without being prompted by you.

Part 5: Debrief

Once you have worked through all the usability tasks, and as you get to the end of the interview, you want to get a general overview of the user’s perspective. Summarise what they saw, and get their overall takeaway from the session. How do the various pieces compare to each other? Think about the pros and cons of various product parts – what did they like and dislike? How does this feel to use? How could this be better?

At this part of the interview, you’re also starting to send some subtle signals that you’re getting to the end of the process, and this allows for a gentle and natural way to wind down and conclude the conversation.

In the “Example Usability Questions Template,” – there are several example questions that you can work from.

Remember, you do not want to lead the witness – by embedding the answer in the question. You want to ask open-ended questions and allow the interviewee to respond as broadly and naturally as possible. So, for example, don’t say something like “Did you think that xxx was great?” but rather “What did you think of xxx?”.

Part 6: Cool Down

At this part of the interview, you’re starting to signal that you are getting to the end of the process. You can use various techniques, like a change in your voice or physical posture. You can put down your pen, notepad, or forms/questionnaires and move things out of the way. You start shifting into “thank you” mode. Say, “This has been great. The feedback has been incredibly useful”, and “I appreciate you coming in to do this.”

What is also useful here is to summarise a couple of the key points the interviewee raised in a very generic way. “It’s super interesting to understand more about how you ……”. You may find that more valuable feedback comes out at this point.

Part 7: Close

At this point, the interview is over. You conclude by thanking them for their participation and giving the incentive (another strong signal that the interview is over). You will be walking back to the reception/exit area with them.

Tip: Don’t stop recording until the person has left the building. Sometimes another conversation will pop up at the end, and more useful information might be there.

Preparation and Logistics

The question is – how exactly do you talk to these people? How do you create a clean, neutral, and somewhat innocuous environment for each participant? If you look at the Interview schedule template, there are several items to work through. Let’s focus on the time around the interview:

8 hours and counting

At the start of the day, as you prepare, you want to get your head, body, physical presence, location, and team into “interview mode.” Here are some useful tips:

  • Don’t wear perfume or cologne.
  • Don’t wear overt branding of your company or any brand or label – dress neutral.
  • Send a reminder with a schedule to your team and notetakers about observing sessions.
  • Prepare the hardware (computer, mobile devices) you’ll use for testing.
    • Make the computer or devices look neutral and generic.
    • Hide all icons on the desktop, any distractions, or visible bookmarks.
    • Clean the keyboard and screen.
    • Install and test prototypes, presentations, and wireframes.
    • Shortcuts: Add bookmarks or desktop shortcuts for prototypes, sites, etc.
    • Set the computer screen and settings to match the target users’ preferred settings.
    • Set an innocuous home page for the browser (e.g., local weather or news).
    • Clear the browser history, cookies, and cache.
    • Minimize the browser or prototypes.
    • Test the screen recorder and/or set up the video camera.
    • Start and test screen-sharing software for observers.
    • Run through all the prototypes under actual test conditions. Then reset them.
  • Print copies of NDAs and interview guides.
  • Clean up the room – erase whiteboards etc.

10 minutes and counting

Right before the start of the interview, you want to be 100% focussed – remember, sometimes interviewees will arrive early too.

  • Post a “Do not disturb” sign on the interview room door.
  • Turn off your cell phone.
  • Turn off all notifications from all devices powered on – e.g., computer, test devices.
  • Have a glass of water and take a bathroom break.
  • Freshen your breath – chew mints or gum.
  • Get into character: curious, open, and objective. Smile! Try to see the world through the participants’ eyes.

The below template contains a more complete user studies schedule, starting from 1 week out until the next day.

Template – DIY user studies – schedule

User studies are weird

Remember, this is a somewhat weird and unnatural situation. You have a person who’s come for an interview; they don’t quite know what you want or what they are doing there. They possibly feel self-conscious, and you sit awkwardly to the side or behind them, watching them struggle through a prototype. You don’t know each other at all, you’re spending a few minutes together, and it is unlikely that you will ever meet again.

Also, it is a one-way engagement; you’re asking all the questions, and the participant works through them. They are not learning anything about you, but you’re learning a lot about them. And.. you’re going to pay them for this. It’s just an odd situation in general.

So note that the things covered previously – about the conversation arc and making participants feel comfortable- are really important and will vastly improve the quality of the data you will collect. Also, it will be more fun for both of you – being awkward isn’t fun for anyone.

It could be helpful to consider yourself as a “host.” Someone is coming to visit and spend some time with you. You’re there to make them feel as comfortable as possible.

Get into character – the “Researcher Persona”

Before the interviewee arrives, you need to get into character. You should be 100% in character by the time they get there. You’re the host, receiving your guest. You’re smiling and friendly. Pro tip: smiling changes the sound of your voice and how you express yourself.

“You should smile whenever you’re doing interviews – including when you’re doing telephonic interviews.”

You’re grateful they’ve made the time available to come and see you, and you’re very happy to see them.

You will make them feel welcome and comfortable and care for them throughout their visit.

You’re there to learn from them and be interested in them.

Everything they say is super interesting; your body language and facial expressions will reflect that.

At the same time, you need to remain neutral. Curious, open, encouraging but neutral – without any strongly held opinions and without leading them. In other words, if they provide you with feedback or an answer, you will find it interesting, but you will not have an opinion on whether the response is good or bad.

“You should say “uh-huh” rather than “That’s good!”. There are no good or bad answers. All answers are good.”

Be very careful not to get defensive. Be aware of your own attachments to your product or certain outcomes. You don’t want to explain or defend any position. Divorce your emotions from the thing that you’re discussing. Feedback about your product is not feedback about you as a person.

Also, ensure you are not pitching the product to the person. You’re not selling the product. You’re not there to enumerate the value proposition, get excited about the features, or anything. If you’re describing anything, you’re doing so in a fairly neutral manner.

Interviewing tips

Here are a couple of additional interview tips:

  • Ask follow-up questions – “Why?”. Also, “silence” is a powerful follow-up
  • When in doubt, ask for clarification
  • Answer questions with questions
  • Keep it personal and concrete – avoid hypotheticals
  • Don’t pitch the product
  • Watch the time (surreptitiously)
  • Be careful when testing your own designs – note that you don’t get attached to the product or the outcome.
  • Watch non-verbal cues (yours and theirs). Facial expressions, body language, and tone. Are they comfortable? Tentative? Nervous? Bored?

Closing the study

Recording information

  • Watch and listen
  • Don’t filter or judge
  • Take users seriously
  • Take notes
  • Periodically note the time
  • After the session, note the top issues
  • Don’t do anything rash (like fix bugs)
  • Summarize and prioritize at the end of the user study

How do you analyze and apply results from user studies?

How to Analyze Usability and Discovery Results

When analyzing the results of usability testing and discovery sessions, look for patterns and trends that emerge from the data. Identify common issues, user needs, and areas of opportunity. Organize and summarize your findings to make it easier for the team to understand and act on the insights.

Making Improvements to the Product Based on Findings

Use the insights from usability testing and discovery sessions to inform design and development decisions. Prioritize the most critical issues and address them accordingly. Implement changes that address user needs, improve usability, and enhance the overall user experience.

Measuring the Success of Changes and Iterating as Needed

Continuously monitor and evaluate the success of the changes made to your product. Use metrics such as task completion rates, user satisfaction scores, and user engagement to assess the impact of the improvements. Iterate and make further adjustments using ongoing usability testing and discovery sessions to inform your decisions.

Conclusion

In conclusion, user research, usability testing, and discovery are essential to successful product development. By mastering these processes, you can create products that address user needs, save valuable resources, and stand out in the market. Establish clear objectives, recruit appropriate participants, and follow a structured interview process. With the right approach to analyzing and applying results, you’ll be well-equipped to continually iterate and improve your product. Keep learning, practicing, and refining your skills to ensure your products’ long-term success.

Templates

References

Agile Practices Change Management Implementing Agile Requirements Management