What makes good user experiences great? That’s a topic that has been discussed and debated ad nauseum over the past couple years. There’s so many ‘best practices’ to learn and follow, and so many notions on what works and what doesn’t. Whether you’re curious about a product or website, the advice out there is honestly more-than-enough to make your head spin. And oftentimes, the best metric to follow isn’t in your Google Analytics. Sometimes the best information comes directly from your customers. Your users. Your site visitors. Those are the folks that can give you the best indication of whether you’re creating a memorable or frustrating experience.
So you’re asking yourself:
- Do users understand how our product works?
- Can potential customers find what they’re looking for on our website?
- How can we tell if our content is actually resonating with our end users?
This article will help to answer your questions and teach you, step by step, how to efficiently gather useful, actionable data from your users.
What’s a usability test? Well, it’s a method to validate an idea or an execution of a concept. But there’s, of course, an art to it. A usability test is loosely structured as follows:
We’ll jump into each one of these attributes more fully throughout this post. And we’ll share a few workshop activity templates so you can try this technique with your product or marketing teams, too. Specifically, we’ll discuss how to:
- Define a test Objective.
- Determine specific and open-ended Tasks.
- Craft specific and open-ended Questions.
Let’s get this party started!
What Do You Hope to Learn?
So, you want to obtain a true reaction from a user, and understand what they are actually thinking—without causing bias in their answers. First things first: When performing a usability test, you should know that the Objective is always known to the team, but hidden from the end user. Basically, the Objective boils down to, “What do I hope to learn?”
This question is critical to answer before starting any user testing project. By answering this question you’ll be able to clearly identify your goals and have a head start on how you’ll use the results. But here’s the trick: you must be as specific as possible.
Let’s try it out. Consider this example Objective: “Can a user create a new account on our site easily?” This question might seem clear at first, but it doesn’t offer any specifics. How do we fix this? Let’s break the account creation task down into digestible chunks to make it easier…
- Do users notice the “Create account” button?
- Are there usability issues on the “Confirm account” page?
- How do users prefer to fill out fields for a new user?
When you deconstruct your Objective, you uncover whether or not it actually works as the driver for a usability test.
Let’s dive in a little deeper.
How to Create an Effective Objective
Obviously there’s some nuance to creating a clear objective for a usability test. Here’s some tips and tricks to get you or your team moving in the right direction:
- Start with a question: By framing your Objective as a question, you are laser-focused on solving a particular problem.
- Go one level deeper: If you think you have a good Objective, see if you can distill it down to an even more specific problem you’re trying to solve. If you cannot, then you’re solid.
- Be specific: What is the exact information you want to uncover? Is it the effectiveness of a particular button? Or watching how someone completes an order?
Let’s try an activity. Here you will find an activity to help you learn how to create an effective Objective. After you finish, you’ll be ready to tackle the next step: Tasks.
Do This, Now Do That: How to Create Tasks
Once you nail down your Objective, the next step is to create the action(s) or flow that you would like to instruct a user to take. Depending on what your Objective is, there are two primary types of Tasks that you can write: Open-ended, and Specific. What’s the difference?
Open-ended Tasks are a bit… well, open-ended. They do not have a definite goal or solution in mind. You may be wondering, what’s an example of an Open-ended Task?
Let’s say your Objective is: “How easy or difficult is it for users to onboard onto my app?”
Here’s an example of an Open-ended Task you would give users to solve this Objective: “Please spend 5 minutes exploring the XYZ app, like you normally would.”
Notice that your user is still being instructed to do something—but it’s vague. Why would anyone want to do this? What advantage does it provide the tester?
Answer: The main advantage is gaining tons of new knowledge that you didn’t have before. For example, learning how a new client’s users interact with an app, or gaining awareness as to how people are using a new feature.
Tips for Open-Ended Task Creation:
- Encourage Users to Explore: If you want to find things that are broken or cause friction for your users, letting them explore freely will uncover issues you may not already be aware of.
- Avoid Exact Instructions: Even if you’re giving your test participants specific tasks and questions, you need to strike a balance. You don’t want to micromanage them, because then you won’t learn anything. Let them do some of the work on their own.
- No Industry Jargon: The average user doesn’t understand terms like “sub-navigation” and “friction,” so don’t include them in your questions. Define key terms or concepts in the questions themselves (unless the goal of your study is to see if they understand these terms/concepts).
Contrary to Open-ended Tasks which are used for exploratory information gathering, Specific Tasks are created to test a particular feature, flow, or object and offer a very narrow, focused view on a single item. Another use case for Specific Tasks is Conversion Optimization, i.e., testing out a very specific flow.
Hopefully now, the differences between Specific and Open-ended Tasks are clear, but to clarify: The main advantage of using Specific Tasks (versus Open-ended) is you get 100% of the information you’re seeking. You’ll achieve deep insights on a nuanced feature, flow, etc. In theory, it will help you to prioritize the backlog of your work, and come up with some new User Stories.
Let’s say your Objective is: “I want to understand if my users can easily use and enjoy the heart rate tracking feature on this Garmin watch.” Here’s an example of a Specific Task you would give to help solve this objective: “Open up the heart rate tracking feature and try to track your heart rate.”
Tips for Specific Task Creation:
- Simpler Tasks: Just like writing a good Objective, the more you can break the task down, the better.
- Leave Breadcrumbs: If you ask for a Specific Task to be completed, but users are on the wrong page, your data will be incorrect. So how do we solve for this? Think about including notation (“You should now be on the Login screen”) or a link to the specific page to test (“Now we’ll go to the Login screen. Make sure that you’re on this page: [link].”)
- Have a Correct Solution: This may seem to straightforward… but there should be a conclusion to the step. For Specific Tasks, the user must be able to achieve what you’re instructing them to do.
How could this Specific Task be improved? Break it into smaller tasks to make it more specific…
“Find a diamond necklace for around $1,275 that is at least 1 total carat weight. Add it to your cart. Include the protection plan to your purchase. Then find a pair of earrings that will match. Add them to your cart. Checkout.”
Now that you have a good handle on Tasks, let’s jump into the next step: Creating Questions.
Riddle Me This: How to Write Questions That Matter
A Question is the follow-up to a Task, and is normally written to uncover even more information. Questions can be simple or complex, and—like Tasks—can be Open-ended or Specific.
Open-ended Questions are specifically crafted so they cannot be answered with a “yes” or “no”. They force the test participant to provide an explanation and uncover opinions or perceptions that may not have been expressed during the completion of the task. Keep in mind, responses may vary drastically from one test participant to the next.
Here’s an example of an Open-ended Question: “What would you expect to be able to do with a fitness app?”
The main advantage of an Open-ended Question is that the tester can ruminate about things they may not have considered. It can open the door to new thoughts and ideas and prompt the user think about the product in a new or different way.
Tips for Open-ended Question Creation:
- Define the Timeline: If you are asking about frequency, such as how often a user visits a particular site, make sure you define the timeline clearly. Always put the timeline at the beginning of the sentence.
- Clearly Define Your Test Objective: When you’re asking Open-ended Questions, you still need to make sure you have a clear objective in mind. (For example, “Can visitors find the product they’re looking for?”) If you don’t know what you want to learn, your test participants may talk aimlessly without uncovering anything useful.
- Avoid Leading Questions – Try not to bias your questions by suggesting a specific response. For example, “How easy was it to find the pricing page?” This is a common trap that even the most experienced researchers fall into it. Make sure all your questions are as objective as possible. A good way to reframe that question is: “How easy or difficult was it to find the pricing page?”
How could these Open-ended Questions be improved?
- “How often do you visit Amazon.com?”
- “How easy was it to find the pricing page?”
- “Was the navigation difficult to use?”
Specific Questions give users clear direction on which features to talk about. Similar to Specific Tasks, Specific Questions allow you narrow your focus and gather targeted pieces of information. They are tremendously useful when looking to solve small issues, or if you’re focusing on a particular action a user should take.
Here’s an good example of a Specific Question: “Did you find the heart rate feature to be helpful? Why or why not?”
The advantage of a specific question is that you can immediately get critical feedback about a very specific item. There’s little room for interpretation, and, if the question is crafted correctly, can yield immediate results.
Tips for Specific Question Creation:
- Don’t Test the Participants: Remember, you are not testing the participants. Your tasks and follow up questions should make them feel like there is no wrong answer. The best way to avoid this is to make sure that all blame is placed on the website being tested.
- Ask Many Specific Questions: When you ask questions about vague or complex concepts, users often don’t know how to answer them. Break concepts up when you’re asking the questions and put them back together when you’re analyzing the results.
How could these Specific Questions be improved:
- “I wasn’t able to find the product I was asked to find. Agree or Disagree?”
- “What is your level of satisfaction with the Padres MLB website?”
Team up with a buddy and let’s tackle another activity here where you’ll now get the opportunity to create your very own Usability Test with a group.
Thanks for reading! I hope that you found this post valuable. You now know how to:
- Define an Objective
- Determine Specific and Open-ended Tasks
- Craft Specific and Open-ended Questions
- Create a kick-ass Usability Test
Plus, you also know and understand the value of Usability Tests. As I mentioned above, one of the greatest benefits of Usability Tests is getting realtime feedback from real users. This yields powerful, meaningful insights that could (and oftentimes does) change the course of your company’s communication and value. Lastly, here’s some great resources that helped me craft this post, workshops and activities.
If you have any questions, drop me a line in the Comments Section. Or drop us a quick note. We love running usability tests with our design partners, and it’s always a super fun collaboration.