What makes good user experiences great? That’s a topic that has been discussed and debated ad nauseum over the past couple years. There’s so many ‘best practices’ to learn and follow, and so many notions on what works and what doesn’t. Whether you’re curious about a product or website, the advice out there is honestly more-than-enough to make your head spin. And oftentimes, the best metric to follow isn’t in your Google Analytics. Sometimes the best information comes directly from your customers. Your users. Your site visitors. Those are the folks that can give you the best indication of whether you’re creating a memorable or frustrating experience.

So you’re asking yourself:

This article will help to answer your questions and teach you, step by step, how to efficiently gather useful, actionable data from your users.

What’s a usability test? Well, it’s a method to validate an idea or an execution of a concept. But there’s, of course, an art to it. A usability test is loosely structured as follows:

  1. Objective
  2. Tasks
  3. Questions

We’ll jump into each one of these attributes more fully throughout this post. And we’ll share a few workshop activity templates so you can try this technique with your product or marketing teams, too. Specifically, we’ll discuss how to:

  1. Define a test Objective.
  2. Determine specific and open-ended Tasks.
  3. Craft specific and open-ended Questions.

Let’s get this party started!

What Do You Hope to Learn?

So, you want to obtain a true reaction from a user, and understand what they are actually thinking—without causing bias in their answers. First things first: When performing a usability test, you should know that the Objective is always known to the team, but hidden from the end user. Basically, the Objective boils down to, “What do I hope to learn?”

This question is critical to answer before starting any user testing project. By answering this question you’ll be able to clearly identify your goals and have a head start on how you’ll use the results. But here’s the trick: you must be as specific as possible.

Let’s try it out. Consider this example Objective: “Can a user create a new account on our site easily?”  This question might seem clear at first, but it doesn’t offer any specifics. How do we fix this? Let’s break the account creation task down into digestible chunks to make it easier…

When you deconstruct your Objective, you uncover whether or not it actually works as the driver for a usability test.

Let’s dive in a little deeper.

How to Create an Effective Objective

Obviously there’s some nuance to creating a clear objective for a usability test. Here’s some tips and tricks to get you or your team moving in the right direction:

  1. Start with a question: By framing your Objective as a question, you are laser-focused on solving a particular problem.
  2. Go one level deeper: If you think you have a good Objective, see if you can distill it down to an even more specific problem you’re trying to solve. If you cannot, then you’re solid.
  3. Be specific: What is the exact information you want to uncover? Is it the effectiveness of a particular button? Or watching how someone completes an order?

Let’s try an activity. Here you will find an activity to help you learn how to create an effective Objective. After you finish, you’ll be ready to tackle the next step: Tasks.

Do This, Now Do That: How to Create Tasks

Once you nail down your Objective, the next step is to create the action(s) or flow that you would like to instruct a user to take. Depending on what your Objective is, there are two primary types of Tasks that you can write: Open-ended, and Specific. What’s the difference?

Open-Ended Tasks

Open-ended Tasks are a bit… well, open-ended. They do not have a definite goal or solution in mind. You may be wondering, what’s an example of an Open-ended Task?

Let’s say your Objective is: “How easy or difficult is it for users to onboard onto my app?”

Here’s an example of an Open-ended Task you would give users to solve this Objective: “Please spend 5 minutes exploring the XYZ app, like you normally would.”

Notice that your user is still being instructed to do something—but it’s vague. Why would anyone want to do this? What advantage does it provide the tester?

Answer: The main advantage is gaining tons of new knowledge that you didn’t have before. For example, learning how a new client’s users interact with an app, or gaining awareness as to how people are using a new feature.

Tips for Open-Ended Task Creation:

Specific Tasks

Contrary to Open-ended Tasks which are used for exploratory information gathering, Specific Tasks are created to test a particular feature, flow, or object and offer a very narrow, focused view on a single item. Another use case for Specific Tasks is Conversion Optimization, i.e., testing out a very specific flow.

Hopefully now, the differences between Specific and Open-ended Tasks are clear, but to clarify: The main advantage of using Specific Tasks (versus Open-ended) is you get 100% of the information you’re seeking. You’ll achieve deep insights on a nuanced feature, flow, etc. In theory, it will help you to prioritize the backlog of your work, and come up with some new User Stories.

Let’s say your Objective is: “I want to understand if my users can easily use and enjoy the heart rate tracking feature on this Garmin watch.” Here’s an example of a Specific Task you would give to help solve this objective: “Open up the heart rate tracking feature and try to track your heart rate.”

Tips for Specific Task Creation:


How could this Specific Task be improved? Break it into smaller tasks to make it more specific…

“Find a diamond necklace for around $1,275 that is at least 1 total carat weight. Add it to your cart. Include the protection plan to your purchase. Then find a pair of earrings that will match. Add them to your cart. Checkout.”

Now that you have a good handle on Tasks, let’s jump into the next step: Creating Questions.

Riddle Me This: How to Write Questions That Matter

A Question is the follow-up to a Task, and is normally written to uncover even more information. Questions can be simple or complex, and—like Tasks—can be Open-ended or Specific.

Open-ended Questions:

Open-ended Questions are specifically crafted so they cannot be answered with a “yes” or “no”. They force the test participant to provide an explanation and uncover opinions or perceptions that may not have been expressed during the completion of the task. Keep in mind, responses may vary drastically from one test participant to the next.

Here’s an example of an Open-ended Question: “What would you expect to be able to do with a fitness app?”

The main advantage of an Open-ended Question is that the tester can ruminate about things they may not have considered. It can open the door to new thoughts and ideas and prompt the user think about the product in a new or different way.

Tips for Open-ended Question Creation:


How could these Open-ended Questions be improved?

Specific Questions

Specific Questions give users clear direction on which features to talk about. Similar to Specific Tasks, Specific Questions allow you narrow your focus and gather targeted pieces of information. They are tremendously useful when looking to solve small issues, or if you’re focusing on a particular action a user should take.

Here’s an good example of a Specific Question: “Did you find the heart rate feature to be helpful? Why or why not?”

The advantage of a specific question is that you can immediately get critical feedback about a very specific item. There’s little room for interpretation, and, if the question is crafted correctly, can yield immediate results.

Tips for Specific Question Creation:


How could these Specific Questions be improved:

Team up with a buddy and let’s tackle another activity here where you’ll now get the opportunity to create your very own Usability Test with a group.

Wrapping Up

Thanks for reading! I hope that you found this post valuable. You now know how to:

  1. Define an Objective
  2. Determine Specific and Open-ended Tasks
  3. Craft Specific and Open-ended Questions
  4. Create a kick-ass Usability Test

Plus, you also know and understand the value of Usability Tests. As I mentioned above, one of the greatest benefits of Usability Tests is getting realtime feedback from real users. This yields powerful, meaningful insights that could (and oftentimes does) change the course of your company’s communication and value. Lastly, here’s some great resources that helped me craft this post, workshops and activities.

If you have any questions, drop me a line in the Comments Section. Or drop us a quick note. We love running usability tests with our design partners, and it’s always a super fun collaboration.

Some Light Reading Material & Extra Resources…

  1. Running Usability Test
  2. How to Write Great User Test Questions
  3. The Key to Asking Good Questions During a User Test
  4. Setting Research Objectives
  5. Why You Need to Test with at Least 5 Users
  6. 7 Tips for Writing Usability Task Scenarios