The Top 5 User Testing Methods | Adobe Blog

by Nick Babich POSTED ON 02-23-2017

In an industry devoted to the people who use our products, services, and apps, user
testing is paramount. The main goal of user testing is to inform the design process from
the perspective of the end user. User-centered design is focused on designing for real
users, and user testing tells us who that person is, in what context theyll use a product,
and what goal they are looking to achieve.

UX researchers have developed many techniques over the years for testing and validating
their ideas, ranging from well-known lab-based usability studies to those that have been
more recently developed, such as unmoderated online UX assessments and guerilla
testing.

Some of the most popular forms of testing are usability testing, focus groups, beta testing,
A/B testing, and surveys:

1. Usability Testing

Usability testing is the process of watching/tracking an actual user while they use your
product to see if its in fact usable. Usability testing is the best way to understand how real
users experience your website or app. Its also flexible for collecting a range of
information about users, and its easy to combine with other techniques. Usability testing
is a cornerstone of UX practice.

When it comes to usability testing, one of the most important decisions youll make is
whether someone should moderate the session.

Moderated Usability Testing

Moderated usability testing is practiced by professionals looking to obtain feedback from

live users. During a moderated test, moderators are live with test participants (either in
person or remotely), facilitating them through tasks, answering their questions, and
replying to their feedback in real time. Live communication with test participants is a
strength of this type of testing, because nothing beats watching participants in real time,
and being able to ask probing questions about what they are doing.

Tip: If you want to use moderated testing, make sure you follow these 20 tips for
moderated usability testing.
When To Use

Moderated tests are recommended during the design phase when a team has a design
that hasnt yet been fully developed. You can run a moderated test to nd the potential
issues of your working prototype. By watching participants reactions on your prototype,
you can gather baseline data that can save you from spending a lot of design and
development time on a product that are difcult to use.

Things To Remember

A moderator can help probe the participant to delve deeper, keep the participant on track,
and help clarify any confusion. However, a very common mistake moderators make is to
tell a participant what to do. There is a ne line between guiding and helping a user. Thus,
you need to nd a balance to keep the participant on task, while not messing with their
natural experience. When this balance is struck properly, even the most convoluted of
tasks can provide rewarding feedback.

Unmoderated Remote Usability Testing (URUT)

Unmoderated remote usability testing, as the name implies, occurs remotely without a
moderator. It offers quick, robust, and inexpensive user testing results.
This method is usually based on the use of usability testing tools that automatically
gather participant feedback and record their behavior.

URUT has following benets:

 Participants complete tasks in their own environment without a moderator present,
which leads to the product being used naturally.
URUT is conducted online much like a survey with pre-determined tasks, so it can be
completed in the participants own time without requiring the hassle of coordinating
schedules.
Unmoderated tests can also be run concurrently, allowing for a much greater volume.
The turn-around time for unmoderated tests is often signicantly faster than that of
moderated tests. Data can be collected in as a little as a few hours depending on
sample size and testing criteria.
The cost is usually quite low since you dont need to pay for moderators or an
equipment setup. You can get maximum value for minimum cost when tasks are
written as clearly as possible.

When To Use

When you need to obtain a large sample in order to prove key ndings from your
initial moderated research.
When you have very specic questions about how people use a user interface for
relatively simple and straightforward tasks.

Things To Remember

URUT should not be used as a replacement for moderated usability testing. Instead,
its best when you use it in conjunction with moderated testing.
The lack of a moderator means less control, less personal observation, and a higher
risk of confusion. Thus, to run a test successfully you need to set clear expectations
for participantsits crucial to ensure that tasks are clear and user-friendly.
Be mindful of how much time participants spend with test. Kyle Soucy suggests an
unmoderated test should be 1530 minutes in duration, and comprised of
approximately 35 tasks, because the dropout rate tends to increase if a test takes
longer.

2. Focus Groups

Focus groups are a tried and true method of communication between a researcher and
users. In a focus group, you bring together 6 to 12 users to discuss issues and concerns
about the features of a UI. The group typically lasts about 2 hours and is run by a
moderator who maintains the groups focus.

Tip: Check out the article The Use and Misuse of Focus Groups on how to effectively
utilize focus groups for user testing.
When To Use

Focus groups can be a powerful tool in system development. This technique can help you
assess user needs and feelings both before a products design and long after its release.
In website or mobile app development, the proper role of focus groups isnt to assess
design usability, but to discover what users want from a producttheir personal thoughts
and preferences.

Things to remember:

Focus groups shouldnt be used as your only source of user testing data. They are a
rather poor method for evaluating interface usability as individuals rarely get the
chance to explore the product on their own.
 Its recommended to run more than one focus group, because the outcome of any
single session may not be representative.

3. Beta Testing

Beta testing allows you to roll out a near-complete product to individuals who are happy
to try it and provide critical feedback. This testing method allows you to ask users
questions after they have the new product, track their usage and have them le bug
reports.

When To Use

You should use this testing when your product is near complete and you want to put it in
the hands of the end users to gather feedback. Beta testing a good way to market your
product and get constructive feedback in order to rene the design to improve the
product.

Things To Remember

Sufcient testing should be carried out before releasing a product to the customers.
Naturally, you dont want your users to nd and report bugs, you simply want their
feedback on product features and usability.

4. A/B Testing

An A/B test is ideal as the appropriate testing method when designers are struggling to
choose between two competing elements. This testing method consists of showing one
of two versions randomly to an equal number of users, and then reviewing analytics on
which version accomplished a specic goal more effectively.
Tip: You can dene stronger A/B test variations through UX research.

When To Use

A/B testing is good when trying to detect small differences in designs. This testing is
particularly valuable when comparing a revised screen to an older version.

Things To Remember

With A/B testing you only nd the best option from among the available variations. These
variations should be selected very carefully. If the variations are only based on internal
experience and opinion, the testing wont nd the optimal design.

5. Surveys

Questionnaires and surveys are an easy way to gather a large amount of information
about users, with minimal time invested. A researcher can create a survey using tools like
Wufoo, SurveyMonkey or Google Forms, send it out, and receive hundreds of responses in
just minutes. The right questions can uncover customer needs, desires, and pains.

Tip: Here is a great step-by-step guide of creating a survey.

When To Use

Surveys can help you accumulate quantitative data about overall user satisfaction or
collect quantitative data to support qualitative research ndings.
Surveys are also good when you need to gather feedback about a brand new feature.

Things To Remember

You cant study user behavior with surveys. If you want to study how your visitors
behave or what usability problems they face during an interaction with your product,
consider other research methods.
Creating a survey looks like a quick and easy task, but in reality it is the opposite. A
signicant amount of time should be dedicated to preparing surveys. Its important to
get the questions right and direct them at the right audience.
Conclusion

User testing is an essential part of the design process its a fantastic way to understand
how your user base interacts with your product. As you just saw, different types of user
testing suit different types of goals. Ultimately, the best format of user testing depends
entirely on what your product is, what youre looking to learn about it, and how much time
you have available. So its up to you to consider which method will best suit your needs in
order to gather the most valuable feedback on the user experience of your product.

Save

Save

Save