Anda di halaman 1dari 6

Team Better Than Yours

Kurt Blancaflor, Luis Gonzales, Tsuki Kaneko-Hall, Neha Nuguru

Usability Study Plan - Final


October 28th
2016
Purpose
TIDAL is a music streaming service that provides High Fidelity sound quality, along with High
Definition music videos and music reviews curated by music journalists. The purpose for this

study is to evaluate the discoverability, organization, and navigation of content on the desktop
(website) version of TIDAL. We will do this by providing tasks to build a playlist and prompt
navigation of different sections of the site.
Our group previously identified in our heuristic evaluations that there were many problem
areas with TIDAL concerning its interface including extraneous registration screens, ambiguous
icons, information organization, behavioral bugs, etc., and we want to test these aspects with
potential users to validate/invalidate our findings. We are conducting this study in order to
eventually come up with design recommendations for improving the way people navigate and
discover music on TIDALs online web platform. We are providing these recommendations to
TIDAL stakeholders in order to help TIDAL become a rising contender in the competitive music
streaming market.

Research Questions
1.

Ambiguity: How well can users recognize iconography? Will they correctly interpret the
icons on the Featured section?

2. Organization: Is Tidals content organized in a way that matches users mental model?
Will users be able to effectively navigate to their destination?

3. Accuracy: Will users effectively discover music that appeals to their preferences as part
of their music streaming experience on TIDAL?

Participant Characteristics
We are looking for participants that are unfamiliar with using TIDAL, but they can be familiar
with another music streaming service. Unfamiliar users will be those that have never used
any streaming services before, while Familiar users will be individuals that use a music

streaming service regularly (~5hrs /week). We hope to recruit 5 participants from each group,
making a total of 10 participants. We also hope to find users that listen to artists featured on
TIDAL in order to test the validity of TIDALs artist suggestion function.

Methods
We plan to use Morae to record user sessions to gather data on the amount of time to
complete tasks, number of screens to complete tasks. Screen recording will be a main method
of recording data. It will be used to gather the efficiency of navigation of our participants by
counting the number of pages it takes for the participant to complete a task. Screen recording
will also be used to measure the efficiency of tasks by recording the amount of time it takes to
complete a task and recording the amount of time a participant spends on each page. Time will
be disregarded for tasks that do not measure navigation efficiency, such as when users are
asked to interpret icons or count the number of suggested artists that they like.

Screening
1.

Participants will be recruited based on if they fit the characteristics that we discussed in
the Participant Characteristics section, and we will set up a meet time that works for
them and the group.

Welcome
1.

Upon arrival, participants will be greeted and eased into the evaluation area.

2. They will be reassured about the nature of the evaluation, more specifically, that it is the
product they will be using that is being tested, rather than them.
3. We will talk to them about how the session will play out. Namely, that they will be given
a set of pages with a task to be completed on each page. We will have them complete
these tasks and then talk about their experience with them in a post-session interview.

Tasks
Post-Test Interview

Task Lists
1.

Exploration of Icons: Explore the home page and take in the icons. As you go, identify
the icons you find.
a. Prompted to main Page.
b. Scroll over Features.
c. Notice that all features have icon on the top right.

d. As you scroll, name the icon you see on each feature and what you think the
icon indicates. You do not need to repeat naming and identifying an icon if it
shows up multiple times in the cue. Only once is fine.
2. Your friends are all raving about a Beyonce song that was released last year. They all
insisted that you go and listen to Beyonces Hold Up. You finally give in and decide to
go listen to the song.
a. Navigate to the Tidal website then,
b. Find Hold Up by Beyonce (example song).
c. Play Hold Up by Beyonce (example song).
3. Your friend was playing Christmas songs when you were visiting their place. You didnt
like the Christmas songs being played. Create a playlist of three of your prefered
Christmas songs and play them.
a. Use the site to find your preferred songs
b. Add songs to a playlist. Name this playlist Xmas Faves
c. After all songs are added, play the playlist.
4. You are beginning to become bored of your music, but you are interested in finding
similar songs. You decide to stretch out to find some new music.
a. Begin from the artist page that corresponds to your favorite artist mentioned in
screening.
b. Find Similar artists.
c. Count how many suggestions for similar artists there are, and tell us the number.
d. How many of the artists prompted have you listened to and enjoyed? Tell us the
number.
Study Design - Users will conduct the tasks in the same order. The tasks do not necessarily
build upon each other, but participants may become more familiar with the platform as they
continue to use it. For example, we will ask users to identify icons first before they are exposed
to them throughout the site. To avoid similar issues that may come up, we will prioritize basic
tasks first.

Background Questions:
1.

Ask what music streaming services they have experience with/currently use.

2. If they currently use music streaming services, ask them how many hours they use them
per week.
3. Find out what kind of music they listen to, including genres, artists, songs, etc. Develop
a list of content to verify that it is on TIDAL later.

Post-Test Interview Questions:

1.

Ask them to rate their experience with each task and with TIDAL overall (on a scale of
1-10) in terms of satisfaction.

2. Ask if there were any tasks or moments where they were confused about what to do or
where to go.
3. Ask about how it was finding songs and accessing their playlists.
4. Ask about their thoughts on the icons they identified and saw during their session.
5. Take them back to some of the pages that they navigated to and ask about their
thought process when going through the pages and carrying out the tasks.
6. Ask about what they think about the content layout of the main page, sidebar, and
subpages.

Test Environment(s), Equipment, and Related Logistics


For the purposes of this study, we will rent HP laptops from the LUTE (Laboratory for Usability
Testing and Evaluation) for users to use TIDAL on since we are analyzing TIDALs desktop
version specifically. We plan on utilizing a study room at Odegaard Library to conduct our study
where we will use the screen recording software, Morae, to track and record the participants
navigation paths. We also hope to measure efficiency and accuracy by timing users and
counting the number of screens they explore as they complete each task. We will use voice
recording applications on our phones in addition to text editing software like Microsoft Word to
take notes for the post-test interviews. These interviews will help us to collect information
about user experiences and their thoughts about TIDALs layout.

Facilitation Approach(es)
We will have users test the product individually. We will seat users at the PC and give them
their first task to complete. We will start recording their interaction and once they are ready to
begin we will time how long it takes them to complete the task. We will record their interaction
time and observe the route they take for each task. We will do this for each task and end the
session by talking to users about their thoughts on the experience. We will ask them about any
observations we made during their session and have them share their opinions about certain
parts of TIDALs features and UI. This will help us to obtain qualitative information about the
motives behind their actions and the factors that influenced their thoughts. In combination with
the rest of the data we collect, we will hopefully gain a better understanding of how TIDALs
layout impacts the user experience.

Data to be Collected

Through our study, we plan on collecting various forms of data including both quantitative and
qualitative data. Quantitative data we plan to collect includes: page click-throughs, time on
pages, number of errors in icon recognition exercise, similar artist suggestion accuracy
percentage, and time to complete specific tasks. In terms of qualitative data, we plan on
obtaining demographic information, familiarity with other streaming services, pain points they
experienced during the usability study, what users enjoyed while using the service and their
overall thoughts of using TIDAL. By aggregating all of this data we will be able to get a
complete picture of the usability, specific to our tasks, of TIDAL.

Data Evaluation
We plan to evaluate how successfully users, both familiar and unfamiliar with streaming music
applications, can find content within the TIDAL desktop application. Symbol ambiguity will be
assessed based on each users ability to correctly interpret TIDALs icons. Organization

success will be determined by the number of screens users navigate to and the time taken to
complete each task. And finally, TIDALs content association accuracy will be evaluated based
on how many similar artist suggestions were in line with user preferences. Success will be

determined by the number of screens users navigate to while completing navigation tasks, the
amount of time taken on the each task, and correctly recognizing icon functionality. A post-test
interview will be used to gather qualitative data from each user. We hope to combine our

qualitative and quantitative user data to determine how well users can navigate TIDAL, as well
as identify possible issues with TIDALs interface.

Reporting
We plan on utilizing the structure of the class for most of our reporting which includes: a project
declaration, project proposal, heuristic evaluation, usability study plan, usability test kit,
preliminary analysis, final presentation and final report. Each of these items is intended to
achieve specific usability study goals. The project declaration was intended to give us a
general idea of the problem space we were thinking about working within. The project
proposal was a more in depth look at the problem space and the specifics regarding our
plan/approach in addressing TIDAL as a service. The heuristic evaluation allowed us to define
our users and their goals along with defining the tasks they would attempt. The heuristic
evaluation also gave us the opportunity to complete walkthroughs, taking tasks step-by-step
through the users point of view to help us identify any problems based off a set of
predetermined heuristics.
The usability study plan was a detailed plan outlining the specifics of our study, so much so,
that another researcher could use it to execute a study comparable to the study our team will

be doing. The usability test kit includes the complete set of non-product items we will use to
interact with participants during our study. The preliminary analysis includes information
regarding our study once they have been completed along with brief descriptions of the
product tested, participants and preliminary analysis of our findings. The final presentation and
report include a complete synthesis and analysis of our study from beginning to end,
aggregated in a visually compelling and clear format to best depict our findings.