Lego Griffin Instruction Manual

Usability Report
Alexander Schwartz, Oliver Tseng, Bailey Wang
November 8, 2018

Abstract:
To save our company from bankruptcy, we need to create a financially successful Lego
set. We conducted a usability test for our Lego Griffin Instruction Manual to ensure
users were satisfied with our product and that the Lego Griffin would be completed
efficiently and effectively. Although this product is intended for 8-year-old children, due
to lack of availability, the users tested were three UC Davis students. User feedback to
our manual is integral in improving our product for release. Regarding our manual,
users had problems with picture perspective, color visibility, and organization. These
issues lessened the effectiveness, efficiency, and satisfaction of our product.
Table of Contents

Table of Contents 1

1. List of Figures 3

2. Glossary 3

3. Executive Summary 3
3.1. Purpose 3
3.2. Summary of Methods 3
3.3. Summary of Results and Recommendations 4

4. Introduction 4
4.1. The Lego Set 4
4.2. Testing Goals 5
4.3. Report Outline 5

5. Methods 5
5.1. Introduction to Methods 5
5.2. Definition 5
5.3. User Profile 5
5.4. Conditions 6
5.5. Survey 6
5.6. Recording 6

6. Results 6
6.1. Introduction of the Results 6
6.2. Observation by Task 7
6.3. Results By Goals and Time 8
6.4. Survey Results 10
6.5. Summary of the Results 10
Positive responses from users: 10
Errors revealed through observation 10
Errors revealed through time 11
Errors revealed through survey 11

7. Discussion 11
7.1. Overview of Recommendations 11

1
 Recommendation 1: Picture Perspective 11
Recommendation 2: Color Visibility 12
Recommendation 3: Organization 12

8. Appendices 12
8.1. Revised Instruction Manual 13
8.2. Original Instruction Manual 13
8.3. Testing Materials 13

2
1. List of Figures
● Table 1​: Goals and Time Results Chart
● Figure 1​: Efficiency Test Graph

2. Glossary
Effectiveness​​: The images and instructions were accurate to build the Lego Griffin.
Efficiency​​: The Lego Griffin was built within or under the recommended time.
Satisfaction​​: The tester felt satisfied and at ease while building the Lego Griffin.
Think Aloud Protocol​​: A method of gathering data by having participants articulate
their thoughts while performing their assigned task. In this case, it was following the
manual to construct the Lego Griffin.
Usability Testing​​: A technique used to evaluate the ease of use of a product by testing
it on users.

3. Executive Summary

3.1. Purpose
Our purpose is to create a Lego manual that will provide the user with a great
experience and prevent our company from financial ruin. User satisfaction is imperative
for the product to be not only viable, but successful on the market. While usability
testing the Lego Griffin instruction manual, we measured users’ effectiveness and
efficiency in completing the project and their overall satisfaction.

3.2. Summary of Methods

The methods we used participant observation, think aloud protocol, and user feedback.
We observed the participant’s physical actions during the build and their reactions to the
steps, such as whether they looked happy or confused. The think aloud protocol provide
us with a sense of what was easy or difficult in construction of the Lego Griffin. We
learned what aspects of the manual were unclear. Finally, user feedback was
conducted in Google Forms. Users reflected on the build and gave suggestions on how
to improve the manual.

3
3.3. Summary of Results and Recommendations
Users gave us valuable feedback of what to improve in the instruction manual. Users
found both shared and individual problems with the manual. The three most glaring
issues were: unclear picture perspective, poor color visibility, and lack of organizational
flow. These problems hindered the effectiveness, efficiency, and satisfaction of the user
while completing the Lego Griffin.

Fortunately, these problems have simple fixes. We resolved the unclear perspective in
our pictures by replacing them with ones that are easier to comprehend from first
glance. Secondly, the color visibility was improved with the addition of light green
arrows instead of dark blue arrows. The light versus dark red color issue in the print
manual is nonexistent in the electronic version. Lastly, the main problem with
organization was the cumbersome list of bricks needed at the beginning of the manual.
Reading the list wasted time and it was not all that valuable, so we moved it to the back
of the manual. Resolving these issues improved the three criteria of effectiveness,
efficiency, and satisfaction.

4. Introduction
We work for the company Struggling Financially. In order to bail us out of financial ruin,
we resolved to create a fantastic Lego set for 8-year-old children. The company will sell
the set to the targeted demographic of 8-year-old children. We created the Lego Griffin
and an accompanying instruction manual that is simple enough for the appropriate user.
Due to lack of clearance to test 8-year-old children by UC Davis policies, we substituted
UC Davis students in their place to test our manual. The methods we used were
participant observation, think aloud protocol, and user feedback to gauge the users’
opinion of our product based on three criteria: effectiveness, efficiency, and satisfaction.
This report contains the results that this testing produced as well as the
recommendations and actions to solve them.

4.1. The Lego Set

We created a Lego Griffin and developed an accompanying instruction manual. The
manual was designed to be as simple and easy to follow as possible. The set is
comprised of 29 pieces, so it can be completed by an 8-year-old.

4
4.2. Testing Goals
Our testing goals were to refine the instruction manual by observing users as they
constructed our product and hearing what they thought throughout the process. This
data allowed us to observe what was easy and what was difficult during the process
through objective users’ point of view. We want to make the product as effective,
efficient, and satisfactory to our users as possible.

4.3. Report Outline

Methods, results, and discussion of the usability test will be covered in this report. The
methods section details the techniques we employed to gather the data. Observations
and feedback constitutes the results section. At the end of the report, we analyse the
recommendations and resolutions to them.

5. Methods
We tested for effectiveness, efficiency, and satisfaction of our Lego Griffin instruction
manual through the usability testing. The methods used are user observations, think
aloud protocol, and user feedback.

5.1. Introduction to Methods

We tested our Lego Griffin instruction manual through observing three different testers
who used to the directions in the Lego Griffing Instructional Manual to complete the
Lego model’s construction. The testers demo-ed the manual to check if it was the most
effective and efficient at providing instructions. The tools used to measure the user’s
experience were the observation sheets and the survey questionnaires.

5.2. Definition
Effectiveness​​: The images and instructions were accurate to build the Lego Griffin.
Efficiency​​: The Lego Griffin was built within or under the recommended time.
Satisfaction​​: The tester felt satisfied and at ease while building the Lego Griffin.

5.3. User Profile

The goal of this experiment is to create a Lego manual suitable for children around the
age of eight. Children will have the most fun experience building the lego without being
5
overwhelmed with complexity. To create the best manual possible, we tested our
classmates at UC Davis.

5.4. Conditions
The test was performed in the Olson room 1 with testers from the UWP104T course,
since we do not have access to children testers. It is important to test the product within
a similar age group (20-25 years) to decrease the variance between age groups in our
sample group. The testers were encouraged to speak their thoughts, while they
completed the lego build. The testers were Christian, Haaken, and Bill who are all our
classmates.

5.5. Survey
To determine the effectiveness of the Lego Griffin manual, we also had our testers
complete a survey. The survey was conducted through a Google Forms. It is important
to clarify and improve the confusing aspects of the manual.

5.6. Recording
We encouraged participants to verbalize their cognitive processes as they were reading
our instruction manual and building the Lego assembly. We wrote down their thoughts
on our observation sheets. The participants provided valuable information such as
recommendations and complaints while completing the tasks and pinpoint which steps
and visuals caused the most difficulty, frustration, and confusion.

6. Results

6.1. Introduction of the Results

We tested our Lego Griffin manual on 3 different individuals in Olson 5 at UC Davis.
Each individual had at least 10 minutes to build the Lego set. During the test we
recorded the individual’s comments, recommendations, their physical observations, and
errors they made. After the build, users filled out a survey to answer a few questions
about the manual. Individual comments, recommendations, and difficulties are noted on
what they say about the manual while they are building. We recorded users’ physical
actions, such as if they picked up the wrong piece or put on a piece incorrectly, etc. The
results in this report are ordered by each individual.

6
6.2. Observation by Task
Christian:
Christian constructed the body differently compared to the other two testers. Instead of
pushing down the 2x6 on the two 2x3 pieces, he decided to place the rear on the 2x3
piece first. He had no problem doing step 1 except for a minor millisecond of figuring out
the color of the neck. He completed the step 2 in 28 seconds. Next, he immediately
knew to create two legs from just looking at the picture, similarly with the shoulders.
Step 2 was completed in 17 seconds and step 3 in 9 seconds. Step 4 was just as simple
for him as well, however, he had to double check what the top of the tail looked like
(which was like a pyramid triangle). He completed step 4 in 34 seconds. When he
started step 5, he noted that the wings are created to make a stair like appearance (as
the bricks were stacked on top of each other created a stair like shape). He completed
the wings in 18 seconds.

Step 6 proved to be a little more difficult. The printing of the manual had problems
showing the arrows, therefore he had trouble figuring out how the shoulders were
supposed to be placed. Without the guidance of the moderator to show where to place
the shoulders. Afterwards, the rest of the build was finished quickly. However, the
placements of the wings were hard to see and he had to count the ‘circles’ on top of the
1x6 to see the location of placement. He completed step 6 in 32 seconds.

Haaken:
Haaken had the hardest time trying to complete the build. He tried to place both the 2x6
and rear on the 2x3 pieces at the same time causing him to struggle just to create the
body. He also had trouble figuring the color of the neck. He completed step 1 in 53
seconds. For step 2, he looked at the manual and got confused, since the steps indicate
2 times, therefore he thought there must be 4 legs in total. From his mistakes on the
legs, he completes step 3 easily. Step 2 was completed in 81 seconds and step 3 in 33
seconds.

Step 4 and 5 were completed with less difficulty than the prior ones. Step 4 seemed to
be the one with the least amount of errors as there did not seem to be any problems.
Step 4 was completed in 34 seconds. Step 5 was difficult to create due to the color of
the bricks once again. Step 5 was completed in 18 seconds.

7
He had many problems doing step 6. He placed the legs backwards and only fixed one
of the legs at the very end. He also had problems placing the wings on the shoulders.
The build was finished in 101 seconds.

Bill:
Bill looked at the material list and piled everything from a neat pile into a messy one.
Afterwards, he did not seem to have any problems in step 1. He completed step 1 in 68
seconds, mainly due to looking at the material list.

Steps 2-5 were easy for him to complete. He understood that in step 2 the legs required
two sets without looking at the legs, similarly with the shoulders in step 3. He completed
the legs in 21 seconds and the shoulders in 14 seconds. He stacked the tail following
the manual. He completed step 4 in 26 seconds. He found that creating the wings were
pretty easy and straightforward. He completed step 5 in 31 seconds.

Despite his quick work on the other portions, he had difficulty completing step 6. The
body and legs kept breaking due to holding the build in his hands. Instead, he
connected the shoulders onto the legs and then placed the body on it. He counted the
length of the shoulder to place the wings. He completed the build in 81 seconds.

6.3. Results By Goals and Time

User Efficiency Effectiveness Satisfaction

Christian 140.62 seconds Read the materials Liked the arrows

Completed the page that showed where
entire product Started from step 1 to to place the pieces
step 6 “The tail is cutoff,
would will to see
more room in the
photo”

Haaken 363.14 seconds Attached the legs Perspectives

Completed the backwards, but only confused him,
entire product fixed one of them thought there were
One set of the Confused that having larger sized pieces
legs were put x2 on both parts of the Complained that
on backwards manual thinking there the arrows are too
should be 4 in total hard to see
Confused on which

8
 colors to use (red vs
dark red)
Perspective of the
shoulder confused him
(arrows not bright
enough)
Had trouble connecting
the legs together with
the body

Bill 161.96 seconds Understood the Smiled throughout

Completed the instructions right away the build
entire product Put wings in a
Got confused downward direction,
and piled then fixed it
everything Had trouble connecting
together out of the legs together with
the assorted the body
piles Assembling the product
often broke it by holding
it up

Steps Goal (seconds) Christian Haaken Bill

Step 1 25 28.11 53.62 68.85

Step 2 15 17.13 81.10 21.04

Step 3 15 9.07 33.58 14.97

Step 4 15 34.78 33.15 26.01

Step 5 25 18.80 60 31.09

Step 6 45 32.73 101.69 81.09

9
6.4. Survey Results
The images were great, however, the color for some of the bricks were harder to see.
Depth perception issues impacted the effectiveness in placing the tail and the wings.
The survey indicates that 66% of the users found the manual was clear to them and will
be to 8 year olds. All of the users thought that the manual was visually appealing as
well.

6.5. Summary of the Results

Positive responses from users:

Overall, the users found most of the manual clear and effective. The manual had a
positive visual appeal and so did the directions.

Errors revealed through observation

The perspective for a few of the steps need to be more clear. Specifically, users had
trouble with attaching the wing and the tail onto the Lego Griffin.

10
Errors revealed through time
One of the users attached backwards onto the body. When he was almost complete he only
realized the front pair was backwards and fixed it, but the hind legs remained backwards for the
finished product.

Errors revealed through survey

The users had difficulties determining which blocks to use in a couple of steps because
they had similar colors. They recommended that we write down which block to use in
the manual.

7. Discussion
This section contains discussion of the ways we can improve the effectiveness,
efficiency, and satisfaction of our Lego Griffin instruction manual through analysing user
recommendations in addition to applying them to the new video format.

7.1. Overview of Recommendations

There was a combination of shared feedback between the participants and valuable
individual recommendations. The overall issues were the perspective of the pictures,
color visibility, and organization.

Recommendation 1: Picture Perspective

The first recommendation stemmed from unclear perspectives of where to place the
Lego pieces in step 5 and 6. Primarily users struggled with the pictures that showed
where to place the shoulders, the wings, and tail. The picture of the shoulders and
wings were poorly angled, whereas the picture of the tail was cut off. Despite being cut
off, all users placed it correctly, but it still posed a problem. Problems with the
perspective of the images hindered both efficiency and effectiveness, as users had
trouble deciphering them and were more prone to commit errors.

To resolve these issues, we opted to create a video for our remediation project. The
video would provide multiple and dynamic perspectives in a matter of seconds rather
than having only a couple two dimensional pictures. Audio instructions will also aid the
visuals.

11
Recommendation 2: Color Visibility
The second recommendation was to make colors more distinct. The light and dark red
pieces in our Lego set seem difficult to differentiate, especially in the print manual. The
print manual turned out less clear than the original electronic one. Colors were faded
and the arrows were darker. The arrows were difficult to see on the dark brown
background of the body pieces. Participants stumbled in assembling the shoulders due
to lack of arrow visibility. Two participants thought the shoulder pieces connected
elsewhere until they figured out the proper placement. Again, both efficiency and
effectiveness were negatively impacted as a result.

Color visibility is improved in the video, as colors do not appear faded like in the printed
manual. Arrows are no longer necessary as well due to the video displaying where to
connect the pieces. The voiceover contains instructions and details what color piece is
attached to which.

Recommendation 3: Organization
The third recommendation was to improve the organization by putting the list of pieces
and the picture of all of them together in the back of the manual. A couple users wasted
time reading out each piece or putting them all in order according to the picture when
they were all grouped up already by body part. The overall pieces and list were
distracting, but overall still necessary to include. Having in in the back of the manual
avoids confusion while still including the integral information in case any piece is
missing or lost.

This organization recommendation will not apply in the video as it only contains a few
second shot of all the pieces organized by body part rather than the less clear picture.
The video has a more concise depiction of all the pieces in the set, excluding the wordy
list of all pieces.

8. Appendices
Revision of the instruction manual is necessary for the Lego Griffin in the toy market. To
increase revenue for this product, we edited the manual to ease the construction of our
Lego Griffin for our eight year old audience.

12
8.1. Revised Instruction Manual
https://docs.google.com/document/d/1QFa9RJKI2Xup1FkPb-Hz2kxWnvu34J2g7HaE7U
NpvcU/edit#heading=h.m5fbvcvopz2h

8.2. Original Instruction Manual

https://docs.google.com/document/d/1wp9zHEWoYXmxajjI2ZY7Y09WQx7DrQ1ImEFhs
e-Bpqk/edit#heading=h.m5fbvcvopz2h

8.3. Testing Materials

https://docs.google.com/document/d/1djmLb7Ndn8lPA9QGXxOEhImwqh1NS5a6KKGjoQyrQsk/
edit

13