Heuristic Evaluation

of the

School of Interactive Arts & Technology Web Site

Jan Castro

Marie Cheung

Diana Luong

IAT432

Design Evaluation

Dear Dr. Hatala,

This report highlights usability issues found within the September 23rd, 2012 version of the proposed new SIAT
website (siat.sfu.ca). An analytical heuristic evaluation method was used to evaluate, find, and rate the severity
of usability problems existing currently in the website. To conduct the evaluations, five tasks were used to
compare the interface usability to Jakob Nielsens original design principles. A few major heuristic violations
were found in the evaluation including: not providing help and feedback in the job submission form which can
severely hinder the completion of said task, several broken links denying potential end users from getting
crucial information, and inconsistent and complicated terminology which can confuse the audience. A list of 5
recommendations are given to help remedy usability issues and to bolster the success of the websites goals.
Recommendations include redesigning the Job Posting form to make it error proof, conducting a thorough
sweep of all links, clearly defining concentrations individually, using consistent terminology especially in the
case of program names, and creating a more visually appealing and categorized navigational structure. The
recommendations outlined in this report has potential in creating an informative, easy to navigate, and
satisfying website.

For more information feel free to contact the team,

Jan Castro (jrc15@sfu.ca), Marie Cheung (mlc9@sfu.ca) and Diana Luong (dla64@sfu.ca)

CONTENTS
Scenario
Methodology
Results & Intepretation
Suggested Immprovements
Apendixes

3
3-5
6 - 11
12
13 - 21

SCENARIO
Section 1

The interface we are evaluating is the newly revamped

School of Interactive Arts & Technology (SIAT) website,
which serves many students and faculty members on a
daily basis. Despite following a very structured layout
based off the Simon Fraser University (SFU) branding
guideline, many issues still arise that need to be dealt
with and fixed. The role of our production evaluation
team is to do a quick usability study by analyzing the

interface based on Nielsens heuristics and identify

design problems in the system. We will then present
our findings in a summary with suggested
improvements to aid in improving the website. Our
objective is to utilize the SIAT website goals in the
brief to consider what errors may prevent the
website from reaching its goals, and choose critical
tasks that best meet these goals to increase usability.

METHODOLOGY
Section 2

In this report, our team of experts used a heuristic

evaluation approach in analyzing the usability of the
new proposed SIAT website. The team consisted of
three team members: Diana Luong, Marie Cheung,
and Jan Castro. Furthermore, these individuals are
current SIAT students and have been practicing
interaction and usability for several years making them
double experts on the matter. A team of three was
formed over having a single evaluator or having over
five evaluators due to having a higher effectiveness in
finding results (see bottom graph http://www.useit.com
/papers/heuristic/heuristic_evaluation.html).

Heuristic evaluation is often used in the design

industry due to its affordability, quickness, and
effectiveness. This type of evaluation is the process
of comparing the current interface with a set of
usability principles. In addition, these usability
principles or heuristics are adapted from Jakob
Nielsens original 9 guiding principles. Below is a
list of said heuristics. Further information on these
adapted heuristics can be found on
http://carmster.com/432/uploads/Main/HeuristicEv
aluation.pdf.

LIST OF GUIDING HEURISTICS:

1. Simple and natural dialog
2. Speak the users language
3. Minimize users memory load

4. Be consistent
5. Provide feedback
6. Provide clearly marked exits

7. Provide shortcuts
8. Deal with errors in a positive manner
9. Provide help

Any violations of these 9 heuristics would be noted by the evaluators (see appendix 2) and used to inform
recommendations on creating a more usable interface. Using these 9 heuristics as the guiding principles of
using the interface, the team began to plan the scenarios and tasks upon which these heuristics will be tested.
Based on the 15 tasks given in the original web design brief
(http://carmster.com/432/index.php?n=Main.SIAT_Web_Site_Brief), 5 Tasks were chosen to conduct the
evaluation, due to their vitalness in obtaining the websites 9 goals.

LIST OF TASKS UTILIZED TO CONDUCT EVALUATION:

Task 1: Describe what kind of school SIAT is. List 2 characteristics of SIAT.
Task 2: Outline what concentrations undergraduate students can study in SIAT, and what degree options are
available to them.
Task 3: Outline which Graduate degree programs are offered in SIAT, and how to apply.
Task 4: You met Bernhard Riecke at a conference in Germany last semester. You are interested in doing your
masters degree at SIAT and you want to find out if he will supervise you. Find his contact information,
supply a list of 3 research areas Prof. Riecke is interested in, and list 1 project he has done in the past.
Task 5: Post a job on the SIAT website.
The first four goals of the website are geared towards providing information about SIAT and what makes SIAT
different from other schools. Naturally we chose task 1 through 3 as they directly pertain to obtaining the vital
information from the website. Furthermore, the first four tasks we chose are highly reminiscent of scenarios of
which we believe the majority of end users will go through. Task 4 was also picked as it could test the ability of
the website to provide information on projects, connecting with contacts, and finding SIAT staff members.
Similarily, task 5 was selected as it was the only task that could test goal number six (see chart below).

TABLE SUMMARIZING WHICH TASKS MATCH WHICH WEBSITE GOALS:

Website goal as outlined in the design brief

Which
tasks
influence
given goal

1. Successfully convey what kind of research school SIAT is; what

the students, professors and community are like; and what the
campus, labs, and resources are like.

Task 1,
Task 2,
Task 3

2. Provide users with clearly written and accessible information as

well as visual cues about our school so they can easily identify the
traits that differentiate SIAT from other schools.

Task 1,
Task 2,
Task 3

3. Provide users with clearly written and accessible information

about our degree programs so they can quickly identify the types of
programs that will interest them, the associated degree requirements,
the admissions information, and the kinds of projects and courses
involved.

Task 1,
Task 2,
Task 3,
Task 4

4. Ensure visitors are able to easily identify the types of research

being done at SIAT and who they can contact in each area.

Task 1,
Task 2,
Task 3,
Task 4

5. Ensure users can easily find faculty, their biography information,

links to their research, and their contact information.

Task 4

6. Provide an interface which encourages faculty, students and staff

in the community to contribute content to the website (submit projects
and publications to be displayed, edit their profile pages, post jobs,
news, notices), and make it easy to find common resources.

Task 5

7. Have a web presence thats lively, rich with visuals and frequently
updated with current information and events.

All Tasks

8. Have an easy to use website that provides effective cues to your

location at all times, provides appropriate feedback to the user, has
minimal errors, and is organized in a way that visitors can intuitively
navigate through the site. Provide a site that people want to come
back to.

All Tasks

9. Increase the number of people finding our website via keyword

searches.

Task 1,
Task 2,
Task 3

After formalizing and completing the criteria of evaluation, each team member individually evaluated the
website to check if it adhered to the heuristics defined earlier. Secondly, once the individual evaluations were
complete, the team gathered to summarize and compile their individual findings. Individual evaluations were
done rather than analyzing as a group because of implicit advantages such as; creating unbiased results,
evaluations not influenced by others, and higher variety in problems found. Using affinity diagramming, the
violations of the heuristics were grouped together. Violations were ranked using a 0-4 scale adapted from
Nielsens severity scale (http://www.useit.com/papers/heuristic/severityrating.html),

SCALE OF VIOLATIONS:
0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix, so should be given high priority
4 = Usability catastrophe: imperative to fix this before product can be released
The five recommendations we later suggest, to employ on the SIAT website, are based off violations with a
usability severity ranking of a 3 or a 4.

RESULTS & INTERPRETATION

Section 3

Severity Ratings

Evaluators

J
M
D
1.

2.

3.

4.

5.

6.

7.

8.

9.

10. 11. 12. 13. 14. 15. 16. 17. 18.

Usability Problems Ordered Based on Expert Consensus & Severity Ratings

From our Heuristics Evaluation, we have results that suggest that improvement can be made in certain
aspects of the SIAT website. We used Nielsons Severity Ratings for Usability Testing to help us determine the
most important findings that would suggest further investigation and potential improvement. We found
broken and redundant links and consistency problems (although not as severe) throughout the entire
website. We also found the Natural and Simple Dialogue and Using the Users Language heuristics violated
quite a few times throughout many of the tasks that we tested. Below is a list of our usability problems
ordered based on expert consensus and severity ratings. On the pages following we have grouped related
problems together and have included examples and screen shots that explain our thoughts.

USABILITY PROBLEMS ORDERED BASED ON

EXPERT CONSENSUS & SEVERITY RATINGS
1. Broken and redundant links. (Severity 4, found by All)
2. Does not provide feedback, and allows you to submit incomplete form. (Severity 4, found by All)
3. Confusing lingo. (Severity 4, found by Marie)
4. Concentrations not clearly defined. (Severity 3, found by Jan and Marie)
5. Users Language - confusing. (Severity 2, found by All)
6. Degree programs information is Inadequate and has no clear navigation. (Severity 2, found by All)
7. Memory Load - Too much writing, lack of efficiency. (Severity 2, found by Marie and Diana)
8. Broken Links (Severity 2, found by Jan and Marie)
9. No help provided to help describe forms field boxes. (Severity 2, found by Jan and Marie)
10. Overall consistency issues. (Severity 1, found by All)
11. Consistency issues - common look and feel and overall use of the website. (Severity 1, found by All)
12. Does not deal with errors in a positive (Severity 1, found by Jan and Marie)
13. Searching - About SIAT doesnt direct to About Us page. (Severity 1, found by Diana)
14. Clear language. (Severity 1, found by Jan)
15. Natural and simple dialogue - Funding is put in the wrong page. (Severity 1, found by Diana)
16. Naturally Bernhard Riecke should be linked on his publications. (Severity 1, found by Jan)
17. Clear exits absent.
18. Users language when finding professors (Severity 1, found by Diana)

BROKEN AND REDUNDANT LINKS

1.

8.

Task 4

Severity Level: 4

Found by 3/3

Task 2

In Task 4, when trying to find out more about Bernard

Riecke, the entire team came to broken links existing
in particular parts of his description. Clicking on
Previous Projects would lead you to this 404 Error
that exists on the Max Planck Institute for Biological
Cybernetics Site. Clicking on the Statement of
Research Interests - PDF would lead you to this Not
Found page as well.

Severity Level: 2

Found by 2/3

In Task 2, when looking for what concentrations and

degree options are available, our team ran into some
links that led us to non-existant pages. Under
Competitive Concentrations in the Course
Planning tab the Reserved Concentration PDF leads
to: Page Not Found. Also, on the main Undergraduate
landing page, the link TechOne Curriculum leads to:
Page Not Available.

INCONSISTENT TERMINOLOGY AND NEED FOR DEFINITIONS

3.

4.

Task 2

Severity Level: 4

Found by 1/3

In Task 2, there is a lack of consistency between

terminology used to explain SIAT. TechOne is also
referred to as SIAT First Year in certain places such as
the side menu. Having two names may cause
confusion. Although Informatics is now referred to as
Interactive Systems, it is still referred to as Informatics
on certain documents such as the B.Sc. or B.A. with
Major in Interactive Arts and Technology and
Informatics Concentration. Competitive
Concentrations could also be defined more clearly.

Task 2

Severity Level: 3

Found by 2/3

Also in Task 2, under Competitive Concentrations

there are no descriptions of what concentrations are
available for students. It would be helpful for students
to see their available choices. Also, as mentioned
above, the link to the pdf Reserved Concentration
PDF which may include this information leads to a
broken page.

5.

14.

Task 1

Severity Level: 2

Task 3

Found by 3/3

In Task 1 (when looking for the SIAT description), as

well as some other sections of the SIAT website,
unfamiliar terminology is used to explain what SIAT is.
Those who are new to SIAT may not understand the
descriptions without looking up the words on their
own, especially when the description is so wordy.
Some words that may be unfamiliar to the reader are:
interdisciplinary, immersive, cross-functional,
conception and implementation.

Severity Level: 1

Found by 2/3

In Task 3, when looking for which graduate programs

are offered and how to apply, in the admissions
section you can find words and abbreviations that may
not be understood by the reader without further
definition such as TOEFL and Out-of-Cycle

NEED FOR FEEDBACK

2.

9.

Task 5

Severity Level: 4

Found by 3/3

In Task 5, when posting a job to the SIAT Job Board,

there is lack of feedback which may result in users
submitting incomplete forms. There is no way the user
can know which boxes are required and which can be
left blank. For example, jobs that you submit that only
have the Company and Position filled in will still
go through even though important information is
missing.

Task 5

Severity Level: 2

Found by 2/3

Also in Task 5, when posting a job to the SIAT Job

Board, there is no help in case people are unsure of
what should be included in the boxes. Users may be
unsure of how descriptive they should be, for example
the Type box may cause confusion to the person
submitting the form on what that exactly means.

DOES NOT NATURALLY GUIDE THE USER THROUGH

6.

13.

Task 3

Severity Level: 2

Found by 3/3

Task 1

In Task 3, when outlining which Graduate degree

programs are offered in SIAT and how to apply, our
team members all found a disconnection between the
descriptions on the Degree Programs page and the
Degree Requirements page. The content does not
guide/link the user through the application but instead
they must look for the requirements elsewhere.

15.

Severity Level: 1

Found by 1/3

In Task 1, to look for the About SIAT page, our team

searched the words About SIAT and What is SIAT.
Neither searches lead to the About Us page but
instead lead to different links including the Dejine
booklet as well as
http://www.siat.sfu.ca/future-undergrad/degree/
which when clicked on results in a failed page.

16.

Task 3

Severity Level: 1

Found by 1/3

In Task 3, when looking at the Degree Programs

under Graduate we found that some categories
could make more sense if they were categorized in a
different way. For example Funding is found under
Degree Programs when it should be found elsewhere.

Task 4

Severity Level: 1

Found by 1/3

In Task 4, when trying to find more about Bernard

Riecke, our team clicked on this publication found on
the home page hoping itd eventually lead to his
profile. It lead to a publication by Bernard Riecke but
no link to connect this publication to his profile.

18.

Task 4

Severity Level: 1

Found by 1/3

In Task 4, when looking for Bernard Riecke, some

people maybe mistakenly click on the Faculty + Staff
Resources link first thinking that this would lead to
Faculty information. Users language when finding
professors - one can accidentally click on Faculty +
Structure resources instead of People

CONSISTENCY BETWEEN PAGES

10.

11.

Task 2

Severity Level: 1

Found by 3/3

In Task 2, when looking at the Course Planning tab

we found inconsistencies between the pages. For
example the headers and links found under SIAT
Major are not as clean and easy to read as the
headers and links found under SIAT Joint Major.

Task 3

Severity Level: 1

Found by 3/3

In task 3, when looking at the Graduate pages and

comparing them to the Undergraduate pages we
found inconsistencies between the two. The right side
bar found in the Program Overview of the
Graduate tab only shows social media while the tab
in Program Overview of the Undergraduate tab
has not just social media but projects as well. There
are also other pages in the Graduate tab which
have no blue bar (Degree Programs) or a blue bar
with nothing in it (Course Outlines). Also, the
Undergraduate description under Program Overview
has links such as BA or BSc Degree while the
Graduate description under Program Overview does
not link to any pages when it mentions Honours
Degrees or Joint Majors

INCREASE CLARITY OF INFORMATION

7.

12.

Task 1

Severity Level: 2

Found by 2/3

In Task 1, when looking to define what kind of school

SIAT is and its characteristics, the definition provided is
very wordy and may be hard for the reader to get an
idea of what SIAT is.

17.

Task 4

Severity Level: 1

Found by 1/3

In Task 4, when mistakenly signing into the Faculty +

Staff Resources, we found that there is no exit for the
user. Since the program sent a notice saying
Credentials not authorized for service, the user
would think to press back on their browser. Instead
this window pops up asking Are you sure you want to
send a form again?. This means the user is unable to
go back to the SIAT homepage.

Task 5

Severity Level: 1

Found by 2/3

In Task 5, when submitting a job to the Industry Job

Board, the system thanks the user even though the
user did not completely fill out the form. In our case
we only filled in Company and Position but did not
include any contact information. The program still
sends the job post through, thanks the user, and says
that it will be in touch shortly.

SUGGESTED IMPROVEMENTS
Section 4
1. Terminology (Consistent terminology/lingo)
We suggest to improve efficiency and memorability of searching for relevant information by providing
definitions to keywords or simplifying them, and using consistent terminology throughout the program
website to avoid confusion. This will help to reduce user frustration and improve accessibility of information.

2. Hyperlinks (Thorough sweep of links - in particular the faculty page)

We recommend to conduct a thorough sweep of the links, in particular the individual faculty profiles, to aid
in users search of information. In particular, we were unable to finish our task of finding more information
of a faculty members previous projects because a link was broken.

3. Job Posting (Foul-proof job form)

Another suggestion is to provide definitions for fields and add required links in order to help filter out job
postings such as spam as well as aid in user input to provide consistent information to students. Although
one of the website goals is to provide students to contribute to the website, the only form of contribution we
see is the job posting. At the very basis of the form, we feel that although it is straightforward to use, it does
not provide much information on what should be written in each field of the form and there seems not to be
any way to filter out the postings and important information that should be filled out is not clear.

4. Clearly Defined Concentrations

(Clearly defining concentrations individually and have them linked)
Define concentrations by providing clear explanations of each concentration, highlighting their respective
names, and linking them to proper information (i.e. PDFs) is another recommendation that we have on the
interface. Although the concentrations are not as heavily promoted anymore, it is still valuable information
for those who are interested in concentrations to provide them a guide of how to choose their courses to
those who need it.

5. Navigational architecture (Navigational architecture does not need

to be exactly the same as the one provided in the branding guidelines.)

Define concentrations by providing clear explanations of each concentration, highlighting their respective
names, and linking them to proper information (i.e. PDFs) is another recommendation that we have on the
interface. Although the concentrations are not as heavily promoted anymore, it is still valuable information
for those who are interested in concentrations to provide them a guide of how to choose their courses to
those who need it.
(See: envision and beedie websites)

APPENDIXES
Analysis & Raw Data

Appendix 1: Analysis of heuristic evaluation method for usability studies.

The heuristic evaluation method for usability studies was straightforward and easy to conduct and given the
short time frame (less than a week), and we were able to identify various critical problems within the website.
However, we believe that there were many issues we were not able to flush out and find due to our limited time
frame. Hence, we were unable to determine solutions that meet all of the SIAT website goals. Although we
individually identified certain issues, some of the issues were not seen as critical or did not fit in the teams
specific categories, and were not included as a specific concern to be highlighted and could potentially be an
important factor to improving the website. Additionally, it was hard to categorize some usability issues such as
missing links because some of the issues were very specific, yet stood out in the evaluating process. When
compiling the individual data it was important to be able to see the connection between the individual
findings. For example, two evaluators essentially found the same problem; however, their individual notes
would be worded differently thus the evaluators could potentially not compile the problems together. An
approach we found useful was affinity diagramming the problems we each individually found onto a
whiteboard for the whole team to see. This allowed the team to be on the same page on ideas and moving
forward. In the future I would highly suggest conducting the evaluations individually rather than doing the
evaluation altogether. This approach scaffolds results to be unbiased, have more variety, and is efficient.
Working through the interface together step by step could create groupthink as well as divert originality. Some
of the evaluators felt constrained on what feedback/problems existed as they tried to fit their findings
accordingly to the heuristics outlined.

Appendix 2: Raw data. All original data sheets and data summary sheet from heuristic
evaluation should be included here.
Marie Cheung
Tasks & specific
location in task

Heuristic broken

Usability defeat
description

Evaluators comments
regarding defect

Task 1: Describe what

kind of school SIAT is.
List 2 characteristics of
SIAT.

Memorability - Affects
users memory load.

Text is lengthy and

wordy, and not easy to
scan quickly and read.

Information should be
more concise by
highlight keywords and
visuals that are relevant
to SIAT (i.e. no
outdated stock
silhouettes)

Terminology is not
consistent - many
documents still use the
word Informatics.

Update the wording.

Satisfaction.
Efficiency.

Task 2: Outline what

concentrations
undergraduate students
can study in SIAT, and
what degree options are
available to them.

Consistency.
Simple and natural
dialogue (Users
conceptual model).
Efficiency.

Not easy to find

concentrations.

Fix broken link to

Competitive
Concentration.
Clearly outline
concentrations.

Task 3: Outline which

Graduate degree
programs are offered in
SIAT, and how to apply.

Satisfaction of content .
Consistency.

Task 4: You met

Bernhard Riecke at a
conference in Germany
last semester. You are
interested in doing your
masters degree at SIAT
and you want to find out
if he will supervise you.
Find his contact
information, supply a
list of 3 research areas
Prof. Riecke is
interested in, and list 1
project he has done in
the past.

Simple and natural

dialogue.

Task 5: Post a job on

the SIAT website.

Terminology.
Feedback.
Help.

Information is lengthy
and not pleasant to
read.

Information to be placed
more clearly like in a
box.

Text in Admissions
page on the bottom half
of page is italicized.

Italicize only important

text.

Broken link to Previous

Projects

Fix or remove the

broken link if not
absolutely necessary.

Redundant to have two

similar links to same
project (Projects and
iSpace Lab Vision)

Choose one of the two

links, and label it with
iSpace.
Project category should
be better defined.
Perhaps with a line
above and below that
area.

Unclear what is meant

by Type.

Have a description of
what each form
requires.

Jan Castro
Tasks & specific
location in task

Heuristic broken

Usability defect
description

Evaluators comments
regarding defect

Task 1: Describe what

kind of school SIAT is.
List 2 characteristics of
SIAT.

Speak the Users

Language

Wording is quite lengthy

and can be found
difficult to understand

Simplify language to
make it easily
understandable to
people not familiar with
SIAT

Simple and Natural

Dialogue

The infographic of the

cross which includes
art, design, science,
technology is confusing
to understand and only
clarified with the
descriptive paragraph

Maybe redesign the

infographic to show that
siat is a cross between
art and science with
technology as
foundation and design
for people as

beneath it

motivation. Or switch
the paragraph to be
above the image

Simple and Natural

Dialogue

Broken Links, eg first

year curriculum. Also
more information should
be given about the
concentrations

Properly fix all broken

links. Provide a clear
definition and space for
explanation for the
concentrations

Use the Users

Language

Confusing, inconsistent
and unnatural choice of
diction eg competitive
concentrations

Remove the word

competitive and just
simply use
concentrations.
Concentrations is used
ubiquitously throughout
the university and end
users will better
understand it

Be Consistent

Overall type (ie

keywords being bolded
sometimes) and
hierarchy issues. Some
pages layout doesnt
function the same as
other pages

Important keywords
should be bolded and
hierarchy revisited.
Layouts should function
the same across the
website. eg the right bar
should have similar
information across the
website (but not the
exact same information)

Use the Users

Language

Exclusive language,
which could not be
readily known by users,
found and not defined,
ie TOEFL and

Provide
explanations/definitions
for specific
words/phrases. eg
TOEFL (insert TOEFL

Task 2: Outline what

concentrations
undergraduate students
can study in SIAT, and
what degree options are
available to them.

Task 3: Outline which

Graduate degree
programs are offered in
SIAT, and how to apply.

Out-of-cycle

description here)

Be Consistent

Page layout and type

hierarchy isnt
consistent

Reformat pages to look

and function like similar
pages. eg Graduate
pages should be similar
to Undergraduate pages

Simple and Natural

Dialogue

Unable to simply click

on the three graduate
programs offered by
SIAT to get more
information

The words Master of

Science, Master of
Arts, and Doctorate in
Philosophy(PhD) should
link to an individual
page or paragraph
defining the desired
program

Task 4: You met

Bernhard Riecke at a
conference in Germany
last semester. You are
interested in doing your
masters degree at SIAT
and you want to find out
if he will supervise you.
Find his contact
information, supply a
list of 3 research areas
Prof. Riecke is
interested in, and list 1
project he has done in
the past.

Simple and Natural

Dialogue

Broken Link found, link

name: Previous
Projects. No link to
Bernards page on a
paper he published
(which can be accessed
from the homepage).

Correct links. Add links

to Bernards page if
Bernard is mentioned on
pages.

Task 5: Post a job on

the SIAT website.

Provide Help

Underneath the single

line entry text forms of
Type, Salary, and
Contact Notes its a bit
confusing of what
information should be
included.

Provide examples of
what to include ie
underneath Type have
a eg, full time, part
time

Provide Feedback

No feedback given

After submitting, if
information isnt filled
out correctly and/or if
entry lines are left
unfilled, user should be
prompted to correct the
errors and fill in the
necessary information
before being able to
successfully submitting
the job

Deal With Errors In A

Positive Way

I was able to submit an

incorrect and
incomplete form but the

Create an error
message informing
users that the form they

Deal With Errors In A

Positive Way

I was able to submit

an incorrect and
incomplete form but the
system still thanked me
for submitting the job
successfully

Create an error
message informing
users that the form
they submitted is
missing information or
is incorrect

Diana Luong - http://ow.ly/dZsfz

Tasks & specific
location in task

Heuristic broken

Usability defeat
description

Evaluators comments
regarding defect

Task 1: Describe what

kind of school SIAT is.
List 2 characteristics of
SIAT.

Simple & Natural

Dialogue

Navigation from "Core

Ideas" (which is what
it automatically lands
on) to "Community"
can be easily unnoticed
because it is on the
left side. Uses very big
words that may not be
understood by people
who are new to SIAT
and have not studied
similar studies before

Have a link more

visible, making it natural
for the user to switch
from "core ideas"
to "community"

Speak the User's

Language

Description is very
wordy, may be hard for
reader to understand

More visual
descriptions would
be better and make it
easier for readers to
understand

Provide Shortcuts

If the user knows

exactly what they're
looking for they should
be able to go there
quickly. (About Me has
to be accessed before
Community can be
accessed)

Changing the menus

- allowing it to quickly
show options without
waiting/clicking

Provide Help

Although the search bar

allows you to search
only SIAT pages,
looking up the terms"
About SIAT" or "What
is SIAT" does not lead
to the SIAT about me
page.

Making search
keywords work better

Task 2: Outline
what concentrations
undergraduate students
can study in SIAT, and
what degree options
are available to them.

Task 3: Outline which

Graduate degree
programs are offered in
SIAT, and how to apply.

Task 4: You met

Bernhard Riecke at a
conference in Germany
last semester. You are
interested in doing your
masters degree at SIAT
and you want to find
out if he will supervise
you. Find his contact
information, supply
a list of 3 research
areas Prof. Riecke is
interested in, and list 1
project he has done in
the past.

Simple & Natural

Dialogue

Confusing with so
many links: admitted
within certain times
and with different
concentrations means
different outlines,
Header for "SIAT Major
Course Planner/Grad
Check" is same font
weight as other bolded
points in the paragraph

Clearer distinctions of
titles

Be Consistent

The layout/titles for the

Joint Majors is more
clear than those in
Major

Making it more
consistent and making
the Titles of the Major
planners more clear

Simple & Natural

Dialogue

No direct connection
from the descriptions
of each program on
the main page to the
requirements

Quick link from

that section to the
requirements

Simple and Natural

Dialogue /Speak the
User's Language

Navigation doesn't
seem to be
clear, "Funding"
is under "Degree
Programs" Seems
like it should be found
elsewhere

Making sure links under

categories make sense

Be Consistent

Some pages have

headers and some do
not, spacing, numbers

Simple and Natural

Dialogue / Speak User's
Language & No Exit

Initially clicked
on "Faculty + Staff
Resources". Mistakenly
thought that Faculty
was found here. Lead
me to a site that asked
me to sign in -> Then
was not allowed to sign
in. Realized I should
click on "People"

Making titles more clear

Task 5: Post a job on

the SIAT website.

Deal with errors in a

positive manner, Speak
the User's Language

Broken Links, Click

on Projects and lead
to different site that
opened in a new tab.
404 Error in Max Planck
Institute

Fix links, Letting users

know how they got
there

Deals with errors in a

positive manner

Submitted a test job,

and all that needed
to be filled in was
Company and Position,
it went through!

Ensuring all information

is filled in by not letting
jobs be submitted
unless all required
information is filled
in: Especially contact
information

Summary
Deal with errors in a
Broken Links, Click
Found
by manner,Evaluators
Severity
positive
Speak oncomments
Projects and lead
Evaluators:
regardingtodefect
Rating
the User's Language
different site that
opened in a new tab.
Task 1:
Marie, Diana.
More visual
and
concise
404
Error
in Max2Planck
Memory Load descriptions,
and
Institute
Too much writing,
relevant information.
lack 5:
of Post
efficiency.
Task
a job on
Deals with errors in a
Submitted a test job,
the SIAT website.
positive manner
and all that needed
to be filled in was
Company and Position,
Very big words
that
is
it went
through!
Users Language Jan, Diana,
confusing to people not
2
confusing.
Marie.
familiar to SIAT.
Usability defect
description

Searching - About
SIAT doesnt direct
Summary
to About Us page.

Diana.

Searching terms in
search engine should
lead to correct pages.

Fix links, Letting users

Recommendations
know
how they got
there
Instead of lengthy
paragraphs, highlight
keywords, update
visual and
make
Ensuring
all information
them
relevant
(no
is filled in by not letting
silhouettes,
update
jobs
be submitted
infographic.)
unless
all required
information is filled
Show,contact
Less Tell.
in: More
Especially
information
Add relevant terms
and keywords to
pages.

Usability defect
description

Found by
Evaluators:

Evaluators comments
regarding defect

Severity
Rating

Recommendations

Task 1:
Memory Load Too much writing,
lack of efficiency.

Marie, Diana.

More visual and concise

descriptions, and
relevant information.

Users Language confusing.

Jan, Diana,
Marie.

Very big words that is

confusing to people not
familiar to SIAT.

Instead of lengthy
paragraphs, highlight
keywords, update
visual and make
them relevant (no
silhouettes, update
infographic.)

More Show, Less Tell.

Diana.

Searching terms in
search engine should
lead to correct pages.

Add relevant terms

and keywords to
pages.

Searching - About
SIAT doesnt direct
to About Us page.

Task 2:
Concentrations not
clearly defined.

Jan, Marie.

Concentrations should
be defined with proper
explanation.

Defining
concentrations in
its own space (i.e.
individually defined)

Broken Links.

Jan, Marie.

Links should be properly

linked.

Fix Reserved
Concentrations PDF
link and TechOne
Curriculum link

Confusing lingo.

Marie.

Consistent names need

to be used.

TechOne should
be properly named,
and all documents
with Informatics
should be Interactive
Systems. Competitive
Concentrations should
be concentrations or
more clearly defined.

Overall consistency
issues.

Diana, Jan,
Marie.

Headers and titles

unclear on certain
pages.

Important keywords
should be bolded
i.e. concentrations.
Course planning title
should be larger.

Task 3:
Consistency issues
- common look and
feel and overall use
of the website.

Jan, Marie,
Diana.

Content should be
consistent with the rest
of SIAT website.

Jan, Marie,
Diana.

Website should guide

you through the process
(i.e. of how to apply.)

Make header and right

sidebar consistent with
the rest of the website
(i.e. In Undergrad
sidebar, it shows
Projects. Master
of Science, etc
should be linked
similar to Bachelor of
Science in undergrad
page.
Inconsistent spacing in
Admissions page, and
italicize only what is
necessary.

Jan.

Language should be
inclusive and definitions
made clear.

Diana.

Information should be
categorized better.

Degree programs
information is
Inadequate and
has no clear
navigation.
Clear language.

Natural and simple

dialogue - Funding
is put in the wrong
page.

Recommended
programs should have
their own pages and
be linked properly.
Terms like TOEFL and
Out-of-cycle should
be clear of what they
mean i.e. TOEFL
(description).
Funding should have
its own or on different
page.

Task 4:
Broken and
redundant links.

Jan, Marie,
Diana.

Links should be linked

properly and useful.

Remove redundant
links and fix broken
links (i.e. Previous
Projects, and iSpace)

Naturally Bernhard
Riecke should
be linked on his
publications.

Jan.

All authors of
publications should be
linked to profiles.

Link Bernhard
Rieckes publication to
his profile.

Clear exits absent.

Diana.

User is stuck on page

because cant sign into
faculty resource page.

Provide back button.

Users language
when finding
professors - one
can accidentally
click on Faculty
+ Structure
resources instead
of People

Diana.

When users are

finding specific faculty
members, they will
naturally look for the
word faculty.

Changing the wording

faculty + staff (i.e.
have separate links
for staff, faculty, and
resources.)

Jan, Marie.

There should be help in

case people arent sure
what to write.

Include examples of
what to put in.

Jan, Marie,
Diana.

User should be prompt

on required boxes.

Include asterisks
and do not allow
incomplete forms
to be submitted
unless asterisks are
complete.

Jan, Marie.

Should do more than

just thank you, if you fill
in incomplete form.

Change messaging.

Task 5:
No help provided
to help describe
forms field boxes.
Does not provide
feedback, and
allows you to
submit incomplete
form.
Does not deal with
errors in a positive
way.