Halley Hill September 29, 2013 303- Intro to Web Publishing Usability Analysis Based on User Testing Usability

test for NYTimes.com Choosing Participants About my site NYTimes.com is the online edition of the New York Times newspaper. It covers a variety of topics, from world news to sports to entertainment. The websites biggest topics cover politics and the U.S. Government. Along with these topics, which are also in the New York Times print edition of the newspaper, the website features videos and user interactions to accompany stories and articles. (NYTimes.com) About My Sites Audience The median age for the NYTime.com is 44 years old. This symbolizes that there is a fairly spread out age distribution in the users that explore NYTimes.com. Most of the coverage of the website deals with political and national news which attracts many users who have an average yearly income of $74,151. With this kind of income information, it indicates that NYTimes.com users have a high level of education and/or job status. My Chosen Testers I have chosen Daryl Hill, a high experienced user, and Linda Morrison, a low experienced user. Daryl was chosen because he uses the Internet on a daily basis as he works on computers for a living. He gets all of his news and updates from the Internet, and prefers using the web to find news rather than reading the newspaper. Daryl is 45 years old and fits the age of the audience for the New York Times well. His yearly salary is fairly close to the statistic of $74,151 that was given. Linda Morrison, 62, was chosen based on her low user experience profile. Linda prefers to read the newspaper, and will only go online for specific tasks. She fits well for this experiment because she is part of the generation that is just now learning how to use the Internet, and NYTimes.com could benefit by catering their website to her needs. Usability Test A: Daryl Hill Testing Environment Physical Environment: Daryl browses the Internet in his home office. He preferred to browse the Internet with the lights off so that he can see his monitor better, and kept the room fairly quiet. As he is a computer technician, he has three computers on desks that are on at all times in his office. This contributes to a steady buzzing of the room. Daryls two dogs come in and out of the room at their leisure, and sometimes disrupt him from his task at hand. He also keeps his cell phone next to his keyboard, in case he receives a call or text message. Technical Environment: Daryl uses his desktop computer that has an Intel 3.1 GHz i5 processor and runs on Windows 7. His screen resolution is set to 1280x1024. His Internet connects through a cable going into the computer. The connection speed of his Internet is 15 MB/sec. Daryl uses a wired mouse to browse on his computer and scrolled using the

scroll wheel built into the mouse. For this test, he used Mozilla Firefox version 23.0.1. His browser add-ons include Adobe Acrobat, iTunes Application Detector, Java Deployment Toolkit, Microsoft Office 2010, Nero Kwik Media Helper, Shockwave Flash, Silverlight Plug-in, and VLC Web Plug-In. Test Results Task 1 Analysis: After reading the task to Daryl, he went to Google.com to find his answer. Once on the Google.com homepage, he typed in North Korea nuclear sus and looked at the drop down of narrower search criteria that Google provided for him and clicked North Korea Nuclear suspension. This gave him search results that had news articles relating to North Korea. Daryl found the second news article result labeled North Korea Nuclear Suspension by Time.com and clicked the article title to further read about the suspension. He successfully completed task one. Task 2 Analysis: After directing him to NYTimes.com and reading off task 2, Daryl scrolled halfway through the front page of the website using his scrolling wheel on his mouse and then went to the first column on the left hand side of the page. He clicked on an article titled Children and guns and read it. He successfully completed task two. Task 3 Analysis: After task 2 was completed Daryl went back to the homepage of NYTimes.com to complete the third task. He began to whistle as he started on task number three. He hovered with his mouse over the different sections that were above the fold of the front page of the website and then scrolled down, stated that he was looking to see if they have a search thing. At that moment, an Advertisement popped up in which he had to manually close out of, saying under his breath G-d D#^$ advertisements! Once he had clicked out of the pop up ad, and scrolling back up to the top of the homepage of the NYTimes.com, he found the search bar located in the upper left hand corner of the page. He typed in sting tickets and pressed enter. This brought up a variety of articles related to the singer Sting, but nothing of what he wanted. Staying on that page, he then added anspocker theatre to the end of his sting tickets search. After pressing the enter key on his keyboard, this brought a bolded text line that asked Did you mean: sting tickets anspacher theater in which he clicked, which led him to the first article on the search results page that told him the dates of his concerts and that tickets ranged in the thousands. Daryl successfully completed task number three. Task 4 Analysis: Before assigning task four, Daryl went back to the homepage of NYTimes.com. He typed in maxim purchase into the search bar in the top left hand corner of the page. After browsing through the first page of results for which the search maxim purchase brought up, he could not find what the task was asking for. He decided to read every headline and scroll to the very bottom of the page, where he clicked the 2 that indicated to go to the second page of the search results. He clicked on the first article that appeared on the second page of the search results and was unable to find who purchased Maxim Magazine. He clicked on the back arrow in his Firefox browser, located in the top left hand corner of the screen, and went back to the second page of his search results for maxim purchase, letting out a sigh of frustration as he did so. Daryl did not complete task number four.

Task 5 Analysis: Daryl started task 5 from his search results page for task 4. He scrolled to the very top of the page and clicked on the sports tab of the navigation bar which was then on the very top of the page, instead of the left hand side, which is where it is located on the homepage. This led him to the homepage of the Sports section of NYTimes.com. After glancing over the topics that were covered on the homepage, Daryl clicked on the search bar in the upper left hand corner and typed in Heinrick Stinson, which led to a search page that said in bolded text letters Did you mean Henrik Stenson. Daryl clicked on this suggestion, which led him to the first article the search brought up. This article told him that Henrik Stenson is a golfer who recently played in the BMW Championship. Daryl successfully completed task five. Task 6 Analysis: Daryl started this task on the article about Henrik Stenson and clicked on the jobs tab on the upper right hand side of the page in the navigation bar. There he searched for digital media in the search bar and pressed enter on the keyboard, which found a job listing for jobs, which met the criteria for digital media. Daryl failed to find a job for NYTimes.com or to e-mail the posting to a friend. Task 7 Analysis: Daryl went back to the homepage for task number seven. For this task, he searched bill cunningham in the search bar in the top left corner of the homepage screen and pressed enter on his keyboard. This search brought up a page that led to multiple articles about Bill Cunningham and once Daryl scrolled to the middle of the page, where he found a collage of videos by Bill Cunningham. Daryl clicked on the link Bill Cunningham- Getting Real and watched the video. He successfully completed task seven. Task 8 Analysis: Daryl started at the homepage for task eight. Once the question was read, he typed in new york times sept 23 2013 into the search box in the upper left hand corner of the page and pressed enter on the keyboard. When the search results appeared, he quickly scrolled through the list of articles all the way to the bottom of the page without finding what he wanted. He scrolled back to the top of the search results page using the scroll wheel on his mouse and added the words front page to his search of new york times sept 23 2013. After pressing enter, he clicked on the first article that appeared in the results, which lead him to an opinion page. He clicked the back arrow located in the upper left hand corner of his browsing window, which led him to his search result list again. He then added, print edition into his search of new york times sept 23 2013 front page. This search brought up a blank page with no results found. Daryl did not complete task eight. Task 9 Analysis: The advertisements that appeared on the homepage of NYTimes.com did not accommodate to Daryls needs, according to him. His advertisements included a Citi bank mobile advertisement, which was located above the fold on the right hand side of the page just below the navigation bar, as well as an advertisement for Sothebys Business Institute of Art, located near the bottom of the homepage. After claiming that he didnt know how to turn ads off, Daryl scrolled from the top of the homepage of NYTimes.com to the bottom of the homepage looking for an options link. After being unsuccessful in his search, he decided to hover over the ad with his mouse, which led to

an appearance of a text on the advertisement, which read, dont show. Daryl clicked this text and the advertisement disappeared. He successfully completed task nine. Task 10 Analysis: Daryl started this task on the homepage of NYTimes.com. He began to look for Total Recall by searching total recall in the search box in the upper left hand corner of the page. After pressing enter on the keyboard, he glanced at the results that were given and chose to click on the sponsored link in a blue box that was located right above the search results for NYTimes.com. This led him to WebCrawler.com, which gave him a list of places to download Total Recall to a mobile phone. Daryl unsuccessfully completed this task.

Daryls completion rate for the usability test of NYTimes.com was 60%. This is below average for a successful completion rate, which is 78%. This is somewhat surprising, since he fits the demographic of NYTimes.com very well. Usability Test B: Linda Morrison Testing Environment Physical Environment: Lindas preferred browsing environment is on her kitchen island. This is a good spot for her because due to an open layout in her house, she is still able to watch television and talk to her husband. The lights above the kitchen island are turned on as she browses. Her dog and grandchildren were a distraction on the day of her usability test, which is a normal distraction in her everyday life. The television was kept on while she browsed the Internet, and she occasionally stopped to catch what was going on in the living room. Technical Environment: Linda uses an HP laptop with an A8 Vision AMD Processor. She keeps the laptop plugged in to the wall to charge at all times. She does not like using the track pad that is built into the laptop itself, so she always carries a portable USB wireless mouse to plug in. Her laptop uses Windows 8, and has a screen resolution set to 1600 x 900. Since she tends to not use a desk or office when using her laptop, Linda uses her home Wi-Fi to connect to the Internet. She uses Internet Explorer, and does not have any browser plug-ins. Test Results Task 1 Analysis: Linda starts task one on Google.com. She searches North Korea into the search bar and clicks on the search button with her mouse. She then scrolls through the results for the search North Korea and finds an article relating to Koreas nuclear weapon suspension that is posted on NYTimes.com. She clicks on the link and reads about their weapon suspension. Linda successfully completed task one. Task 2 Analysis: Once I have told Linda what site we would be testing, she accessed NYTimes.com by searching nytimes into Google.com. She finds the link to the website and is relocated to the homepage. She then looks at the very first featured article on the homepage, which is located in the center of the screen above the fold. The article is called The ghost of Mayors past and is accompanied with a cartoon picture. Linda has successfully completed task two.

Task 3 Analysis: Linda looks for task three by scanning above the fold of the homepage of NYTimes.com looking for Anspacher theatre. After an unsuccessful glance of the homepage, she locates the search bar located in the upper left hand corner. She types in ann sparker theater and presses enter on her keyboard, which does not bring up any search results or suggestions. She becomes frustrated noting, Ugh, it should have brought it up! I dont see anything! She goes back to the search bar from her search results page and types in sting tickets in new york and sighs because none of the search results have come up with what she is looking for after scrolling through the list on the first page of results. She scrolls back up to the top of the search results page and sees the blue box with sponsored links. She hovers over and clicks on the info.com banner for more information on Sting. This then leads her to Sting.com, where her browser freezes and she is unable to get out of the page. Linda unsuccessfully completed task three. Task 4 Analysis: After reading task four aloud, Linda wonders if she should go to Media or the Public and editor tabs in the navigation bar, but decides to click on the magazine tab on the left hand side of the homepage and scrolled through the magazine section of NYTimes.com. Scrolling from the top to the bottom of the page, she makes note that there is nothing there about Maxim and scrolls back to the top to the search bar in the top left hand corner. She types in maxium magazine and is lead to a page that says in bolded text Did you mean: Maxim Magazine, which she clicks on. After clicking on this suggestion, she is brought to a page that has no search results related to the topic of a Maxim purchase. Linda failed to complete this task. Task 5 Analysis: Linda begins this task on the homepage of NYTimes.com. She uses the search box located in the top left hand corner of the page to search Heindrick Stenston and presses the search button next to the box. This brings up articles related to Henrich Stenson and she is able to find that he is a golfer. She scrolls through the top and bottom of the page to look for an article relating to his recent participation in events, and clicks on a sponsored link located in a blue box at the top of the search results page that leads to info.com. Linda was unable to find the latest event Henrich Stenson participated in, but was able to find which sport he played. She partially completed this task. Task 6 Analysis: Linda stays on the Heindrick Stenston search results page for this task. She deletes the Heindrick Stenston search with the backspace bar on her keyboard and types in NEW YORK TIMES MEDIA JOBS in all caps. After pressing enter on the keyboard, she comes across the sort by text option at the top of the search results list that allowed her to sort her search by newest, oldest, and relevance. She chose to sort by newest and then clicked on the sponsored link for localjobindex.com. Linda was unable to find a digital media job for NYTimes.com. Task 7 Analysis: For this task, Linda started on NYTimes.com homepage. She went to the search bar in the upper left hand corner of the page and searched bill cunningham, after pressing enter she was led to a search result page that shows many options of articles and videos that relate to Bill Cunningham. She browses through the first few options and then decides to narrow her search by retyping into the search box bill Cunningham getting real. She clicks on the first search result, which leads to a playlist of Bill Cunningham videos. The first one that plays is Bill Cunningham: Forever Plaid

and she notices a banner of videos that are up next in the playlist. The next one is Bill Cunningham- Getting Real and she is able to hover over it, click it with her mouse, and watch the video. Linda passes task seven. Task 8 Analysis: Linda starts at the homepage of NYTimes.com for this task. She uses the search box located in the top left hand corner of the page and types in print edition 9/23/13 and presses enter on her keyboard, which does not bring up any results. She sighs, becoming frustrated with the search process. Linda was unable to complete task eight. Task 9 Analysis: Linda believes the ads that are located on the homepage of the NYTimes.com are catered to her needs. Above the fold of the front page, the Citi bank advertisement lets her know that there is a mobile app for her to use. On the very bottom of the homepage, an advertisement for Sothebys Business Institute of Art is located in the bottom right hand corner. This advertisement does not pertain to Linda. To opt out of the ads, Linda scrolls and browses through the homepage to find something that says opt out. When this is unsuccessful, she becomes frustrated and is unable to finish the task. Task 10 Analysis: For the last task, Linda starts above the fold of the homepage of NYTimes.com. She types in the search box total recall the game and hits enter on her keyboard. When the results come up, she notices the sponsored links in the blue box in the center of the page. She clicks on the link to Yideo.com, which gives her a link to a free trial of the game. Linda has passed the last task.

Lindas completion rate was 45%. This falls below the average successful completion rate of 78%. Testing Conclusions Similarities & Differences: The two testers for the NYTimes.com usability test were similar in that they both used Google.com to access their news about North Korea for task one. They also utilized the search bar on NYTimes.com to complete most of their tasks. The differences came in how they perceived what the web pages were giving them. Daryl was better at problem solving when he came into a problem with his tasks. Linda wanted to have an easy answer for every question she had with her task, and found great difficulties when it was not the first article that popped up after her search. Another difference was that Daryl experienced one pop up advertisement while browsing NYTimes.com, and Linda did not. This could be due to a pop up blocker on Lindas browser, or due to the differences in browsers the two testers used. Completion rate: Compared to the standard 78% successful completion rate, both testers were below the average. Daryl, the high experienced user who uses computers for a living, only reached a completion rate of 60%. Although he searched in depth for what he was looking for, the website still lacked the help needed to get to a satisfying result 40% of the time. He comes close to a successful completion of 78%, and would have had an easier time with the tasks had the NYTimes.com had a more informative layout. For Linda, it was difficult for her to find what she was looking for when the search field did

not give her better search options to choose from besides what she was typing in. This resulted in her low completion rate of 45%. Errors: A big error that Linda came across was that occasionally, her searches on NYTimes.com would come up with no results. She was not given a narrative search option, and was left to go back to the home page and start over using a different search. This could have been a prevention on the sites end by having a better way to track what a user is trying to search for, or as Google.com does, give search options as they type in their search as to avoid misspelling. Recommendations: To improve the site, NYTimes.com should make a better user interface for the user to use. Although it looks like a print newspaper would, it is not very intuitive when it comes to where things are located. Daryl, a prime audience member for the site, was at first unable to come across the search bar. For Daryl, a more user-friendly and intuitive layout would have helped him find what he was looking for. The navigation bars changing locations from page to page made Daryl take longer time to find the tab and section of the website he was looking for. According the Jakob Nielsons Heuristics for good usability, an error prevention system is an effective way to eliminate conditions in which the user is unable to get out of. This would help in Lindas case when she would search for a certain criteria and nothing would come up. An error message saying no search found would help a user understand what is going on, instead of wasting their time as they wait for something to happen. When using the search feature of the website, for older users, the suggested links blue box becomes a distraction, and they are unable to find what they are needing without going off onto another website. This could be avoided by placing the suggested links box in a lower spot on the page, possibly below the fold.