Evaluation Framework for User Interface Design

AnEvaluationFramework
Basedontheslidesavailableatwww.idbook.com
Theaims

Explainkeyevaluationconcepts&terms Describetheevaluationparadigms& techniquesusedininteractiondesign Discusstheconceptual,practicaland ethicalissuesthatmustbeconsideredwhen designingevaluations IntroducetheDECIDEframework
Evaluationparadigm
Anykindofevaluationisguidedexplicitely orimplicitelybyasetofbeliefs,whichare oftenunderpinedbytheory.Thesebeliefs andthemethodsassociatedwiththemare knownasanevaluationparadigm
Userstudies
Userstudiesinvolvelookingathowpeople behaveintheirnaturalenvironments,orin thelaboratory,bothwitholdtechnologies andnewones
Evaluationparadigms

Quickanddirty Usabilitytesting Fieldstudies Predictiveevaluation
Quickanddirty
Quickanddirtyevaluationdescribesthe commonpracticeinwhichdesigners informallygetfeedbackfromusersor consultantstoconfirmthattheirideasare inlinewithusers'needsandareliked Quickanddirtyevaluationsaredoneany time Theemphasisisonfastinputtothedesign processratherthancarefullydocumented findings
Usabilitytesting
Usabilitytestinginvolvesrecordingtypical users'performanceontypicaltasksin controlledsettings. Astheusersperformthesetaskstheyare watched&recordedonvideo&theirkey pressesandmouseclicksarelogged Thisdataisusedtocalculateperformance times,identifyerrors&helpexplainwhy usersdidwhattheydid Usersatisfactionquestionnaires& interviewsareusedtoelicitusers'opinions
Fieldstudies

Fieldstudiesaredoneinnaturalsettings Theaimistounderstandwhatusersdo naturallyandhowtechnologyimpactsthem Indesign,fieldstudiescanbeusedto:

Identifyopportunitiesfornewtechnology Determinedesignrequirements Decidehowtobestintroducenewtechnology Evaluatetechnologyinuse
Predictiveevaluation
Expertsapplytheirknowledgeoftypical usersoftenguidedbyheuristics,topredict usabilityproblems Anotherapproachinvolvestheoretically basedmodels Akeyfeatureofpredictiveevaluationisthat usersneednotbepresent Relativelyquickandinexpensive
Overviewoftechniques

Observingusers Askinguserstheiropinions Askingexpertstheiropinions Testingusers'performance Modelingusers'taskperformancetopredict theefficacyofauserinterface IMPORTANT:sometechniquesareusedin differentwaysindifferentevaluation paradigms
DECIDE:aframeworktoguide evaluation

Determinethegoalstheevaluationaddresses Explorethespecificquestionstobeanswered Choosetheevaluationparadigmandtechniquesto answerthequestions Identifythepracticalissues Decidehowtodealwiththeethicalissues Evaluate,interpretandpresentthedata
Determinethegoals
Whatarethehighlevelgoalsofthe evaluation? Whowantsitandwhy? Thegoalsinfluencetheparadigmforthe study Someexamplesofgoals:

Identifythebestmetaphortobasethedesign Checktoensurethatthefinalinterfaceis consistent Investigatehowtechnologyaffectsworking practices
Explorethequestions
Allevaluationsneedgoals&questionstoguide themsotimeisnotwastedonilldefinedstudies Forexample,thegoaloffindingouthowmany customersprefertopurchasepaperairlinetickets ratherthaneticketscanbebrokendowninto subquestions: Whatarethecustomers'attitudestothesenew tickets? Aretheyconcernedaboutsecurity? Istheinterfaceforobtainingthempoor? Whatquestionsmightyouaskaboutthedesignof acellphone?
Choosetheevaluationparadigm &techniques
Theevaluationparadigmstrongly influencesthetechniquesused,howdatais analyzedandpresented E.gfieldstudiesdonotinvolvetestingand modeling
Identifypracticalissues
Forexample,howto:

Selectusers Stayonbudget Stayingonschedule Findevaluators Selectequipment
Decideonethicalissues

Developaninformedconsentform Participantshavearightto:

Knowthegoalsofthestudy Whatwillhappenwiththefindings Privacyofpersonalinformation Nottobequotedwithouttheiragreement Leavewhentheywish Betreatedpolitely Reviewdocumentation,approve
Uvichumanethicsboard
Evaluate,interpretandpresent data
Howdataisanalyzedandpresented dependsontheparadigmandtechniques used Thefollowingalsoneedtobeconsidered:

Reliability:canthestudybereplicated? Validity:isitmeasuringwhatyouthought? Biases:ittheprocesscreatingbiases? Scope:canthefindingsbegeneralized Ecologicalvalidity:istheenvironmentofthe studyinfluencingite.gHawthorneffect(oreasy midtermeffect)
PilotStudies

Asmalltrialrunofthemainstudy Theaimistomakesureyourplanisviable Pilotstudiescheck:

Thatyoucanconducttheprocedure Theinterviewscripts,questionnaires, experiments,etc.workappropriately It'sworthdoingseveraltoironoutproblems beforethemainstudy Askcolleaguesifyoucan'tsparerealusers
Keypoints
Anevaluationparadigmisanapproachthatis influencedbyparticulartheoriesandphilosophies Fivecategoriesoftechniqueswereidentified: observingusers,askingusers,askingexperts,user testing,modelingusers TheDECIDEframeworkhassixparts: Determinetheoverallgoals Explorethequestionsthatsatisfythegoals Choosetheparadigmandtechniques Identifythepracticalissues Decideontheethicalissues Evaluatewaystoanalyzeandpresentdata DOAPILOTSTUDY

Evaluation Framework for User Interface Design

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Evaluation Framework for User Interface Design

Diunggah oleh

Hak Cipta:

Format Tersedia

AnEvaluationFramework

Explainkeyevaluationconcepts&terms Describetheevaluationparadigms& techniquesusedininteractiondesign Discusstheconceptual,practicaland ethicalissuesthatmustbeconsideredwhen designingevaluations IntroducetheDECIDEframework

Anykindofevaluationisguidedexplicitely orimplicitelybyasetofbeliefs,whichare oftenunderpinedbytheory.Thesebeliefs andthemethodsassociatedwiththemare knownasanevaluationparadigm

Userstudiesinvolvelookingathowpeople behaveintheirnaturalenvironments,orin thelaboratory,bothwitholdtechnologies andnewones

Quickanddirty Usabilitytesting Fieldstudies Predictiveevaluation

Fieldstudiesaredoneinnaturalsettings Theaimistounderstandwhatusersdo naturallyandhowtechnologyimpactsthem Indesign,fieldstudiescanbeusedto:

Identifyopportunitiesfornewtechnology Determinedesignrequirements Decidehowtobestintroducenewtechnology Evaluatetechnologyinuse

Expertsapplytheirknowledgeoftypical usersoftenguidedbyheuristics,topredict usabilityproblems Anotherapproachinvolvestheoretically basedmodels Akeyfeatureofpredictiveevaluationisthat usersneednotbepresent Relativelyquickandinexpensive

Observingusers Askinguserstheiropinions Askingexpertstheiropinions Testingusers'performance Modelingusers'taskperformancetopredict theefficacyofauserinterface IMPORTANT:sometechniquesareusedin differentwaysindifferentevaluation paradigms

Determinethegoalstheevaluationaddresses Explorethespecificquestionstobeanswered Choosetheevaluationparadigmandtechniquesto answerthequestions Identifythepracticalissues Decidehowtodealwiththeethicalissues Evaluate,interpretandpresentthedata

Whatarethehighlevelgoalsofthe evaluation? Whowantsitandwhy? Thegoalsinfluencetheparadigmforthe study Someexamplesofgoals:

Identifythebestmetaphortobasethedesign Checktoensurethatthefinalinterfaceis consistent Investigatehowtechnologyaffectsworking practices

Theevaluationparadigmstrongly influencesthetechniquesused,howdatais analyzedandpresented E.gfieldstudiesdonotinvolvetestingand modeling

Selectusers Stayonbudget Stayingonschedule Findevaluators Selectequipment

Knowthegoalsofthestudy Whatwillhappenwiththefindings Privacyofpersonalinformation Nottobequotedwithouttheiragreement Leavewhentheywish Betreatedpolitely Reviewdocumentation,approve

Howdataisanalyzedandpresented dependsontheparadigmandtechniques used Thefollowingalsoneedtobeconsidered:

Reliability:canthestudybereplicated? Validity:isitmeasuringwhatyouthought? Biases:ittheprocesscreatingbiases? Scope:canthefindingsbegeneralized Ecologicalvalidity:istheenvironmentofthe studyinfluencingite.gHawthorneffect(oreasy midtermeffect)

Asmalltrialrunofthemainstudy Theaimistomakesureyourplanisviable Pilotstudiescheck:

Thatyoucanconducttheprocedure Theinterviewscripts,questionnaires, experiments,etc.workappropriately It'sworthdoingseveraltoironoutproblems beforethemainstudy Askcolleaguesifyoucan'tsparerealusers

Anda mungkin juga menyukai