Advancing usability and functionality of electronic dental records

Titus Schleyer, DMD, PhD INFO-I 667 October 10, 2013

Big Hole Valley, Montana, 9/2001

The plan
What do dentists actually do? Dental care process, documentation and decision-making Learn (or practice) a skill: heuristic evaluation Usability of four electronic dental record systems Designing a novel interface based on a cognitive task analysis
2 of 68

What do dentists actually do?

3 of 68

Patient care in dentistry and medicine

Patient
Initial visit # of diagnoses # of therapies Tx type Life&death situations Dentistry doesnt feel sick lots of data few many mechanical/ biological few Medicine feels sick few data many few biological/ mechanical many

4 of 68

Dental care process

Emergency patient

Assessment and diagnosis

Regular patient

Treatment plan

Health status maintenance

Treatment

5 of 68

Clinical documentation
medical and dental history, incl. chief complaint extraoral and intraoral exam radiographs images problem list treatment plan progress notes
6 of 68

Medical history

7 of 68

Medication history

8 of 68

Hard tissue exam and radiology report

9 of 68

Periodontal exam

10 of 68

Radiographs

11 of 68

Images

me

12 of 68

Treatment plan

13 of 68

Progress notes

14 of 68

Differences in Delivery System

# practitioners Generalist/ specialist Solo/group practice Hospital affiliation Annual $ Dentistry 166,000 70/30 62/38 few $60b Medicine 813,000 35/65 ? many $1,200b
15 of 68

Based on what you know and/or have learned so far, what are possible research issues for electronic dental records? [Please take about 5 min. to record your observations.]

16 of 68

Usability of electronic dental records

Grand Tetons, Wyoming, 10/2001

17 of 68

Learn (or practice) a skill: Heuristic evaluation

Ligonier, Pennsylvania, 10//2004

18 of 68

What is heuristic evaluation?

small set of evaluators examines the interface judge its compliance with usability principles (heuristics) heuristic violations merged into master list

Nielsen J, Mack RL. Usability Inspection Methods, John Wiley & Sons, Inc, New York 1994.

19 of 68

Significance of heuristic evaluation

discount usability engineering method identifies usability problems in the interface with reference to recognized usability principles cheap and easy to conduct

20 of 68

Heuristic evaluator training

Madison River Valley, Montana T. Schleyer 2001

21 of 68

Visibility of system status

charting MOD amalgam

22 of 68

Match between system and real world

23 of 68

User control Flexibility and efficiency of use

report preview

24 of 68

Consistency and standards

25 of 68

Error prevention

extracting a tooth

26 of 68

Recognition rather than recall

switching from hard tissue to perio chart

27 of 68

Flexibility and efficiency of use

28 of 68

Esthetic and minimalist design

29 of 68

Help users with errors

family schedule

30 of 68

Help and documentation

chart legend

31 of 68

Your turn!
32 of 68

Match between system and real world

33 of 68

Error prevention

prescribing Amoxicillin

34 of 68

Visibility of system status

moving an appointment

35 of 68

Esthetic and minimalist design

36 of 68

Flexibility and efficiency of use

accessing the perio chart

37 of 68

Consistency and standards Flexibility and efficiency of use

saving a prescription

38 of 68

Help users with errors

family schedule

39 of 68

Consistency and standards

40 of 68

Limitations of heuristic evaluation

tells you about problems, but not how to fix them finds different types of errors than usability testing not very reliable*

*W. D. Gray and M. C. Salzman. Damaged merchandise? A review of experiments that compare usability evaluation methods. Hum-Comput Interact 13 (3):203-261, 1998.

41 of 68

Usability of electronic dental record systems

Research questions:
Which usability problems are common in dental software? What is the rate of completed, incorrectly completed and incomplete user tasks?

Thyvalikakath T et al. A usability evaluation of four commercial dental computer-based patient record systems. J Am Dent Assoc 2008 Dec;139(12):1632-42

42 of 68

Record a missing tooth correct path

43 of 68

45 of 68

Results

46 of 68

User adverse events

70 60 50 40 30 20 10 0
ES PW DX SD

47 of 68

Usability of electronic dental records

Significant usability problems due to:
complex information design mismatch between system and user model difficulty in finding functionality

significant cognitive effort required that did not contribute to task completion

48 of 68

Designing a novel interface based on a cognitive task analysis

Research questions:
What cognitive processes do dentists engage in and what information do they use when they treat a patient? How can a new design support these processes better than existing EDRs?

Thyvalikakath T et al. Advancing cognitive engineering methods to support user interface design for electronic health records (in review)

49 of 68

Methods
think-aloud method with 3 standardized patient cases 5 dental faculty and 5 general dentists analyze the sessions to determine:
information that dentists requested sequence in which they reviewed information instances of concurrent review of information items

50 of 68

Results
Average of patient data accessed in 10 high complexity cases
35 Average number of times each patient data accessed 30

25

20

15

10

0 0% Start of 10% 20% 100% Average session time in percentage (normalized) for a case of high complexityEnd of session
Social History Medical Consult Radiographs Oral Pathology Consult Medical History Dental History Hard Tissue Chart Treatment Plan Notes

30%

40%

50%

60%

70%

80%

90%

Patient Demographics Medications Intra-oral Images Periodontal Exam

51 of 68

Results (cont.)
Average of patient data accessed in 10 low complexity cases
20 Average number of times each patient data accessed 18 16 14 12 10 8 6 4 2 0 0% Start 10% 20% 100% End of Average session time in percentage (normalized) for a case of low complexity sessio
Medical History Radiographs Treatment Plan Notes Dental History Hard Tissue Chart Extra-oral Images Periodontal Exam

30%

40%

50%

60%

70%

80%

90%

Patient Demographics Intra-oral Images Financial Informaton

52 of 68

Visualization of information artifacts used over time

53 of 68

Prototype: The DMD Project

54 of 68

A novel approach to designing EDRs

55 of 68

56 of 68

57 of 68

58 of 68

59 of 68

60 of 68

61 of 68

62 of 68

63 of 68

Results

scale: 0 (least satisfaction) - 9 (most satisfaction)

64 of 68

Results

65 of 68

A final thought
"There is a strong temptation always to keep our eyes on the faraway, glittering objectives, and forget the rubbish accumulated at our feet. It is a wrench to turn from the great problems that flatter our dignity by their mere contemplation to give our attention to the relatively insignificant but very effective thongs that bind our ankles at this moment. But unless we can do that, and rid ourselves of immediate impediments, we shall stay planted exactly in one spot. It is not merely that we wont be able to advance toward all-important, distant goals; we may be overwhelmed where we stand by tides of disaster we are unable either to turn back or escape."
66 of 68 ~Walter Dorwin Teague~

Thank You for Your Attention!

Questions, comments?
Thanks to:

Please visit us at:

http://www.regenstrief.org
H. Spallek T. Thyvalikakath H. Torres-Urquidy

Twitter
P. Hernandez

Facebook /titusschleyer
J. Irwin A. Acharya

Scribd
Margate, New Jersey, 8/1999

M. Song

67 of 68

Thank you for your attention!

Download this talk: Contact info:
schleyer@regenstrief.org

68 of 68

References
Thyvalikakath TP, Schleyer TK, Monaco V. Heuristic evaluation of clinical functions in four practice management systems: a pilot study. J Am Dent Assoc. 2007 Feb;138(2):209-10, 212-8. Thyvalikakath T et al. A usability evaluation of four commercial dental computer-based patient record systems. J Am Dent Assoc 2008 Dec;139(12):1632-42 Thyvalikakath T et al. Advancing cognitive engineering methods to support user interface design for electronic health records (in review) http://www.dmdproject.com
69 of 68