Behavioral Economics Decision Support

Teaching Bayesian Reasoning

Birte Grger

Agenda
Teaching Bayesian Method in Less Than Two Hours

1. 2. 3. 4. 5. 6.

Bayesian Method/Inference Information Formats Teaching Methods Training Effectiveness Studies and Experiments Results and Conclusion

Bayesian Method/Inference
Bayes Rule in Theory

Named after Thomas Bayes, published 1763 Describing conditional probabilities (A|B) given another event (B) Update beliefs in light of new evidence Transfer prior probability P(A) into posterior probability

Bayesian Method/Inference
The Problems

Studies show: Bayesian inference is alien to human inference

Neglect or overweighing of base rates (conservatism) Cognitive illusions = systematic deviations

Studies attempting to teach Bayesian reasoning with no success

Information Formats
Probability vs. Natural Frequencies

Cognitive algorithms work on information information needs representation format Mathematical probability and percentage = recent developments Input format for human minds: natural frequencies

Information Formats
Crucial Theoretical Results

1. Bayesian computations = simpler, when information represented in natural frequencies 2. Natural frequencies = corresponding to the information format encountered throughout most of our evolutionary development

Information Formats
Example Comparison Mammography Problem The probability that a woman who undergoes a mammography will have breast cancer is 1%. If a woman undergoing a mammography has breast cancer, the probability that she will test positive is 80%. If a woman undergoing a mammography does not have cancer, the probability that she will test positive is 10%.

Ten of every 1,000 women who undergo a mammography have breast cancer. Eight of every 10 women with breast cancer who undergo a mammography will test positive. Ninety-nine of every 990 women without breast cancer who undergo a mammography will test positive.

Teaching Methods
Overview

Teaching: showing people how to construct frequency representations Mechanism: tutorial, practices, feedback

Rule Training

Frequency Grid

Frequency Tree

Teaching Methods
Rule Training

Explanation how to extract numerical information by computer system Translation of base-rate information in components of Bayes formula Insert probabilities Calculation of result

Teaching Methods
Rule Training

Teaching Methods
Frequency Grid

Representation cases by squares Indicate squares according to base rates

Shaded percentage of population Circled pluses (+) for hit rate on shaded squares Circled pluses for false alarm rate on non-shaded squares

Calculate ratio: pluses in shaded squares divided by all circled pluses

Teaching Methods
Frequency Grid

Teaching Methods
Frequency Tree

Constructing reference class and breaking-down into four subclasses System: explanation how to obtain frequencies Inserting into corresponding nodes Calculation by dividing number of true positive by sum of all positives

Teaching Methods
Frequency Tree

Training Effectiveness
Evaluation

Explanation of program and instructions Answer format/solution as a formula Systematically varied order of problems Scoring criteria
strict
Match exact value Obscure fact that participants created sound but inexact response

liberal
Match value +/- 5% Increased possibility including non-Bayesian algorithms

Training Effectiveness
Measures

Comparing solution rates

At baseline (w/o training Test 1) Immediately after training (Test 2) About a week after training (Test 3) 1 to 3 months after training (Test 4)

Traditional: steep decay curve Expectation now: decay not as quick with frequency training

Studies and Experiments

Structure

Study 1a 62 University of Chicago students 4 groups in 3 training methods and one w/o training as control All 4 tests with 10 problems each Old and new problems High attrition rates (increasing # of participants)

Study 1b 56 Free University of Berlin students Prevent high attrition rates with later payments and bonus based on results 2 groups with the different frequency trainings Reduced number of problems No attrition

Study 2 72 University of Munich students Issue of used graphical aids in frequency conditions Longer period of time between Test 3 and 4 Use also graphical aid for rule training probability tree

Studies and Experiments

Results Study 1a

Substantial improvement in Bayesian reasoning

High level of transfers: average performance in new problems almost as god as in old problems Increase in median number of inferences in the frequency grid condition

Studies and Experiments

Results Studies 1b and 2

Study 1b

Study 2

Conclusion
Teaching Bayesian Reasoning is possible

Prove that Bayesian computations are simpler using natural frequencies Environmental change illusions Idea: teach people to represent information according to cognitive algorithms Translation in representation format = major tool for helping to attain insight High immediate effects, better transfer to other problems and long-term stability