EPID 600; Class 11

Screening

University of Michigan School of Public Health

1
 The New York Times
Sunday, October 31, 1999; pg. 5

Bedtime stories. Telephone bills. Life as usual. It ends quickly with the trauma of a breast biopsy –
even though most breast biopsies turn out to be benign. This fact has inspired clinical trials of an
adjunctive breast screening device designed to distinguish benign from malignant lesions without a
breast biopsy. So, life can return to normal for a little sooner for everyone. We invite you to help us.
If you’re scheduled for a breast biopsy, ask your doctor about participating in our clinical trials. 2
 Why screen?

To find people with the disease (or at risk of the disease)

who don’t know it
In other words…to find people who are pre-symptomatic

3
 Why try to find asymptomatic diseased
people?
To treat disease
To cure disease
To prevent disease spread
To slow down disease progress
To study disease natural history

4
 Different from identifying people at risk
but without disease
Identifying people at risk of disease but without disease is
done to prevent the disease altogether, to delay disease
onset, or to study the “precondition” state

5
 Additional thought...

Why else might we encourage screening or promote a

specific screening test?

6
 Additional thought...

Why else might we encourage screening or promote a

specific screening test?

Because we want to do something

Because we can
For money, fame, and glory

7
 A digression (1)....

What is a disease?
Colon cancer
Myocardial infarction

What is a condition?
High blood pressure
High cholesterol

What is a marker?
High Prostate Specific Antigen
8
 A digression (2)...

Binary tests
Yes vs No

Continuous measures
Multiple values; may require the choice of a cutoff point

9
 A note: primary vs. secondary
prevention
Primary prevention
Screening that aims to identify risk factors or etiologic
factors for disease so that disease occurrence can be
prevented

Secondary prevention
The early detection of disease in the hope of improving
prognosis

10
 Natural history of a disease

Detectable
Preclinical Phase
(DPCP)

Onset Detectable Symptomatic Death

by
Screening

11
 Screening

Disease
?

Pos
POSITIVES
Test

Neg
NEGATIVES

12
 Screening

DISEASE
Yes No

Pos TP FP
TEST
Neg FN TN

13
 Sensitivity (Sn)

Probability of test positive if disease is present

TP true positives
Sn = =
TP + FN everyone with disease

14
 Specificity (Sp)

Probability of a negative test if disease is not present

TN true negatives
Sp = =
TN + FP everyone without disease

15
 Sensitivity and specificity

Sensitivity and Specificity are characteristics of TEST itself,

i.e., how good is the test
Changing cutoffs generally increases one at the expense of
the other

16
 Changing cutoffs

0 6 8 10 12 14
Disease

No Disease

Disease

Test Yes No

Pos (+) TP FP

Neg (–) FN TN
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No

Pos (+) 5 FP

Neg (–) FN TN
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No

Pos (+) 5 3
Neg (–) FN TN
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No

Pos (+) 5 3

Neg (–) 1 9
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No

Pos (+) 5 3

Neg (–) 1 9
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No
Sn = 5/(5+1) = 0.83
Pos (+) 5 3

Neg (–) 1 9 Sp = 9/(9+3) = 0.75

 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No

Pos (+) TP FP

Neg (–) FN TN
 Changing cutoffs

0 6 8 10 12 14
Disease

No disease

Disease

Test Yes No
Sn = 3/(3+3) = 0.50
Pos (+) 3 1

Neg (–) 3 11 Sp = 11/(11+1) = 0.92

Number Screened

Non-
Cases Cases

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Score on Screen
25
 Overlapping Area

Screening Level Set at >5

Number Screened

Screening Level Set at >7

Non-
Cases
Cases

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Score on Screen
26
 Issues about sensitivity vs. specificity

What is “gold standard” that actually determines if disease is

present or not?
Cost of false positives and false negatives
Anxiety/emotional distress
Inconvenience
Subsequent testing and mortality

27
Classification of test results

Disease

yes no

TP FP

FN TN

Sensitivity = Specificity =
TP TN
TP + FN FP +TN
28
 Characteristics of tests

Validity (accuracy)
How close does the test result get to the correct (true) number

Reliability (precision)
How close are repeat measurements on the same sample?

29
 Validity vs Reliability

Baby scale examples

• Well calibrated • Well calibrated • Scale 6oz off • Scale 6oz off
scale scale
• Allowed to settle • Not allowed to
• Allowed to settle • Not allowed to before measurement settle before
before measurement settle before recorded measurement
recorded measurement recorded
recorded

X
X
XX X X X X X
XXX X XX
XX
X
X
X
Truth = 8lbs Biased = 7lbs 6oz

Valid and Valid but not Not valid but Not valid and not
reliable reliable reliable reliable
30
 Four sources of variability

Biological variation
Test method itself
Intra-observer
Inter-observer

31
Example...blood pressure variability

BP Patient A Patient C

Lowest 86/47 123/78

Highest 126/79 153/107

Casual 108/64 137/103

32
 Question addressed so far...

If we screen a population, what percent of people with

the disease, and without the disease, will be correctly
identified by our test?

How well does the test work in a population?

33
 The clinical question however is

If a specific patient has a positive test, what is the

probability that this patient really has the disease?

34
 Screening...

DISEASE
Yes No

Pos TP FP
TEST
Neg FN TN

35
 Positive predictive value

Likelihood that disease is present IF test is positive

TP true positives
PPV = =
TP + FP all positives

36
 Negative predictive value

Likelihood that disease is NOT present IF test is negative

TN true negatives
NPV = =
TN + FN all negatives

37
 Classification of screening test results

Predictive TP
TEST pos TP FP Value
(positive) TP + FP
(Screening
Survey) Predictive
neg TN
FN TN Value
(negative) FN +TN

38
 PPV and NPV

PPV and NPV are characteristics of test and of disease

prevalence

PPV is influenced by disease prevalence and more by the

specificity of test*
The greater the prevalence and the specificity, the greater is the
PPV

NPV is influenced by disease prevalence and more by the

sensitivity of test*
The lower the prevalence and the greater the sensitivity, the
greater is NPV

*when disease is rare

39
 PPV, example 1

Disease Prevalence = 1%

True Status
Sick Not-Sick Total

+ 99 495 594
Test Result
- 1 9405 9406

Total 100 9900 10,000

Test Sensitivity = 99% Positive 99

Predictive = = 17%
Test Specificity = 95% Value 99 + 495

40
 PPV, example 2

Disease Prevalence = 5%

True Status
Sick Not-Sick Total

+ 495 475 970

Test Result
- 5 9025 9030

Total 500 9500 10,000

Test Sensitivity = 99% Positive 495

Predictive = = 51%
Test Specificity = 95% Value 495 + 475

41
 Relationship of disease prevalence
to predictive value of a positive test

True Status
Case Non-Case Total

Test Sensitivity = 99% + TP FP

Test Result
Test Specificity = 95% - FN TN

Total 10,000

Prevalence Rate = 1% Predictive Value (positive) = 17%

Prevalence Rate = 5% Predictive Value (positive) = 51%

42
 Classification of screening test results

Disease

yes no

Predictive TP
TEST pos TP FP Value
(positive) TP + FP
(Screening
Survey) Predictive
neg TN
FN TN Value
(negative) FN +TN

Sensitivity = Specificity =
TP TN
TP + FN FP +TN
43
 Epidemiologic approach to the
evaluation of screening programs
Key question: do patients benefit from early detection of disease?

1.  Can the disease be detected early?

2.  What are the sensitivity and specificity of the test?
3.  What is the predictive value of the test?
4.  How serious is the problem of false-positive results?
5.  What is the cost of early detection in terms of funds, resources, and
emotional impact?
6.  Are the subjects harmed by the screening tests?
7.  Do the individuals in whom disease is detected early benefit from
the early detection, and is there an overall benefit to those who are
screened?

44
 Mammography and mortality reduction

The US recommends annual screening for breast cancer for women

above age 40
From a public health perspective it may be argued that this is
justifiable only if screening reduces breast cancer mortality
If screening is offered to all women in the target group, no well
defined control group is available
A study was done in Denmark to examine the varying estimates of
breast cancer mortality reduction based on different control groups

45
Olsen et al. Estimating the benefits of mammography screening: the impact of study design. Epidemiology. 2007; 18: 487-492
 Mammography and mortality reduction

The study population included all women invited to screen

in Copenhagen from April 1991 to March 2001
The women were followed for breast cancer mortality
Person years at risk counted as date of first invitation until
date of death, emigration from Denmark, or end of follow-
up (March 2001)

46
Olsen et al. Estimating the benefits of mammography screening: the impact of study design. Epidemiology. 2007; 18: 487-492
 Mammography and mortality reduction

Control group 1: Concurrent regional. Women in the same

age group living in Denmark from April 1991-2001, outside
the region of organized screening programs
Control group 2: Local historical. These were women from
the same age group living at any time between April 1981
and March 1991 (10 years before the program)
Control group 3: Historical-regional. These women were in
the same age group and living in Denmark, from
1981-1991, living outside of the region that later
implemented organized screening programs

47
Olsen et al. Estimating the benefits of mammography screening: the impact of study design. Epidemiology. 2007; 18: 487-492
 Mammography and mortality reduction

1. Local historical. This analysis showed a reduction of

20%; the “lesser benefit” was probably due to the increase
in incidence in breast cancer over time
2. Concurrent regional. This analysis yielded a reduction in
breast cancer mortality of 9%. Breast cancer incidence and
mortality was higher in Copenhagen than in the rest of
Denmark before screening.
3. Historical regional. This analysis estimated a 25%
decrease in breast cancer mortality. This controlled for time
and region. Probably the best method.

48
Olsen et al. Estimating the benefits of mammography screening: the impact of study design. Epidemiology. 2007; 18: 487-492
 Factors influencing epidemiologic approach to
the evaluation of screening programs

1.  Natural history of disease

2.  Pattern of disease progression
3.  Methodologic issues
4.  Study designs for evaluation of screening
5.  Problems in assessing sensitivity and specificity of tests
6.  Interpreting study results that show no benefit of
screening
7.  Cost benefit analysis of screening

49
 Natural history

To discuss
methodologic
issues involved in
evaluating the
benefit of screening,
we need to
understand natural
history of disease

50
Natural history

51
Pattern of disease progression

52
 Methodologic issues

There are concerns particular to screening and an understanding of

why decisions about whether or not to use screening tests are
controversial requires consideration of the biases that can arise with
screening

Detection
Lead time bias
Length time bias

53
 Detection

Screening appears to have a positive effect since disease precursor is

detected in persons who would not ultimately develop symptoms or die
from the disease

Screening dx

Initiation Death from

Disease NO Clinical other causes
Detectable Symptoms
by Screening NO Complications
from disease

54
 Detection

Screening appears to have a positive affect since disease

precursor is detected in persons who would not ultimately
develop symptoms or die from the disease

Example: Blood pressure screening leads to people with high

blood pressure being told that they have hypertension. While
people with hypertension are more likely to develop diseases
such as stroke, not all of them will.

55
 Lead-time bias

Survival appears to be increased among screen-detected cases

because diagnosis was made earlier in the disease

Screening dx Usual dx

Death
Initiation Disease Clinical Complications
detectable symptoms from the disease
by screening

56
 Lead-time bias

Screening for lung cancer with chest X-rays is an example of lead time
bias. When tumors can be detected earlier, screening will seem to
prolong life compared to persons who are not screened and in whom
disease is detected later

Lead Time Bias

Positive Screening
Outcomes

57
Lead-time bias and 5 year survival

58
 Length-time bias

People with a more protracted preclinical phase have a greater

probability of coming to screening. If a protracted preclinical phase is
associated with a better prognosis or survivorship, then screening may
actually look better than it is because of its affiliation with a protracted
preclinical phase.

Death
Initiation Disease Detectable Clinical Complications from the
by Screening Symptoms disease

Death
Initiation Disease Clinical Complications from the
Detectable Symptoms disease
by Screening
59
 Length-time bias example

Example: Length-time bias may occur when carcinomas-in-situ are

picked up with breast screening. These may be slow-growing
precursors to cancer. Their early detection and treatment may appear
to improve mortality from the disease.

60
 Epidemiologic study designs to evaluate
screening
Non randomized studies
Case-control
Individuals with and without disease are compared; controls should be
representative of the population from which disease cases emerged
Cohort
Compare the rate of disease in those who chose to be
screened vs. who choose not to be screened

Randomized studies
Randomized trials
Most evidence about the efficacy of screening comes from non-
experimental designs: randomize to screening vs. no screening and
compare rates of disease
61
 Problems in assessing the Sensitivity
and Specificity of tests

New screening programs are

frequently initiated after a
screening test becomes
available for the first time.
Usually claims are made (by
manufacturers of test kits,
investigators etc.) that the test
has high Sn and Sp. However,
not always easy to demonstrate.

62
 Interpreting study results that show no
benefit of screening
The apparent lack of benefit may be inherent in the natural history of
the disease (e.g., the disease has no detectable preclinical phase or
an extremely short detectable preclinical phase).
The therapeutic intervention currently available may not be any more
effective when it is provided earlier than when it is provided at the time
of usual diagnosis.
The natural history and currently available therapies may have the
potential for enhanced benefit, but inadequacies of the care provided
to those who screen positive may account for the observed lack of
benefit (that is, there is efficacy, but poor effectiveness).

63
 Cost-benefit analysis of screening

Cost issues when evaluating screening include financial but

also non-financial issues.

1.  There must be good evidence that each test or procedure

recommended is medically effective in reducing morbidity and
mortality
2.  The medical benefits must outweigh risks
3.  The costs of each test or procedure must be reasonable compared
to expected benefits
4.  The recommended actions must be practical and feasible

Source: American Cancer Society 64

 Screening conclusions

Screening assumes that we can do something with the

positive screen
There are real costs of false negatives and false positives
We should not be screening “just because we can”

65