Signal/noise analysis to compare tests for measuring visual field loss,

and its progression.

short title Signal/noise ratios in perimetry

words, figures, tables 3,650 words, 11 figures, 3 tables. IOVS proofs signed off Aug.’09

section & previous Section code GL, previously shown at the International Perimetric Society
presentations (IPS) meeting in Portland, Oregon in 2006

keywords perimetry, visual field, sensitivity, progression, glaucoma, FDT,

frequency doubling, Humphrey-Matrix

www.wordle.net (courtesy of J Feinstein)

authors & affiliation Paul H Artes & Balwantray C Chauhan

Ophthalmology and Visual Sciences
Dalhousie University,
Rm 2035, West Victoria,
1276 South Park St, Halifax, Nova Scotia
B3H 2Y9, Canada
paul@dal.ca

support No commercial relationships,

support from the E A Baker Foundation (PHA), and the
Canadian Institutes of Health Research (BCC, MOP-11357)

1
 Artes & Chauhan, Signal/noise ratios in perimetry

Abstract

Purpose
To describe a methodology for establishing signal/noise ratios (SNRs) for different perimetric
techniques, and to compare SNRs of Frequency-Doubling Technology (FDT2) perimetry and standard
automated perimetry (SAP).

Methods
Fifteen patients with open-angle glaucoma (median MD, -2.6 dB, range +0.2 to -16.1 dB) were tested 6
times with FDT2 and SAP (SITA Standard program 24-2) within a 4 week period. Signals were
estimated from the average superior-inferior difference between the Mean Deviation (MD) values in 5
mirror-pair sectors of the Glaucoma Hemifield Test, and noise from the dispersion of these differences
over the 6 repeated tests. SNRs of FDT2 and SAP were compared by mixed effects modelling.

Results
There was a moderate correlation between the signals of FDT2 and SAP (r2=0.68, p<0.001) but no
correlation of the noise (r2=0.01, p=0.16). Although both signal as well as noise estimates were larger
with FDT2 compared to SAP, 60-70% of sector pairs showed higher SNRs with FDT2. The SNRs of
FDT2 were between 20% and 40% larger than those of SAP (p=0.01). There were no meaningful
differences between parametric and non-parametric estimates of signal, noise, or SNR.

Conclusion
The larger SNRs of FDT2 suggest that this technique is at least as efficient as SAP at detecting localised
visual field losses. Signal/noise analyses may provide a useful approach for comparing visual field tests
independent of their dB-scales and may provide an initial indication of sensitivity to visual field change
over time.

2
 Artes & Chauhan, Signal/noise ratios in perimetry
Introduction

Visual field loss and its progression are hallmarks in glaucoma, optic neuritis, and other diseases.1
However, clinically important “signals” in the visual field are often small compared to the variability
between successive tests (“noise”). This applies to the detection of abnormalities with single
examinations as well as to the measurement of change over time with serial examinations.
Consequently, tests may have to be repeated several times before one can be certain that damage is
either present or absent, and at least 5 tests are required to detect change with any confidence.2-7 With
typical variability, at least 3 years of 6-monthly tests are needed to detect sight-threatening rates of
visual field progression.8, 9

Previous investigators have aimed to reduce variability through optimised threshold strategies,10-14
closer control over factors such as response bias, fatigue, and attention,15-19 and most notably through
new types of stimuli.20-24 Studies on retest variability, and on response variability estimated by
psychometric functions, have shown that several of the newer techniques do not suffer from the large
increase in variability in damaged fields seen with SAP but have nearly uniform variability across their
dynamic range.25-29

However, it is challenging to compare visual field data between one technique and another. It is
difficult, for example, to comparing threshold estimates from SAP to those of motion perimetry, and
although the technique’s scales can be made to appear similar by empirical “correction factors”, this is
more likely to conceal the problem rather than to solve it. Second, when visual fields change over time,
a 10 dB change with one technique may translate into a smaller or larger change with another technique,
and this relationship may not be constant but vary with the degree of damage. Moreover, techniques
differ in their measurement ranges – in a damaged area of the visual field one technique may still
provide useful threshold estimates while another only measures absolute losses (0 dB) that are not
informative for determining change.28 In combination, these issues limit the usefulness of current
methods for comparing different techniques.

Direct evidence that one technique performs better than another can only be obtained through
comparative studies with substantial numbers of patients, but even these studies do not always give
conclusive results. Because normal reference data for different techniques are usually obtained from
different samples of healthy controls, it can be difficult to compare probability maps from different
techniques.30, 31 Longitudinal studies are needed to compare effectiveness in measuring change over
time, but such studies are costly and often take several years.32-34 They may also be difficult to interpret
because no single technique provides an ideal reference standard.35 A method is therefore needed that
can provide clues to the potential merit of a new technique within a relatively short period of time.

3
 Artes & Chauhan, Signal/noise ratios in perimetry

In this work we propose a simple extension for analyses of retest data and demonstrate how perimetric
techniques may be compared even if they use different types of stimuli that do not share the same dB-
scale. The underlying rationale of this analysis is to compare the ability of the techniques to measure
systematic differences within a visual field, for example the asymmetries between the superior and
inferior areas that are characteristic of glaucomatous visual field loss.36 Differences in sensitivity, or
deviation from normal, between the superior and inferior mirror-pairs of sectors within a visual field can
be interpreted as a signal, and the variability of these differences from test to test can be interpreted as
noise. By estimating the ratio between signal and noise (signal/noise ratio, SNR), the ability of the
technique to identify localised visual field loss may be expressed independent of the dB-scale of the
instrument, enabling a paired comparison to be made between perimetric techniques.

Methods
Data
Fifteen patients with open-angle glaucoma (mean age, 66.3 years, range, 56.1-80.6 years) who had early
to moderately advanced visual field loss with SAP (median MD, -2.6 dB, range +0.2 to -16.1 dB) were
recruited from the clinics of the QEII Health Sciences Centre (Halifax, Nova Scotia, Canada). Inclusion
criteria were a clinical diagnosis of open-angle glaucoma, refractive error within 5 D equivalent sphere
or 3 D astigmatism, visual acuity better than or equal to 6/12 (+0.3 logMAR), and prior experience with
FDT1 perimetry and SAP. During a period of 4 weeks, one eye of each patient was tested 6 times with
FDT2 (24-2 threshold test) and 6 times with SAP (SITA Standard; 24-2 test), in randomized order. The
protocol was approved by the Queen Elizabeth II Health Science Centre Research Ethics Committee,
and all participants had given written informed consent. Full details of this dataset are described in a
previous paper on threshold and variability properties of the Humphrey-Matrix (FDT2) perimeter.26

Analysis
The threshold values of FDT2 and SAP were transformed to total deviation values using reference data
from healthy volunteers.37 For each test, we then averaged the total deviation values within the 10 visual
field sectors of the Glaucoma Hemifield Test 36 to obtain 10 sectoral Mean Deviation (sMD) values
(Fig. 1). Reduced major axis (RMA) regression 38 was used to estimate the relationship between the
sMDs of FDT2 and SAP. Unlike ordinary-least-squares regression, which assumes that the x-values are
from an “independent” variable measured without error, the line fitted by RMA regression minimizes
the residuals in both vertical as well as horizontal directions; it is therefore a more appropriate method
for establishing the slope of the relationship between FDT2 and SAP.

4
 Artes & Chauhan, Signal/noise ratios in perimetry

Fig. 1)

Stimulus locations of FDT2

(grey squares) and SAP (black
dots). The five visual field
sectors in the superior
hemifield which are compared
with their mirror images in the
lower hemifield are outlined in
red. The location of the blind
spot is shown by the black
square and white dot.

Signal / Noise analysis

Because each patient had been examined 6 times, there were 6 sMD values, one for each sector, for both
SAP and FDT2. For each patient, we calculated the superior-inferior difference in sMD between the
mirror sectors in each test, and in all 30 combinations of the 6 tests, so that 36 differences were obtained
for each patient, each sector, and each of the two techniques (SAP and FDT2). Estimates of signal and
noise were then derived from the mean and the population standard deviation (SD) of the distribution of
differences (n=36), respectively (Fig. 2).

Because a single outlying data point can have an unduly large influence on both the mean and the SD,
we also computed non-parametric estimates by replacing the mean with the median, and the SD with the
Median Absolute Deviation (MAD)39 of the differences. This is described in the Appendix, along with
Bland-Altman comparisons of parametric and non-parametric estimates.

Signal, noise, and signal/noise ratios (SNRs) were then compared between SAP and FDT2, in each of
the 5 pairs of sectors and each of the 15 patients. All analyses were performed in the freely available
open-source environment R.40, 41 The nlme library was used to estimate statistical significance and
confidence intervals; patients were treated as random factors to adjust for the non-independence of the 5
sector pairs within each individual patient.42

5
 Artes & Chauhan, Signal/noise ratios in perimetry

Fig. 2)

Example illustrating how signal and noise were derived. a) In the upper and lower arcuate sectors, total
deviation values of each test were averaged to sector MDs. b) From the 6 repeated tests, 6 sMDs were
obtained in each of the sectors. c) By enumerating all possible combinations of the 6 test, a distribution of 36
differences between the upper and lower sectors was obtained , and d) estimates for signal (black arrow
head) and noise (curly bracket) were derived from the average and the dispersion of this distribution,
respectively.

6
 Artes & Chauhan, Signal/noise ratios in perimetry

Results
The sectoral MD values in the 10 visual field sectors ranged from -27.8 dB to +1.5 dB (median, -2.3 dB)
with SAP, and from -26.2 dB to +3.3 dB (median, -2.4 dB) with FDT2. The relationship between SAP
and FDT accounted for 69% of the variance in the data (Spearman rank correlation, p<0.001, Fig. 3),
and, for sectors with SAP sMD better than -10 dB, the relationship between SAP and FDT2 appeared
linear (Tukey’s test for additivity,43 p=0.22) with a slope of 2.1 (RMA regression, 95% C.I, 1.9-2.3). For
sectors with more advanced damage, however, the relationship between the sMDs of both techniques
became progressively weaker and deviated significantly from that observed at less damaged sectors
(p<0.001, Tukey’s test). Despite the averaging of 6 repeated tests with both techniques, the data
exhibited a large degree of scatter (Fig. 3).

Fig. 3)

Relationship between sectoral

MD values obtained with SAP
and FDT2.

Each data point represents the

mean of 6 repeated tests with
FDT2 and SAP.

For sectors with SAP MDs

better than -10 dB (filled dots),
RMA regression estimated a
slope of 2.1 (95% C.I., 1.9,
2.3).
The 1:1 line is shown for
comparison.

Signal and noise estimates were derived from the mean and the SD of the superior-inferior differences
(Fig. 2), with both FDT2 and SAP. Three selected examples are shown in Figs. 4-6.

7
Artes & Chauhan, Signal/noise ratios in perimetry

Fig. 4)

Example A

In this visual field with mild

damage, a 2 dB asymmetry
between the nasal superior
and inferior sectors was
measured with FDT2 but not
with SAP.

The dispersion of the

differences (noise) was similar
with SAP and FDT2 (0.9 and
0.7 dB, respectively), and
FDT2 had a larger SNR (3.0
compared to 0.2 with SAP).

The vertical grey bar (noise)

represents ±2 SD to indicate if
the mean of the differences is
significantly different from
zero (dashed grey line).

8
Artes & Chauhan, Signal/noise ratios in perimetry

Fig. 5)

Example B

In this extensively damaged

visual field, both SAP and
FDT2 revealed a small
difference (2.8 and 4.2 dB,
respectively) between the
superior and inferior
paracentral sectors.

With SAP, the large variability

of both sectors (SD, 3.7 dB)
made it difficult to distinguish
this signal (SNR, 0.8). With
FDT2, the asymmetry was
more clearly apparent (SNR,
3.1), chiefly because of lower
variability (1.3 dB).

The vertical grey bar (noise)

represents ±2 SD to indicate if
the mean of the differences is
significantly different from
zero (dashed grey line).

9
Artes & Chauhan, Signal/noise ratios in perimetry

Fig. 6)

Example C

In this visual field, both SAP

and FDT indicated substantial
damage.

With SAP, an asymmetry of 6

dB was clearly apparent in the
nasal sectors (SNR > 2),
despite large variability in the
more damaged inferior sector.

Despite lower variability (1.3

dB), there was no detectable
signal with FDT2 (SNR, 0.3).

The vertical grey bar (noise)

represents ±2 SD to indicate if
the mean of the differences is
significantly different from
zero (dashed grey line).

10
 Artes & Chauhan, Signal/noise ratios in perimetry
There was a moderately close relationship between the signals of both techniques (Fig 7a, r2 = 0.52,
p<0.001), but no such relationship between the noise estimates (r2=0.01, p=0.16, Fig. 7b).

Fig. 7)

Relationship between
superior/inferior
differences (signal) of
SAP and FDT2 (a) and
noise (b). Axes are
drawn on a square-
root scale to emphasize
mid-range values.

Although both signal and noise appeared numerically larger with FDT2 compared to SAP (Table 1), a
direct comparison between the estimates is difficult to interpret because they are expressed in
instrument-specific dB-units.

Table 1. Distribution of signal and noise estimates from FDT2 and SAP. P-values and confidence intervals were
established by mixed effects modelling because each patient contributed 5 estimates.

Table 1 FDT2 (dB) SAP (dB) FDT2>SAP FDT2/SAP p-value

signal mean, median 3.5, 1.9 2.8, 1.2 51/75 (68%) 1.40 [1.06, 1.85] 0.02
[range] [0.1 – 15.4] [0.0 – 16.6]

noise mean, median 1.7, 1.6 1.5, 1.3 48/75 (64%) 1.17 [0.97, 1.42] 0.10
[range] [0.6 – 3.0] [0.5 – 5.2]

However, by calculating the ratio between signal and noise, the instrument-specific units in numerator
and denominator cancel each other such that the resulting signal/noise ratio (SNR) is independent of the
dB-scale of the instrument.

There was a moderately close association between the SNRs of SAP and FDT2 (r2=0.68, p<0.001, Fig.
8). Of the 75 sector pairs, 46 (61%) had a larger SNR with FDT2. On average, the SNRs of FDT2 were
19% larger (95% C.I., -6%, 52%, p=0.14) than the corresponding SNRs of SAP.

11
 Artes & Chauhan, Signal/noise ratios in perimetry
Fig. 8)

Relationship between signal-to-noise

ratios with SAP and FDT2.

The points labelled A, B, C correspond

to the examples in Figs. 4, 5, and 6.

Axes are drawn on a square-root scale

to emphasize mid-range values.

Sector pairs within which there were no differences in damage contribute little information, but they
may dilute genuine differences between the techniques if they are included in an overall comparison. To
reduce this dilution while avoiding any selection bias, we compared the SNRs of SAP and FDT2 for all
those pairs in which either SAP or FDT2 had an SNR greater than 0.5, 1.0 and 2.0 (Table 2).

Table 2: Comparison between SNRs of FDT2 and SAP. A ratio >1 in column 2 means that FDT2 provided a
larger SNR than SAP. Column 3 gives the number of pairs in which the SNR of FDT2 was greater than that of
SAP. P-values and confidence intervals were established by mixed effects modelling because each patient
contributed 5 estimates.

SNR Ratio of SNR (FDT2/SAP) SNR FDT2 > SAP / total p-value

>0.5 1.20 [0.93 - 1.56] 66/71 (93%) p=0.15

>1.0 1.43 [1.05 – 1.94] 34/50 (68%) p=0.03

>2.0 1.39 [1.04 – 10.6] 22/30 (73%) p=0.02

Of the sector pairs with SNRs > 0.5 with either FDT2 or SAP, between 68-93% had larger SNRs with
FDT2 compared to SAP (Table 2, column 3). However, these findings were statistically significant
(p<0.05) only for the subset of SNRs which were greater than 1.0. In these sector-pairs, the SNRs of
FDT2 were approximately 40% larger than the corresponding SNRs of SAP (Table 2, column 2).

12
 Artes & Chauhan, Signal/noise ratios in perimetry
Discussion

The aim of this paper was to develop an approach for comparing perimetric techniques independent of
their measurement scales, and to apply this methodology to visual field data from the Humphrey-Matrix
(FDT2) perimeter.

In a previous study,26 we demonstrated that the variability characteristics of both techniques were
qualitatively different – threshold estimates from FDT2 had nearly uniform variability across the
measurement range of the instrument, while those of SAP showed an exponential increase in variability
with decreasing sensitivity. Similar results were obtained in other studies, both with FDT127, 29and more
recently with FDT2.28 We were then, however, unable to make a quantitative comparison of the
variability, because both instruments use different types of stimuli and different definitions of the dB-
scales. The lack of a general method for comparing threshold data from different perimetric tests
motivated the signal/noise analyses performed in this paper. By relating systematic differences within a
visual field (signal) to the precision with which such differences can be measured (noise), techniques
can be compared independent of their underlying measurement units.

Our data showed a substantial correlation between the signals of the two techniques (r2=0.52), but no
such correlation for the noise. The lack of a relationship between the noise estimates of FDT2 and SAP
clarifies why, in some patients, either technique may have systematic advantages for measuring losses
that are less detectable with the other.26, 31, 44 While our dataset of 15 patients is too small to carry out a
meaningful subgroup analysis, we suggest that the signal/noise methodology proposed in this paper
provides a useful framework for studying systematic differences between different perimetric
techniques. It is particularly important to establish factors that contribute to the large scatter apparent in
Fig. 3. Because 6 tests had been averaged for each data point, it is unlikely that this scatter can be
explained solely by measurement variability of FDT2 and SAP.

For sectors with SAP MDs better than -10 dB, the slope of the relationship between the sectoral MD
values of FDT2 and SAP was 2.1 (Fig. 3). This closely mirrors the findings reported in our earlier paper
in which we compared threshold estimates from individual test locations, and it is also in agreement
with the slope of 2.0 expected from the different definitions of the dB-scale of both techniques.45 With
FDT2, a dB is defined as -20 log10 of Michelson contrast such that a change of 20 dB corresponds to a 1
log unit change in contrast, whereas with SAP, a dB is defined as -10 log10 of Weber contrast, such that
a change of 20 dB corresponds to a change of 2 log units. Importantly, the empirically determined slope
~2 would suggest that the magnitude (in dB) of early and moderate visual field losses, and changes over
time, could be up to twice as large with FDT2 as compared to SAP. In our data, the signals (superior-
inferior differences between the mirror-pairs of visual field sectors), were on average only 40% larger

13
 Artes & Chauhan, Signal/noise ratios in perimetry
with FDT2 compared to SAP, but this average would have been reduced by sector pairs between which
there were no meaningful differences in damage and therefore no measurable signals.

Approximately 70% of sector pairs with an SNR > 1.0 with either SAP or FDT2 had higher signal/noise
ratios with FDT2 (Table 2). This is evidence of an overall gain and confirms that the benefits of the
larger FDT2 signals are not offset by the larger variability of this technique (Table 1). For pairs with
SNRs > 1.0, the SNRs of FDT2 were approximately 40% larger than those of SAP. This difference is
similar in magnitude to the improvement in SNR that would be expected from repeating a test, since
performing the same test twice can reduce the measurement variability, in theory, by a factor of √2
(1.41). A difference of this magnitude would mean a substantial net improvement in the detectability of
early changes, for cross-sectional detection of visual field loss as well as for longitudinal measurement
of visual field progression. For the former, empirical investigations on total- and pattern deviation
probability maps31, 44, 46 with FTD2 and SAP, as well as global visual field indices such as pattern
standard deviation,47 are in agreement with our results.

How may signal/noise ratios, calculated from retest data, help to estimate performance in measuring
progression? The rationale for using gradients in space as a surrogate for changes over time is illustrated
in Fig. 9. Progression of visual field loss is a change in sensitivity over time, and the usefulness of a test
for following patients over time depends on how well its data reflect these changes. In contrast,
signal/noise ratios estimated from tests performed within a short period of time express the detectability
of differences within a visual field at that particular time.

Fig. 9)

a) The SNR measured between locations At1

and Bt1 measures how reliably the technique
represents the gradient of damage in space
(vertical arrow). b) If B deteriorates over
time, such that its deviation at Bt2 becomes
equal to that of At1, the gradient in time
between Bt1 and Bt2 is equal to that between
At1 and Bt1. The ability to detect the change
from Bt1 to Bt2 should be similar to that
measured by the SNR between At1 and Bt1.

SNRs express how reliably a technique reflects gradients of damage within a visual field, and this is a
function of the depth of loss, the variability of the measurements, and the dynamic range of the
technique. A larger SNR therefore does not necessarily mean that the technique is more sensitive than
another, nor does a more sensitive technique necessarily provide a larger SNR (Fig. 10).

14
 Artes & Chauhan, Signal/noise ratios in perimetry
Fig. 10)

Relationship between
signals of 2 techniques A
and B when A is more
sensitive than B.

a) A reveals superior loss

(signal at nasal step,
vertical arrow). There is
no signal with B.

b) A reveals more extensive

loss than B but has
reached the limit of its
dynamic range; its signal
is small compared to B.

c) Owing to its larger

dynamic range, B
continues to provide signal
even though both superior
and inferior sectors are
damaged.

The signal/noise methodology proposed in this paper has several limitations. To obtain robust estimates
of signal and noise, multiple tests have to be performed. Nevertheless, a rigorous protocol with at least 5
examinations per patient has advantages also for the derivation of test-retest intervals, because a large
number of combinations of test-retest examinations can be analysed.26, 48 SNRs depend on the sample of
patients and cannot be compared across different studies. They also depend on the somewhat arbitrary
choice of where in the visual field the signal and noise distributions are derived from. In this study, we
used the superior-inferior sectors of the Glaucoma Hemifield Test, and therefore our finding of larger
SNRs with FDT2 may strictly apply only to those analyses which make use of a similar clustering. In
principle, however, other pairs of test locations or pairs of clusters could not be chosen. Finally, SNRs
can be estimated only if focal losses are present in the visual field; diffuse reductions in sensitivity do
not contribute a signal. As a consequence, the method is unsuitable for evaluating techniques that
predominantly uncover diffuse loss.

The assumption that gradients in space can be used as a first approximation for change over time
appears reasonable but is, as yet, untested. Signal/noise estimates from test-retest studies will therefore
not replace longitudinal studies for investigating new visual field tests’ ability to monitor patients with
glaucoma, but they may provide early insight into properties that cannot be gained from analyses of test-
retest variability. They may help in hypothesis-building and in planning effective longitudinal studies of
new visual field tests.
15
 Artes & Chauhan, Signal/noise ratios in perimetry
Appendix

Mean and SD are parametric estimates of central tendency and dispersion and can be highly affected by
outliers, particularly so in small samples. Compared to mean and SD, the non-parametric median and
Median Absolute Deviation (MAD) are more robust to outliers, but they are also less efficient (more
variable) when there are no outliers. To investigate whether there were meaningful differences between
parametric and non-parametric estimates of signal, noise, and SNRs, we computed non-parametric
estimates of signal and noise from the median and MAD. The MAD was scaled by a factor of 1.483 to
make it similar to the SD in a Normal distribution.49

It should be noted that the enumeration of all possible 36 differences between the superior and inferior
sMDs as explained in Fig. 2 of the Methods section needs to be performed only to compute the non-
parametric estimates. For the parametric estimates, the mean of the 36 differences is identical to the
difference between the means of the 6 sMDs in the superior and inferior sectors (1), and the SD of the
differences is identical to that obtained from pooling the SDs of the superior and inferior sectors (2).

(1) xdiff = xsup ! xinf

(2) ! diff = 2 (! sup ) 2 + (! inf ) 2

The Bland-Altman plots in Fig. 11 show good overall agreement between the parametric and non-
parametric estimates of signal and noise. With both FDT2 and SAP, noise estimates >2 dB were
systematically smaller with the non-parametric method (Fig. 11 c, d; p<0.10). The discrepancies
observed in the SNR ratios of FDT2 increased with the magnitude of the SNR (Fig. 11f). However,
none of our findings changed substantially when the analyses were performed with the non-parametric
SNRs instead of the parametric alternatives.

16
Artes & Chauhan, Signal/noise ratios in perimetry
Fig. 11)

Bland-Altman plots of
parametric and non-
parametric estimates of
signal, noise, and
signal/noise ratio with
SAP and FDT2.

Points on the horizontal

line show perfect
agreement between
parametric and non-
parametric estimates, and
the shaded area encloses
95% of the differences.

17
 Artes & Chauhan, Signal/noise ratios in perimetry

References

1. Anderson DR, Patella VM. Automated Static Perimetry: Mosby St. Louis; 1999.
2. Spry PGD, Johnson CA. Identification of progressive glaucomatous visual field loss. Survey of
Ophthalmology 2002;47:158-173.
3. Keltner JL, Johnson CA, Levine RA, et al. Normal visual field test results following
glaucomatous visual field end points in the Ocular Hypertension Treatment Study. Archives of
Ophthalmology 2005;123:1201-1206.
4. Keltner JL, Johnson CA, Quigg JM, Cello KE, Kass MA, Gordon MO. Confirmation of visual
field abnormalities in the ocular hypertension treatment study. Archives of Ophthalmology
2000;118:1187-1194.
5. Heijl A, Leske MC, Bengtsson B, et al. Measuring visual field progression in the early manifest
glaucoma trial. Acta Ophthalmologica Scandinavica 2003;81:286-293.
6. Gardiner SK, Crabb DP. Examination of different pointwise linear regression methods for
determining visual field progression. Investigative Ophthalmology and Visual Science
2002;43:1400-1407.
7. Vesti E, Johnson CA, Chauhan BC. Comparison of Different Methods for Detecting
Glaucomatous Visual Field Progression. Investigative ophthalmology & visual science
2003;44:3873-3879.
8. Gardiner SK, Crabb DP. Frequency of testing for detecting visual field progression. British
Journal of Ophthalmology 2002;86:560-564.
9. Chauhan BC, Garway-Heath DF, Goñi FJ, et al. Practical recommendations for measuring rates
of visual field change in glaucoma. British Journal of Ophthalmology 2008;92:569-573.
10. Bengtsson B, Olsson J, Heijl A, Rootzén H. A new generation of algorithms for computerized
threshold perimetry, SITA. Acta Ophthalmologica Scandinavica 1997;75:368-375.
11. Vingrys AJ, Pianta MJ. A new look at threshold estimation algorithms for automated static
perimetry. Optometry and Vision Science 1999;76:588-595.
12. Anderson AJ, Johnson CA. Comparison of the ASA, MOBS, and ZEST threshold methods.
Vision Research 2006;46:2403-2411.
13. Turpin A, McKendrick AM, Johnson CA, Vingrys AJ. Performance of efficient test procedures
for frequency-doubling technology perimetry in normal and glaucomatous eyes. Investigative
Ophthalmology and Visual Science 2002;43:709-715.
14. Schiefer U, Pascual JP, Edmunds B, et al. Comparison of the new perimetric GATE strategy
with conventional full-threshold and SITA standard strategies. Investigative ophthalmology &
visual science 2009;50:488-494.
15. Frisén L. Computerized perimetry: Possibilities for individual adaptation and feedback.
Documenta Ophthalmologica 1988;69:3-9.
16. Frisen L. Perimetric variability: importance of criterion level. Doc Ophthalmol 1988;70:323-
330.
17. Kutzko KE, Brito CF, Wall M. Effect of Instructions on Conventional Automated Perimetry.
Investigative ophthalmology & visual science 2000;41:2006-2013.
18. Wall M, Woodward KR, Brito CF. The effect of attention on conventional automated perimetry
and luminance size threshold perimetry. Investigative ophthalmology & visual science
2004;45:342-350.

18
 Artes & Chauhan, Signal/noise ratios in perimetry
19. Miranda MA, Henson DB. Perimetric sensitivity and response variability in glaucoma with
single-stimulus automated perimetry and multiple-stimulus perimetry with verbal feedback. Acta
Ophthalmol Scand 2007.
20. Anderson RS. The psychophysics of glaucoma: Improving the structure/function relationship.
Progress in Retinal and Eye Research 2006;25:79-97.
21. McKendrick AM. Recent developments in perimetry: test stimuli and procedures. Clinical &
experimental optometry : journal of the Australian Optometrical Association 2005;88:73-80.
22. Monhart M. What Are the Options of Psychophysical Approaches in Glaucoma? Survey of
Ophthalmology 2007;52.
23. Wall M. What's New in Perimetry. Journal of Neuro-Ophthalmology 2004;24:46-55.
24. Fogagnolo P, Rossetti L, Ranno S, Ferreras A, Orzalesi N. Short-wavelength automated
perimetry and frequency-doubling technology perimetry in glaucoma. Progress in Brain
Research; 2008:101-124.
25. Hot A, Dul MW, Swanson WH. Development and evaluation of a contrast sensitivity perimetry
test for patients with glaucoma. Investigative ophthalmology & visual science 2008;49:3049-
3057.
26. Artes PH, Hutchison DM, Nicolela MT, LeBlanc RP, Chauhan BC. Threshold and variability
properties of matrix frequency-doubling technology and standard automated perimetry in
glaucoma. Investigative ophthalmology & visual science 2005;46:2451-2457.
27. Chauhan BC, Johnson CA. Test-retest variability of frequency-doubling perimetry and
conventional perimetry in glaucoma patients and normal subjects. Investigative Ophthalmology
and Visual Science 1999;40:648-656.
28. Wall M, Woodward KR, Doyle CK, Artes PH. Repeatability of Automated Perimetry: A
Comparison between Standard Automated Perimetry with Stimulus Size III and V, Matrix, and
Motion Perimetry. Invest Ophthalmol Vis Sci 2009;50:974-979.
29. Spry PGD, Johnson CA, McKendrick AM, Turpin A. Variability components of standard
automated perimetry and frequency-doubling technology perimetry. Investigative
Ophthalmology and Visual Science 2001;42:1404-1410.
30. Anderson AJ, Johnson CA. Anatomy of a supergroup: does a criterion of normal perimetric
performance generate a supernormal population? Investigative ophthalmology & visual science
2003;44:5043-5048.
31. Sakata LM, DeLeon-Ortega J, Arthur SN, Monheit BE, Girkin CA. Detecting visual function
abnormalities using the Swedish interactive threshold algorithm and matrix perimetry in eyes
with glaucomatous appearance of the optic disc. Archives of Ophthalmology 2007;125:340-345.
32. Chauhan BC, House PH, McCormick TA, LeBlanc RP. Comparison of Conventional and High-
Pass Resolution Perimetry in a Prospective Study of Patients With Glaucoma and Healthy
Controls. Am Med Assoc; 1999:24-33.
33. Johnson CA, Adams AJ, Casson EJ, Brandt JD. Progression of early glaucomatous visual field
loss as detected by blue- on-yellow and standard white-on-white automated perimetry. Archives
of Ophthalmology 1993;111:651-656.
34. Bayer AU, Erb C. Short wavelength automated perimetry, frequency doubling technology
perimetry, and pattern electroretinography for prediction of progressive glaucomatous standard
visual field defects. Ophthalmology 2002;109:1009-1017.
35. Haymes SA, Hutchison DM, McCormick TA, et al. Glaucomatous Visual Field Progression with
Frequency-Doubling Technology and Standard Automated Perimetry in a Longitudinal
Prospective Study. Investigative ophthalmology & visual science 2005;46:547-554.
19
 Artes & Chauhan, Signal/noise ratios in perimetry
36. Asman P, Heijl A. Glaucoma Hemifield Test. Automated visual field evaluation. Archives of
Ophthalmology 1992;110:812-819.
37. Anderson AJ, Johnson CA, Fingeret M, et al. Characteristics of the Normative Database for the
Humphrey Matrix Perimeter. Invest Ophthalmol Vis Sci 2005;46:1540-1548.
38. Warton DI, Wright IJ, Falster DS, Westoby M. Bivariate line-fitting methods for allometry.
Biological Reviews 2006;81:259-291.
39. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of
clinical measurement. Lancet 1986;1:307-310.
40. R Development Core Team. R: A language and environment for statistical computing. Vienna,
Austria: R Foundation for Statistical Computing 2004.
41. Ihaka R, Gentleman R. R: A Language for Data Analysis and Graphics. Journal of
Computational and Graphical Statistics 1996;5:299-314.
42. Pinheiro JC, Bates DM. Mixed-Effects Models in S and S-Plus: Springer; 2000.
43. Weisberg S. Applied Linear Regression: John Wiley & Sons New York; 1985.
44. Racette L, Medeiros FA, Zangwill LM, Ng D, Weinreb RN, Sample PA. Diagnostic Accuracy of
the Matrix 24-2 and Original N-30 Frequency-Doubling Technology Tests Compared with
Standard Automated Perimetry. Invest Ophthalmol Vis Sci 2008;49:954-960.
45. Sun HAO, Dul MW, Swanson WH. Linearity Can Account for the Similarity Among
Conventional, Frequency-Doubling, and Gabor-Based Perimetric Tests in the Glaucomatous
Macula. Optometry and Vision Science 2006;83:E455.
46. Leeprechanon N, Giangiacomo A, Fontana H, Hoffman D, Caprioli J. Frequency-Doubling
Perimetry: Comparison With Standard Automated Perimetry to Detect Glaucoma. American
journal of ophthalmology 2007;143:263-271.e261.
47. Medeiros FA, Sample PA, Zangwill LM, Liebmann JM, Girkin CA, Weinreb RN. A Statistical
Approach to the Evaluation of Covariate Effects on the Receiver Operating Characteristic
Curves of Diagnostic Tests in Glaucoma. Invest Ophthalmol Vis Sci 2006;47:2520-2527.
48. Artes PH, Iwase A, Ohno Y, Kitazawa Y, Chauhan BC. Properties of perimetric threshold
estimates from full threshold, SITA standard, and SITA fast strategies. Investigative
Ophthalmology and Visual Science 2002;43:2654-2659.
49. Huber PJ. Robust Statistics: Wiley-Interscience; 2004.

20