Quality of individually calibrated customary printers for

assessment of typical dental diagnoses on glossy paper

prints: a multicenter pilot study
Ralf K. Schulze, DrMedDent,a Dirk Schulze, DrMedDent,b Kai Voss, DrMedDent,c
Michael Rottner, DrMedDent,d Hans-Peter Keller, Dipl-Phys, Dr-Ing,e
Karin Dollmann, Dipl-Ing,f Burkhard Maager, DrMedDent,g and Matthias Wedel, Dipl-Ing,h
Mainz, Germany
JOHANNES GUTENBERG UNIVERSITY

Objective. The objective of this study was to compare dental radiographs printed on glossy paper from calibrated low-
cost printers with monitor display.
Study design. Three typical intraoral radiographs were selected and a questionnaire was developed with questions
assessing accuracy and subjective quality. A test pattern was designed for printer calibration. After calibration,
radiographs were printed on glossy paper with 3 ink-jet and 2 thermo-sublimation printers. Sixteen raters evaluated the
printed radiographs, 9 of them also on standardized viewing monitors. Subjective ratings were compared, and an
ROC-analysis based on expert-consensus monitor readings was performed.
Results. Low inter-rater reproducibility (mean Cohen’s Kappa monitor: 0.49; printers: 0.44), but high diagnostic
accuracy was found for all printers (areas [Az] underneath the ROC curves: 0.725 to 0.884). The overlap between the
95% Az confidence intervals of the mean indicate no significant differences.
Conclusion. Our preliminary findings indicate that dental radiographs may be evaluated on glossy paper prints of
calibrated customary printers. (Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2008;106:578-86)

As digital radiographic technology becomes more and
more commonplace, the need for simple and reliable
transfer of digital images from one office to another
increases. Although today personal computers are used
throughout almost every working environment, a few
offices still operate without any digital technology.
These offices are not capable of managing a digital
image file, even if it is encoded in typical digital image-
file formats. According to German regulations,1,2 the
producer of a digital radiograph must provide it to a
a
Assistant Professor, Departments of Oral Surgery (and Oral Radi-
ology), Johannes Gutenberg University, Mainz, Germany.
b
Head of Section for Dentomaxillofacial Radiology, Department of
Craniomaxillofacial Surgery, Albert-Ludwigs-Universität, Freiburg,
Germany.
c
Private practice, Schleswig-Holstein Dental Association, Germany.
d
Private practice, Bavarian Dental Association, Germany.
e
Head of Dental Standardization Committee, Deutsches Institut für
Normung (German Institute for Standardization), Pforzheim, Ger-
many.
f
Carestream Health, Deutschland GmbH, Stuttgart, Germany.
g
Private practice, Germany.
h
Chairman of Standardization Committee, Siemens AG Medical So-
lutions, Erlangen, Germany.
Received for publication Mar 15, 2007; returned for revision Aug 7,
2007; accepted for publication Sep 14, 2007.
1079-2104/$ - see front matter
© 2008 Mosby, Inc. All rights reserved.
doi:10.1016/j.tripleo.2007.09.004
dentist in an appropriate way. Since a digital file is
certainly not appropriate for offices not yet equipped
digitally, alternative solutions must be found. Cost-
effective hardcopies on paper would provide a useful
alternative for this purpose.
Budget printers considered here include typical off-
the-shelf ink-jet technology, plus thermo-sublimation
printers belonging to the mid-range price segment. For
comparison, professional dry or wet laser radiographic
printers typically range between $25,000 and $55,000,
which is obviously out of range for a regular dental
office considering the limited number of prints. Not
much scientific information has been published on the
topic after introduction of a prototype ink-jet printer for
printing of various radiographs by Kirkhorn et al. in
1994.3 The same group published encouraging results
from a Receiver Operating Characteristics (ROC)
analysis for simulated tumors in a chest phantom.4 In
dental radiography, we are aware of only 2 subjec-
tive image quality studies5,6 and 2 accuracy studies
using paper prints from customary printers.7,8 On the
other hand, expensive wet or dry radiographic laser
imagers explicitly constructed for the production of
high-quality hardcopies are well established, e.g., in
digital mammography9 or for pulmonal nodule de-
tection.10,11 Because of the reasons explained above,
however, they do not fall within the scope of this
article.

578
OOOOE
Volume 106, Number 4
According to German regulations,1,2 a dental office
using digital radiographic technology is required to
have at least one diagnosis monitor of initially con-
trolled, standardized quality with respect to size, lumi-
nescence, resolution, and distortion. An obligatory
daily and monthly quality check using the well-known
Society of Motion Picture and Television Engineers
(SMPTE) test pattern includes contrast assessment as
well as geometric reproduction and color display, re-
spectively.2 Today, the monitor display is considered as
the gold standard for diagnosis on digital radiographs.
Therefore, hardcopies have to be compared against
monitor display. Up-to-date printers offer a variety of
settings for print demands such as photographs, line
graphics, or simple text printout. We aimed to achieve
a high standardization level for the prints by establish-
ing a reasonable calibration process that is easy to apply
during routine daily work. This approach differs fun-
damentally from that of other work groups,7,8 who
printed the radiographs without prior calibration of the
printers.
The aims of this pilot study were to compare (1)
accuracy and (2) subjective image quality of different
customary printers for paper prints as compared to
standardized monitor display for typical dental radio-
graphic diagnoses. Using this pilot study as an initial
indicator, provided that the outcome seems promising,
we plan on pursuing additional studies to obtain a more
complete overview of the printing performance of bud-
get printers.
METHODS
Radiographs
Typical critical dental radiographic features were
identified, which should be displayed on the intraoral
test radiographs. They included the periodontal liga-
ment space, interproximal caries, apical lesions, and the
marginal bone level. From a set of digital intraoral
radiographs from the patient records of 2 university
dental schools and 4 private dental offices, 3 CCD-
based direct digital radiographs each presenting differ-
ent features were selected (Fig. 1). The radiographs had
to be exposed in sufficient geometry and should display
a normal contrast range. Two radiographs contained
872 ⫻ 668 square pixels of 0.039-mm length (Full size,
Sirona Dental Systems AG, Bensheim, Germany), 1
radiograph (#1 in Fig. 1) contained 1590 ⫻ 1024 square
pixels of 0.0195-mm length (Visualix EHD, Kavo Den-
tal Gendex Dental Systems, Lake Zurich, IL).
Printers
On the basis of availability, we selected 5 different
printers for this study. Three printers (Canon S 500
[CS5], Epson Stylus Photo RX 600 [EPS], and Tetenal
Schulze et al 579
Fig. 1. Test radiographs selected from the patient records of
2 dental clinics. Interproximal caries was assessed on radio-
graph #1. The periodontal ligament space was prompted on
all radiographs, whereas periapical lesions were evaluated
only on radiographs #2 and #3.
HP Color Inkjet Printer CP 1700 [THP]) operate with
ink-jet technology. CS5 and EPS represent typical bud-
get off-the-shelf printers. Two (Kodak 1400 DPP
[KDP] and Codonics Horizon [CHO]) thermo-sublima-
tion printers were also included representing a more
established technique for medical grayscale printing
(e.g., ultrasound images). While the KDP printer is a
budget semiprofessional thermo-sublimation printer
costing less than $650, the CHO is an expensive pro-
fessional thermo-sublimation printer costing roughly
$25,000. We selected this printer as a quality control
for comparison with the budget printers. Technical
specifications that were made available to us are de-
tailed in Table I.
Printer calibration
The basic idea was to generate a test pattern including
test features for easy and timely assessment of the princi-
ple printing quality. By application of this test pattern, an
observer should be able to evaluate optical resolution,
gray scale or color resolution, color gradient, and geomet-
ric reproduction of the printer. A test pattern fulfilling the
requirements was generated (Fig. 2).
The test pattern was printed at 600 dpi on glossy
paper (Distributed Medical Imaging Paper, Eastman
Kodak Company, Rochester, NY) using Irfanview-
Software (www.irfanview.com). Alternatively, the
highest possible resolution of the printer was selected
(Table I). The initial idea was that each printer had to
pass all test features to qualify for radiograph printing.
By default, the initial printer settings included ”photo-
print,” ”black-and-white (grayscale) modus” and ”high-
quality print.” If adjustable, ”error diffusion” was acti-
vated whereas ”dithering” was deactivated. If possible,
the gamma curve was adapted directly and the print
intensity was altered. By application of the test pattern
and step-by-step modification of the printer settings
specified above, the pattern was printed on glossy paper
until it had reached the best possible quality. If black
and white was not explicitly available, color mode was
allowed also (Table I).
 OOOOE
580 Schulze et al October 2008

Table I. Printer specifications

Printer Technology Mode applied Maximum resolution, dpi
S 500; Canon Inc., Tokyo, Japan Bubble ink jet Color 2400 ⫻ 1200
HP Inkjet Printer CP 1700; Tetenal AG & Co. KG; Thermo ink jet 256 shades of gray 2400 ⫻ 1200
Norderstedt, Germany
Stylus Photo RX 600; Seiko Epson Corporation, Bubble ink jet 256 shades of gray 2400 ⫻ 4800
Nagano, Japan
Codonics Horizon CI; Siemens AG, Munich, Thermo-sublimation 256 shades of gray 320 ⫻ 320
Germany
Kodak 1400 DPP, Kodak GmbH, Stuttgart, Germany Color thermo-sublimation 256 shades of gray 301 ⫻ 301

Fig. 2. Test pattern developed for printer calibration. Test features included horizontal and vertical optical resolution (upper left
corner), color gradient (lower half and upper left corner), contrast resolution (circular frame around image plus scripture
”Bildqualität”) as well as geometric reproduction (bars in the middle).

Radiograph printing Radiograph evaluation

Using the parameters identified from calibration
and Irfanview software, the radiographs were printed
in landscape format fitted to the DINA4 (print size
approximately 27 cm ⫻ 19 cm) glossy paper speci-
fied above. THP, CHO, and KDP had to be printed on
glossy paper specifically supplied for these printers.
A questionnaire originally comprising 11 major
questions subdivided into 35 single questions was de-
veloped (Appendix 1). Out of this original question-
naire, 4 major questions subdivided into 14 single ques-
tions that had been designed for evaluation of accuracy
(interproximal caries, apical translucency, length of
OOOOE
Volume 106, Number 4
root canal filling) as compared to the monitor mean.
Here, the certainty of decision was assessed on a
3-point confidence scale (1 ⫽ pathology definitely
present, 2 ⫽ uncertain, 3 ⫽ pathology definitely not
present). Another 4 major questions subdivided into 14
single questions had been prepared for subjective rank-
ing of image quality on a 3-point scale (1 ⫽ diagnos-
tically acceptable, 2 ⫽ uncertain, 3 ⫽ diagnostically
not acceptable). The remaining 3 questions (1.4, 3.1,
and 3.3) provided a fourth answer option not following
this scale. Thus, they were not evaluated in this study.
A total of 16 observers (11 from 3 different univer-
sity dental schools; 5 from different private offices)
participated in the study. This sample comprised 12
general dental practitioners, 3 oral surgeons, and 1
medical radiologist, with their working experience
ranging between 1 and 20 years. Please note that in
Germany there is no specialization in dental radiology.
Nine observers (1 radiologist, 2 oral surgeons, and 6
general practitioners) from 2 university dental schools
also evaluated the radiographs on 2 standardized mon-
itors complying with German regulations2: 6 of them
on a 17-inch cathode ray tube (CRT) monitor (Vision
Master Pro 413, IIyama Electric Co. Ltd., Nagano,
Japan) with an image size of 25.5 cm ⫻ 19.5 cm; the
remaining 3 on a 19-inch liquid crystal display (LCD)
monitor (SCENICVIEW P19-1S, Fujitsu Siemens, Mu-
nich, Germany; image size: 27.5 cm ⫻ 21 cm). By
German law,1 standardized viewing monitors have to
be available in every office using digital radiography.
These monitors have to be labeled after an initial thor-
ough evaluation performed by authorized personnel. To
pass this evaluation, the monitors have to provide suf-
ficient contrast, luminance (intensity), optical resolu-
tion, and geometric accuracy, together with additional
requirements all detailed in the guidelines.2 A daily
plus a monthly quality check by means of the well-
known SMPTE test pattern is mandatory. Evaluations
were all made in a darkened, quiet room. Paper prints
were viewed on a white cardboard support using indi-
rect light. The observers were blinded with respect to
printer type and study design. Altogether, we evaluated
28 ⫻ 16 ⫽ 448 observations from each of the 5 printers
and 28 ⫻ 9 ⫽ 252 observations from the calibrated
monitors.
Establishment of a ”weak gold standard”
In addition, 2 experts in radiographic diagnostics
(D.S., R.S.) with a working experience of more than 10
years rated the images independently on either of the 2
standardized and calibrated viewing monitors detailed
above. This was also done in a quiet dimmed-light
environment. In case of disagreement (50% of the
cases), a consensus was reached during a telephone
Schulze et al 581
Table II. Kappa values for assessment of inter-rater
reproducibility
Monitor Print out
Mean 0.49 0.44
SD 0.14 0.14
Median 0.51 0.46
Minimum 0.19 0.04
Maximum 0.85 0.74
Values are based on all observations per category.
conference. This consensus reading was subsequently
taken as a ”weak gold standard” representing the
”truth” with respect to the monitor observations.
Statistical data analysis
All statistics were performed with SPSS (Version
12.0.1) for Windows (SPSS Inc., Chicago, IL). As
explained earlier, 4 questions (subdivided into 14 single
questions) were used for accuracy assessment as com-
pared to the mean monitor observations. An ROC anal-
ysis was calculated, with the area (Az) beneath the
ROC curve indicating diagnostic accuracy.
Inter-rater agreement was assessed using Cohen’s
kappa. To obtain symmetrical cross-correlation tables
required for a Kappa analysis, a positive and a negative
rating category were computed. The undecided rating
”uncertain (2)” was assigned to the negative category.
The monitor evaluation is based on all observations
from the 9 observers.
RESULTS
The analysis of the test-pattern prints revealed that 2
printers (CS5, THP) had difficulties with the criterion
”color gradient,” where either stripe artifacts were pro-
duced or, even worse, an inverse gradient was recog-
nizable. A low-contrast detail depicted by the final
letter of the writing “Bildqualität,” was reproduced by
none of the printers. THP, however, even failed to
reproduce the last 2 letters. All other features of the
pattern were printed sufficiently.
Inter-rater reproducibility was low for this investiga-
tion (Table II), with a mean kappa value of 0.49 (⫾
0.14) for the monitor versus 0.44 (⫾ 0.14) for the
printout observation. A separate evaluation of the ques-
tions used for accuracy analysis revealed very similar
values (printer: 0.45 ⫾ 0.16; monitor: 0.42 ⫾ 0.20). For
both evaluations, the Wilcoxon test indicated no signif-
icant difference (P ⫽ .53).
Interproximal caries had to be detected in 1 radio-
graph, where 50% of the surfaces according to the
consensus monitor evaluation either surely had a lesion
or the observers were uncertain. For this question, KDP
had the highest mean Az (0.893), followed by CS5
 OOOOE
582 Schulze et al October 2008

Table III. Mean Az values and their 95% confidence

intervals (CI) for all printers
95% CI, lower 95% CI, upper
Printer Mean Az boundary boundary
Kodak 1400 DPP 0.828 0.772 0.884
Epson Stylus Photo 0.802 0.742 0.862
RX 600
Canon S 500 0.801 0.741 0.861
Codonics Horizon 0.796 0.736 0.856
Tetenal HP Color 0.786 0.725 0.847
Inkjet Printer CP
1700

Fig. 3. Combined ROC curves over all observers and obser-

vations separated for each printer. The 95% confidence inter-
vals of the mean area underneath the ROC curves (Az) clearly
overlapped, indicating no significant difference among the
printers.

(0.886), CHO (0.849), EPS (0.835), and THP (0.831).

Apical translucencies according to the consensus mon-
itor ”gold standard” were certainly present in only 2 of
7 apices. A fraction of 4/7 was rated uncertain, whereas
1/7 was rated to certainly show no apical translucency.
Again, we observed highest average Az-values for
KDP (0.750), followed by CHO (0.729), CS5 (0.728),
EPS (0.713), and KDP (0.685).
Combining all available questions, the ROC analysis
revealed mean Az-values between 0.786 (THP) and
0.828 (KDP) for the printout as compared to the mon-
itor mean (Fig. 3, Table III). A clear overlap of the 95%
confidence intervals (CIs) of the mean Az is evident
from Table III. Fig. 4. The 95% confidence intervals of the mean ratings for
Regarding subjective ranking of image quality, all subjective assessment of print-out quality. Obviously, all ex-
95% CIs of the mean ranking ranged between 1.38 and cept the Tetenal printer were ranked on average between 1.5
1.95 (Fig. 4). According to the verbalization introduced and 1.6 (i.e., between ”diagnostically acceptable” and ”un-
above, this indicates a diagnostically acceptable qual- certain”). The interval of the Tetenal printer does clearly not
ity. Clearly, all 95% CIs except of the THP consider- overlap that of all other printers, with a mean ranking of 1.85.
ably overlap, indicating no significant difference
among the printers CS5, EPS, CHO, and KDP with
respect to subjective image quality. The THP printer,
however, received significantly lower subjective rat- from digital radiographs to pure digital files. This may
ings. seem somewhat outdated, since of course the digital
image file should rather be diagnosed on the computer
DISCUSSION monitor. There are instances, however, where low-cost
Printing dental radiographs with customary printers hardcopies may be useful. A simple example is that
ubiquitously available in modern working environ- files exported in common image file formats (JPEG,
ments could be a cost-efficient alternative to the pure TIFF), because of different compression algorithms,
digital file, e.g., for transferal to other offices. Even may not open correctly in every image software. In
though computers are available in almost every work- addition, not every dentist is a computer expert. Fur-
ing environment, many dentists still prefer hard copies thermore, official German regulations1,2 require appro-
OOOOE
Volume 106, Number 4
priate radiograph transfer also in instances where the
recipient is not yet using digital technology. For exam-
ple in Germany, ink-jet printers typically cost between
$120 and $500, with an estimated cost per glossy-paper
print between $1 and $4. Using budget nonproprietary
ink can further reduce these costs. Although the overall
print costs are relatively high, this has to be compared
to the high expense for dry or wet laser printers com-
monly used in medical radiography. The quality of
paper prints from digital radiographs has been assessed
by relatively few studies,3,6-8 where only7,8 are con-
cerned with accuracy evaluation on dental radiographs.
Two printers (CS5, EPS) used in this study were typical
off-the-shelf ink-jet printers and 1 (KDP) was a budget
semi-professional thermo-sublimation photo printer.
The remaining 2 printers (THP, CHO) were constructed
particularly for radiograph printing. The printers were
selected by (1) availability (CS5, EPS) and (2) with
respect to their design purpose (THP, KDP), with all 4
belonging to the budget and medium price segment
ranging between $150 and $650. The very expensive
CHO printer had been included as a control. Our basic
idea was to create a rather simple test pattern to cali-
brate printers in a way that, regardless of their actual
printing technique, they were capable of accurately
printing dental radiographs. Because of the large vari-
ety of settings for the different printers, our approach
was not to specify particular settings but rather specify
the required printing features, including fundamental
print properties like contrast, spatial resolution, color
gradient, and geometric accuracy. A test pattern was
specifically designed to enable printing-quality assur-
ance in the dental office. In the instance of a hardcopy
request, to reduce time consumption once a sufficient
setting configuration of the printer is found, it should be
recorded and used per default in the subsequent in-
stances of printing. Methods to enhance printing quality
typically include a cleaning procedure of the nozzles
followed by the adjustments specified earlier.
We only selected a few radiographs displaying typical
dental radiographic features such as interproximal carious
lesions, apical translucencies, and the periodontal liga-
ment space. They had been selected from the patient
records of 2 university hospitals and 4 dental offices. The
selection process beforehand was applied to (1) provide
typical images with relevant quality and to (2) reduce the
number of variables. Bearing in mind the low inter-rater
reproducibility, variation among several images may have
clouded the printer effect. In addition, the observer burden
due to the overall number of observations has to be taken
into account. For assessment of accuracy, only interprox-
imal carious lesions and apical translucencies in the set of
3 radiographs were prompted. Subjective rating of image
Schulze et al 583
quality was based on the display of the periodontal liga-
ment space, marginal bone level, and the pulp cavity.
Inter-rater reproducibility was low, a drawback that is
common knowledge in radiographic image interpreta-
tion,12,13 particularly for enamel approximal lesions,14
which may have been suspected in our images. Also, the
observers had not been specifically trained for the partic-
ular radiographic tasks. To compensate for this well-
known handicap, we decided to use a relatively high
number of experienced observers from many centers,
which also should eliminate bias due to familiarization
with center-specific image quality.10 Owing to the multi-
center design of the study, we had to use 2 viewing
monitors instead of only 1. Since the initial and follow-up
calibration required by German regulations1,2 provides a
high level of standardization, however, we believe that
this drawback should be of negligible influence.
A ”weak gold standard” was established using the
monitor ratings of 2 experts obtained in a consensus
conference. We are well aware that while the test under
evaluation represents the gold standard, considerable error
may be introduced.15,16 On the other hand, it is difficult to
establish a ground truth when using patient radiographs.
Since the monitor evaluation itself is considered the cur-
rent gold standard for digital radiograph viewing, how-
ever, testing against this standard should provide mean-
ingful results. It should be noted here, that no ”ground
truth” is really representing truth in a universal sense;
rather it represents the best possible information on the
object one can obtain on a state-of-the art level. To avoid
bias because of varying image size,17 the monitor display
had to approximately match the size of the prints. We
observed rather high Az values of around 0.8, indicating
good diagnostic performance as compared to the monitor.
The clear overlap of the 95% CIs of the mean indicate
nonsignificant differences between them.18 Neither the
high-cost color thermo-sublimation printer (CHO) nor the
radiographic image ink-jet printer Tetenal performed bet-
ter than their budget competitors. Actually, the signifi-
cantly lower subjective ratings (Fig. 4) plus the objec-
tively worse reproduction of test pattern features of the
Tetenal printer seem to be in line with the objective data
also located on the low end of the scale. Yet, in light of the
obvious overlap of the Az 95% CIs, this at most may
indicate a trend. Another interesting aspect is that the less
expensive ink-jet printers Canon S500 and Epson Stylus
Photo were ranked in the upper half in the objective
evaluation.
Our findings are in agreement with those of other
authors. Benediktsdottir and Wenzel7 conclude for the
assessment of position and morphology of mandibular
third molars, that ink-jet prints on glossy paper ”may be as
accurate as the original monitor-displayed digital pan-
oramic images.” Another study indicates that paper prints

584 Schulze et al
of digitized film radiographs compare well to their film-
based origin with respect to caries detection.8 Guerrant
and colleagues5 showed that thermo-sublimation prints
subjectively revealed equal diagnostic utility as monitor
display for various anatomic features. Yet subjective scor-
ings were found to be significantly better for expensive
direct-thermal film prints when compared to budget ink-
jet prints on glossy paper.6 From a physical point of view,
the dynamic transmission range of transparent media is
inherently larger than the dynamic range of reflection
media such as printing paper.19 There are 2 reasons for the
surprisingly good performance of the paper prints. First,
intrinsic contrast enhancement applied per default by the
manufacturer’s software is a likely factor. Since contrast
on reflection media is measured quite differently from that
of a transmission medium such as the monitor, however,
direct comparison is relatively complicated. Yet the re-
sults of the studies indicate that, if contrast enhancement is
applied, it seems to work for radiographic evaluation also.
Second, for prints acquired in color mode, because of the
superior color perception of humans the contrast is artifi-
cially enhanced. Unfortunately, not all manufacturers al-
low the user to control for the printing mode (color or
black and white). Consequently, we set aside this issue
according to our basic approach, that no matter what the
actual settings may be, as long as the test pattern features
are reproduced adequately, the print quality should be
sufficient. Obviously, our preliminary findings plus those
from other authors5,7,8 support this assumption. Because
some test features required a priori were not reproduced
sufficiently by all printers, we conclude that the pattern
introduced here may be even too demanding. Further
work has to be done in reducing the test features to the
necessary minimum.
CONCLUSION
The results of this preliminary study are promising
regarding the diagnostic accuracy on glossy-paper
prints obtained from budget printers for typical dental
radiographic diagnoses. Because of the preliminary na-
ture of this study and the rather thin databases available
in the literature, however, further studies are required
before a final statement about the safety of such low-
cost hardcopies in clinical use is feasible.
REFERENCES
1. Bundesregierung BRD. Verordnung zur Änderung der Röntgen-
verordnung und anderer atomrechtlicher Verordnungen. Bundes-
druckerei GmbH, Berlin, Germany, 2002.
2. Richtlinie zur Durchführung der Qualitätssicherung bei Röntge-
neinrichtungen zur Untersuchung oder Behandlung von Men-
schen - Qualitätssicherungs-Richtlinie (QS-RL). Gemeinsames
Ministerialblatt, Carl Heymanns Verlag KG, Cologne, Germany,
2004.
3. Kirkhorn T, Kehler M, Nilsson J, Lyttkens K, Andersson B,
Holmer N-G. Demonstration of digital radiographs by means of
OOOOE
October 2008
ink jet-printed paper copies: pilot study. J Digit Imaging
1994;5:246-51.
4. Lyttkens K, Kirkhorn T, Kehler M, Andersson B, Ebbesen A,
Hochbergs P, et al. Evaluation of the image quality of ink-jet
printed paper copies of digital chest radiographs as compared
with film: a receiver operating characteristic study. J Digit Im-
aging 1994;7:61-8.
5. Guerrant GH, Moore WS, Murchison DF. Diagnostic utility of
thermal printed panographs compared with corresponding com-
puter monitor images. Gen Dent 2001;49:190-6.
6. Gijbels F, Sanderink GC, Pauwels H, Jacobs R. Subjective image
quality of digital panoramic radiographs displayed on monitor
and printed on various hardcopy media. Clini Oral Invest
2004;8:25-9.
7. Benediktsdottir IS, Wenzel A. Accuracy of digital panoramic
images displayed on monitor, glossy paper, and film for assess-
ment of mandibular third molars. Oral Surg Oral Med Oral
Pathol Oral Radiol Endod 2004;98:217-22.
8. Otis LL, Sherman RG. Assessing the accuracy of caries diagno-
sis via radiograph. Film versus print. J Am Dent Assoc
2005;136:330.
9. Funke M, Obenauer S, Hermann KP, Fischer U, Grabbe E.
Softcopy- versus Hardcopybefundung in der digitalen Mammog-
raphie. Radiologe 2002;42:265-9.
10. Ishigaki T, Endo T, Ikeda M, Kono M, Yoshida S, Ikezoe J,
Murata K, Matsumoto M. Subtle pulmonary disease: detection
with computed radiography versus conventional chest radiogra-
phy. Radiology 2005;201:51-60.
11. Kosuda S, Kaji T, Kobayashi H, Watanabe M, Iwasaki Y, Ku-
sano S. Hard-copy versus soft-copy with and without simple
image manipulation for detection of pulmonary modules and
masses. Acta Radiol 2000;41:420-4.
12. Goldmann M, Pearson AH, Darzenta N. Endodontic success—
Who’s reading the radiograph? Oral Surg 1972;33:432-7.
13. Pascoal A, Lawinski CP, Honey I, Blake P. Evaluation of a
software package for automated quality assessment of contrast
detail images— comparison with subjective visual assessment.
Phys Med Biol 2005;50:5743-57.
14. Wenzel A. Digital radiography and caries diagnosis. Dentomax-
illofac Radiol 1998;27:3-11.
15. Wenzel A, Hintze H. Comparison of microscopy and radiogra-
phy as gold standards in radiographic caries diagnosis. Den-
tomaxillofac Radiol 1999;28:182-5.
16. Wenzel A, Hintze H. Editorial review. The choice of gold stan-
dard for evaluating tests for caries diagnosis. Dentomaxillofac
Radiol 2001;1999:132-6.
17. Versteeg KH, Sanderink GCH, Geraets WGM, van der Stelt PF.
Impact of scale standardization on images of digital radiography
systems. Dentomaxillofac Radiol 1997;26:337-43.
18. Henderson RA. Assessing test accuracy and its clinical conse-
quences: a primer for receiver operating characteristic curve
analysis. Ann Clin Biochem 1993;30:521-39.
19. Weinstein LM, Fitzer MS, Fitzer PM. Detail enhancement in
prints of radiographs. Use of a linear radial transmission filter.
Radiology 1975;115:726-8.
APPENDIX 1. (QUESTIONNAIRE)
Dear observer,
You will be presented with 3 intraoral radiographs
(No. 1, 2, 3). Please, position them separately on the
white cardboard enclosed in such a way that the number
is clearly readable in the left upper corner of the
image. The radiographs should be evaluated one after
OOOOE
Volume 106, Number 4 Schulze et al 585

another using an indirect lighting environment. Please tooth 47 □ definitely caries present
answer the questions in the questionnaire for each sin- mesially □ uncertain
gle radiograph according to their numbering, and check □ definitely no caries present
the answer best matching your decision for each 1.3. How would you judge the radiographic diagnos-
question (check only 1 answer per question!). tic significance of the marginal bone level separately
Please note: If questions refer to structures (teeth, for each interproximal region?
etc.) only partly visible on the radiographs, the ques- 1.3.1
tions are only applicable to that particular part of between tooth 44 □ diagnostically acceptable
and □ uncertain
the structure. tooth 45 □ diagnostically not acceptable
1.3.2
RADIOGRAPH NO. 1 between tooth 45 □ diagnostically acceptable
From the left to the right in this radiograph teeth no. and □ uncertain
tooth 46 □ diagnostically not acceptable
47, 46, 45, and 44 are (partly) visible.
1.3.3
1.1. How do you judge the radiographic diagnostic
between tooth 46 □ diagnostically acceptable
significance of the periodontal ligament gap for each and □ uncertain
tooth? tooth 47 □ diagnostically not acceptable
1.1.1 1.4. How would you judge the radiographic diagnos-
tooth 47 □ diagnostically acceptable tic significance of the filling in tooth 46 in the distal
□ uncertain approximal region?
□ diagnostically not acceptable
□ there is certainly a gap between filling and tooth
1.1.2 □ uncertain
tooth 46 □ diagnostically acceptable □ there is certainly no gap between filling and tooth
□ uncertain □ no filling is visible in tooth 46
□ diagnostically not acceptable
1.1.3
tooth 45 □ diagnostically acceptable
□ uncertain
RADIOGRAPH NO. 2
□ diagnostically not acceptable From the left to the right you (partly) see teeth no.
1.1.4 24, 25, 26, and 27.
tooth 44 □ diagnostically acceptable 2.1. How would you judge the diagnostic signifi-
□ uncertain cance of the radiographic periodontal ligament gap
□ diagnostically not acceptable separately for each tooth?
1.2. Please indicate your decision about the presence 2.1.1
or absence of an interproximal caries lesion separately tooth 24 □ diagnostically acceptable
for each interproximal region. □ uncertain
1.2.1 □ diagnostically not acceptable
tooth 44 □ definitely caries present 2.1.2
distally □ uncertain tooth 25 □ diagnostically acceptable
□ definitely no caries present □ uncertain
1.2.2 □ diagnostically not acceptable
tooth 45 □ definitely caries present
2.1.3
mesially □ uncertain tooth 26 □ diagnostically acceptable
□ definitely no caries present □ uncertain
1.2.3 □ diagnostically not acceptable
tooth 45 □ definitely caries present
2.1.4
distally □ uncertain tooth 27 □ diagnostically acceptable
□ definitely no caries present □ uncertain
1.2.4 □ diagnostically not acceptable
tooth 46 □ definitely caries present
2.2. How would you judge the radiographic diagnos-
mesially □ uncertain tic significance of the marginal bone level separately
□ definitely no caries present for each single interproximal region?
1.2.5 2.2.1
tooth 46 □ definitely caries present between tooth 24 □ diagnostically acceptable
distally □ uncertain and □ uncertain
□ definitely no caries present tooth 25 □ diagnostically not acceptable
1.2.6 2.2.2
 OOOOE
586 Schulze et al October 2008

between tooth 25 □ diagnostically acceptable (Cont’d)

and □ uncertain 3.1.3
tooth 26 □ diagnostically not acceptable
2.2.3 tooth 21 □ diagnostically acceptable
□ uncertain
between tooth 26 □ diagnostically acceptable
□ diagnostically not acceptable
and □ uncertain
□ no periodontal ligament gap present
tooth 27 □ diagnostically not acceptable
2.3.Please indicate your certainty about the presence 3.1.4
or absence of an apical translucency separately for each tooth 22 □ diagnostically acceptable
□ uncertain
tooth. □ diagnostically not acceptable
2.3.1 □ no periodontal ligament gap present
tooth 24 □ apical translucency definitely present 3.1.5
□ uncertain
tooth 23 □ diagnostically acceptable
□ apical translucency definitely not present
□ uncertain
2.3.2 □ diagnostically not acceptable
tooth 25 □ apical translucency definitely present □ no periodontal ligament gap present
□ uncertain 2. Please indicate your certainty about the presence
□ apical translucency definitely not present
or absence of an apical translucency separately for each
2.3.3
tooth.
tooth 26 □ apical translucency definitely present
□ uncertain
3.2.1
□ apical translucency definitely not present tooth 11 □ apical translucency definitely present
2.3.4 □ uncertain
□ apical translucency definitely not present
tooth 27 □ apical translucency definitely present
□ uncertain 3.2.2
□ apical translucency definitely not present tooth 21 □ apical translucency definitely present
2.4. The root canal filling of tooth 26 at the palatal □ uncertain
□ apical translucency definitely not present
root is:
3.2.3
□ definitely too long
□ uncertain tooth 22 □ apical translucency definitely present
□ definitely not too long □ uncertain
□ apical translucency definitely not present
3.3. How would you judge the diagnostic signifi-
cance of the pulp chamber of tooth 11?
RADIOGRAPH NO. 3 □ diagnostically acceptable
From the left to the right you see (at least partly) □ uncertain
teeth no. 12, 11, 21, 22, and 23. □ diagnostically not acceptable
□ no pulp chamber is visible for tooth 11
3.1. How would you judge the diagnostic significance
of the periodontal ligament gap separately for each tooth? Thank you very much for your help!
3.1.1
tooth 12 □ diagnostically acceptable
□ uncertain
□ diagnostically not acceptable Reprint requests:
□ no periodontal ligament gap present
Ralf Kurt Willy Schulze, DDS, Dr med dent habil
3.1.2 Assistant Professor
tooth 11 □ diagnostically acceptable Department of Oral Surgery (and Oral Radiology)
□ uncertain Johannes Gutenberg University
□ diagnostically not acceptable Augustusplatz 2
□ no periodontal ligament gap present 55131 Mainz, Germany
rschulze@uni-mainz.de