Journal of Clinical Epidemiology 61 (2008) 324e330

REVIEW ARTICLES

Efficient ways exist to obtain the optimal sample size in clinical trials
in rare diseases
J.H. van der Leea,*, J. Wesselinga,b, M.W.T. Tanckc, M. Offringaa,d
a
Department of Pediatric Clinical Epidemiology, Emma Children’s Hospital (ECH), Academic Medical Center (AMC), University of Amsterdam,
Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
b
Department of Pediatrics, Red Cross Hospital, Beverwijk, The Netherlands
c
Department of Clinical Epidemiology, Biostatistics and Bioinformatics, AMC, University of Amsterdam, The Netherlands
d
Department of Neonatology, ECH AMC, University of Amsterdam, The Netherlands
Accepted 12 July 2007

Abstract
Objective: Recruitment of pediatric patients in randomized clinical trials is hampered by the rarity of many conditions and by ethical
constraints. The objective of this paper is to give an overview of design options to obtain a statistically valid result while including a
minimum number of subjects.
Study Design and Setting: Overview and discussion of several approaches to conduct valid randomized clinical trials in rare diseases
and vulnerable populations.
Results: Sequential designs have been developed as efficient ways to evaluate accumulating information from a clinical trial, thereby
reducing the average size of trials. Different sequential procedures exist, including group sequential designs, boundaries designs, and adap-
tive designs. The sample size attained at the end of the trial is unknown at the start. The sample size for a given set of a, b, and effect size
may turn out to be larger than with a classical fixed sample size approach. Simulations have shown that on average, sample sizes are
smaller.
Conclusion: There are several possibilities to optimize the number of subjects in a clinical trial. The rarity of many disorders in
children and the ethical requirements in this patient population should not obstruct the performance of well-designed research to support
clinical decision making. Ó 2008 Elsevier Inc. All rights reserved.
Keywords: Epidemiologic research design; Sample size; Randomized controlled trials; Ethics; Rare diseases; Child

1. Introduction by legislation, which has been in effect since January 26,

2007 (http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?
At present, the evidence base for treatments in clinical
uri5CELEX:32006R1901:EN:NOT) [2]. As a result, an in-
pediatrics is, at best, weak [1]. Expectations on the effec-
crease in the number of drug trials in children is expected.
tiveness of most drug interventions in children are based
These developments have sparked the debate on various
on extrapolations from results of studies in adults. How- design issues in pediatric clinical research, including the
ever, responses to drug treatments and other interventions
‘‘optimal’’ number of patients to be included in these trials.
in children are often unpredictably different from adults
This debate is relevant because the number of eligible chil-
[1]. The need for clinical trials in children has been increas-
dren for pediatric trials is often small because many condi-
ingly recognized by pediatricians and pediatric research
tions are relatively uncommon in children, and recruitment
groups from all over the world. A recently published policy
of children into trials is challenging because the threshold
statement points out the importance of performing random-
for gaining consent is often high and complex. One poten-
ized controlled trials on health care interventions in chil-
tial solution is the application of techniques that minimize
dren [1]. Meanwhile, the European Union has taken the number of subjects in a randomized clinical trial. In the
measures to encourage pharmaceutical trials in children
present paper, we will present some readily available alter-
native approaches to conduct valid research while optimiz-
* Corresponding author. Tel.: þ31-20-566-3299; fax: þ31-20-696-
ing or minimizing the number of subjects to be included.
5099. The aim of this paper is to give an overview of several rel-
E-mail address: j.h.vanderlee@amc.uva.nl (J.H. van der Lee). atively unfamiliar design options that may be used to obtain
0895-4356/08/$ e see front matter Ó 2008 Elsevier Inc. All rights reserved.
doi: 10.1016/j.jclinepi.2007.07.008
 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330
a statistically valid result while including a minimum num-
ber of subjects to expand the readers’ knowledge of alterna-
tive possibilities to consider for designing future pediatric
trials. Some examples of recent pediatric trials that have
used these approaches are presented. Although this paper
is prompted by the challenges surrounding pediatric drug
trials, the methods described here can be applied to any dis-
cipline performing clinical trials in small or vulnerable
populations.
2. Optimal number of subjects in pediatric clinical
trials
From a statistical point of view, the number of subjects
in a trial should be as large as possible, thus increasing the
precision of effect estimation of the intervention (i.e., nar-
rower confidence intervals [CIs]) and enhancing the possi-
bility of a justified rejection of the null hypothesis (usually
stated as ‘‘There is no difference in the effect of the inter-
ventions’’) in favor of the alternative hypothesis (‘‘There is
a difference in the effect of the interventions’’). This prob-
ability of reaching a true positive conclusion is called the
power of a study [3].
When designing any clinical trial, the power should be
maximized. This can be achieved by maximizing the num-
ber of subjects included in the study. However, as discussed
above, there are practical and ethical restraints to the inclu-
sion of large numbers of subjects, especially in the pediatric
population.
Contrary to the statistical viewpoint, from an ethical
point of view the number of children participating in a trial
should be as small as possible. Not only to protect partici-
pating children from receiving the less favorable inter-
vention unnecessarily, when the information gathered is
already sufficient to conclude which intervention is supe-
rior, but also to start treating patients not participating in
the trial according to the evidently best intervention
strategy as soon as possible.
It is essential to conduct trials in children, but how can
the optimal number of participants be established given
the aforementioned constraints? Sample size calculations
offer a possibility to estimate the optimal number of pa-
tients needed to obtain a power of 80% or 90%. The prob-
lem of limited availability of subjects to be included in
a trial has led to different strategies used by researchers
to improve power. Several strategies are use of 1) compos-
ite or 2) surrogate outcomes, 3) the crossover design, 4) re-
peated measurements, 5) analysis of covariance instead of
simple comparison of outcomes in two groups, 6) conduct-
ing an under-powered trial for a later meta-analysis, 7) the
prospective meta-analysis approach, 8) use of one-sided in-
stead of two-sided hypothesis testing, and 9) inflation of the
minimal clinically relevant difference. The first five strate-
gies have to do with truly improving the power; strategies
#6 and #7 make use of meta-analysis techniques, strategy
#8 changes the assumptions underlying the trial, and thus
325
will lead to results that answer a somewhat different re-
search question, and strategy #9 (mis)uses the fact that
a minimal clinically relevant difference is arbitrary and that
there is no standardized way to its determination [4]. All
these strategies have their pros and cons, which have to
be weighed carefully in the design phase of each trial. More
detailed discussion of these approaches can be found in nu-
merous articles and textbooks (e.g., [4e7]).
3. Alternative approaches minimizing the number
of subjects to be included
3.1. Sequential designs
The principles of randomized clinical trials were origi-
nally derived from agricultural research, in which the statis-
tical methods of analysis of variance and regression
analysis were developed [8]. An essential difference be-
tween agricultural and clinical trials is the time period dur-
ing which data are gathered. In an agricultural field trial, all
crops are harvested simultaneously at the end of the growth
season, whereas in clinical trials it usually takes weeks or
months, sometimes even years before all subjects are in-
cluded, and consequently the gathering of the outcome data
is extended over a similarly long time period. Thus, it is
possible that before the end of the inclusion phase of a clin-
ical trial enough information has already been assembled,
though usually not analyzed, to decide which intervention
is superior. Because subjects will be included and random-
ized until the sample size that was determined in advance
has been reached, this may lead to inefficiency, that is, in-
appropriate inclusion of subjects, unnecessarily prolonging
the trial duration, and, more importantly, to allocation of
trial subjects to the ‘‘inferior’’ intervention at a time when
the evidence of its inferiority might be available. Sequential
designs were developed to overcome these problems. Vari-
ous sequential procedures exist, and they can be (roughly)
divided into two types, namely those derived from the
repeated significance test approach, also called group se-
quential designs, and those derived from the boundaries ap-
proach. Because there is no uniform terminology, we make
a distinction between ‘‘design’’ and ‘‘analysis’’ [9]. Thus,
within the boundaries design a distinction can be made
between ‘‘continuous sequential analysis’’ and ‘‘group se-
quential analysis.’’ Recently, a modification of the group
sequential design has been proposed, called the adaptive
design. We will discuss the group sequential design, the
boundaries design, and adaptive design in more detail
below.
3.1.1. Group sequential design, repeated significance
testing
In this approach, a series of conventional statistical anal-
yses, so-called interim analyses, is carried out at various
predetermined time points on the accumulating data. The
basis for this methodology was laid by Armitage in the
326 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330
1970s [10,11]. To ensure an overall type I error probability
(a), the significance thresholds at the various time points
are adjusted to allow for the repetition (a*). Various
methods for a-adjustment have been proposed, either with
the same a* for all analyses, Pocock’s method [12], or with
different a* at each analysis, for example, O’Brien and
Fleming [13], and Lan and DeMets [14]. In contrast to
the earlier methods of Pocock and O’Brien and Fleming,
the so-called ‘‘a-spending function’’ developed by Lan
and DeMets [14] that characterizes the rate at which the
a is spent and thus determines the a* at each interim anal-
ysis, only depends on the number of past and current in-
terim analyses, and not on the number of future interim
analyses. This approach enhances the flexibility of the de-
sign, because the number of (interim) analyses does not
have to be predetermined. Further modifications included
the possibility of stopping early when there is no relevant
difference between the intervention groups (stopping for fu-
tility) [15,16]. Because, in general, the assumed effect size
is too optimistic, the probability of early stopping is very
low, depending on the type of a-spending that is used.
Therefore, it is unrealistic to plan a trial that is too large
to be conducted, in the hope that it can be stopped at an
early stage.
An example of repeated significance testing is a con-
trolled clinical trial to investigate whether the survival
and neurological outcomes of pediatric patients requiring
airway management in an out-of-hospital emergency situa-
tion differed between those treated with bagevalveemask
ventilation or with endotracheal intubation [17]. A group
sequential design was used with an a correction according
to O’Brien and Fleming to allow for early stopping if the
outcome in one of the treatment groups would be much bet-
ter or worse than that in the other group [13]. The trial was
designed to have 80% power to detect an increase in ‘‘sur-
vival to hospital Emergency Room’’ from 5% to 10% with
a two-sided a of 0.05, resulting in a required sample size of
800 patients, with three interim analyses after each 200 pa-
tients. The interim analyses did not result in stopping the
trial, and the trial was continued until 830 patients were in-
cluded. The final intention-to-treat analysis showed no sig-
nificant difference between the two airway management
strategies for both outcomes. The clinical interpretation of
these results gave rise to some discussion [18e21]. How-
ever, this is outside the scope of this paper.
Another example, in which the effect size estimation
appeared too conservative in retrospect, is a randomized clin-
ical trial to investigate the difference in incidence of injection
pain during intravenous induction of anesthesia in children
between a new formulation (EtomidateeÒLipuro) and the
existing standard of propofol with added lidocaine [22].
The required sample size, calculated based on an expected
proportion of 25% in the propofolelidocaine group and 5%
in the EtomidateeÒLipuro group, a of 0.05 and power of
90%, was reported to be 110. On request of the Ethics Com-
mittee, an interim analysis was planned after the inclusion
of 80 patients, with a nominal value a* 5 0.02. The rationale
for the interim analysis was to prevent unnecessary injection
pain in children if there was a difference in pain incidence
between the two groups. The underlying considerations for
the timing and a* of the interim analysis were not reported.
The study was ended when this interim analysis showed a
significantly lower incidence of injection pain in the Etomi-
dateeÒLipuro group (5%; 95% CI 5 0.61%e16.9%) than
in the propofolelidocaine group (47.5%; 95% CI 5 31.5%e
63.9%) (P 5 0.0007) [22].
3.1.2. Boundaries design
This approach relies on a graphical rule, where a V sta-
tistic, representing the amount of information gathered in
the course of a trial, is plotted on the X-axis, and a Z statis-
tic, representing the effect size, is plotted on the Y-axis.
Prior to the start of the experiment, the boundaries are cal-
culated based on the alternative hypothesis and the desired
levels of the type I and II errors (a and b, respectively). Ex-
amples of boundaries are shown in Fig. 1. At each analysis,
which can be done after each patient (continuous sequential
analysis) or after a fixed or variable number of patients
(group sequential analysis), the two statistics Z and V are
calculated based on the data accumulated thus far, and plot-
ted, creating a so-called sample path. This is illustrated in
Fig. 2. Inclusion and randomization of subjects is continued
as long as the sample path remains between the boundaries
(continuation region). A conclusion is reached when a
boundary is crossed. In case of a one-sided test, that is, in-
vestigating only whether the experimental treatment leads
to a better outcome than the control treatment, a single pair
of boundaries is plotted (Fig. 1aec). Crossing of the upper
boundary leads to rejection of the null hypothesis; crossing
of the lower boundary leads to nonrejection of the null hy-
pothesis. Two pairs of boundaries are plotted symmetrically
around the horizontal axis in case of a two-sided test, in
which both possibilities of better and worse outcomes of
the experimental compared to the control treatment are in-
vestigated (Fig. 1d). Crossing of the upper or lower bound-
ary leads to the conclusion that the experimental treatment
is superior or inferior, respectively; crossing of the bound-
aries between both pairs of boundaries leads to nonrejection
of the null hypothesis.
Tests of this type are descendants of the sequential prob-
ability ratio test (SPRT) of Wald [8,23]. Initially, these
methods could only be used to compare a single proportion
or mean to a hypothetical value, or for the comparison of
two proportions (paired observations). These restrictions
hampered a successful application of these types of tests.
After modifications by Whitehead [24,25], comparison of
two independent groups with respect to continuous, bino-
mial (e.g., alive/dead), or censored outcomes (survival data)
was possible, thus increasing the applicability of the
methods.
In an SPRT, the boundaries are parallel (Fig. 1a), giving an
infinite ‘‘continuation region.’’ Therefore, it is theoretically
 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330 327

Fig. 1. (aec) Boundaries for a one-sided superiority trial with one-sided a 5 0.05 and b 5 0.10, using a) a single sequential probability ratio test (SPRT), b)
a single truncated SPRT with a maximal sample size of 600, c) a single triangular test. (d) Boundaries for a two-sided trial with two-sided a 5 0.05 and
b 5 0.10, using a double triangular test. All four trials were designed to detect a difference between means of 0.25 standard deviations. The vertical dotted
line represents the V value corresponding with the equivalent fixed sample size VFIXED.

possible that a trial requires an almost infinite value of V
(and thus n), which renders this test impracticable for
clinical trials. Alternatives that are more useful in clinical
trials are the truncated SPRT (Fig. 1b) and the triangular
test (TT) (Fig. 1c) [8,26]. In the truncated SPRT, a maxi-
mum value (L) of V is determined after which the trial
Fig. 2. One-sided superiority triangular test and sample path of trial of
Bellissant et al. [30].
stops, irrespective of the result. This truncation point L
is chosen more or less arbitrarily, but it has to be beyond
the amount of information VFIXED required for the equiv-
alent fixed sample design to correct for multiple hypothe-
sis testing. The space between the parallel boundaries is
influenced by the choice of L. In other words, the choice
of L has consequences for the sample size. For a given a,
b, and effect size, the boundaries are closer to each other
with increasing L. In case of the TT (Fig. 1ced) the
boundaries are convergent, resulting in a finite ‘‘continua-
tion region.’’ Therefore, the optimal properties of the
SPRT and TT differ: the average sample size reduction is
larger with the SPRT when the actual effect size is (much)
larger than expected and smaller when the actual effect
size is smaller than expected (Fig. 1a vs. 1c) [27]. When
the actual and expected effect sizes are similar, the ex-
pected sample size reductions are about equal. In most
trials, the actual effect size turns out to be smaller than
expected. In those cases the TT is more efficient [27].
In contrast to the classical fixed sample size design, the
eventual amount of information, that is, number of patients,
needed to complete a trial is unknown at the start of a se-
quential trial. Although correction for multiple testing
results in a larger maximal possible sample size (the
amount of information which is represented by the apex
of the triangle in Fig. 1c and d), the average sample size
328 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330
needed to complete a trial using a sequential method in
simulations was always smaller than that of the correspond-
ing fixed design, irrespective of the effect size or power
[27]. The 90th percentiles of the sample size distributions
of the sequential designs were in the same order of magni-
tude as the corresponding fixed designs, when the actual
effect was close to the expected effect, but larger when
the actual effect was smaller than expected, especially for
the SPRT.
In principle, Z and V can be calculated after each indi-
vidual patient, but generally Z and V are calculated after
the data of a number of patients have become available, that
is, group sequential analysis. Given the possibility that in-
termediate points, if plotted, could have lain outside the tri-
angular region during long gaps between inspections, and
thus opportunities for stopping might have been missed,
an adjustment of the stopping boundaries is made, resulting
in a so-called Christmas tree shape (Fig. 2). After a bound-
ary has been crossed, an adjusted point estimate and CI of
the effect can be calculated with the computer program
PEST or EaSt [28,29]. Due to the sequential nature of the
analysis, the CIs are wider than those obtained with conven-
tional fixed sample size methods of analysis.
Bellissant et al. described a TT to assess the efficacy of
metoclopramide on gastroesophageal reflux in infants [30].
The trial was designed to detect a mean benefit on a contin-
uous outcome scale of 0.5 with an expected standard devi-
ation of 0.5 with 95% power and a one-sided a of 0.05. In
a fixed design, 23 patients per treatment arm would have to
be included. The authors anticipated that recruitment would
be difficult and wanted to stop the study as soon as suffi-
cient information was collected and decided, therefore, to
use the TT. After 3 years and 9 months and inclusion of
39 children, the trial ended in futility, because the lower
boundary was crossed (Fig. 2). The observed benefit of me-
toclopramide over placebo was approximately 0.2 instead
of 0.5.
3.2. Adaptive or flexible design
Adaptive designs share a number of features with se-
quential designs, in which the null hypothesis is tested at
a sequence of interim analyses [31]. However, in contrast,
the design of an adaptive trial can be changed based on full
knowledge gained from the interim analyses. When modifi-
cations are made, a new phase of trial starts, and data accu-
mulated in (an) earlier phase(s) is no longer combined with
data from the new phase. All phases are analyzed sepa-
rately, and the P-values of the different phases are then
combined using a predefined rule. Examples of combina-
tion rules are the product criterion of Fisher [32] and the in-
verse normal method by Lehmacher [33]. The emphasis in
these designs is more on flexibility of the design than on
minimization of the average sample size.
Different adaptations are possible including reassess-
ment of sample size (see critical reflection by [34]),
selection of treatments (e.g., [35,36]), adaptation of end-
points (e.g., [37,38]), or inserting or deleting interim anal-
yses. See for a more extensive description of this design the
papers by, for example, Bauer and Brannath, Posch et al.,
Chang et al., Shen and Cheng [39e42]. However, many
features, for example, definition of stopping rules [43] or,
interpretation of the results when primary endpoints have
been changed, for instance, are still subject for debate
[44]. The term ‘‘adaptive design’’ is a comprehensive term
comprising many possible design adaptations [44]. It
should not be confused with the more specific term ‘‘adap-
tive treatment allocation design,’’ where the allocation ratio
can be adapted based on preliminary results from the trial
[45]. To our knowledge no pediatric trials with an adaptive
design have been published so far. Because it maximizes
the efficiency of data gathering from individual patients,
this approach deserves more attention [46].
4. Discussion
The basic approach of trying to minimize the number of
subjects needed to obtain enough information either to de-
cide which intervention is best, or to decide that the inclu-
sion of patients in the trial can be stopped because of
futility [47,48] is appealing. There is no simple general rule
to decide which of the aforementioned approaches to re-
duce the necessary number of children to be included in
a trial is most appropriate in specific circumstances. The
trial about ventilation techniques in the emergency out-of
hospital setting in the example above might have been
stopped earlier because of futility if a boundaries design
had been used instead of the O’Brien and Fleming method
[17,48]. Although on average the boundaries approach
leads to inclusion of less subjects than the fixed sample size
approach [24,27], there is no guarantee for any individual
trial that this will be the case.
An important prerequisite for the boundaries approaches
is that the time between inclusion and outcome measure-
ment is short in comparison to the accrual rate. If many
patients can be included in a short period, using effects
measured in only a part of those patients to decide when
to stop including patients makes no sense. The boundaries
approaches can also be used for survival analysis. However,
if the median survival time is long compared to the rate of
inclusion, again, the analysis will be too late to make a
sensible decision about ending the inclusion of patients.
Although this does not lead to a reduction in sample size, it
may lead to a reduction in total trial duration.
It is evident that in a sequential design, the boundaries
are designed based on prior knowledge and assumptions,
and that these boundaries should not be changed based on
information gathered during the sequential analyses. Never-
theless, some mid-trial design reviews are possible [49].
In this paper, we have also discussed the adaptive de-
signs, in which the assumptions are deliberately changed
 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330
during the trial, based on information gathered in a so-
called internal pilot. The advantage of this type of design
is that all information gathered is used for the analysis.
For these designs in particular, the prospective recording
of assumptions and adaptations ‘‘along the way’’ is of ut-
most importance for the credibility of the final conclusions
of a report. This can be guaranteed if all randomized trials,
including sequential and adaptive designs, are registered in
a prospective trial register, with documentation of their ap-
proaches to sample size calculation. Furthermore, to pre-
clude bias due to changing experimental conditions, it is
essential that the data analysis is performed independently
from the actual performance of the trial, that is, inclusion
and treatment of patients, and assessment of the outcome.
An independent data monitoring committee should decide
about sample size adaptations, continuing or stopping the
inclusion of subjects without giving any information on
the details of the analysis to those involved in the trial per-
formance. A decision to stop a trial should not be made
light-heartedly. It will be much more difficult to defend fu-
ture trials on the same intervention after one trial has been
stopped early.
One of the obstacles for designing and publishing a se-
quential trial with a boundaries approach is probably that
this approach is, so far, relatively unknown among re-
searchers and journal editors. We suggest that, in addition
to the CONSORT statement for randomized clinical trials
[www.consort-statement.org], a list of criteria is developed
to assess the validity of sequential trials using the bound-
aries approach, and to establish minimum criteria for
reporting this type of trial.
An objection that was brought forward by an expert was
that he worried about the external validity of a sequential
trial, because the results obtained in such a small sample
may be due to selection bias (Jan Tijssen, oral communica-
tion, April 2006). In our opinion, this argument could be
used against all randomized controlled trials, not specifi-
cally those using the boundaries approach. Yet another ob-
jection might be that attempts to reduce the sample size in
a trial also reduce the ability of detecting side effects or ad-
verse effects. If side effects are expected to be a substantial
issue, the trial should be powered in such a way that they
can be reasonably evaluated. Apart from that, there are
many side effects or adverse effects that are too infrequent
to be detected even in large trials. Clinicians always have to
be on the alert for possible unwanted effects.
In conclusion, the approaches to optimize the number of
subjects in a trial described in this paper show that limita-
tions in the available numbers of patients should not be ac-
cepted as a prime reason not to conduct a trial to answer
a clinically relevant question. There are several possibilities
to minimize the sample size necessary to yield a valid re-
sult. Nevertheless, it is important to remain critical and alert
for false-positive results, especially when effect sizes are
larger than expected and when vested (financial) interests
may be at stake [50]. Replication of results in different
329
settings will still be necessary. However, the rarity of many
disorders in children and the ethical requirements in this
patient population should not obstruct the performance of
well-designed research to support clinical decision making.
References
[1] Caldwell PH, Murphy SB, Butow PN, Craig JC. Clinical trials in
children. Lancet 2004;364:803e11.
[2] Clinical trials in children, for children. Lancet 2006;367:1953.
[3] Altman DG. Practical statistics for medical research. London:
Chapman & Hall; 1991.
[4] Schulz KF, Grimes DA. Sample size calculations in randomised
trials: mandatory and mystical. Lancet 2005;365:1348e53.
[5] Bland JM, Altman DG. One and two sided tests of significance. BMJ
1994;309:248.
[6] Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical
trials. 3rd ed. New York: Springer-Verlag; 1998.
[7] Sackett DL, Cook DJ. Can we learn anything from small trials? Ann
N Y Acad Sci 1993;703:25e31.
[8] Whitehead J. The design and analysis of sequential clinical trials.
Chichester, UK: John Wiley and Sons; 1997.
[9] Sebille V, Bellissant E. Sequential methods and group sequential de-
signs for comparative clinical trials. Fundam Clin Pharmacol
2003;17:505e16.
[10] Armitage P. Sequential methods in clinical trials. Am J Public Health
1958;48:1395e402.
[11] Armitage P. Sequential medical trials. Oxford: Blackwell Scientific
Publications; 1975.
[12] Pocock SJ. Group sequential methods in the design and analysis of
clinical trials. Biometrika 1977;64:191e9.
[13] O’Brien PC, Fleming TR. A multiple testing procedure for clinical
trials. Biometrics 1979;35:549e56.
[14] Lan KKG, DeMets DL. Discrete sequential boundaries for clinical
trials. Biometrika 1983;70:659e63.
[15] Emerson SS, Fleming TR. Symmetric group sequential test designs.
Biometrics 1989;45:905e23.
[16] Pampallona S, Tsiatis AA. Group sequential designs for one-sided
and two-sided hypothesis testing with provision for early stopping
in favor of the null hypothesis. J Stat Plan Infer 1994;42:19e35.
[17] Gausche M, Lewis RJ, Stratton SJ, Haynes BE, Gunter CS,
Goodrich SM, et al. Effect of out-of-hospital pediatric endotracheal
intubation on survival and neurological outcome: a controlled clinical
trial. JAMA 2000;283:783e90.
[18] Cristofani C. Out-of-hospital endotracheal intubation of children.
JAMA 2000;283:2791.
[19] Eckstein M. Out-of-hospital endotracheal intubation of children.
JAMA 2000;283:2790.
[20] Nieman C, Merlino J, Polk JD, Kovach B, Mancuso C, Fallon WF Jr.
Out-of-hospital endotracheal intubation of children. JAMA 2000;283:
2790e1.
[21] Sagel JS. Out-of-hospital endotracheal intubation of children. JAMA
2000;283:2791e2.
[22] Nyman Y, Von HK, Palm C, Eksborg S, Lonnqvist PA. Etomidate-
Lipuro is associated with considerably less injection pain in children
compared with propofol with added lidocaine. Br J Anaesth 2006;97:
536e9.
[23] Wald A. Sequential analysis. New York: Wiley); 1947.
[24] Whitehead J, Jones DR. The analysis of sequential clinical trials.
Biometrika 1979;66:443e52.
[25] Whitehead J, Stratton I. Group sequential clincial trials with triangu-
lar continuation regions. Biometrics 1983;39:227e36.
[26] Anderson TW. A modification of the sequential probability ratio test
to reduce the sample size. Ann Math Stat 1960;31:165e97.
330 J.H. van der Lee et al. / Journal of Clinical Epidemiology 61 (2008) 324e330
[27] Sebille V, Bellissant E. Comparison of four sequential methods
allowing for early stopping of comparative clinical trials. Clin Sci
(Lond) 2000;98:569e78.
[28] PEST 4.4 Operating manual. Reading, UK: The University of Reading;
2004.
[29] Cytel Software Corporation. EaSt: A software package for the design
and interim monitoring of group sequential clinical trials. Cambridge,
MA: Cytel Software Corporation; 1992.
[30] Bellissant E, Duhamel JF, Guillot M, Pariente-Khayat A, Olive G,
Pons G. The triangular test to assess the efficacy of metoclopramide
in gastroesophageal reflux. Clin Pharmacol Ther 1997;61:377e84.
[31] Wassmer G. Basic concepts of group sequential and adaptive group
sequential test procedures. Stat Pap 2000;41:253e79.
[32] Bauer P, Kohne K. Evaluation of experiments with adaptive interim
analyses. Biometrics 1994;50:1029e41.
[33] Lehmacher W, Wassmer G. Adaptive sample size calculations in
group sequential trials. Biometrics 1999;55:1286e90.
[34] Jennison C, Turnbull BW. Mid-course sample size modification in
clinical trials based on the observed treatment effect. Stat Med
2003;22:971e93.
[35] Hommel G. Adaptive modifications of hypotheses after an interim
analysis. Biometrical J 2001;43:581e9.
[36] Kelly PJ, Stallard N, Todd S. An adaptive group sequential design for
phase II/III clinical trials that select a single treatment from several.
J Biopharm Stat 2005;15:641e58.
[37] Bauer P, Kieser M. Combining different phases in the development
of medical treatments within a single trial. Stat Med 1999;18:
1833e48.
[38] Kieser M, Bauer P, Lehmacher W. Inference on multiple endpoints in
clinical trials with adaptive interim analyses. Biometrical J 1999;41:
261e77.
[39] Bauer P, Brannath W. The advantages and disadvantages of adaptive
designs for clinical trials. Drug Discov Today 2004;9:351e7.
[40] Chang M, Chow SC, Pong A. Adaptive design in clinical research:
issues, opportunities, and recommendations. J Biopharm Stat 2006;16:
299e309.
[41] Posch M, Koenig F, Branson M, Brannath W, Dunger-Baldauf C,
Bauer P. Testing and estimation in flexible group sequential designs
with adaptive treatment selection. Stat Med 2005;24:3697e714.
[42] Shen Y, Cheng Y. Adaptive design: estimation and inference with cen-
sored data in a semiparametric model. Biostatistics 2007;8:306e22.
[43] van Houwelingen HC. On ‘‘Bayesian monitoring’’. J Clin Epidemiol
1999;52:713e4.
[44] Committee for medicinal products for human use (CHMP). Reflec-
tion paper on methodological issues in confirmatory clinical trials
with flexible design and analysis plan. London: European Medicines
Agency (EMEA); 2006.
[45] Coad DS, Ivanova A. The use of the triangular test with response-
adaptive treatment allocation. Stat Med 2005;24:1483e93.
[46] Hirtz DG, Gilbert PR, Terrill CM, Buckman SY. Clinical trials in
childrendhow are they implemented? Pediatr Neurol 2006;34:436e8.
[47] Whitehead J. Stopping rules for clinical trials. Control Clin Trials
2004;25:69e70.
[48] van der Tweel I, van Noord PA. Early stopping in clinical trials and ep-
idemiologic studies for ‘‘futility’’: conditional power versus sequential
analysis. J Clin Epidemiol 2003;56:610e7.
[49] Whitehead J, Whitehead A, Todd S, Bolland K, Sooriyarachchi MR.
Mid-trial design reviews for sequential clinical trials. Stat Med
2001;20:165e76.
[50] Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH,
Briel M, et al. Randomized trials stopped early for benefit: a system-
atic review. JAMA 2005;294:2203e9.