Abstract
Objectives: To describe the health care settings, purposes, and study reporting quality of the 2 2 cluster randomized controlled facto-
rial trial design.
Study Design and Setting: This study is a systematic review. We searched Medline, Embase, Cochrane Library, and Web of Knowl-
edge for articles published up to May 2012. Cluster randomized controlled 2 2 factorial trials in health, evaluating at least one complex
intervention, were included. Two authors independently reviewed and extracted data from the studies.
Results: Twenty-nine studies covering a wide range of clinical areas and health care settings were included. The cluster design was
mostly used to minimize contamination. The factorial design was mostly used to assess the effects of two interventions in the same study
and to explore interactions between interventions. However, although most studies explored the presence or absence of intervention inter-
actions, they were often either not sufficiently powered to detect any interactions or did not provide information on whether the study was
sufficiently powered to detect any interactions. There was a considerable variability in the reporting of a number of study characteristics and
methodological aspects. Study quality was also variable within and across studies.
Conclusion: The design has been used in a wide range of health care settings and clinical areas to minimize contamination, assess the
effects of two interventions in the same study, and explore intervention interactions. There is need for improvement on and guidelines for
the reporting of factorial trials. 2014 Elsevier Inc. All rights reserved.
Keywords: Systematic review; 2 2; Factorial; Cluster randomized; Randomized controlled trial; Complex interventions
the factorial design: factorial, 2 2 and 2 by 2 [2,3]. The Reviews and Dissemination [6] to take account of cluster
full search strategy for Medline is shown in Appendix, and randomization. Our quality assessment criteria therefore
this was adapted accordingly for the other databases. We comprised the following items: accounting for the cluster
also hand searched the reference lists of previous reviews design in sample size calculation and in the analysis, ade-
of factorial design studies [2,3]. We limited the search to quacy of sequence generation, recruitment of patients
English language. However no publication date limits or before cluster allocation to study groups, evidence of clus-
geographic limits were applied. ter imbalances, participant attrition, and cluster attrition.
Table 1. Continued
Sample size;
Randomization unit cluster size;
Study Country (cluster-cluster/split plot) number of clusters Participants
Watson 2008 UK Split plot (GP/individual patients) 200; d; 91 Patients over 16 years consulting
with pain in one or both
shoulders for 12 months
Wesson 2008 Kenya Cluster-cluster (public health 611; d; 45 Family planning (FP) providers and
clinics) community-based distribution
agents at rural FP clinics
Zillich 2008 USA Cluster-cluster (primary care 284; 5.9 (standard Continuously enrolled Medicaid
prescribers) deviation [SD], 8.4, members at high risk of CVD, not
range, 1e51) per receiving statin medication in
practice and 2.2 (SD, the 18 months before the
2.0, range, 1e13) intervention
per provider; 48
in the United States [20,23,30,31,33,34,36], three in France system interventions at a population level [18], and the nature
[9,24,32], two in Canada [11,17], two in The Netherlands of the intervention which meant randomization had to be at
[12,35], and one each in Pakistan [18], Denmark [22], Ger- cluster level [27]. Twenty studies gave at least one reason for
many [27], and Kenya [29]. The clusters varied widely and using a factorial design and the reasons were to assess the
included family and/or general practitioner practices, hospi- effects of two interventions independently in the same
tals, wards within hospitals, communities, and pharmacies. study [10,12e16,19e24,26,28,29,31e35] and to explore
Most studies recruited adult patients. interactions between the interventions [10,12e15,20,21,24,
Of the 29 included studies, 23 described the studies as 29,31e33,35].
randomized in both the title and the abstract, whereas this
was stated only in the abstract for three studies, only in 3.5. Assessment of intervention interaction effects
the title for two studies, and neither the title nor the abstract
for one. The cluster design was specified in both the title Results for the assessment of interaction effects are summa-
and the abstract by 13 studies, only in the title by one, only rized in Table 3. Eight studies specified an a priori interaction
in the abstract by seven, and neither in the title nor in the hypothesis [11,12,14,17,18,23,25,28]; however, none of these
abstract in the remaining eight. Only five studies were studies clearly specified the rationale for that hypothesis. Thir-
described as a factorial design in both the title and the ab- teen studies stated exploring interactions as the rationale for
stract, whereas this was stated only in the abstract for 22 using a factorial design [10,12e15,20,21,24,29,31e33,35],
studies, only in the title for one, and neither in the title of these, five studies assumed no interaction in the sample size
nor in the abstract for one. calculation [12,13,15,24,29], one study included interaction
effects in the sample size calculation [35], one study reported
not being statistically powered to detect any interactions [14],
3.3. Areas of health research in which the design has
been used and it was unclear in six studies whether the study had been
powered to explore interactions [10,20,21,31e33]. Of the 21
The included studies covered a wide range of clinical studies that assessed the presence or absence of interactions
conditions, areas of health research, and interventions. [9,12e16,18,20,21,23,25,26,28e30,32e37], only three
These are shown in Table 2. The interventions generally studies were reported to be statistically powered to detect an
aimed to improve at least one of the following: knowledge interaction [9,18,35] and five assumed no interaction in
transfer, guideline implementation, disease prevention, the sample size calculation [12,13,15,26,29]. Statistically sig-
health care professional practice and quality of care, refer- nificant interaction effects were observed in 4 of the 21 studies
rals, treatment decisions, health care utilization, and the assessing the presence or absence of an interaction [13,
management of specific diseases. 18,20,35].
of the 91 recruited clusters, 51 did not recruit any partici- that assessed the presence or absence of interactions actu-
pants and 19 withdrew from the study [37]; and for one ally found a statistically significant interaction. In their re-
the attrition rate was not clearly specified [35]. Participant view, McAlister et al. reported that only 2 of 29 trials
follow-up rates, or the proportion of participants for which actually found a statistically significant interaction and
outcome data were available at the end of the follow-up seven had clinically significant interaction ratios (1.25
period, were 100% for seven studies [12,17,19,26,28, or 0.8) [2]. They concluded that investigators conducting
30,36], 75e99% for 16 studies [9,11,14,15,18,21,23e25,27, factorial design trials are appropriately choosing to test on-
28,31e34,37], 50e74% for two studies [10,13], !50% for ly those interventions that do not have potential for substan-
three studies [16,20,35], and data on follow-up rates were tive interactions. For our present review, however, it is
not reported for the remaining two. difficult to conclude whether this is the case or whether
the lack of interactions was due to the majority of studies
being underpowered to detect any interaction. In relation
to this, we recommend that investigators specify their a pri-
4. Discussion
ori hypothesis of whether an interaction exists or does not
The 2 2 cluster randomized factorial design has been exist between interventions, as well as a rationale for such
used in a wide range of health care settings and clinical a hypothesis. A factorial design should not be used if a sub-
areas: mostly for educational interventions and patient stantive intervention interaction is suspected (or sought) un-
assessment and disease and/or condition management stra- less the trial is adequately powered. It is a good design if
tegies. The design has mostly been used in developed coun- one is confident there is no interaction or alternatively if
tries and with cluster randomization for both interventions the study specifically wants to estimate the interaction
rather than the split plot design. The most commonly cited and powers the study accordingly.
reason for the factorial design was efficiency (ie, to assess Torgerson suggests that when considering cluster
the effects of two interventions independently in the same randomization over individual randomization as a way to
study), followed by exploring interactions between the in- minimize contamination, researchers have to be certain that
terventions, as has been observed in other reviews [2,3]. for their trial, contamination is real rather than a theoretical
The cluster design is chosen mostly as a means of mini- possibility [40]. However, many trial reports do not give de-
mizing contamination. tails on why contamination is a real possibility. For
Some researchers have recommended reporting of esti- example, of the seven studies that cited the possibility of
mates of interaction between interventions for factorial contamination, only one gave details: once general practi-
design trials as desirable as unrecognized interactions could tioners within a practice had been trained in new communi-
distort results and their interpretation [38,39]. From our re- cation skills they could not switch at random between using
view, however, most studies either reported not being suffi- these skills and usual consulting practice [12].
ciently powered to detect any interaction or did not provide One of the problems associated with cluster randomiza-
information on the matter. Only 4 of the 21 studies (19%) tion is the possibility of recruitment bias where the person
1090 N.D. Mdege et al. / Journal of Clinical Epidemiology 67 (2014) 1083e1092
recruiting participants for the study can use their foreknowl- 5. Conclusion
edge of treatment allocation to select patients to enroll into
In conclusion, the 2 2 cluster randomized factorial
the trial [41]. Moreover, in cluster randomized trials in
design has been used for evaluating complex interventions
which participants are able to opt out of or avoid the trial
in a wide range of health care settings and clinical areas to
intervention, participants can differentially refuse consent
minimize contamination, assess the effects of two interven-
to participate in the trial. This could result in differences
tions in the same study, and explore intervention interac-
in participant characteristics between the various arms of
tions. However, although most studies explored the
the trial, thereby defeating the objective of randomization.
presence or absence of intervention interactions, they were
Methods of minimizing the possibility of selection bias in
cluster randomized trials have been proposed and these often either not sufficiently powered to detect any interac-
tions or did not provide information on whether the study
include identifying participants before randomization or uti-
was sufficiently powered to detect an interaction. There is
lizing an independent person, preferably blind to allocation,
need for improvement on the following: (1) clear provision
to recruit participants [41]. A review of 113 cluster random-
of information on whether the study was powered enough
ized trials published in MEDLINE in 2008 found that about
to detect an interaction if one is suspected or if the study
40% of those trials had recruited participants before cluster
intends to estimate the interaction, (2) the inclusion of data
randomization [42] compared with only approximately 7%
on whether any cluster imbalances were present at baseline
(2 of 29) for our present review. Moreover, for the studies
included in our review, in which cluster imbalances were in the study reports, and (3) incorporating methods for
minimizing cluster imbalances and their impact during
evident (13 studies), they were rarely adjusted for in the
study design and analysis. The variability in the reporting
analysis. There is therefore a need to improve on incorpo-
of some methodological aspects such as motivation for
rating methods for minimizing cluster imbalances and their
using the design, a priori hypothesis for intervention
impact when designing and analyzing 2 2 cluster random-
interactions including a rationale, and other aspects such
ized factorial studies. Our review also demonstrates the need
as sequence generation supports calls from other
to improve on the reporting of any cluster imbalances and
researchers for the need for guidelines for reporting of
methods used for sequence generation.
Although almost all studies specified they were random- factorial trials [2,3].
ized and factorial designs in the title and/or abstract, approx-
imately a third (8 of 29) did not specify that they were a Appendix
cluster design. Six of the eight studies that did not specify
Search strategy for MEDLINE (OVID SP)
the cluster design were published between 2006 and 2009,
which is after the first CONSORT statement extension to
cluster randomized controlled trials (CRCTs) published in 1946ePresent
2004, which recommended the identification of CRCTs as 1. factorial.mp.
cluster randomized in the title or abstract of reports [43]. 2. 2 by 2.mp.
Other reviews have also found that the majority of factorial 3. 2 x 2.mp.
design studies identify themselves as such in the title or ab- 4. or/1-3
stract [2,3], whereas the percentage of CRCTs that clearly 5. randomized controlled trial.pt.
identified themselves as cluster randomized in the title or ab- 6. controlled clinical trial.pt.
stract is low [44]. Using currently existing search strategy for 7. (randomized or randomised or rct$).ti,ab.
identifying RCTs in electronic databases such as MEDLINE 8. placebo$.ti,ab.
yields high sensitivity for identifying CRCTs [44]. However, 9. (crossover$ or cross over$ or cross-over$ or (doubl$
specific reference to cluster randomized trial in the title or adj blind$) or (singl$ adj blind$)).ti,ab.
abstract of trial reports it also important as it facilitates sim- 10. randomly.ti,ab.
ple and efficient identification of that subset of trials when 11. trial$.ti,ab.
required [44], such as for our study in which we were inter- 12. clinical trials as topic.sh.
ested in the cluster randomized controlled factorial design 13. or/32-39
and not just randomized controlled factorial design. 14. 4 and 13
Factorial design studies are difficult to identify through
electronic searches because of the absence of MeSH terms References
or consistent text words [2,3]. Our search might therefore [1] Torgerson DJ, Torgerson CJ. Designing randomised trials in health,
have missed some studies, particularly those that did not education and the social sciences: an introduction. New York, NY:
use any design-specific words or phrases in their title or ab- Palgrave Macmillan; 2008.
stract identifying them as 2 2 factorial cluster random- [2] McAlister FA, Straus SE, Sackett DL, Altman DG. Analysis and report-
ing of factorial trialsda systematic review. JAMA 2003;289:2545e53.
ized controlled designs. We however supplemented the [3] Montgomery AA, Astin MP, Peters TJ. Reporting of factorial trials of
electronic search with hand searching of previous reviews complex interventions in community settings: a systematic review.
of factorial design studies. Trials 2011;12:179.
N.D. Mdege et al. / Journal of Clinical Epidemiology 67 (2014) 1083e1092 1091
[4] Victora CG, Habicht J-P, Bryce J. Evidence-based public health: of training in information management, evidence-based medicine,
moving beyond randomized trials. AJPH 2004;94:400e5. both or neither: the PIER trial. Br J Gen Pract 2002;52:818e24.
[5] Higgins JPT, Green S, editors. Cochrane handbook for systematic re- [22] Larsen ER, Mosekilde L, Foldspang A. Vitamin D and calcium sup-
views of interventions 5.1.0 [updated March 2011]. The Cochrane plementation prevents osteoporotic fractures in elderly community
Collaboration; 2008. Available at. www.cochrane-handbook.org. Ac- dwelling residents: a pragmatic population-based 3-year intervention
cessed December 3, 2011. study. J Bone Miner Res 2004;19:370e8.
[6] Centre for Reviews and Dissemination. Systematic reviews: CRDs [23] Mannheimer SB, Morse E, Matts JP, Andrews L, Child C,
guidance for undertaking reviews in health care. York, UK: Univer- Schmetter B, et al. Sustained benefit from a long-term antiretroviral
sity of York; 2009. adherence intervention. Results of a large randomized clinical trial. J
[7] Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA group. Acquir Immune Defic Syndr 2006;43(Suppl 1):S41e7.
Preferred Reporting Items for Systematic Reviews and Meta- [24] Ravaud P, Giraudeau B, Logeart I, Larguier JS, Rolland D, Treves R,
Analyses: the PRISMA Statement. BMJ 2009;339:b2700. et al. Management of osteoarthritis (OA) with an unsupervised home
[8] Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, based exercise programme and/or patient administered assessment
et al. Developing and evaluating complex interventions: the new tools. A cluster randomised controlled trial with a 22 factorial
Medical Research Council guidance. BMJ 2008;337:a1655. design. Ann Rheum Dis 2004;63:703e8.
[9] Amsallem E, Kasparian C, Cucherat M, Chabaud S, Haugh M, [25] Richards SH, Bankhead C, Peters TJ, Austoker J, Hobbs FD,
Boissel JP, et al. Evaluation of two evidence-based knowledge trans- Brown J, et al. Cluster randomised controlled trial comparing the
fer interventions for physicians. A cluster randomized controlled effectiveness and cost-effectiveness of two primary care interventions
factorial design trial: the CardioDAS Study. Fundam Clin Pharmacol aimed at improving attendance for breast screening. J Med Screen
2007;21:631e41. 2001;8:91e8.
[10] Bahrami M, Deery C, Clarkson JE, Pitts NB, Johnston M, Ricketts I, [26] Thomas RE, Croal BL, Ramsay C, Eccles M, Grimshaw J. Effect of
et al. Effectiveness of strategies to disseminate and implement clin- enhanced feedback and brief educational reminder messages on lab-
ical guidelines for the management of impacted and unerupted third oratory test requesting in primary care: a cluster randomised trial.
molars in primary dental care, a cluster randomised controlled trial. Lancet 2006;367:1990e6.
Br Dent J 2004;197:691e6. [27] Twardella D, Brenner H. Effects of practitioner education, practi-
[11] Brock G, Carrier S, Alarie P, Pommerville P, Casey R, Harris S, et al. tioner payment and reimbursement of patients drug costs on smoking
The effect of physician and patient education when combined with cessation in primary care: a cluster randomised trial. Tob Control
vardenafil treatment in Canadian males with erectile dysfunction: 2007;16:15e21.
an open-label, factorial-designed, cluster randomized clinical trial. [28] Watson MC, Bond CM, Grimshaw JM, Mollison J, Ludbrook A,
J Sex Med 2008;5:705e15. Walker AE. Educational strategies to promote evidence-based com-
[12] Cals JWL, Butler CC, Hopstaken RM, Hood K, Dinant G-J. Effect of munity pharmacy practice: a cluster randomized controlled trial
point of care testing for C reactive protein and training in communi- (RCT). Fam Pract 2002;19:529e36.
cation skills on antibiotic use in lower respiratory tract infections: [29] Wesson J, Olawo A, Bukusi V, Solomon M, Pierre-Louis B,
cluster randomised trial. BMJ 2009;338:b1374. Stanback J, et al. Reaching providers is not enough to increase
[13] Cheater FM, Baker R, Reddish S, Spiers N, Wailoo A, Gillies C, et al. IUD use: a factorial experiment of academic detailing in Kenya. J
Cluster randomized controlled trial of the effectiveness of audit and Biosoc Sci 2008;40:69e82.
feedback and educational outreach on improving nursing practice [30] Zillich AJ, Ackermann RT, Stump TE, Ambuehl RJ, Downs SM,
and patient outcomes. Med Care 2006;44:542e51. Holmes AM, et al. An evaluation of educational outreach to improve
[14] Clarkson JE, Turner S, Grimshaw JM, Ramsay CR, Johnston M, evidence-based prescribing in Medicaid: a cautionary tale. J Eval
Scott A, et al. Changing clinicians behavior: a randomized Clin Pract 2008;14:854e60.
controlled trial of fees and education. J Dent Res 2008;87:640e4. [31] Blalock SJ, DeVellis BM, Patterson CC, Campbell MK,
[15] Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Orenstein DR, Dooley MA. Effects of an osteoporosis prevention
Effect of audit and feedback, and reminder messages on primary-care program incorporating tailored educational materials. Am J Health
radiology referrals: a randomised trial. Lancet 2001;357:1406e9. Promot 2002;16:146e56.
[16] Fletcher AE, Price GM, Ng ES, Stirling SL, Bulpitt CJ, Breeze E, [32] Daucourt V, Saillour-Glenisson F, Michel P, Jutand MA, Abouelfath A.
et al. Population-based multidimensional assessment of older people A multicenter cluster randomized controlled trial of strategies to
in UK general practice: a cluster randomised factorial trial. Lancet improve thyroid function testing. Med Care 2003;41:432e41.
2004;364:1667e77. [33] Jones LA, Weymiller AJ, Shah N, Bryant SC, Christianson TJH,
[17] Herbert CP, Wright JM, Maclure M, Wakefield J, Dormuth C, Brett- Guyatt GH, et al. Should clinicians deliver decision aids? Further
MacLean P, et al. Better Prescribing Project: a randomized controlled exploration of the statin choice randomized trial results. Med Decis
trial of the impact of case-based educational modules and personal Making 2009;29:468e74.
prescribing feedback on prescribing for hypertension in primary care. [34] Ling BS, Schoen RE, Trauth JM, Wahed AS, Eury T, Simak DM,
Fam Pract 2004;21:575e81. et al. Physicians encouraging colorectal screening: a randomized
[18] Jafar TH, Hatcher J, Poulter N, Islam M, Hashmi S, Qadri Z, et al. controlled trial of enhanced office and patient management on
Community-based interventions to promote blood pressure control compliance with colorectal cancer screening. Arch Intern Med
in a developing country: a cluster randomized trial. Ann Intern 2009;169:47e55.
Med 2009;151:593e601. [35] Riemersma-van dar Lek RF, Swaab DF, Twisk J, Hol EM,
[19] Jiwa M, Skinner P, Coker AO, Shaw L, Campbell MJ, Thompson J. Hoogendijk WJ, Van Someren EJ. Effect of bright light and melatonin
Implementing referral guidelines: lessons from a negative outcome on cognitive and noncognitive function in elderly residents of group care
cluster randomised factorial trial in general practice. BMC Fam Pract facilities: a randomized controlled trial. JAMA 2008;299:2642e55.
2006;7:65. [36] Scholes D, Grothaus L, McClure J, Reid R, Fishman P, Sisk C, et al.
[20] Kinney ED, Kennedy J, Cook CA, Freedman JA, Lane KA, Hui SL. A randomized trial of strategies to increase chlamydia screening in
A randomized trial of two quality improvement strategies imple- young women. Prev Med 2006;43:343e50.
mented in a statewide public community based, long-term care pro- [37] Watson J, Helliwell P, Morton V, Adebajo A, Dickson J, Russell I,
gram. Med Care 2003;41:1048e57. et al. Shoulder acute pain in primary healthcare: is retraining effec-
[21] Langham J, Tucker H, Sloan D, Pettifer J, Thom S, Hemingway H. tive for GP principals? SAPPHIREda randomized controlled trial.
Secondary prevention of cardiovascular disease: a randomised trial Rheumatology 2008;47:1795e802.
1092 N.D. Mdege et al. / Journal of Clinical Epidemiology 67 (2014) 1083e1092
[38] Lubsen J, Pocock SJ. Factorial trials in cardiology: pros and cons. Eur [42] Giraudeau B, Caille A, Le Gouge A, Ravaud P. Participant informed
Heart J 1994;15:585e8. consent in cluster randomized trials: review. PLoS One 2012;7:e40436.
[39] Brittian W, Wittes J. Factorial designs in clinical trials: effects of non- [43] Campbell MK, Elbourne DR, Altman DG, CONSORT group. CONSORT
compliance and subadditivity. Stat Med 1989;8:161e71. statement: extension to cluster randomised trials. BMJ 2004;328:702e8.
[40] Torgerson DJ. Contamination in trials: is cluster randomisation the [44] Taljaard M, McGowan J, Grimshaw JM, Brehaut JC, McRae A,
answer? BMJ 2001;322:355e7. Eccles MP, et al. Electronic search strategies to identify reports of cluster
[41] Hahn S, Puffer S, Torgerson DJ, Watson J. Methodological bias in randomized trials in MEDLINE: low precision will improve with adher-
cluster randomised trials. BMC Med Res Methodol 2005;5:10. ence to reporting standards. BMC Med Res Methodol 2010;10:15.