Advantage of meta-analysis
Pooling of data from individual studies leads to an increase in sample size, and an increase in power which is particularly important when the size of effect is small or there is a relatively low event rate. The increase in sample size not only means an increase in power, but also an increase in the precision in the estimate of effect
Appraisal of SW
3. What were the inclusion/exclusion criteria? The criteria for selecting or rejecting studies should be clearly stated and appropriate. The process* by which articles are assessed for relevance should also be recorded. 4. How was the validity of the primary studies assessed? The process* by which validity assessment was undertaken and the criteria used to assess the quality of the primary studies should be clear. It should also be apparent how the results of the validity assessment are used within the reviews data synthesis.
Appraisal of SW
5. How were data extracted from the primary studies? The process* by which data was extracted from the primary studies should be transparent. 6. Are the characteristics of the included studies clearly displayed? A table illustrating the study characteristics of each included primary study should be presented.
Appraisal of SW
7. Does the review examine differences/similarities between the included studies and their results? Heterogeneity between studies should be explored and the reasons for any variations discussed. Heterogeneity can be explored statistically, graphically or through a narrative. 8. Was the synthesis of the data carried out appropriately? Was data pooled qualitatively or statistically? If statistical pooling (meta-analysis) was used, was it used appropriately? 9. Were the results interpreted appropriately? Any conclusions, implications for research or practice should follow on logically from the results.
A clear protocol
Describing the background to the work, hypothesis to be tested and methodology to be used Allowing peer (and often consumer) review of the question to be asked, and methods to be used, so that these can be improved. It also limits vague inclusion criteria that may preferentially allow in studies with good results, and data dredging where lots of analyses are tried out, but only those with significant results reported.
Extracting data
Ideally the process is independently duplicated, based on prior decisions (in the protocol), comprehensive (on a form designed for the review, and may involve contacting authors to fill in any gaps in published reports) Clearly tabulated to allow transparency and possibly corrections at a later date.
Pooling of data
Narrative or statistical pooling or metaanalysis? Narrative or meta-analytic comparisons and sub-groupings should be prespecified in the protocol (to avoid multiple analyses being carried out with only the statistically significant ones being published).
What is meta-analysis?
To pool extracted numerical data, weighted so that larger studies, or those with less variability, contribute more to the outcome. This pooling provides an answer with greater precision that each included study on its own The pictorial representation of a metaanalysis is called a forest plot (see example).
Statistical analysis
Fixed effects (where it is assumed that the true outcomes of the various studies are the same) Random effects methodologies (where the true outcomes are assumed to vary a little with differing study inclusion, dose, duration etc). Where fixed effects meta-analysis produces a result that is statistically heterogeneous it is usual to switch to random effects meta-analysis. Statistical heterogeneity of studies (large differences in their results, suggesting differing true outcomes) is ideally explored through subgrouping or metaregression.
CL policy
It is Cochrane policy that reviewers revisit their review and update it within two years of it being published on The Cochrane Library. There are several stages to the Cochrane peer review process, including the assessment of protocols, evaluation of the reviews methodology and content by editors, peer reviewers and potential end users / consumers. It has been suggested that the existence of such a thorough refereeing process ultimately leads to Cochrane reviews being less prone to bias than systematic reviews and meta-analysis published in paper-based journals.
Ways to contribute
The Oral Health Group welcomes all those interested in contributing to the work of the group. There are several options for participation, either as a lead reviewer, assisting as a co-reviewer, handsearching a journal to identify RCTs, or by becoming a member of the panel of peer reviewers or consumers. For further details or an information pack please refer to the groups website: www.cochrane-oral.man.ac.uk Contact: Emma Tavender, Co-ordinator, Cochrane Oral Health Group, MANDEC, University Dental Hospital of Manchester, Higher Cambridge Street, Manchester M15 6FH. Tel: +44 161 275 7818, Fax: +44 161 275 7815, Email: emma.tavender@man.ac.uk