Student BMJ: Performing a literature review

1 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

Education

A necessary skill for any doctor

By: Gulraj S Matharu, Christopher D Buckley
Published: 01 febrero 2012
DOI: 10.1136/sbmj.e404
Cite this as: Student BMJ 2012;20:e404
Respond to this article
In this article:
Formulating a question
Collecting the evidence
Appraising the evidence
Writing up

Figures:
Fig 1 Results from a broad literature search using the term angiotensin converting enzyme inhibitor
Fig 2 Example of a specific literature search. The search terms used were angiotensin converting enzyme inhibitor and
hypertension. The limits applied to this search were all randomised controlled trials carried out in humans, published in
English over the past 10 years, with the search terms appearing in the title of the study only
Fig 3 Results from a specific literature search (using the search terms and limits from figure 2)

Boxes:
Box 1: Definitions of common terms in the literature7
Box 2: Different study types7
What causes disease, which drug is best, does this patient need surgery, and what is the prognosis? Although experience
helps in answering these questions, ultimately they are best answered by evidence based medicine. But how do you assess
the evidence? As a medical student, and throughout your career as a doctor, critical appraisal of published literature is an
important skill to develop and refine. At medical school you will repeatedly appraise published literature and write literature
reviews. These activities are commonly part of a special study module, research project for an intercalated degree, or another
type of essay based assignment.
Top

Formulating a question
Literature reviews are most commonly performed to help answer a particular question. While you are at medical school, there
will usually be some choice regarding the area you are going to review.
Once you have identified a subject area for review, the next step is to formulate a specific research question. This is arguably
the most important step because a clear question needs to be defined from the outset, which you aim to answer by doing the
review. The clearer the question, the more likely it is that the answer will be clear too. It is important to have discussions with
your supervisor when formulating a research question as his or her input will be invaluable. The research question must be
objective and concise because it is easier to search through the evidence with a clear question. The question also needs to be
feasible. What is the point in having a question for which no published evidence exists? Your supervisors input will ensure you
are not trying to answer an unrealistic question. Finally, is the research question clinically important? There are many research
questions that may be answered, but not all of them will be relevant to clinical practice. The research question we will use as
an example to work through in this article is, What is the evidence for using angiotensin converting enzyme (ACE) inhibitors
in patients with hypertension?
Top

Collecting the evidence

After formulating a specific research question for your literature review, the next step is to collect the evidence. Your
supervisor will initially point you in the right direction by highlighting some of the more relevant papers published. Before
doing the literature search it is important to agree a list of keywords with your supervisor. A source of useful keywords can be
obtained by reading Cochrane reviews or other systematic reviews, such as those published in the BMJ.[1] [2] A relevant
Cochrane review for our research question on ACE inhibitors in hypertension is that by Heran and colleagues.[3] Appropriate
keywords to search for the evidence include the words used in your research question (angiotensin converting enzyme
inhibitor, hypertension, blood pressure), details of the types of study you are looking for (randomised controlled trial,
case control, cohort), and the specific drugs you are interested in (that is, the various ACE inhibitors such as ramipril,
perindopril, and lisinopril).
Once keywords have been agreed it is time to search for the evidence using the various electronic medical databases (such as
PubMed, Medline, and EMBASE). PubMed is the largest of these databases and contains online information and tutorials on

08/03/2012 08:56 a.m.

Student BMJ: Performing a literature review

2 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

how to do literature searches with worked examples. Searching the databases and obtaining the articles are usually free of
charge through the subscription that your university pays. Early consultation with a medical librarian is important as it will
help you perform your literature search in an impartial manner, and librarians can train you to do these searches for yourself.
Literature searches can be broad or tailored to be more specific. With our example, a broad search would entail searching all
articles that contain the words blood pressure or ACE inhibitor. This provides a comprehensive list of all the literature, but
there are likely to be thousands of articles to review subsequently (fig 1). 1 In contrast, various search restrictions can be
applied on the electronic databases to filter out papers that may not be relevant to your review. Figure 2 gives an example of
a specific search. 2 The search terms used in this case were angiotensin converting enzyme inhibitor and hypertension. The
limits applied to this search were all randomised controlled trials carried out in humans, published in the English language over
the last 10 years, with the search terms appearing in the title of the study only. Thus the more specific the search strategy,
the more manageable the number of articles to review (fig 3), and this will save you time. 3 However, this method risks your
not identifying all the evidence in the particular field. Striking a balance between a broad and a specific search strategy is
therefore important. This will come with experience and consultation with your supervisor. It is important to note that
evidence is continually becoming available on these electronic databases and therefore repeating the same search at a later
date can provide new evidence relevant to your review.

Fig 1 Results from a broad literature search using the term angiotensin converting enzyme inhibitor

08/03/2012 08:56 a.m.

Student BMJ: Performing a literature review

3 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

Fig 2 Example of a specific literature search. The search terms used were angiotensin converting enzyme inhibitor and
hypertension. The limits applied to this search were all randomised controlled trials carried out in humans, published in
English over the past 10 years, with the search terms appearing in the title of the study only

08/03/2012 08:56 a.m.

Student BMJ: Performing a literature review

4 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

Fig 3 Results from a specific literature search (using the search terms and limits from figure 2)
Reading the abstracts (study summary) of the articles identified in your search may help you decide whether the study is
applicable for your reviewfor example, the work may have been carried out using an animal model rather than in humans.
After excluding any inappropriate articles, you need to obtain the full articles of studies you have identified. Additional relevant
articles that may not have come up in your original search can also be found by searching the reference lists of the articles
you have already obtained. Once again, you may find that some articles are still not applicable for your review, and these can
also be excluded at this stage. It is important to explain in your final review what criteria you used to exclude articles as well
as those criteria used for inclusion.
The National Institute for Health and Clinical Excellence (NICE) publishes evidence based guidelines for the United Kingdom
and therefore provides an additional resource for identifying the relevant literature in a particular field.[4] NICE critically
appraises the published literature with recommendations for best clinical practice proposed and graded based on the quality of
evidence available. Similarly, there are internationally published evidence based guidelines, such as those produced by the
European Society of Cardiology and the American College of Chest Physicians, which can be useful when collecting the
literature in a particular field.[5] [6]
Top

Appraising the evidence

08/03/2012 08:56 a.m.

Student BMJ: Performing a literature review

5 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

Once you have collected the evidence, you need to critically appraise the published material. Box 1 gives definitions of terms
you will encounter when reading the literature. A brief guide of how to critically appraise a study is presented; however, it is
advisable to consult the references cited for further details.
Top

Box 1: Definitions of common terms in the literature[7]

Prospectivecollecting data in real time after the study is designed
Retrospectiveanalysis of data that have already been collected to determine associations between exposure and
outcome
Hypothesisproposed association between exposure and outcome. If presented in the negative it is called the null
hypothesis
Variablea quantity or quality that changes during the study and can be measured
Single blindsubjects are unaware of their treatment, but clinicians are aware
Double blindboth subjects and clinicians are unaware of treatment given
Placeboa simulated medical intervention, with subjects not receiving the specific intervention or treatment being studied
Outcome measure/endpointclinical variable or variables measured in a study subsequently used to make conclusions
about the original interventions or treatments administered
Biasdifference between reported results and true results. Many types exist (such as selection, allocation, and reporting
biases)
Probability (P) valuenumber between 0 and 1 providing the likelihood the reported results occurred by chance. A P value
of 0.05 means there is a 5% likelihood that the reported result occurred by chance
Confidence intervalsprovides a range between two numbers within which one can be certain the results lie. A confidence
interval of 95% means one can be 95% certain the actual results lie within the reported range
Top

Background
The study authors should clearly define their research question and ideally the hypothesis to be tested. If the hypothesis is
presented in the negative, it is called the null hypothesis. An example of a null hypothesis is smoking does not cause lung
cancer. The study is then performed to assess the significance of the exposure (smoking) on outcome (lung cancer).
Top

Methods
A major part of the critical appraisal process is to focus on study methodology, with your key task being an assessment of the
extent to which a study was susceptible to bias (the discrepancy between the reported results and the true results). It should
be clear from the methods what type of study was performed (box 2).
Top

Box 2: Different study types[7]

Systematic review/meta-analysiscomprehensive review of published literature using predefined methodology.
Meta-analyses combine results from various studies to give numerical data for the overall association between variables
Randomised controlled trialrandom allocation of patients to one of two or more groups. Used to test a new drug or
procedure
Cohort studytwo or more groups followed up over a long period, with one group exposed to a certain agent (drug or
environmental agent) and the other not exposed, with various outcomes compared. An example would be following up a
group of smokers and a group of non-smokers with the outcome measure being the development of lung cancer
Case-control studycases (those with a particular outcome) are matched as closely as possible (for age, sex, ethnicity)
with controls (those without the particular outcome). Retrospective data analysis is performed to determine any factors
associated with developing the particular outcomes
Cross sectional studylooks at a specific group of patients at a single point in time. Effectively a survey. An example is
asking a group of people how many of them drink alcohol
Case reportdetailed reports concerning single patients. Useful in highlighting adverse drug reactions
There are many different types of bias, which depend on the particular type of study performed, and it is important to look
for these biases. Several published checklists are available that provide excellent resources to help you work through the
various studies and identify sources of bias. The CONSORT statement (which stands for CONsolidated Standards Of Reporting
Trials) provides a minimum set of recommendations for reporting randomised controlled trials and comprises a rigorous 25
item checklist, with variations available for other study types.[8] [9] As would be expected, most (17 of 25) of the items focus
on questions relating to the methods and results of the randomised trial. The remaining items relate to the title, abstract,
introduction, and discussion of the study, in addition to questions on trial registration, protocol, and funding.
Jadad scoring provides a simple and validated system to assess the methodological quality of a randomised clinical trial using
three questions.[10] The score ranges from zero to five, with one point given for a yes in each of the following questions.
(1) Was the study described as randomised? (2) Was the study described as double blind? (3) Were there details of subject
withdrawals, exclusions, and dropouts? A further point is given if (1) the method of randomisation was appropriate, and (2)
the method of blinding was appropriate.
In addition, the Critical Appraisal Skills Programme provides excellent tools for assessing the evidence in all study types (box
2).[11] The Oxford Centre for Evidence-Based Medicine levels of evidence is yet another useful resource for assessing the
methodological quality of all studies.[12]
Top

08/03/2012 08:56 a.m.

Student BMJ: Performing a literature review

6 de 7

file:///C:/Documents and Settings/QBEX/Mis documentos/ANALISIS L...

Results
Ensure all patients have been accounted for and any exclusions, for whatever reason, are reported. Knowing the baseline
demographic (age, sex, ethnicity) and clinical characteristics of the population is important. Results are usually reported as
probability values or confidence intervals (box 1).
Top

Discussion
This should explain the major study findings, put the results in the context of the published literature, and attempt to account
for any variations from previous work. Study limitations and sources of bias should be discussed. Authors conclusions should
be supported by the study results and not unnecessarily extrapolated. For example, a treatment shown to be effective in
animals does not necessarily mean it will work in humans.
Top

Writing up
The format for writing up the literature review usually consists of an abstract (short structured summary of the review), the
introduction or background, methods, results, and discussion with conclusions. There are a number of good examples of how
to structure a literature review and these can be used as an outline when writing your review.[13] [14]
The introduction should identify the specific research question you intend to address and briefly put this into the context of
the published literature. As you have now probably realised, the methods used for the review must be clear to the reader and
provide the necessary detail for someone to be able to reproduce the search. The search strategy needs to include a list of
keywords used, which databases were searched, and the specific search limits or filters applied. Any grading of methodological
quality, such as the CONSORT statement or Jadad scoring, must be explained in addition to any study inclusion or exclusion
criteria.[6] [7] [8] The methods also need to include a section on the data collected from each of the studies, the specific
outcomes of interest, and any statistical analysis used. The latter point is usually relevant only when performing
meta-analyses.
The results section must clearly show the process of filtering down from the articles obtained from the original search to the
final studies included in the reviewthat is, accounting for all excluded studies. A flowchart is usually best to illustrate this.
Next should follow a brief description of what was done in the main studies, the number of participants, the relevant results,
and any potential sources of bias. It is useful to group similar studies together as it allows comparisons to be made by the
reader and saves repetition in your write-up. Boxes and figures should be used appropriately to illustrate important findings
from the various studies.
Finally, in the discussion you need to consider the study findings in light of the methodological qualitythat is, the extent of
potential bias in each study that may have affected the study results. Using the evidence, you need to make conclusions in
your review, and highlight any important gaps in the evidence base, which need to be dealt with in future studies. Working
through drafts of the literature review with your supervisor will help refine your critical appraisal skills and the ability to
present information concisely in a structured review article. Remember, if the work is good it may get published.
Gulraj S Matharu, academic foundation doctor, Christopher D Buckley, Arthritis Research UK professor of rheumatology
1

Institute of Biomedical Research, College of Medical and Dental Sciences, School of Immunity and Infection, University of
Birmingham, UK
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.

References
1. The Cochrane Library. www3.interscience.wiley.com/cgibin/mrwhome/106568753/HOME?CRETRY=1&SRETRY=0.
2. British Medical Journal. www.bmj.com/.
3. Heran BS, Wong MMY, Heran IK, Wright JM. Blood pressure lowering efficacy of angiotensin converting enzyme (ACE)
inhibitors for primary hypertension. Cochrane Database Syst Rev 2008;4:CD003823, doi:
10.1002/14651858.CD003823.pub2.
4. National Institute for Health and Clinical Excellence. www.nice.org.uk.
5. European Society of Cardiology. www.escardio.org/guidelines.
6. Geerts WH, Bergqvist D, Pineo GF, Heit JA, Samama CM, Lassen MR, et al. Prevention of venous thromboembolism:
American College of Chest Physicians evidence-based clinical practice guidelines (8th ed). Chest 2008;133:381-453S.
7. Wikipedia. http://en.wikipedia.org/wiki.
8. Moher D, Schulz KF, Altman DG, Egger M, Davidoff F, Elbourne D, et al. The CONSORT statement: revised
recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-4.
9. The CONSORT statement. www.consort-statement.org/.
10. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of
randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1-12.
11. Critical Appraisal Skills Programme (CASP). www.sph.nhs.uk/what-we-do/public-health-workforce/resources/criticalappraisals-skills-programme.
12. Oxford Centre for Evidence-based MedicineLevels of Evidence. www.cebm.net.
13. Van den Bruel A, Thompson MJ, Haj-Hassan T, Stevens R, Moll H, Lakhanpaul M, et al. Diagnostic value of laboratory tests
in identifying serious infections in febrile children: systematic review. BMJ 2011;342:d3082.
14. Awopetu AI, Moxey P, Hinchliffe RJ, Jones KG, Thompson MM, Holt PJ. Systematic review and meta-analysis of the
relationship between hospital volume and outcome for lower limb arterial surgery. Br J Surg 2010;97:797-803.
Cite this as: Student BMJ 2012;20:e404

08/03/2012 08:56 a.m.