Effectiveness Health Care

Journal of International Development
J. Int. Dev. 15, 41–65 (2003)

Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/jid.965
APPROACHES TO OVERCOMING
CONSTRAINTS TO EFFECTIVE HEALTH
SERVICE DELIVERY: A REVIEW
OF THE EVIDENCE
VALERIA OLIVEIRA-CRUZ*, KARA HANSON and ANNE MILLS
Health Policy Unit, London School of Hygiene and Tropical Medicine, London, UK
Abstract: This paper reviews the current evidence base regarding efforts to overcome con-
straints to effective health service delivery in low and middle-income countries. A systematic
literature review was chosen as the approach to gather and analyse existing knowledge about
how to improve the ‘close-to-client’ health system. We focused on three levels of constraints:
community and household, the health services delivery level itself, and health sector policy and
strategic management. In total, 116 studies were reviewed and their main findings presented.
The results should be interpreted with caution due to the considerable limitations in the existing
evidence base. Copyright # 2003 John Wiley & Sons, Ltd.
1 INTRODUCTION
Health systems in low and middle-income countries often face substantial problems
resulting from resource shortages and the inefficient and inequitable use of resources
(Mills, 1997; World Bank, 1993). They continue to be plagued by poor service quality and
low coverage rates, especially for poor populations. There have been strong pressures to
increase health spending in such countries, from a current average of US $131 per capita
per year (Commission on Macroeconomics and Health, 2001). Recently the Commission
on Macroeconomics and Health (CMH) called for a massive effort to scale up2 priority
health interventions and provided an estimate of the minimum cost of financing these
interventions, i.e. US $30 to US $40 per person per year (Commission on Macroeco-
nomics and Health, 2001). Additional resources would allow national governments to
expand access to high priority programmes such as immunization and HIV prevention, and
invest in the urgently needed improvements in the areas of drugs and general supplies,
*Correspondence to: V. Oliveira-Cruz, Health Policy Unit, London School of Hygiene and Tropical Medicine,
Keppel Street, London WC1E 7HT, UK. E-mail: valeria.oliveira-cruz@lshtm.ac.uk
1
Least developed countries.
2
To expand access to and utilization of priority health services or interventions.
Copyright # 2003 John Wiley & Sons, Ltd.

42 V. Oliveira-Cruz et al.
human resource development, and expansion of infrastructure which underpin these

programmes (Commission on Macroeconomics and Health, 2002). However, money
alone, as pointed out by Hanson et al. (2003), will not be sufficient to overcome major
obstacles faced by health systems in such countries. Additional money needs to be spent
effectively and efficiently if it is to result in a significant contribution towards the goal of
improving the health status of the population, in particular of the poor.
The purpose of this review is to gather and analyse existing evidence and experiences,
from country reports in the literature, of overcoming the constraints which affect the
performance of the close-to-client health system. The close-to-client health system was
defined by Working Group 5 of the CMH as consisting of population-based preventive
services, primary health care services and first level (district) referral care (Commission on
Macroeconomics and Health, 2002). Of particular concern for this review is the effec-
tiveness of efforts to improve government performance in terms of efficiency, equity gains
and quality, evidence which allows a better understanding of how to improve health
outcomes. This analysis contributes to recommendations concerning the scaling up of
health interventions to ensure high coverage of the poor (see Hanson et al., 2003, for
further discussion on this point).
The following section describes the categorization of constraints used as the framework
for analysis of the literature findings. The search methodology is presented in Section 3.
The fourth section provides a summary of the findings of the literature review. The main
points brought out in the findings section are discussed in Section 5. The last section
provides some concluding remarks.
2 CATEGORIZATION OF CONSTRAINTS
The constraints to scaling up health interventions reviewed in the literature are analysed
here in terms of the categorization of constraints developed by Hanson et al. (2003). The
categorization distinguishes the different levels at which the constraints operate, as shown
in Table 1. Particularly relevant in terms of policy formulation and implementation is the
degree to which decisions lie within the control of Ministries of Health and government
structures at lower levels of the hierarchy.
Due to the focus of this review on the success of efforts to expand access to priority
interventions, the findings are concentrated on the first 3 levels of constraints, i.e.:
community and household, health services delivery, and health sector policy and strategic
management. While we fully acknowledge the importance of levels 4 and 5, namely public
policies cutting across sectors and environmental and contextual characteristics, an
analysis of the effects of interventions at these levels lies beyond the scope of this review
due to limitations in time and resources.
3 METHODS
The methodology applied in this review is a systematic search of published literature.

Efforts were also made to locate grey literature but due to difficulties in searching and
locating grey literature, a less systematic approach was taken.
The topic for the search was defined as: evidence on the success of efforts to overcome
constraints related to the delivery of health services at the close-to-client level especially
in areas relevant to diseases and health problems of highest priority (malaria, TB,
Copyright # 2003 John Wiley & Sons, Ltd. J. Int. Dev. 15, 41–65 (2003)
Overcoming Constraints to Effective Health Service Delivery 43
Table 1. Categorization of constraints
Levels Constraints
I. Community and —Lack of demand for effective interventions

household level —Barriers to use of effective interventions (physical, financial and social)
II. Health services —Shortage and distribution of appropriately qualified staff

delivery level —Weak technical guidance, programme management and supervision
—Inadequate drugs and medical supplies
—Lack of equipment and infrastructure, including poor accessibility
of health services
III. Health sector —Weak and over-centralized systems for planning and management
policy and strategic —Weak drug policies and supply system
management level —Inadequate regulation of pharmaceutical and private sectors and improper industry
practices
—Lack of intersectoral action and partnership for health between government and civil
society
—Weak incentives to use inputs efficiently and respond to user needs and preferences
—Reliance on donor funding, reducing flexibility and ownership
—Donor practices that damage country policies
IV. Public policies —Government bureaucracy (civil service rules and remuneration
cutting across sectors centralized management system; civil service reforms)
—Poor availability of communication and transport infrastructure
V. Environmental A. Governance and overall policy framework
and contextual —Corruption, weak government, weak rule of law and enforceability of contracts
characteristics —Political instability and insecurity
—Low priority attached to social sectors
—Weak structures for public accountability
—Lack of free press
B. Physical environment
—Climatic and geographic predisposition to disease
—Physical environment unfavourable to service delivery
Source: Hanson et al. (2003)
HIV/AIDS, diarrhoeal diseases, acute lower respiratory tract infections (ALRI), nutri-
tional disorders, immunization preventable diseases, helminth infections, tobacco-related
diseases). The research question was broken down as follows:
* outcomes of interest: efficient use of inputs, increased provider knowledge, client
perception of health service quality (intermediate outcomes); health services
utilization, coverage increase, reduction in morbidity or mortality (final outcomes);
* interventions: management system strengthening, community involvement, improve-
ments in training and supervision, introduction of quality assurance systems;
* population: poor in low and middle income countries; and
* type of studies: descriptive, analytical, comparative and evaluation studies.
A combination of the use of the thesaurus tool and free text terms was used in the
electronic database searches, in order to retrieve the largest number of results possible.
Firstly, key words were used either as guidance to find the appropriate word through the
thesaurus tool for the electronic databases (those databases that have one) or as free text.
Secondly, we combined the key words with terms referring to the classification of
countries or geographical region of interest. Finally, a list of the major diseases or health
problems that were considered relevant was included in the combination of terms
searched. It is important to note that while the focus of the study is the strengthening of
the health system, specific diseases or conditions were used here in order to narrow down
the search to issues of relevance to the overall focus of Working Group 5.
Specific criteria for inclusion of abstracts and articles in the review were as follows:
* Sources: journals, books, reviews, and conferences proceedings and abstracts
* Country classification: low and middle income, less or least developed and developing
* Geographic coverage: Africa, Asia, Latin America
* Target population: poor
* Publication year: after 1990
We selected the following electronic databases for searches: Cabhealth, Medline,
Healthstar, HMIC and Popline.3 Once the electronic search was done, the title, abstracts
and thesaurus (Medical Subheading—MeSH) fields were browsed for relevant terms to
refine the search. We attempted to retrieve as many review papers as possible. In addition,
complementary hand searches were performed due to poor indexation of some journals in
electronic databases.4
The search of grey literature focused on independent (external review) evaluation
reports, documents that were peer reviewed5 and those available on the World Wide Web.
The sites of institutions used for unpublished literature searches were: World Bank, World
Health Organization, Department for International Development, Partnerships for Health
Reform Project, Quality Assurance Project, Health Systems Resources Centre, Population
Services International, and Management Sciences for Health.
In addition to the sources of studies mentioned above, we also received recommenda-
tions from experts in the field.
The relevance of selected and located studies was reviewed against the defined criteria
and focus of the literature review described above. In addition, the overall quality of the
research results of the studies was assessed, in particular regarding the existence of bias,
the methods used, and the potential generalizability. Due to difficulties in finding a large
sample of studies applying a rigorous study design, the main criterion for inclusion was
that the paper should report on an evaluation study or describe an intervention, thus
leaving out opinion and critique papers.
In order to provide an assessment of the strength or power of the reviewed evidence, an
attempt was made to rate the studies in terms of their methodological design and quality
using the following criteria:
1. Descriptive, analytical, comparative or evaluation studies reporting on a type of
intervention addressing health systems constraints of relevance to the review.
2. Peer-reviewed study.
3. Study uses control groups.
4. Study evaluates changes over time.
5. Study reports statistical significance of results.
3
For literature on social marketing.
4
The hand-searched journals were Health Policy and Planning and the International Journal of Health Planning
and Management.
5
We considered internal peer review as acceptable as long as the paper could fulfil at least one of the other
eligibility criteria.
Regarding the first criterion, a simple categorization of the study design was adopted,
identifying them as descriptive/analytical for those that describe and analyse interven-
tions, comparative studies for those that compare two or more interventions or strategies,
and evaluation studies for those that follow a more rigorous scientific method and include
quantitative and/or qualitative methodologies. The fourth criterion is understood here as
studies that use a baseline measure and at least a second measure, or that observe the
effects of an intervention over a certain period of time as opposed to cross-sectional (one-
time) measures. The score for each study was calculated by allocating equal weight to
each of the above criteria.
4 OVERVIEW OF FINDINGS
In total, 116 studies6 were identified.7 These studies covered 46 different low and middle-
income countries: 50 per cent in Africa, 24 per cent in Asia, 22 per cent in Latin America
and 4 per cent elsewhere (Table 2).
Table 2. Distribution of studies by region and country

Africa Asia Latin America and the Caribbean Elsewhere
Tanzania (10) India (9) Peru (5) Jordan (1)

Kenya (9) Indonesia (9) Bolivia (2) Turkey (1)
Nigeria (7) Nepal (6) Brazil (2) Developing
Ghana (5) Bangladesh (3) Ecuador (2) countries (4)1
South Africa (5) China (2) Colombia (1)
Zimbabwe (4) Cambodia (2) Costa Rica (1)
Cameroon (3) Pakistan (2) Guatemala (1)
Uganda (3) Korea (1) Honduras (1)
Zambia (3) Philippines (1) Jamaica (1)
Guinea (3) Sri Lanka (1) Mexico (1)
Burkina Faso (2) Vietnam (1)
Egypt (2)
Chad (1)
Ethiopia (1)
Mali (1)
Senegal (1)
Guinea Bissau (1)
Malawi (1)
Benin (1)
Burundi (1)
Niger (1)
The Gambia (1)
Zaire (1)
Sub-Saharan Africa
(1)2
() Number of studies per country; Some studies cover more than one country; thus the total number of studies
reviewed does not match the number of countries.
1
Review studies.
2
Review study.
6
The number of studies does not necessarily match the number of references as we used, in some cases, more than
one source per study, depending on availability.
7
Only a limited number of studies followed rigorous scientific methods. However, due to the dearth of
information in this area and the relevance of the subject under review, we opted to consider all identified studies
that met a minimum of two of the eligibility criteria outlined in Section 3.
The main types of intervention applied to overcome the constraints facing the close-to-
client health system are classified in line with the categorization of constraints outlined in
Section 2, focusing on the first 3 levels.
At constraint levels I and II, the strength of the evidence8,9 reviewed was rated as ‘good’
and at level III it was rated as ‘weak’. In terms of study design, the majority of the studies
were descriptive/analytical, followed by evaluations.
Study outcomes reported in the reviewed studies were mainly positive. At constraint
level I, reported outcomes were mostly positive and intermediate in nature with some final
outcomes as well. At constraint level II, outcomes were mixed though more positive than
negative, and mostly intermediate in nature. At constraint level III, outcomes were mostly
positive and almost all intermediate.
Due to space limitations, specific findings regarding the types of interventions and
outcomes can be only briefly reported. A detailed summary of the links between constraints,
interventions and improvements, as well as the rating of the strength of the evidence of the
different papers reviewed, is provided elsewhere (Oliveira-Cruz et al., 2001).10 This also
contains, to the extent possible, information extracted from the papers on enabling factors or
conditions that facilitated the achievement of the results described. In Table 3, we show two
examples of the information presented in the background paper. Outcomes reported in
italics refer to failures, and the ones reported in regular font refer to successes.
In the following subsections we present a summary of the main types of interventions11
and outcomes (results) as documented in the reviewed studies.12
4.1 Community and Household Level
Under community and household level constraints 37 studies were identified. These
studies covered the following broad types of interventions: community participation (10
studies,13 including one review), Bamako Initiative programmes (2 studies14), community
health workers (13 studies15), appropriate patient use of drugs (2 studies16, both reviews)
and social marketing (10 studies17).
8
The assessment of the strength of the evidence was made by calculating a simple average of the strength of each
reviewed study according to the quality criteria described in Section 3. The assessment is based on the following
scoring system: weak for studies with 2 points (mainly the descriptive/analytical type of studies), satisfactory for
studies with 3 points, good for studies with 4 points and very good for studies with 5 points.
9
Review studies are not included in our assessment of the evidence they reviewed due to the different assessment
criteria required.
10
For more information see background paper no 15 of CMH Working Group 5 available at http://www3.who.int/
whosis/cmh/cmh_papers/e/papers.cfm?path ¼ cmh,cmh_papers&language ¼ english#
11We recognize, as emphasized later, that this may not reflect the universe of possible or existing interventions to
deal with the various constraints at the different levels specified in the framework.
12
There are instances when the same study is considered under different types of interventions, particularly in the
case of review studies of drug policies or supplies.
13
Hadley and Maher, 2000; Perry et al., 1998, 1999; Moser and Sollis, 1991; Onyejiaku et al., 1990; Ahmed
et al., 1993; Sepehri and Pettigrew, 1996; Pagnoni et al., 1997; Ruebush et al., 1994; Marsh et al., 1996; Stuer,
1998.
14
McPake et al., 1992 and 1993; Levy-Bruhl et al., 1997; Knippenberg et al., 1997.
15
Robinson and Larsen, 1990; Frankenberg and Thomas, 2000; Melville et al., 1995; Stock-Iwamoto and
Korte, 1993; Ayele et al., 1993; Zeitz et al., 1993a; Mehnaz et al., 1997; Bang et al., 1994; Benavides and
Jacoby, 1994; Chaulagai, 1993; Curtale et al., 1995; Faveau et al., 1990; Christensen and Karlqvist, 1990.
16
Foster, 1991; Homedes and Ugalde, 2001.
17
Abdulla et al., 2001; Janowitz et al., 1992; Snow et al., 1999; Hanson and Jones, 2000; Meekers, 2000a;
Meekers, 1998; Van Rossem and Meekers, 1999a; 1999b; Eloundou-Enyegue et al., 1998; Meekers, 2000b.
Table 3. Links between constraints, interventions and outcomes
Country Ranking Constraints Interventions Outcomes Facilitating factors

(reference) of strength or conditions
Study of
design evidence*
Nepal 1. yes —Poor health Community —Inadequate Adequate

(Chaulagai, 2. yes knowledge and health management support management
1993) 3. no skills; volunteers from government health support.
Evaluation 4. yes —Limited in an urban facilities;
study 5. no utilization of setting. —Substantial increase
Strength: 3 health services in coverage.
Brazil 1. yes —Low coverage Contracting —Majority of contracts —Existence of a
(Kravosec 2. yes of HIV/AIDS out to NGOs. met their objectives and liaison unit within
et al., 2000; 3. no programmes reached the specified the government;
Connor, 4. no target populations; —Frequent and
2000) 5. no —NGO contracting effective
Design: Strength: 2 expanded access and communication
Descriptive/ types of services between the
analytical provided, in particular government and
study to vulnerable NGOs;
populations; —Consistent flow
—No incentives for of funds;
NGOs to be cost- —Existing NGO
efficient (cost- capacity to
minimisation); implement contract
—Lack of quality terms;
assessment; —Government
—No significant support to
improvement of NGOs’ strengthening NGO
financial sustainability. capacity when
needed.
*1. Descriptive, analytical, comparative or evaluation study reporting on a type of intervention addressing health
systems constraints of relevance to the review; 2. Peer-reviewed study; 3. Study used control groups; 4. Study
evaluated changes over time; 5. Study reported statistical significance of results.
4.1.1 Community participation

Overall, there was an important lack of consistency with regard to the definition of the
term community participation,18 with different interpretations such as collaboration,
specific targeting, or empowerment (Kahssay and Oakley, 1999).
The main types of intervention reported were as follows:
* Health education (meetings, small group teaching and lectures, home visits, with local
schools).
* Participative approach in planning and implementation by incorporating a community’s
health priorities, carrying out mobilization activities and involving leaders.
* Outreach strategy with regular visits to households (and targeting of high risk groups).
* Training and supervision of nurses, community health workers and/or mothers.
Outcomes reported were mostly positive and included increased health-related aware-
ness and knowledge within the community and increased community empowerment (such
18
For the purposes of this review, we did not attempt to select studies applying similar meanings to the concept of
community participation.
as improved technical and accounting capacity). In addition, there were improvements

regarding intervention coverage levels (EPI and nutritional monitoring), hygiene and
sanitation practices.
4.1.2 Bamako Initiative19

The Bamako Initiative was launched by African ministers of health in 1987 with the
support of UNICEF and WHO. The main purposes of the initiative were to secure
community funding of recurrent costs, enhance the essential drug supply system and
strengthen community participation and control (McPake et al., 1992; Arhin-Tenkorang,
2000). The interventions typically take the form of introducing user charges for drugs in
peripheral health facilities, with community management of the funds raised.
To a certain extent results presented by the different sets of studies are contradictory. For
instance in one study (Levy-Bruhl et al., 1997; Knippenberg et al., 1997) there is positive
mention of regular drug supplies, while the other study identified the scope for
polypharmacy which could be worsened by incentive systems that foster drug sales
(McPake et al., 1992, 1993).
4.1.3 Community health workers (CHWs)

The main types of activities implemented under community health workers were training
sessions on a specific topic (for instance, case detection and management of acute
respiratory infections among children) or a general subject. In most programmes training
sessions were provided as a start-up activity. There were also cases where refresher
courses were carried out at the start of the programmes. Training as a continuous activity
was also reported. Other common interventions included regular supervision, and
provision of supplies such as record forms, drugs, and educational materials.
Although studies reported a wide variety of outcomes, the majority were positive and a
common finding was an increase in coverage of health-related activities carried out by
CHWs. In some cases, there were reports that CHWs had difficulties in maintaining good
quality records.
The studies revealed that while CHWs can play an important role in promoting appropriate
health practices, disseminating information and supporting the government in their tasks of
delivering health care at the community level, they are not substitutes for government health
services and need adequate support in order to perform their tasks effectively.
4.1.4 Appropriate patient use of drugs

Successful interventions in improving patient drug adherence were mainly related to
health education in various forms (mass media, interactive methods, visual aids, written
material) and close involvement and building of trust between patients, health workers
(particularly CHWs) and caretakers.
4.1.5 Social marketing

Social marketing is a technique adapted from the private sector. It uses commercial
marketing methods to pursue social goals (UNAIDS, 1998). Products that have been
distributed using this strategy include condoms, insecticide-treated mosquito nets (ITNs),
19
Studies reporting the assessment of interventions based on the Bamako Initiative are documented here
only in terms of their community participation component. No systematic review of the Initiative was
performed, as community ﬁnancing and user fee programmes as strategies for improving health systems
resource availability were reviewed by the CMH Working Group 3 on domestic resource mobilization.
vitamin A and oral rehydration salts. The establishment of such programmes usually
includes market research, training of agents, establishment of a distribution system using
fixed or mobile units, promotion of the product, and IEC activities. The main disadvantage
of social marketing is its inability to reach the very poor. In order to address equity concerns,
a possible strategy is the use of a voucher system to deliver targeted subsidies, such as that
tested in Tanzania which distributed vouchers through antenatal clinics, allowing women to
purchase mosquito nets at a discounted price (Schellenberg et al., 1999).
Mixed outcomes were reported, though with more positive than negative ones. Common
outcomes included an increase in ownership of mosquito nets and familiarity with socially
marketed condoms.
4.2 Health Services Delivery Level
Under the health services delivery level, a total of 46 studies were identified. The main
types of interventions covered by the studies were: human resource development (15
studies20), service organization (8 studies21), service delivery by private providers (5
studies22), improvement of drug supplies (user fees for increasing drug supplies (4
studies23), prescribing practices (10 studies24, including 2 reviews), and drug retailer
training (4 studies25)).
4.2.1 Human resource development

There was a strong focus on training as the main type of intervention to improve staff
skills, followed by supervision or follow-up. Other types of intervention included the
introduction of quality assurance methods as well as performance and incentive schemes.
Outcomes reported were mixed, even if in general they were more positive than
negative. Key outcomes included improved skills (mainly in relation to counselling) and
performance (levels of activity, proportion of patients being examined and given health
education, history-taking and disease classification). There were also some reports of final
outcomes such as increased immunization coverage, service utilization and TB cure rates.
The main result from performance and incentive schemes was the ability to attract medical
doctors to serve in remote areas.
4.2.2 Service organization and delivery
Service organization. Interventions included mainly quality assurance methods and more
comprehensive service reorganization changes (structural and functional rearrangements
of services, implementation of clinical protocols, and provision of easier access to
emergency services).
The main outcomes reported were positive and refer to the reduction of waiting times
for patients and the increase in average contact time (patient with health professionals).
20
Kim et al., 1992; Özek et al., 1998; Walker et al., 2002; Smith et al., 2000; Faxelid et al., 1997; Robinson
et al., 2001; Kelly et al., 2001; Ruck and Darwish, 1991; MMWR, 1998; Kim et al., 2000; Kelley et al., 2000;
Zeitz et al., 1993b; Loevinsohn et al., 1995; Chomitz et al., 1997; Chernichovsky and Bayulken, 1995.
21
Mahomed and Bachman, 1998; Hermida et al., 1996; Fenney-Lynam et al., 1994; Ammari, 1991; Santillan
and Figueroa, 2000; Wilkinson, 1997; Zhenxuan, 1995; Parent and Coppieters, 2001.
22
Bitran, 1995; Kanji et al., 1995; Mills A et al., 1997; Mudyarabikwa and Madhina, 2000; Gilson, 1995.
23
Chalker, 1995; Haddad and Fournier, 1995; Fryatt et al., 1994; Litvack and Bodart, 1993.
24
Laing et al., 2001; Ofordi-Adjei and Arhinful, 1996; Chalker, 2001; Angunawela et al., 1991; Santoso,
1996; Hadiyono et al., 1996; Perez-Cuevas et al., 1996; Bexell et al., 1996; Birrel and Birrel, 2000; Foster,
1991.
25
Kafle et al., 1992; Marsh et al., 1999; Oshiname and Brieger, 1992; Ross-Degnan et al., 1996.
There were also reports of improved health service efficiency (improved use of physicians’
time) and improved staff attitudes towards work. In relation to the interventions
implementing more comprehensive changes, there were improvements in service quality
(reduced perinatal and maternal mortality rates).
Delivery by private providers.26 The evidence that encouraging private providers to

deliver health services results in higher quality, and greater effectiveness and efficiency,
than that of services in government institutions is rather mixed.
Two studies reported better quality in non-governmental services but pointed to the high
level of heterogeneity among service providers (Bitran, 1995; Kanji et al., 1995). In
Zimbabwe and South Africa, lower unit costs were found in privately run hospitals, but no
substantive quality difference was found in the first country and slightly superior quality
for non-governmental providers in the second country (Mills et al., 1997). In Tanzania, it
was shown that the performance of non-governmental units was somewhat variable
(Gilson, 1995). In Zimbabwe, an incentive programme to increase the participation of
the private sector produced negative outcomes (Mudyarabikwa and Madhina, 2000).
4.2.3 Improvement of drug supplies
User fees for increasing drug supplies.27 The main type of intervention was the establish-
ment of user fee schemes (with or without community management).
The main positive outcome reported was the improvement of drug supplies. There were,
however, some contradictions: firstly, in relation to utilization rates (which increased in 2
studies (Litvack and Bodart, 1993;28 Fryatt et al., 1994) and decreased in one (Haddad and
Fournier, 1995)); and secondly, in relation to prescribing practices (which improved in one
study (Chalker, 1995) and deteriorated in another one (Fryatt et al., 1994)).
Prescribing practices. Interventions were mainly educational activities (largely training

but also seminars, small group discussions and newsletters).
The overall trend was of positive outcomes. Prescribing practices were reported to have
improved in terms of decreased average number of drugs per patient/prescription, correct
selection of drugs (for instance reducing the use of non-rehydration medications and
antibiotics) and adequate prescription dosages. Although improvements were reported in
the majority of the studies, they were not uniform across all studies.
Drug retailer training. The main type of intervention reported by the studies reviewed
was the organization of training courses. With respect to outcomes, all studies reported
positive results. There were accounts of increased knowledge among trainees and
increased sales of drugs (oral rehydration salts and antimalarials).
4.3 Health Sector Policy and Strategic Management Level
Under the health sector policy and strategic management level constraints, a total of 33
studies were identified. The primary categories of interventions covered by the studies
26
In this review private providers are understood to include both the private-for-profit sector and the non-profit
sector, which includes non-governmental organisations.
27
As mentioned above, user fees were not assessed in this review. Here we focus on the strategy in relation to the
possibilities it creates for improving drug availability.
28
Referred to probability of increase.
under this level included: management strengthening (10 studies29), monitoring and health
information systems (15 studies,30 including a review), contracting (3 studies31), drug
policy and supply systems (4 studies,32 including a review), and coordination and
regulation of the private and pharmaceutical sectors (1 study33).
4.3.1 Management strengthening

Various interventions were reported. The main types were as follows:
* Workshops for identification and prioritization of managerial problems
* Introduction of regular planning and evaluation cycles
* Quality assurance methods such as: action-based learning strategy (problem analysis,
action research, problem solving and review)
* Regular meetings
* Establishment of routine communication systems
* Training activities.
Outcomes reported were mostly very positive. Reported improvements in relation to
management skills and performance were as follows:
* More rational use of resources and increased funds available (better planning,
budgeting and expenditure)
* Improved co-ordination and integration of programmes and activities (interaction of
disease-control programmes, curative and preventive activities)
* Improved methods of work: delegation, problem solving, regular sharing of lessons
(better communication)
* Improved staff morale and development of team spirit
* Improved data collection, reporting and use
* Strengthened supervision
* Improved transport strategy: allocation, enhanced selection and training of drivers,
sharing of means of transport by team members based on defined priorities
* Increased community participation in health service management or greater under-
standing of the need for this.
In addition, there were some reports of final outcomes including increased utilization of
outpatient and reproductive services and immunization coverage levels.
4.3.2 Monitoring and health information systems

The main types of interventions identified here were the introduction of management
information and disease surveillance systems (including associated training of health
workers) and Lot Quality Assurance Sampling34 (LQAS).
29
Kanlisi, 1991; Barnett and Ndeki, 1992; Ribeiro, 1994; Ranken, 1990; Atherton et al., 1999; Abubaker et al.,
1999; Lynam et al., 1993; Omaswa et al., 1997; Conn et al., 1996; Sandiford et al., 1994.
30
Robertson et al., 1997; Rosero-Bixby et al., 1990; Singh et al., 1996a; Singh et al., 1996b; Oyoo et al., 1991;
Chae et al., 1994; Johnston and Stout, 1999; Osibogun et al., 1996; Satia et al., 1994; Schmidt and Rifkin, 1996;
Santillan and Figueroa, 2001; Singh et al., 1995; Lanata et al., 1990; John et al., 1998; Husein et al., 1993.
31
McPake and Hongoro, 1995; Kravosec et al., 2000; Connor, 2000; Soeters and Grifﬁths, 2000.
32
Dong et al., 1999; Von Massow et al., 1998; Krause et al., 1998; Foster, 1991.
33
Bhat, 1996; Bhat, 1999; DFID Health Systems Resource Center, 2000.
34
Lot Quality Assurance Sampling is a survey methodology originally developed by industry to determine the
quality of lots of shipments by assessing samples of a few elements of each lot (Rosero-Bixby et al., 1990). It is
considered a useful monitoring tool for managers and in contrast to EPI survey methodology, it can
identify small population units with poor performance (Singh et al., 1996a).
Outcomes reported were mostly positive, except for the LQAS type of intervention
where outcomes were mixed. Common outcomes included improved planning and
management due to the use of local routine data. To a lesser extent, there were reports
of reduced morbidity (from vaccine preventable diseases, severe malnutrition and
malaria). In terms of negative results, there were reports of difficulties in handling data
collection and processing of information.
4.3.3 Contracting
Contracting out health services in low and middle-income countries to private sector
providers is suggested as a mechanism for increasing efficiency (McPake and Hongoro,
1995; Mills, 1998; Palmer, 2000).
Some of the pitfalls of this strategy include lack of government capacity in many
countries to design and monitor contracts, lack of technical and administrative capacity
among providers (Mills, 1998; Barnett et al., 2001), lack of competition for contracts
(McPake and Hongoro, 1995; Mills, 1998), and high transaction costs (Brugha et al., 2000).
Outcomes reported were mixed, though with more positive than negative ones. The
main type of outcome reported was the expansion of access or service utilization.
4.3.4 Drug policy and supply systems

The main type of intervention reported was a comprehensive approach to drug policies,
which included competitive bidding, management control, distribution strategies, educa-
tional activities in rational drug use and community participation.
Reported outcomes included accounts of more regular supplies on the positive side and
instances of incorrect use of drugs (e.g. sales without prescription) on the negative side. In
addition, there is some indication of positive achievements in the production of generic
highly active anti-retroviral therapy (HAART) for AIDS treatment in Brazil (Brazilian
Ministry of Health, 2001).35
4.3.5 Co-ordination and regulation of the private and pharmaceutical sectors

Only one study was identified under this section, relating to consumer protection
legislation in India (Bhat, 1996; Bhat, 1999; DFID Health Systems Resource Centre,
2000). This strategy is used as a means of dealing with issues such as medical malpractice
and negligence.
Increased awareness among patients and physicians and improved information flow
were reported as positive outcomes. On the negative side, there was a low success rate for
consumers’ legal claims.
5 DISCUSSION
5.1 Limitations of the Review
The overall quality of the studies leaves much to be desired, both in terms of design and
implementation, which means that the generalizability of the results and recommendations
35
Due to the high costs of these types of drugs, the government decided to use laboratories affiliated with public
universities to produce generic versions. This was feasible since the country passed its international patent law only
in 1996. Due to this initiative, some pharmaceutical companies have reduced their prices in order to compete with
the local production of public laboratories. For more information see background papers of the CMH Working
Group 4—Health and the International Economy available at www.cmhealth.org [accessed 5 June 2002]
is limited. There are rarely any measures undertaken in the interventions to compare
results with appropriate control groups, and sample sizes are often inadequate. Use of
randomization is rare. In addition, interviewers are not always blinded and the statistical
significance of results is often not reported. While some studies have documented factors
that facilitated the implementation and success of the assessed interventions, confounding
factors that affected the outcome of the interventions are rarely acknowledged, much less
presented and discussed. The potential for confounding is significant, and needs to be
taken into account in interpreting the results.
On the whole, most studies reported successful experiences. This could be due to
publication bias, i.e. editors tend to prefer positive findings, and researchers themselves
tend to self-censor and submit only these studies. The result is that we miss the opportunity
to learn important lessons from interventions that do not work successfully.
We recognize that this review was not able to capture the whole spectrum of published
and unpublished literature in this field and is most lacking with regard to the latter. There is
also a bias towards literature published in the English language, as access to non-English
language databases is rather limited. There is a vast literature of country level reports of
evaluations of donor-funded projects carried out by consultants that could have provided
further evidence for this review. However, it is difficult to access this type of literature,
because it is often treated as confidential by sponsoring organizations. It is also rarely
subject to critical review. Despite these limitations, we believe that the studies reviewed
here offer an overview of various attempts to overcome constraints facing health systems
at the close-to-client level. Further complementary work in this field will make valuable
contributions to the debate and provide improved information to policy makers.
This review encompasses a wide range of issues with various subtopics. While the
evidence retrieved describes specific issues within different subtopics, the number of
papers on each subtopic is limited. Due to the variety of types of intervention under the
different constraints levels, we had to take into account multiple outcomes. This makes it
difficult to generalize the results in terms of policy recommendations. An additional
problem related to generalizability is the short length of follow up of the interventions,
with data frequently collected only once after the introduction of the intervention. Short
lengths of follow up mean there is little information on the longer-term impact and
sustainability of the interventions.
Finally, the task of providing analytical input to thinking through how best to scale up
interventions to the national level and improve the health of the poor is constrained by the
very limited evidence on large-scale projects. The great majority of the interventions
covered only a district or sub-national region. Only very rarely are large scale experiences
documented. An exception is the case of the Orissa Health and Family Welfare Projects
where strengthening of training and education took place on a large scale with success
(Martinez et al., 1995).
5.2 What Seems to Work
In spite of the limitations outlined above, we believe that this review offers a valuable
profile of the evidence base in this area. The main points drawn from the reviewed studies
are discussed below.
The evidence suggests that community participation through active involvement of
leaders and members, including community health worker programmes, seems to be an
effective tool for improving performance. Effective community participation appears to

arise from the combination of ‘push’ by the community, and ‘pull’ from the existence of a
space or opportunity for them to participate, opened by IEC or other mechanisms.
Collaboration between the parties involved, community and health workers and autho-
rities, is essential for the success of the approach, as shown in Vietnam (Chalker, 2001).
There is some evidence, though not very strong, that quality assurance methods
contribute to performance improvements. According to Zeitz et al. (1993b), quality
assurance management relies on the concept that processes are responsible for the
achievement of productive work. In addition, ownership by staff in the identification,
analysis and solution of problems is an important characteristic of quality assurance
techniques (Hermida et al., 1996).36 Bottom up and participatory approaches seem to be
major features of quality assurance techniques.
The evidence supporting the success of interventions based on management strengthen-
ing is methodologically weak, based mostly on descriptive studies. Yet it provides some
suggestions, for example, that more autonomy for health managers at district level is
needed. For example, Sandiford et al. (1994) argue that the isolated implementation of
management strengthening strategies, such as training of health managers, procedures for
regular planning and evaluation, improved information systems and decentralization, can
only provide marginal gains in terms of performance improvements. System-wide changes
are required and managers and health professionals need more autonomy and performance
incentives to be able to improve outcomes. There is also some indication that approaches
using participatory methods and promoting ownership are more likely to succeed.
Another strategy is the formulation of clearer definitions of problems, which then
facilitates the identification of solutions and shows managers that the lack of resources is
partially amenable to change at the local level (Conn et al., 1996). In the case of The
Gambia, before problem analysis skills were improved, some managers mentioned that
they would not know what to do if more resources were made available.
A large number of interventions undertaken to deal with management systems, staff
issues, community participation, drug supplies and irrational drug use have concentrated
on training and supervision, yielding improvements in performance and quality of health
services.
Some intervention programmes include training activities complemented by super-
vision. Some programmes also carry out training activities alongside other intervention
packages such as community mobilisation and management strengthening, which may or
not include supervision. Nevertheless, training programmes are sometimes carried out as a
single strategy.
Intermittent training activities can deal with a specific problem but may not create the
capacity to deal with new issues. Faxelid et al. (1997), assessing a training programme in
Zambia, confirms that quality improvements need a more process-oriented and continuous
approach and that training is not the sole solution. Thus, training should be seen as a means
and not an end. Governments and their partners have to be able to provide training and
education opportunities that respond to the changes over time in disease patterns and the
development of the health system. In addition, training programmes need to be integrated
into broader staff management policies, such as systematic supervision and performance
appraisals. The abuse of training activities as a means of topping up low salaries is a well
recognized problem. This again reinforces the view that training should be part of an
36
Paraphrasing Brown et al., 1992.
integral and overarching human resource policy in which issues concerning salaries and
incentives are also addressed.
The evidence also suggests that better results can be achieved where an integrated
approach to drug policy is developed and implemented, which includes an essential drugs
policy, well managed selection, procurement, distribution and dispensing policies,
improved prescribing practice and community/patient education. This finding is in line
with those of Homedes and Ugalde (2001) which reviewed 45 studies of interventions to
improve patients’ drug use. They argue that focusing on only one side of the problem will
not bring about the necessary changes. The World Health Organization (2000) and the
World Bank (1994) also support integrated drug policies.
In addition, drug supply needs to be viewed as part of the health care system. The
overall effectiveness and efficiency of a well-managed drug supply policy will only be
achieved if other complementary elements of the system are also well managed, such as
staff training, supervision and incentives.
5.3 What Facilitates
Various facilitating factors are identified in the literature reviewed. However, difficulties
are encountered when trying to assess the impact and mode of causality of these factors.
They have been reported here as additional, explanatory factors for the success or failure
of the interventions. Table 4 presents a list of the main type of facilitating factors
according to the different levels used to categorize constraints.
Further facilitating factors revealed by an assessment undertaken by the World Bank of
its portfolio in the areas of health, nutrition and population corroborates the type of factors
identified in our review (Johnston and Stout, 1999). The Bank’s assessment showed that
projects able to achieve a high level of institutional development had the following
Table 4. Categorization of facilitating factors
Levels Facilitating factors
I. Community and household level —Community participation

II. Health services delivery level —Staff motivation
—Team work
—Frequent communication
—Supervision and feedback mechanisms
III. Health sector policy and —Liaison units or group of facilitators for driving and
strategic management level maintaining
change processes
—Frequent communication
—Effective technical and managerial support to strengthen the
capacity of community health workers, staff, and NGOs
—Participative, bottom up approaches involving community,
managers and staff
IV. Public policies cutting —Decentralization and autonomy at regional and local levels
across sectors —Intersectoral collaboration, partnerships (with clear definition
of roles of each partner and democratic involvement)
V. Environmental characteristics —Political and macroeconomic stability
—Commitment, leadership and ownership of all partners
(government, staff, community)
features in common: continuous commitment, thorough analysis of constraints, flexible

implementation, and a positive governance and macroeconomic environment.
5.4 Research Priorities
This review of efforts to overcome constraints to health service delivery at the close-to-
client level has provided evidence of various gaps and weaknesses. It points to the vital and
urgent need for further investigation to enhance our understanding on this issue and better
inform policy makers.
We argue that it is of paramount importance that more empirical evidence, based on
methodologically sound research, is produced in order to help guide decision-making and
planning. As discussed in Section 5.2, the conclusions of the present undertaking are
constrained by the lack of good quality data and analysis. To generate this evidence there
is need for more technical collaboration and assistance in order to foster evaluative
research in developing countries where limited financial resources and analytical capacity
present an obstacle to developing a broader research base that could provide local
decision-makers with context-specific evidence.
Few studies present the costs of the intervention. More data on the relative costs and
effectiveness of alternative approaches to service strengthening are needed. There is scope
for improving resource use by building more economic analyses into assessments of
interventions.
Further studies are needed in order to elucidate the extent to which facilitating factors
affect the performance of the close-to-client health system and under which circum-
stances. The explanation of the mechanisms of operation and sustainability of these factors
will provide important answers in our efforts to find ways to overcome constraints. Some
factors such as decentralisation have already been the focus of investigation (Bossert,
1998; Araujo, 1997; Mills, 1994; Collins and Green, 1994; Gilson et al., 1994; Mills,
1990) but empirical analysis of its impact with respect to reduced morbidity or mortality is
still pending. Other factors, such as the role of leadership in the context of low and middle-
income countries, are also priority areas for further research. Existing literature on this
subject is concentrated on the business sector and developed countries (Goleman, 2000;
Johnson, 1996) with very limited investigation in low and middle-income countries
(Santillan and Figueroa, 2001; Lenneiye, 2000).
Finally, the evidence is insufficient and vague with respect to the extent to which
interventions reach the poor and disadvantaged, and the success of interventions
specifically focused on these groups. It is crucial that more studies in this area are
undertaken in order to deepen our understanding of how to reach this section of the
population.
6 CONCLUDING REMARKS
Some of the successful interventions presented in this review may seem remarkable for
their simplicity, such as patient flow analysis (Hermida et al., 1996; Fenney-Lynam et al.,
1994; Ammari et al., 1991; Santillan and Figueroa, 2000; Parent and Coppieters, 2001)
while others are quite innovative, for instance interactive group discussions involving
patients and prescribers, which resulted in reduced use of unnecessary injections (Perez-
Cuevas et al., 1996). Part of the achievements are due not to the technique itself but to the
process through which the changes were made possible, by applying participative
approaches, gaining staff commitment and involving the community. This serves to
highlight that while various strategies to improve health are necessary, quality
improvements and efficiency gains can be achieved through low cost and simple methods.
Account should be taken of the context, process and facilitating factors which need to be
better understood with respect to their triggering and nurturing mechanisms.
Attempting to weigh the relative importance of the different constraints and the impact
of interventions—i.e. those which are simply helpful and ‘enabling’ versus those which
are more essential and ‘necessary’—highlights the interconnected and complementary
nature of the components of health systems. To be able to say something meaningful about
the relative contribution of these components to health outcomes requires systematic and,
optimally, quantitative data from studies of the functioning of such systems. A ‘soft’
element, like staff organization, cannot be considered, a priori, as being of lesser
importance and impact than a ‘hard’ element like drug availability.
Finally, it can be argued that changes of the type presented in this review, in the micro
level structure and functioning of the health system (such as improvements in manage-
ment, information systems, and planning and evaluation), are inherently limited in the
degree of impact they can have within a given institutional framework at the macro level.
Accordingly, macro level constraints (such as financing, government personnel policies
and administrative centralization), although more difficult to alter and requiring longer
time frames, must also be included in any analysis of ways to achieve significant and
sustainable improvements in health outcomes.
ACKNOWLEDGEMENTS
This paper is part of a study commissioned by the World Health Organization’s

Commission on Macroeconomics and Health. We are grateful for helpful comments
from an anonymous reviewer, and from David Peters and Javier Martinez to earlier
versions of this work. Hanson and Mills are members of the Health Economics and
Financing Programme, and Oliveira-Cruz of the Health Systems Development Pro-
gramme, both of which receive support from the UK Department for International
Development. The UK Department for International Development (DFID) supports
policies, programmes and projects to promote international development, but the views
and opinions expressed are those of the authors alone.
REFERENCES
Abdulla S, Armstrong-Schellenberg J, Nathan R, Mukasa O, Marchant T, Smith T, Tanner M,

Lengeler C. 2001. Impact on malaria morbidity of a programme supplying insecticide treated nets
in children aged under two years in Tanzania: community cross sectional study. British Medical
Journal 322: 270–273.
Abubaker WA, Al-Assaf AF, Cleaver VL, et al. 1999. Quality assurance and iron deficiency in
Egypt. International Journal for Quality in Health Care 11(2): 163–168.
Ahmed N, Zeitlin MF, Beiser AS, Super CM, Gershoff S. 1993. A longitudinal study of the impact of
behavioural change intervention on cleanliness, diarrhoeal morbidity and growth of children in
rural Bangladesh. Social Science and Medicine 37(2): 159–171.
Ammari N, Zahra NA, Dreesch N. 1991. Streamlining clinic management. World Health Forum
12(3): 479–482.
Angunawela II, Diwan VK, Tomson G. 1991. Experimental evaluation of the effects of drug
information on antibiotic prescribing: a study in outpatient care in an area of Sri Lanka.
International Journal of Epidemiology 20: 558–564.
Araújo J. 1997. Attempts to decentralise in recent Brazilian health policy: issues and problems,
1988–1994. International Journal of Health Services 27(1): 109–124.
Arhin-Tenkorang D. (Draft, November 2000). Mobilizing resources for health: the case for user
fees revisited. [Available at http://www.cmhealth.org/docs/user_fees.pdf accessed on 20 May
2001].
Atherton F, Mbekem G, Nyalusi I. 1999. Improving service quality: experience from the Tanzania
Family Health Project. International Journal for Quality in Health Care 11(4): 353–356.
Ayele F, Desta A, Larson C. 1993. The functional status of community health agents: a trial of
refresher courses and regular supervision. Health Policy and Planning 8(4): 379–384.
Bang AT, Bang RA, Sontakke PG, Search Team. 1994. Management of childhood pneumonia by
traditional birth attendants. Bulletin of the World Health Organisation 72(6): 897–905.
Barnett C, Connor C, Putney P. 2001. Contracting non-governmental organisations to combat HIV/
AIDS. Special Initiative Report No. 33. Partnerships for Health Reform Project, Abt Associates
Inc: Bethesda, Maryland.
Barnett E, Ndeki S. 1992. Action-based learning to improve district management: a case study from
Tanzania. International Journal of Health Planning and Management 7: 299–308.
Benavides BM, Jacoby ER. 1994. An operational evaluation of the Community Oral Rehydration
Units in Peru. Health Policy and Planning 9(4): 438–443.
Bexell A, Lwando E, von Hofsten B, Tembo S, Eriksson B, Diwan V. 1996. Improving drug use
through continuing education: a randomised control trial in Zambia. Journal of Clinical
Epidemiology 49(3): 355–357.
Bhat R. 1996. Regulating the private sector: the case of the Indian Consumer Protection Act. Health
Policy and Planning 11(3): 265–279.
Bhat R. 1999. Characteristics of private medical practice in India: a provider perspective. Health
Birrel G, Birrel KG. 2000. Assessment of a 1-year teaching programme in Zanzibar, Tanzania. The
Lancet 356(September 23): 1084.
Bitran R. 1995. Efficiency and quality in the public and private sectors in Senegal. Health Policy and
Planning 10(3): 271–283.
Bossert T. 1998. Analysing the decentralisation of health systems in developing countries: decision
space, innovation and performance. Social Science and Medicine 47(10): 1513–1727.
Brazilian Ministry of Health. 2001. National drug policy. Brazilian Programme of Sexually
Transmissible Diseases and AIDS, Brasilia, Distrito Federal.
Brown LD, Franco LM, Rafeh N, Hatzell T. 1992. Quality Assurance of Health Care in Developing
Countries. Center for Human Services: Bethesda, MD.
Brugha R, Smith E, Zwi A. 2000. Working with Private Sector Providers for Better Health Care: An
Introductory Guide. LSHTM and Options: London.
Chae YM, Kim SI, Lee BH, Choi SH, Kim IS. 1994. Implementing health management information
systems: measuring success in Korea’s health centers. International Journal of Health Planning
and Management 9(4): 341–348.
Chalker J. 1995. Effect of a drug supply and cost sharing system on prescribing and utilization: a
controlled trial from Nepal. Health Policy and Planning 10(4): 423–430.
Chalker J. 2001. Improving antibiotic prescribing in Hai Phong province, Viet Nam: the ‘antibiotic-
dose’ indicator. Bulletin of the World Health Organization 79(4): 313–320.
Chaulagai CH. 1993. Urban community health volunteers. World Health Forum 14: 16–19.
Chernichovsky D, Bayulken C. 1995. A pay-for-performance system for civil service doctors: the
Indonesian experiment. Social Science and Medicine 41(2): 155–161.
Chomitz K, Setiadi G, Azwar A, Ismail N, Widiyarti. 1997. What do Doctors Want? Two Empirical
Estimates of Indonesian Physicians’ Preferences Regarding Service in Rural and Remote Areas.
World Bank: Washington, DC.
Christensen PB, Karlqvist S. 1990. Community health workers in a Peruvian slum area: an
evaluation of their impact on health behavior. Bulletin of the Pan-American Health Organisation
24(2): 183–196.
Collins C and Green A. 1994. Decentralisation and primary health care: some negative implications
in developing countries. International Journal of Health Services 24(3): 459–475.
Commission on Macroeconomics and Health. 2001. Macroeconomics and Health: Investing in
Health for Economic Development. Report of the Commission on Macroeconomics and Health.
World Health Organization: Geneva.
Commission on Macroeconomics and Health. 2002. Improving Health Outcomes of the Poor: The
Report of Working Group 5 of the Commission for Macroeconomics and Health. World Health
Organization, Geneva.
Conn CP, Jenkins P, Touray SO. 1996. Strengthening health management: experience of district
teams in The Gambia. Health Policy and Planning 11(1): 64–71.
Connor C. 2000. Contracting non-governmental organizations for HIV/AIDS: Brazil case study.
Special Initiative Report No. 30. Partnerships for Health Reform Project, Abt Associates Inc:
Bethesda, MD.
Curtale F, Siwakoti B, Lagrosa C, LaRaja M, Guerra R. 1995. Improving skills and utilization of
community health volunteers in Nepal. Social Science and Medicine 40(8): 1117–1125.
DFID Health Systems Resource Center. 2000. Making the most of the private sector. Report of
workshop organised on behalf of the UK Department of International Development in May, 2000.
Dong H, Bogg L, Rehnberg C, Diwan V. 1999. Drug policy in China: pharmaceutical distribution in
rural areas. Social Science and Medicine 48(6): 777–786.
Eloundou-Enyegue PM, Meekers D, Calves AM. 1998. From awareness to adoption: the effect of
AIDS education and condom social marketing on condom use in Tanzania (1993–1996). PSI
Research Division Working Paper 17. Population Services International (PSI): Washington, DC.
Faveau V, Wojyniakk B, Chakraborty J, Sarder AM, Briend A. 1990. The effect of maternal and child
health and family planning services on mortality: is prevention enough? British Medical Journal
301: 103–107.
Faxelid E, Ahlberg BM, Freudenthal S, Ndulo J, Krantz I. 1997. Quality of STD care in Zambia. Impact
of training in STD management. International Journal for Quality in Health Care 9(5): 361–366.
Fenney-Lynam P, Smith T, Dwyer J. 1994. Client flow analysis: a practical management technique
for outpatient clinic settings. International Journal of Health Care Quality Assurance 6(2): 179–
186.
Foster S. 1991. Supply and use of essential drugs in sub-Saharan Africa: some issues and possible
solutions. Social Science and Medicine 32(11): 1201–1218.
Frankenberg E, Thomas D. 2000. Women’s health and pregnancy outcomes: does access to services
make a difference? DRU-2329-NIH, Prepared for National Institutes of Health. Labor and
Population Program, Rand Working Paper Series 00–04.
Fryatt RJ, Rai P, Crowley SP, Gurung YB. 1994. Community financing of drug supplies in rural
Nepal: evaluating a ‘fee per item’ drug scheme. Health Policy and Planning 9(2): 193–203.
Gilson L, Kilima P, Tanner M. 1994. Local government decentralisation and the health sector in
Tanzania. Public Administration and Development 14: 451–477.
Gilson L. 1995. Management and health care reform in sub-Saharan Africa. Social Science and
Medicine 40(5): 695–710.
Goleman D. 2000. Leadership that gets results. Harvard Business Review March-April: 78–90.
Haddad S, Fournier P. 1995. Quality, cost and utilisation of health services in developing countries.
A longitudinal study in Zaı̈re. Social Science and Medicine 40(6): 743–753.
Hadiyono JEP, Suryawati S, Danu S, Sunartono, Santoso B. 1996. Interactional group discussion:
results of a controlled trial using a behavioural intervention to reduce the use of injections in
public health facilities. Social Science and Medicine 42(8): 1177–1183.
Hadley M, Maher D. 2000. Community involvement in tuberculosis control: lessons from other
health care programmes. International Journal of Lung Diseases 4(5): 401–408.
Hanson K, Jones C. 2000. Social marketing of insecticide treated mosquito nets, Tanzania: end of
phase 1 social and economic analysis. Technical assistance to PSI Tanzania, final report. London
School of Hygiene and Tropical Medicine for the Malaria Consortium. Unpublished mimeo.
Hanson K, Ranson K, Oliveira-Cruz V, Mills A. Expanding access to priority health interventions: a
framework for understanding the constraints to scaling up. Journal of International Development
15(1): 1–14.
Hermida J, Laspina C, Idrovo F. 1996. Reducing patient waiting time through quality assurance
methods in La Troncal, Ecuador. Bulletin of the Pan-American Health Organisation 30(2): 118–
124.
Homedes N, Ugalde A. 2001. Improving the use of pharmaceuticals through patient and community
level interventions. Social Science and Medicine 52: 99–134.
Husein K, Adeyi O, Bryant J, Cara NB. 1993. Developing a primary health care management
information system that supports the pursuit of equity, effectiveness and affordability. Social
Sciences and Medicine 36(5): 585–596.
Janowitz B, Suazo M, Fried DB, Bratt JH, Bailey PE. 1992. Impact of social marketing on
contraceptive prevalence and cost in Honduras. Studies in Family Planning 23(2): 110–117.
John TJ, Samuel R, Balraj V, John R. 1998. Disease surveillance at district level: a model for
developing countries. The Lancet 352: 47–50.
Johnson K. 1996. Building capacity through collaborative leadership. International Journal of
Health Planning and Management 11: 339–344.
Johnston and Stout S. 1999. Investing in Health: Development Effectiveness in the Health, Nutrition
and Population Sector. Operations Evaluation Department. The World Bank: Washington, DC.
Kafle KK, Gartoulla RP, Pradhan YMS, Shrestha AD, Karkee SB, Quick JD. 1992. Drug retailer
training: experiences from Nepal. Social Science and Medicine 35(8): 1015–1025.
Kahssay HM, Oakley P. 1999. Community involvement in health development: a review of the
concept and practice. World Health Organization: Geneva.
Kanji N, Kilima P, Lorenz N, Garner P. 1995. Quality of primary outpatient services in Dar-es-
Salaam: a comparison of government and voluntary providers. Health Policy and Planning 10(2):
186–190.
Kanlisi N. 1991. Strengthening district health systems in Ghana: the experience of Ejisu District.
Tropical Doctor 21: 98–100.
Kelley E, Geslin C, Dijibrina S, Boucar M. 2000. The impact of QA methods on compliance with the
integrated management of childhood illness Algorithm in Niger. Operations Research Results
1(2). Published for the U.S. Agency for International Development (USAID) by the Quality
Assurance Project (QAP): Bethesda, MD. [Available at http://www.qaproject.org, accessed May

2001].
Kelly P. 2001. Local problems, local solutions: improving tuberculosis control at the district level in
Malawi. Bulletin of the World Health Organization 79(2): 111–117.
Kim YM, Putjuk F, Kols A, Basuki E. 2000. Improving provider-client communication: reinforcing
IPC/C training in Indonesia with self-assessment and peer review. Operations Research Results
1(6). Published for the U.S. Agency for International Development (USAID) by the Quality
Assurance Project (QAP): Bethesda, MD. [Available at http://www.qaproject.org, accessed May
2001].
Kim YM, Rirnon J, Winnard K, Corso C, Mako IV, Lawal S, Babalola S, Huntington D. 1992.
Improving the quality of service delivery in Nigeria. Studies in Family Planning 23(2): 118–127.
Knippenberg R, Alihonou E, Soucat A, Oyegbite K, Calivis M, Hopwood I, Niimi R, Diallo MP,
Conde M, Ofosu-Amaah S. 1997. Implementation of the Bamako Initiative: strategies in Benin
and Guinea. International Journal of Health Planning and Management 12(S1): S29–S47.
Krause G, Benzler J, Heinmueller R, Borchert M, Koob E, Ouattara K, Diesfeld J. 1998.
Performance of village pharmacies and patient compliance after implementation of an essential
drug programme in rural Burkina Faso. Health Policy and Planning 13(2): 159–166.
Kravosec K, Connor C, Putney P, Barnett C. 2000. Public sector contracting of NGOs to deliver HIV/
AIDS services in Guatemala and Brazil. XIII International Aids Conference, abstract of poster
presentation, TuPeD3819.
Laing R, Hogerzeil HV, Ross-Degnan D. 2001. Ten recommendations to improve use of medicines in
developing countries. Health Policy and Planning 16(1): 13–20.
Lanata CF, Stroh GJR, Black RE, Gonzales H. 1990. An evaluation of lot quality assurance sampling
to monitor and improve immunization coverage. International Journal of Epidemiology 19(4):
1086–1090.
Lenneiye NM. 2000. Leadership for kufufuka kwa Africa: African transformation as a leadership
challenge. IDS Working Paper 119. Institute of Development Studies.
Levy-Bruhl D, Soucat A, Osseni R, Ndiaye JM, Dieng B, De Bethune X, Diallo AT, Conde M, Cisse
M, Moussa Y, Drame K, Knippenberg R. 1997. The Bamako Initiative in Benin and Guinea:
improving the effectiveness of primary health care. International Journal of Health Planning and
Management 12(S1): S49–S79.
Litvack J, Bodart C. 1993. User fees plus quality equals improved access to health care: results of a
field experiment in Cameroon. Social Science and Medicine 37(3): 369–383.
Loevinsohn BP, Guerrero ET, Gregorio SP. 1995. Improving primary health care through systematic
supervision: a controlled field trial. Health Policy and Planning 10(2): 144–153.
Lynam P, Rabinovitz LM, Shobowale M. 1993. Using self-assessment to improve the quality of
family planning clinic services. Studies in Family Planning 24(4): 252–260.
Mahomed H, Bachman MO. 1998. Block appointments in an overloaded South African health
centre: quantitative and qualitative evaluation. International Journal of Health Care Quality
Assurance 11(4): 123–126.
Marsh VM, Mutemi W, Some ES, Haaland A, Snow RW. 1996. Evaluating the community education
programme of an insecticide-treated bed net trial on the Kenyan coast. Health Policy and Planning
11(3): 280–291.
Marsh VM, Mutemi WM, Muturi J, Haaland A, Watkins WM, Otieno G, Marsh K. 1999. Changing
home treatment of childhood fevers by training shop keepers in rural Kenya. Tropical Medicine
and International Health 4(5): 383–389.
Martinez J, Martineau T. 1998. Rethinking human resources: an agenda for the millennium. Health
Martinez J, Aitken JM, Martineau T, Hoare G. March 1995. Orissa: health sector strategic review.
Key findings and recommendations. A study conducted for the Overseas Development Admin-
istration. Unpublished mimeo.
McPake B, Hongoro C. 1995. Contracting out of clinical services in Zimbabwe. Social Science and
Medicine 41(1): 13–24.
McPake B, Hanson K, Mills A. 1993. Community financing of health care in Africa: an evaluation of
the Bamako Initiative. Social Science and Medicine 36(11): 1383–1395.
McPake B, Hanson K, Mills A. 1992. Implementing the Bamako Initiative in Africa: a review and
five case studies. PHP Departamental Publication, No. 8, London School of Hygiene and Tropical
Medicine.
Meekers D. 2000a. Going underground and going after women: trends in sexual risk behaviour
among gold miners in South Africa. International Journal of STD & AIDS 11: 21–26.
Meekers D. 2000b. The role of social marketing in STD/HIV protection in 4600 sexual contacts in
urban Zimbabwe. PSI Research Division Working Paper 34. Population Services International
(PSI): Washington, DC.
Meekers D. 1998. The effectiveness of targeted social marketing to promote adolescent reproductive
health: the case of Soweto, South Africa. PSI Research Division Working Paper 16. Population
Services International (PSI): Washington, DC.
Mehnaz A, Billoo AG, Yasmeen T, Nankani K. 1997. Detection and management of pneumonia by
community health workers—a community intervention study in Rehri Village, Pakistan. Journal
of the Pakistan Medical Association 47(2): 42–45.
Melville B, Fidler T, Mehan D, Bernard E, Mullings J. 1995. Growth monitoring: the role of
community health volunteers. Public Health 109: 111–116.
Mills A. 1990. Decentralisation of health services: a review. In Health System Decentralisation
Concepts, Issues and Country Experience, Mills A, Vaughan P, Smith D, Tabibzadeh I (eds).
World Health Organization: Geneva.
Mills A. 1994. Decentralisation and accountability in the health sector from an international
perspective: what are the choices? Public Administration and Development 14: 281–292.
Mills A. 1997. Improving the efficiency of Public Sector Health Services in developing countries:
bureaucratic versus market approaches. In Marketizing Education and Health in Developing
Countries: Miracle or Mirage? Colclough C (ed.). Clarendon Press: Oxford.
Mills A. 1998. To contract or not to contract? Issues for low and middle income countries. Health
Mills A, Hongoro C, Broomberg J. 1997. Improving the efficiency of district hospitals: is contracting
an option? Tropical Medicine and International Health 2(2): 116–126.
MMWR. 1998. Health worker performance after training in integrated management of child-
hood illness—Western Province, Kenya. MMWR—Morbidity and Mortality Weekly Report
47(46).
Moser C, Sollis P. 1991. Did the project fail? A community perspective on a participatory primary
health care project in Ecuador. Development in Practice (an Oxfam Journal) 1(1): 19–33.
Mudyarabikwa O, Madhina D. 2000. An assessment of incentive setting for participation of private-
for-profit health care providers in Zimbabwe. Small Applied Research No. 15. Partnerships for
Health Reform Project, Abt Associates Inc: Bethesda, MD.
Ofordi-Adjei D, Arhinful DK. 1996. Effect of training on the clinical management of malaria by
medical assistants in Ghana. Social Science and Medicine 42(8): 1169–1176.
Oliveira-Cruz V, Hanson K, Mills A. 2001. Approaches to overcoming health systems constraints at
the peripheral level: a review of the evidence. Commission on Macroeconomics and Health,
Working Group 5: Improving Health Outcomes of the Poor. Unpublished mimeo. Available
at: http://www3.who.int/whosis/cmh/cmh_papers/e/papers.cfm?path ¼ cmh,cmh_papers&langu-

age ¼ english#
Omaswa F, Burnham G, Baingana G, Mwebesa H, Morrow R. 1997. Introducing quality manage-
ment into primary health care services in Uganda. Bulletin of the World Health Organization
75(2): 155–161.
Onyejiaku EE, Holzemer WL, Morrow HM, Olabode MA, Rogers S. 1990. Evaluation of a primary
health care project in Nigeria. International Nursing Review 37(3): 265–270.
Oshiname FO, Brieger WR. 1992. Primary care training for patent medicine vendors in rural Nigeria.
Social Science and Medicine 35(12): 1477–1484.
Osibogun A, Jaksic Z, Idowu JA, Alausa OK, Oluwole FA. 1996. For better data, better utilized.
World Health Forum 17(3): 274–276.
Oyoo A, Burstrom B, Forsberg B, Makhulo J. 1991. Rapid feedback from household surveys in PHC
planning: an example from Kenya. Health Policy and Planning 6(4): 380–383.
Özek B, Saat Z, Temiz AT, Kinzie B. 1998. On-the-job training through follow-up visits to improve
the quality of family planning services. European Journal of Contraceptive and Reproductive
Health Care 3(4): 201–206.
Pagnoni F, Convelbo N, Tiendrebeogo J, Cousens S, Esposito F. 1997. A community-based
programme to provide prompt and adequate treatment of presumptive malaria in children.
Transactions of the Royal Society of Tropical Medicine and Hygiene 91: 512–517.
Palmer N. 2000. The use of private-sector contracts for primary health care: theory, evidence and
lessons for low-income and middle-income countries. Bulletin of the World Health Organization
78(6): 821–829.
Parent F, Coppieters Y. 2001. A process of change in first-line health services in Chad. Health Policy
and Planning 16(1): 122–123.
Perez-Cuevas R, Guiscafre H, Munoz O, Reyes H, Tome P, Libreros V, Gutierrez G. 1996. Improving
physician prescribing patterns to treat rhinopharyngitis. Intervention strategies in two health
systems of Mexico. Social Science and Medicine 42(8): 1185–1194.
Perry H, Robison N, Chavez D, Taja O, Hilari C, Shanklin D, Wyon J. 1999. Attaining health for all
through community partnerships: principles of the census-based, impact-oriented (CBIO)
approach to primary health care developed in Bolivia, South America. Social Science and
Medicine 48(8): 1053–1067.
Perry H, Robison N, Chavez D, Taja O, Hilari C, Shanklin D, Wyon J. 1998. The census-based,
impact-oriented approach: its effectiveness in promoting child health in Bolivia. Health Policy
and Planning 13(2): 140–151.
Ranken J. 1990. Strengthening local health management. Health Policy and Planning 5(3): 292–
296.
Ribeiro MC. 1994. Better management for better health. World Health 47(6): 14–15.
Robertson SE, Anker M, Roisin AJ, Macklai N, Engstrom K, LaForce FM. 1997. The lot quality
technique: a global review of applications in the assessment of health services and disease
surveillance. World Health Statistics Quarterly 50: 199–209.
Robinson JS, Burkhalter BR, Rasmussen B, Sugiono R. 2001. Low-cost on-the-job peer training of
nurses improved immunisation coverage in Indonesia. Bulletin of the World Health Organization
79: 150–158.
Robinson SA, Larsen D. 1990. The relative influence of the community and the health system on
work performance: a case study of community health workers in Colombia. Social Science and
Medicine 30(10): 1041–1048.
Rosero-Bixby L, Grimaldo C, Raabe C. 1990. Monitoring a primary health care programme with lot
quality assurance sampling: Costa-Rica, 1987. Health Policy and Planning 5(1): 30–39.
Ross-Degnan D, Soumeral SB, Goel PK, Bates J, Makhulo J, Dondi N, Sutoto, Adi D, Ferraz-Tabor
L, Hogan R. 1996. The impact of face-to-face educational outreach on diarrhoea treatment in
pharmacies. Health Policy and Planning 11(3): 308–318.
Ruck NF, Darwish OA. 1991. Motivating health workers through nutrition training: an example from
Egypt. Health Policy and Planning 6(2): 130–140.
Ruebush II TK, Zeissig R, Koplan JP, Klein RE, Godoy HA. 1994. Community participation in
malaria surveillance and treatment III. An evaluation of modifications in the volunteer colla-
borator network of Guatemala. American Journal of Tropical Medicine and Hygiene 50(1): 85–98.
Sandiford P, Kanga GJ, Ahmed AM. 1994. The management of health services in Tanzania: a plea
for health sector reform. International Journal of Health Planning and Management 9: 295–308.
Santillan D, Figueroa ME. 2000. Implementing a client feedback system to improve the quality of
NGO health services in Peru. Operations Research Summary. Published for the U.S. Agency for
International Development (USAID) by the Quality Assurance Project (QAP): Bethesda, MD.
[Available at http: //www.qaproject.org, accessed May 2001].
Santillan D, Figueroa ME. In press. Improving client satisfaction in Peruvian health clinics.
Operations Research Summary. Quality Assurance Project. [Available at http: //www.qaproject.-
org, accessed May 2001].
Santoso B. 1996. Small group intervention vs. formal seminar for improving appropriate drug use.
Social Science and Medicine 42(8): 1163–1168.
Satia JK, Mavalankar DV, Sharma B. 1994. Micro-level planning using rapid assessment for primary
health care services. Health Policy and Planning 9(3): 318–330.
Schellenberg JR, Abdulla S, Minja H, Nathan R, Mukasa O, Marchant T, Mponda H, Kikumbih N,
Lyimo E, Manchester T, Tanner M, Lengeler C. 1999. KINET: a social marketing programme of
treated nets and net treatment for malaria control in Tanzania, with evaluation of child health and
long-term survival. Transactions of the Royal Society of Tropical Medicine and Hygiene 93(3):
225–231.
Schmidt DH, Rifkin SB. 1996. Measuring participation: its use as a managerial tool for district
health planners based on a case study in Tanzania. International Journal of Health Planning and
Management 11: 345–358.
Sepehri A, Pettigrew J. 1996. Primary health care, community participation and community-financing:
experiences of two middle hill villages in Nepal. Health Policy and Planning 11(1): 93–100.
Singh J, Jain DC, Sharma RS, Verghese T. 1996a. Further observations on comparison of
immunisation coverage by Lot Quality Assurance Sampling and 30 Cluster Sampling. Southeast
Asian Journal of Tropical Medicine and Tropical Health 27(2): 267–273.
Singh J, Jain DC, Sharma RS, Verghese T. 1996b. Evaluation of immunisation coverage by lot
quality assurance sampling compared with 30-cluster sampling in a primary health centre in India.
Bulletin of the World Health Organization 74(3): 269–274.
Singh J, Sharma RS, Goel RK, Verghese T. 1995. Concurrent evaluation of immunization
programme by lot quality assurance sampling. Journal of Tropical Pediatrics 41: 215–220.
Smith JB, Coleman NA, Fortney JA, Johnson JDF, Blumhagen DW, Grey TW. 2000. The impact of
traditional birth attendant training on delivery complications in Ghana. Health Policy and
Planning 15(3): 326–331.
Snow RW, McCabe E, Mbogo CNM, Molyneux CS, Some ES, Mung’Ala VO, Nevill CG. 1999. The
effect of delivery mechanisms on the uptake of bed net re-impregnation in Kifili District, Kenya.
Health Policy and Planning 14(1): 18–25.
Soeters R, Griffiths F. 2000. Can government health workers be motivated? Experimenting with
contract management: the case of Cambodia. Centre D’Etudes et de Recherches sur le Devel-
oppement International (CERDI), International Conference, 30 November–01 December 2000.
Stock-Iwamoto C, Korte R. 1993. Primary health workers in Northeast Brazil. Social Science and
Medicine 36(6): 775–782.
Stuer F. 1998. Enhancing health programme efficiency: a Cambodian case study. Health Policy and
Planning 13(3): 263–276.
UNAIDS. 1998. Social marketing: an effective tool in the global response to HIV/AIDS. UNAIDS
Best Practice Collection, Key Material: Geneva.
Van Rossem R, Meekers D. 1999a. An evaluation of the effectiveness of targeted social marketing to
promote adolescent reproductive health in Guinea. PSI Research Division Working Paper 23.
Population Services International (PSI): Washington, DC.
Van Rossem R, Meekers D. 1999b. An evaluation of the effectiveness of targeted social marketing to
promote adolescent and young adult reproductive health in Cameroon. PSI Research Division
Working Paper 19. Population Services International (PSI): Washington, DC.
Von Massow F, Korte R, Cheka C, Kuper M, Tata H, SchmidtEhry B. 1998. Financially independent
primary health care drug supply system in Cameroon. Tropical Medicine and International Health
3(10): 788–801.
Walker D, McDermott JA, Fox-Rushby J, Tanjung M, Nadjib M, Widiatmoko D, Achadi E. 2002. An
economic analysis of a midwifery training programme in South Kalimantan, Indonesia. Bulletin of
the World Health Organization 80(1): 47–55.
Wilkinson D. 1997. Reducing perinatal mortality in developing countries. Health Policy and
Planning 12(2): 161–165.
World Bank. 1993. World Development Report 1993: Investing in Health. World Bank: Washington,
DC.
World Bank. 1994. Better Health in Africa: Experience and Lessons Learned. World Bank:
Washington, DC.
World Health Organisation. 2000. WHO medicines strategy: 2000–2003. Essential Drugs and
Medicines Policy Department, World Health Organisation. WHO Policy Perspectives on Med-
icines, no. 1.
Zeitz PS, Harrison LH, Lopez M, Cornale G. 1993a. Community health worker competency in
managing acute respiratory infections of childhood in Bolivia. Bulletin of the Pan-American
Health Organisation 27(2): 109–119.
Zeitz PS, Salami CG, Burnham G, Goings SAJ, Tijani K, Morrow RH. 1993b. Quality assurance
management methods applied to a local-level primary health care system in rural Nigeria.
International Journal of Health Planning and Management 8: 235–244.
Zhenxuan X. China, 1995. Lowering maternal mortality in Miyn County, Beijing. World Health
Statistics Quarterly 48: 11–14.

Effectiveness Health Care

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Effectiveness Health Care

Diunggah oleh

Hak Cipta:

Format Tersedia

Journal of International Development

J. Int. Dev. 15, 41–65 (2003)

Copyright # 2003 John Wiley & Sons, Ltd.

human resource development, and expansion of infrastructure which underpin these

The methodology applied in this review is a systematic search of published literature.

Table 1. Categorization of constraints

I. Community and —Lack of demand for effective interventions

II. Health services —Shortage and distribution of appropriately qualified staff

Source: Hanson et al. (2003)

Table 2. Distribution of studies by region and country

Tanzania (10) India (9) Peru (5) Jordan (1)

4.1 Community and Household Level

Table 3. Links between constraints, interventions and outcomes

Country Ranking Constraints Interventions Outcomes Facilitating factors

Nepal 1. yes —Poor health Community —Inadequate Adequate

4.1.1 Community participation

as improved technical and accounting capacity). In addition, there were improvements

4.1.2 Bamako Initiative19

4.1.3 Community health workers (CHWs)

4.1.4 Appropriate patient use of drugs

4.1.5 Social marketing

4.2 Health Services Delivery Level

4.2.1 Human resource development

Delivery by private providers.26 The evidence that encouraging private providers to

Prescribing practices. Interventions were mainly educational activities (largely training

4.3 Health Sector Policy and Strategic Management Level

4.3.1 Management strengthening

4.3.2 Monitoring and health information systems

4.3.4 Drug policy and supply systems

4.3.5 Co-ordination and regulation of the private and pharmaceutical sectors

5.1 Limitations of the Review

5.2 What Seems to Work

effective tool for improving performance. Effective community participation appears to

5.3 What Facilitates

I. Community and household level —Community participation

features in common: continuous commitment, thorough analysis of constraints, flexible

5.4 Research Priorities

This paper is part of a study commissioned by the World Health Organization’s

Abdulla S, Armstrong-Schellenberg J, Nathan R, Mukasa O, Marchant T, Smith T, Tanner M,

Assurance Project (QAP): Bethesda, MD. [Available at http://www.qaproject.org, accessed May

at: http://www3.who.int/whosis/cmh/cmh_papers/e/papers.cfm?path ¼ cmh,cmh_papers&langu-

Anda mungkin juga menyukai