Social Research in Developing Nations

Methods For Social Researchers In Developing Countries
Home TOC Parts Glossary Links References Contact Us Help
Welcome
Focus group: Dar ElSalam Camp, near Omdurman, Sudan

This site has been prepared specifically for beginning researchers in developing
countries. It is based on a book by the same title published by The Ahfad University for
Women.. We have created this Web site to make the contents of Methods for Social
Researchers in Developing Countries available free to researchers in developing
countries, where books are too expensive for faculty, students, or even for libraries to
buy. Of course, we hope this site may be useful to researchers in other areas of the world
as well. All material on this site may be freely downloaded or copied. There are no
restrictions on use of what we present. Restrictions on use, however, may apply to the
contents of some of the sites we refer to.
To help you get started, we suggest you read the Overview, which provides the structure
of the site and briefly describes the content of each of its 20 chapters. The Help section
will also help you get started. This section describes how you can move from one page
or section of the site to another pages or sections; ways you can find specific information
and different ways you can read the contents of the site; and, for those who are not
familiar with the Internet, a brief introduction to the Internet and Web sites.
http://www.srmdc.net/ (1 of 2) [2/11/2008 8:39:05 AM]

Methods For Social Researchers In Developing Countries
In addition, we call your attention to some Web sites that provide a wealth of
information and links to additional and alternative approaches to conducting social
research.
Email:
Adjunct Professor
The Ahfad University for Women (AUW)
Omdurman, Sudan
Executive Director
The Sudan-American Foundation for Education, Inc.
and the Ahfad University for Women
http://www.srmdc.net/ (2 of 2) [2/11/2008 8:39:05 AM]

Contact Us
Contact Us
We hope you will find the contents of this site useful. If you have, we would like to hear
what you liked or found particularly helpful. If any part is not clear or helpful to you, we
want to know about that too. We also welcome suggestions of links we might add to this
site, as a way of further helping other researchers find useful information. We also want
to know about any link we provide that that no longer works. We will use your feedback
as we update and revise this site. Just send an email to the address below. Thank you.
Dr. Lee G. Burchinal
Executive Director
Sudan-American Foundation for Education, Inc.
4141 N. Henderson Road, Suite #1205
Arlington, VA 22203
USA
E -mail:
http://www.srmdc.net/contactus.htm (1 of 2) [2/11/2008 8:39:14 AM]

Contact Us
Hosted by the Sudan-American Foundation for Education, Inc.

http://www.srmdc.net/contactus.htm (2 of 2) [2/11/2008 8:39:14 AM]

Table of Contents
Table of Contents
❍ Overview
❍ Selected Social Research Methods Web Sites
❍ Help
■ Introduction
■ Navigating this site
■ Reading this site
■ Internet searching
■ Aids
Part One: Thinking as a Researcher

Chapter 1. Understanding Scientific Inquiry
■ Introduction
■ Alternatives to Scientific Inquiry
■ Scientific Inquiry
■ Limits of Scientific Inquiry
■ The Research Process
■ Aids
Chapter 2. The Sudan Fertility Survey: An Introduction to Research
■ Introduction
■ The Research Question
■ Designing the Study
■ Collecting the Data
■ Analyzing the Data
■ Interpreting the Results
■ Generalizing the Results
■ Aids
Chapter 3. Learning the Language of Research
http://www.srmdc.net/toc.htm (1 of 7) [2/11/2008 8:39:20 AM]

Table of Contents
❍ Introduction
❍ Concepts and Variables
❍ Theory
❍ Hypotheses
❍ Logic of Scientific Inquiry
❍ Cause and Effect
❍ Aids
Part Two: Planning Research
Chapter 4. Selecting a Question to Investigate
■ Introduction
■ Your initial research question
■ Reviewing the literature
■ Preparing references
■ Refining your list of references
■ Searching the Internet
■ Your final research question
■ Aids
Chapter 5. Creating a Research Design
❍ Design and the purpose of the research
❍ Quantitative and qualitative data
❍ Design alternatives
❍ Aids
Chapter 6. Developing Measures
❍ Overview
❍ Steps in measurement
❍ An illustration
❍ Level of measurement
❍ Validity
❍ Reliability
❍ Relationship between validity and reliability
❍ Reactivity
❍ Ethical considerations

Table of Contents
❍ Aids
Chapter 7. Constructing Composite Measures: Scales and Indexes
❍ Introduction
❍ Scales
❍ Likert scales
❍ Bogardus social distance scales
❍ Guttman scales
❍ Indexes
❍ Additional considerations
❍ Typologies
❍ Measurement error
❍ Aids
Chapter 8. Selecting a Sample

❍ Introduction
❍ Probability sampling
❍ Simple random sample
❍ Systemic random sample
❍ Stratified random sample
❍ Cluster sample
❍ Creativity in sampling
❍ Weighted samples
❍ Problems to watch for
❍ Nonprobability sampling
❍ Sample size
❍ Aids
Part Three: Creating Data
Chapter 9. Designing Experiments
■ Introduction
■ Seeking causal relationships
■ Alternative explanations
■ Internal validity
■ The classical experiment
■ Quasi-experimental designs

Table of Contents
■ External validity
■ Further variations
■ Strengths and limitations
■ Aids
Chapter 10. Surveys and Questionnaire Construction
❍ Introduction
❍ Characteristics of surveys
❍ Planning a survey
❍ Questionnaire construction
❍ The final questionnaire
❍ Qualitative surveys
❍ Internet-based surveys
❍ Response rate
❍ Strengths and limitations
❍ Improving survey results
❍ Aids
Chapter 11. Interviewing
❍ Introduction
❍ Steps in interviewing
❍ Conducting interviews
❍ Completing post interviewing tasks
❍ Telephone interviews
❍ Validity and reliability issues
❍ Sources of error
❍ Aids
Chapter 12. Using Available Data
❍ Introduction
❍ Sources
❍ Analyzing available records
❍ Secondary analyses
❍ Content analysis
❍ Social indicators

Table of Contents
❍ Aids
Chapter 13. Observing Behavior
❍ Introduction
❍ Causal observation
❍ Participant observation: Overview
❍ Conducting a participant observation study
❍ Establishing validity
❍ Generalizing
❍ Personal requirements
❍ Case studies
❍ Focus groups
❍ Aids
Chapter 14 Performing Evaluation Research
❍ Introduction
❍ Illustration
❍ Distinctive features
❍ Conducting an evaluation
❍ Qualitative evaluations
❍ Problems to watch for
❍ Ethical concerns
❍ Aids
Chapter 15. Using a MultiMethod Design and Conducting Rapid Rural

Appraisals
❍ Multimethod approach
❍ Rapid rural appraisal
❍ Ethical issues
❍ Aids
Part Four: Analyzing Data
Chapter 16. Preparing Data for Analysis
❍ Introduction

Table of Contents
❍ Hand versus compute analysis

❍ Coding
❍ Data entry
❍ Aids
Chapter 17. Conducting Univariate Analyses
❍ Frequency distributions
❍ Analyzing single variables
❍ Presenting univariate data
❍ Central tendency
❍ Variability
❍ Standard deviation and the normal distribution
❍ Computer analysis reminder
❍ Aids
Chapter 18. Tabular Bivariate and Multivariate Analyses

❍ Introduction
❍ Bivariate analysis: nominal variables
❍ Bivariate analysis: ordinal variables
❍ Statistical tests for association
❍ Aids
Chapter 19. Performing Inferential Statistical Analyses
❍ Samples and populations
❍ Probability theory and statistical inference
❍ Inferring a population mean
❍ Tests of statistical significance
❍ Tests of differences between means
❍ Coefficient of correlation
❍ Caution with association
❍ Chi square
❍ Other tests of significance
❍ Caution in using statistical test results
❍ Aids
Part Five: Telling What You Did

Table of Contents
Chapter 20. Reporting Your Research

❍ Introduction
❍ Improving your writing
❍ Guidelines for clear writing
❍ Writing your report
❍ Final checking
❍ Oral presentations
❍ Using the mass media
❍ Aids
❍ Glossary
❍ Web Site Index
❍ References

SRMDC Overview
Overview
This section provides an outline of the contents of the site.
The site has five main parts. Part 1, "Thinking Like a Researcher," introduces you to
social research and the way researchers look at the world around them. Part 2 describes
what you need to know to plan a research project. In Part 3 you will learn about ways of
collecting data. Ways of analyzing data are described in Part 4. The last step in a
research project presenting your results is covered in Part 5.
Part 1 introduces you to scientific inquiry and the research process. Scientific inquiry is
the focus of Chapter 1. Chapter 1 also describes the steps in the research process.
Chapter 2 describes social research as it was carried out in the form of the Sudan
Fertility Survey and presents some of the important findings from this study. Becoming
a social researcher also requires learning the language of social research. Chapter 3 will
give you a basic research vocabulary. Together, the three chapters of Part 1 will give
you a foundation for planning and conducting your own research.
Part 2, "Planning Research," describes how to plan a research project, beginning with
the selection of a problem to investigate. Chapter 4 discusses this important step. Also
included in Chapter 4 is a closely related step in the research process that of reviewing
the research literature on the topic you choose to investigate. The other four chapters of
Part 2 describe how to design a research project. These include decisions you need to
make in preparing an overall design for your project (Chapter 5), developing measures
for the data you will collect (Chapter 6 and Chapter 7), and preparing a plan for
selecting the persons from whom you will collect data (Chapter 8).
Alternative ways of collecting data are described in Part 3, "Creating Data." These are:
conducting experiments (Chapter 9); constructing questionnaires for use in surveys
(Chapter 10); conducting interviews (Chapter 11); analyzing available data (Chapter
12); observing behavior (Chapter 13); and conducting evaluation studies (Chapter 14).
Chapter 15 presents the multi-method approach to data collection. One specific
multi-method of research, called Rapid Rural Appraisal, is also described in Chapter 15.
One or a combination of the methods described in these chapters should meet your
immediate and most future research needs.
Part 4 contains four chapters on how to organize and analyze data. Chapter 16
http://www.srmdc.net/overview.htm (1 of 3) [2/11/2008 8:39:22 AM]

SRMDC Overview
describes the important step of preparing data for analysis. The next two chapters
describe ways of analyzing your data. Chapter 17 covers the analysis of data for each
characteristic you want to know about. Researchers refer to the characteristics they study
as variables. Ways of analyzing of two or more variables are covered in Chapter 18.
Chapter 19 presents the logic and methods for using statistical tests in data analysis.
Finally, Part 5, with only one chapter, Chapter 20, describes how to prepare for and
write a report describing your research project. Part 5 also discusses additional ways
you can report your research results through publication in a professional journal, by
giving talks based on your research, by preparing a newspaper or magazine article, or by
appearing on a radio or television program.
To help you get an idea of what each chapter contains, we list the main topics covered in
the chapter in a column on the left side of each page of each chapter. By clicking on any
topic, you can go directly to that topic. Also, figures and diagrams are used to show the
sequence of steps in completing a research task. As mentioned earlier, key words used in
research are shown in italics when they are first used and are defined in the Glossary.
When you discover a term that you are not sure about, you can find its meaning in the
Glossary. In addition, a list of "Main points" is included as a summary at the end of each
chapter.
Throughout the book, we emphasize the important roles of decision-making and
creativity in research.Research does not just happen.The direction and success of any
research project depends on the decisions you make at each step in your project. To help
you make intelligent decisions, we show the choices that frequently have to be made and
give you bases for making them. We also try to show that research is a creative process.
By your choices, you create the data you will use to answer your research question.
Also, you decide how to analyze, present, and interpret your research results. Each of
these steps offers an opportunity to creatively apply what you will have learned.
As you read, we hope you will see that research is a demanding process as well; one that
requires knowledge, skill, and careful decision-making. We also hope you will see that
research is an exciting, creative, and rewarding activity.
We hope you will find the contents of this site useful. If you do, we would like to hear
what you liked or found particularly helpful. If any part is not clear or helpful to you, we
want to know about that too. You can send comments, suggestions, and criticisms to us
by clicking on Contact Us on any page where it appears. We also welcome your
suggestions of links we might add to this site, as a way of further helping other
researchers find useful information. Click on Contact Us and send us the address of any
Web site you think we should add. We also want to know about any link we provide that
that no longer works. We will use your feedback as we update and revise this site.
Best wishes in any research you undertake.
NEXT TOC

SRMDC Overview


Part One: Thinking Like a Researcher

Chapter 1
As part of your preparation in becoming a researcher, you will need to develop a scientific
Understanding
perspective toward the world around you. This includes how you look at everyday social
Scientific
relationships. The purpose of Part One is to help you develop such a perspective.
Inquiry
Social research, like all other scientific research, is based on scientific inquiry. This special way
Chapter 2 of looking at the world began about 500 years ago as a way of understanding the natural universe.
The Sudan Since then, methods of scientific inquiry have spread to all fields of human knowledge. Today,
Fertility Survey: methods of the scientific inquiry are accepted throughout the world as the most reliable way of
An Introduction understanding events and processes in the natural world. This includes how social behavior
to Research occurs and changes.
Chapter 3 Chapter 1 presents the origins of scientific inquiry and its application in social research.
Learning the Chapter 1 also describes the assumptions underlying scientific inquiry and the norms that guide
Language of researchers in their work. Both are combined in the steps of the research process. Chapter 1
Research outlines these steps. The rest of this site fills in the details for completing each step in the
research process. In your own research, you will repeat these steps.
Another way to learn how researchers think is to follow the course of a research project from start
to finish. Chapter 2 will allow you to do this. In this chapter, we present a brief description of
one study, known as the Sudan Fertility Survey. (Fertility is a measure of the number of children
born to some group of women). The Sudan Fertility Survey provided the most accurate
information available on fertility among women in northern Sudan at the time of the survey. It
also provided important information about conditions that may affect fertility and, therefore, the
size of the future population of Sudan. Our brief description of the Sudan Fertility Survey should
help you understand how an investigation is planned and carried out. We also hope you will see
the value of the social research, as shown in this one study, and how research results can help
explain the processes of social change.
Becoming a researcher also requires learning the language of social research. In Chapters 1 and
2 you will begin to acquire a research vocabulary. Chapter 3 continues your introduction to the
language of social research. This chapter presents the conceptual basis for research, how
hypotheses are developed and used, the forms of reasoning used in research, and the criteria for
judging whether one condition might be the cause of change in something else.
When you complete Part One you will be ready to take the first step in the research process that
of selecting a question to investigate.
PREV NEXT TOC
Hosted by the SUDAN-AMERICANFOUNDATION FOR EDUCATION, INC.

http://www.srmdc.net/parts/1.htm (1 of 2) [2/11/2008 8:39:30 AM]


Part Two: Planning Research
Chapter 4 Part Two: Planning Research

Selecting a In Part One, we introduced you to how researchers think. In Part Two, we show how researchers
Question plan research projects. The first step is to decide what you want to study. Chapter 4 deals with
to Investigate this important step. Part of the process of selecting a problem to investigate is learning about
existing research on the problem. In research, this process is known as reviewing the literature.
Chapter 5 Chapter 4, therefore, includes details on how to plan and conduct a review of the literature. With
Creating a the growing importance of the Internet and Web sites as sources of social research information,
Research Chapter 4 also provides an introduction to Internet searching and provides some selected social
Design
research Web sites.
Chapter 6 After you know what you want to study, you can begin to plan how you are going to carry out
Developing your study. This is the second main step in research and consists of formulating a design or plan
Measures for the project. Four chapters are devoted to this process. Chapter 5 addresses the main
decisions that have to be made in preparing a research design, including the definition of the
Chapter 7 concepts or variables you intend to study. Chapter 6 and Chapter 7 describe ways of
Constructing measuring concepts and variables. The important topic of selecting a sample, including different
Composite kinds of samples you might want to use, is discussed in Chapter 8.
Measures:
Scales and After you finish Part Two you should have a pretty good idea of how to select a question to
Indexes investigate and how to prepare a research design to provide data for answering your research
question.
Chapter 8
PREV NEXT TOC
Selecting a
Sample
http://www.srmdc.net/parts/2.htm [2/11/2008 8:39:36 AM]


From the beginning of this book, you have read about the collection of data. Part Three, with its
Chapter 9 seven chapters, presents the main ways data are created and used in social research. The idea of
Designing creating instead of collecting data may strike you as odd. Most researchers use terms like
Experiments collecting or gathering data. In fact, we have throughout this book. Why, then, do we speak of
creating data? Simple. As the researcher you create your research project. You decide which
Chapter 10 variables to study. You also decide how to measure each of these variables, and which method or
Surveys and methods you will use to obtain these measurements. In addition, you will choose how to analyze
Questionnaire your data and what conclusions you will draw from your analyses.
Construction
We hope you approach the following chapters with this view in mind. With this perspective, you
Chapter 11 may appreciate even more your own vital role in how your data are created and, therefore, how
your decisions and actions affect the outcome of your research.
Interviewing
Accepting the idea that you create your data also means that you and you alone are responsible
Chapter 12 for the accuracy of your data. We hope this sobering thought will increase your interest in
Using Available learning how to create, record, and analyze your data as well as possible.
Data
Part Three provides detailed descriptions and step-by-step guidance for applying the most
Chapter 13 frequently used methods of conducting social research. We begin with a discussion of the
experiment the most rigorous form of scientific inquiry. Chapter 9 describes how experiments
Observing
Behavior are planned and conducted in the social sciences. The following two chapters are devoted to
conducting surveys. Chapter 10 describes how to construct a questionnaire. Chapter 11
Chapter 14 describes how to conduct interviews using a questionnaire. Not all research, however, depends on
Performing collecting new data. Creative researchers find ways to reanalyze available data to answer new
Evaluation research questions. This approach to research is covered in Chapter 12. Observation is mankind's
Research oldest technique for learning about our environment. Observation as a scientific process is
described in Chapter 13. Social scientists are frequently asked to conduct evaluations of medical,
Chapter 15 educational, or other social development efforts. Chapter 14 describes ways of planning and
Using a Multi- conducting evaluations of programs. Part Three ends with Chapter 15, which has two main parts.
Method Design The first part compares the strengths and limitations of each of the research methods presented in
and Conducting the preceding chapters and discusses the advantages of using a multimethod approach to
Rapid Rural collecting data. The second part of this chapter describes how to plan and conduct a Rapid Rural
Appraisals Appraisal, a specific form of multimethod research.
After you complete Part Three, you will be ready to prepare a research design and, using that
design, to collect data to answer your research question. Part Four, which follows, describes
how to organize and analyze the data you will collect.
PREV NEXT TOC



Parts Four: Analyzing Data
Part 4. Analyzing Data

After you have collected your data, you are ready to analyze it. But first, you have to prepare
your data for analysis. In Chapter 16, "Preparing Data for Analysis," we describe ways of
preparing data for analysis by hand or with the help of a statistical analysis package. After the
Chapter 16 data are prepared, analysis begins by describing each key variable, one at a time. In statistics, this
Preparing Data process is known as univariate analysis; "uni" means one and "variate" refers to variable.
for Analysis Techniques of univariate analysis are covered in Chapter 17. After completing univariate
analyses, researchers frequently move on to the simultaneous analysis of two or more variables or
Chapter 17 what is called bivariate analysis. "Bi" refers to two and, with variate, means two variables. The
Techniques of simultaneous analysis of three or more variables is called multivariate analysis. "Multi" means
Univariate many, but in research done by a lone researcher, such as you may be, typically only three or four
Analyis variables are analyzed together. Chapter 18, "Conducting Bivariate and Multivariate Analyses,"
shows how to conduct bivariate and multivariate analyses. Results of univatiate, bivariate, and
Chapter 18 multivariate analyses are frequently reported in the form of tables or graphs. Therefore, we also
Conducting show how to construct and "read" tables and graphs.
Bivariate and
Multivariate Chapter 19, "Performing Inferential Statistical Analyses," explains the logic behind using data
Analyses from a sample to estimate or infer characteristics of the population from which the sample was
selected. Two frequently used forms of inference are explained and illustrated. These are: (1)
Chapter 19 estimating the value of a population mean, using data from a sample; and (2) conducting a
Performing statistical test of significance for a relationship between two or more variables in a population.
Inferential Each is based on probability theory. Chapter 19, therefore, includes an introduction to
Statistical probability theory and its use in making statistical inferences. This chapter also describes how to
Analyses conduct some frequently used tests of statistical significance.
PREV NEXT TOC


Part Five: Writing Your Report
Part 5. Writing Your Report

You have spent a lot of time and effort in conducting your research and analyzing your data. Now
it is time to write your report.
In Chapter 20, the last chapter on this site, we provide some guidance on how to prepare for and
Chapter 20 to write your result. We also provide links to some Web sites that you might find useful.
Reporting Your
Experienced researchers often use multiple ways to communicate their results to various
Researchs
audiences. These include preparing a manuscript for submission to a journal, giving an oral
report, and reporting your results through the mass media. We offer some suggestions for
reporting your results in these additional ways.
We wish you success in writing your report and in sharing your result findings with others.
PREV NEXT TOC


Chapter 1 Page 1
Chapter 1: Understanding Scientific Inquiry

Introduction
Introduction
Alternatives
to scientific Social research is based on the methods of scientific inquiry. This way of looking at the world
inquiry around us is relatively new, beginning only about 500 hundred years ago. Previously, people
followed other paths to what was accepted as true. Their beliefs about nature and mankind were
based mainly on traditions passed down from one generation to the next and the unquestioned
Scientific
acceptance that what they were taught was true.
inquiry
Scientific inquiry began in Europe in the 16th century when a few men challenged the traditional
Limits of and authoritarian beliefs of their time and began analyzing what they observed about nature.
scientific Their early research led to what we now call scientific inquiry. All scientific research, including
inquiry social research, is based on the assumptions underlying scientific inquiry and the norms scientists
use in conducting and evaluating research. Much of this chapter is devoted to an examination of
the assumptions and norms of scientific inquiry.
The research
process Scientific inquiry, however, has limits. It cannot be used to discover everything we would like to
know. Scientific inquiry, for example, cannot answer questions about what is right or wrong in a
Aids moral sense; nor can it settle differences in religious or philosophical beliefs. Its use is limited to
answering questions only about things we can observe with our senses. Gaining an understanding
of scientific inquiry and its limits is essential for planning and conducting research.
Alternatives to scientific inquiry
Humans, with brains and feelings much like ours, have roamed the earth for perhaps 100,000
years or longer. Through all this time, humans have lived in social groups - families, tribes, and
now nations - and, over time, have developed a large number of different beliefs about the natural
world and how social life should be organized. These beliefs, common to all societies, include
how the universe and our world were created; why the sun, moon and stars move as they do; how
mankind came to exist; why crops grow or fail to do so; on what basis leaders of the group should
be selected; and thousands of other things that each society accepts as true and proper.
Some of the beliefs from these earlier times still persist and can influence our ability to conduct
scientific inquiry. Two forms of beliefs, in particular, can undermine a scientific approach to
discovering new information. These are the acceptance of traditional beliefs and those based on
the authority of persons. As researchers, we have to guard against accepting either form of beliefs
without testing them to see if they are true. Unquestioned acceptance of traditional and
authoritarian beliefs can influence your thinking as you prepare to do research. Also, each of us,
even with research training, can make certain errors in reaching conclusions from what we
observe. One kind of error is the uncritical acceptance of traditional beliefs.
PREV NEXT
1 Definitions for key terms, shown in italics, are provided in the Glossary.
http://www.srmdc.net/chapter1/1.htm (1 of 2) [2/11/2008 8:39:50 AM]

Chapter 1 Page 1

Chapter 1 Page 2
Traditional beliefs
Introduction
Traditional beliefs are accepted as true simply because previous generations passed them on to
Alternatives the next generation as true. Traditional beliefs are not necessarily true or false; right or wrong;
to scientific good or bad. From a social point of view, some traditional beliefs have positive results.
inquiry Traditional medicines, based on local plants, have cured illnesses in many societies for
generations. Scientists are now examining these "folk" remedies to find new medicines. Other
Scientific traditional beliefs, however, have negative effects. In some societies, women follow the belief of
inquiry "eating down" eating less than normal or, at least, not increasing their food intake during
pregnancy. This practice is based on the belief that eating down will limit the size of the baby
Limits of and, thus, avoids a difficult delivery (Brems and Berg, 1989; Harrison, 1992, p. 21; and El Hiday
and Zumrawi, 1992, p. 31). In addition, societies have a number of beliefs about food that should
scientific
be avoided during pregnancy. In many regions of India, for example, animal foods (eggs, fish,
inquiry meat), which are quite nutritious for pregnant women, are perceived as harmful during pregnancy
(Nag, 1994).
The research
process Apart from their possible negative social effects, social scientists have serious objections to
accepting traditional beliefs as true. In traditional societies, beliefs rest on the power of the group
Aids to make members of the group believe them. Where tradition prevails, there is no procedure for
testing the truthfulness of beliefs: They simply have to be accepted as true. To question a belief
could bring scorn or even punishment from other members of the group.
As a researcher, we urge you to analyze the basis of statements you have been told are true. The
fact that "everybody says" this or that is true is not a scientific basis for accepting what is true.
Accepting statements based on tradition could prevent you from conducing scientifically
acceptable research. In contrast, scientists base their conclusions, the statements expressing what
was learned from research, only on what has been observed, and not on the basis of what is
generally thought to be true.
Beliefs based on authority
Authoritarian beliefs rest on the fact that members of a group accept a statement as true because
persons in powerful positions in the group say the statement is true. These persons may be tribal
or religious leaders, elders, military officials, political leaders, teachers or scientists. Challenging
a statement by a prominent authority requires courage and judgment. Some authorities, like
scientists or teachers, spend a great deal of time and effort to master a field of knowledge. We
rely on these specialists to provide knowledge for our use. But even scientists sometimes disagree
on certain points. Further, a scientist recognized as an authority in one field may act like an
expert in another field, even when the scientist has no exceptional knowledge in that field and
could be plain wrong! To prevent or correct errors scientists demand evidence in support of any
statement about our natural world, regardless of who says it is true.
In The Nature of Inquiry, Professor Darden, University of Maryland, United States, shows how
even highly respected scientists can be wrong. She provides a readable and thoughtful essay in
response to the question: "Are there cases of famous scientists who did good work but also made

Chapter 1 Page 2
honest mistakes?" In answering this question, Professor Darden also illustrates the self correcting
processes of science.
Relying on experience
Each of us has a variety of personal experiences as we grow up and interact daily with others. We
not only learn from these experiences, but we also develop our own unique ways of processing
these experiences in our brains. Through these processes, we reach conclusions about events all
the time. To illustrate, based on your experiences with family members, friends and others, you
have probably decided that most people are fair and honest. The expansion of your conclusion
from the limited experience you have had with a relatively small number of persons to "most"
persons is an example of forming a broader more general statement or what scientists call
drawing a generalization.
The purpose of social research is to produce generalizations about people and their interactions in
groups. In forming generalizations from the data they collect, social scientists guard against
making errors that we all sometimes make in drawing conclusions from everyday experiences.
The names of authors and dates of publications are used in research reports to identify
publications. The full reference for these and other publications we cite are listed in the list of
References. See Chapter 4 for details on how to prepare and cite references.
PREV NEXT
1 The names of authors and dates of publications are used in research reports to identify
publications. The full reference for these and other publications we cite are listed in the list of
References. See Chapter 4 for details on how to prepare and cite references.

Chapter 1 Page 3
One form of error is over-generalizing. This occurs when we arrive at some conclusion or make a
Introduction generalization based on limited information. Let's say you meet some fellow students from a
Alternatives certain area of your country and all said they came from large families. On this basis, you might
be tempted to tell others that all families in that area have large numbers of children. If you did,
to scientific
you would be over-generalizing because you don't have data about all or even a large number of
inquiry the families in that area. Some families might include only a few or even no children.
Researchers limit their conclusions to what their data clearly support.
Scientific
inquiry Another kind of common error is to reach a conclusion before all the data are examined. This is
called premature closure. Let's go back to the example of family sizes. Suppose you decided to
Limits of investigate the actual sizes of the families living in a certain area. Let's say you asked the heads of
20 families out of several hundred living in the area how many persons reside in their households.
scientific
You find that most have large numbers of members, but some do not. If you stopped collecting
inquiry data at that point and concluded that, indeed, most families are large you would probably be
making the error of premature closure. In effect, you closed your mind to new information. To
The research safely generalize about sizes of families in the area, you would need to collect data from a much
process larger number of families representing the entire group of families.
Aids We are also prone to selective observation, another common error in forming conclusions based
on experience. As the name implies, we select certain experiences and ignore others. Often, the
experiences we select are ones that agree with our values or prejudices. We look for what we
expect to find. Using the family size example yet again, suppose you had been told repeatedly
that people living in the area in question all want to have many children. You meet a few students
from the area and find they also want to have large families, which reinforces what you expected
to hear. In addition, suppose you also meet an equal number of students from the area who say
they want only a few children. Without being consciously aware of it, you might disregard the
reports that are contrary to your expectation and form your conclusion based on only on the
reports that agreed with what your expected to hear.
As researchers, we have to guard against generalizing beyond the data we have; closing our
minds to additional data; or selecting only certain findings and ignoring other data when we are
forming generalizations. In later chapters, you will learn how to avoid these kinds of errors.
Scientific inquiry
Origins of the scientific approach
Beginning in about the 16th century in Europe, a small number of men began to question
traditionally accepted beliefs about the natural world. One of these early scientists was Galileo
Galile, an Italian. As students, men like Galileo and others before him, learned what educated
persons had been taught for over a thousand years. Beliefs about the natural world were then
based on what Aristotle, a Greek philosopher, had written about 1,000 years earlier. One of these
"truths" was that the earth was the center of the universe; another was that the sun revolved
around the earth.
But the early scientists began to question these beliefs. Based on their observations of the

Chapter 1 Page 3
movements of the sun, planets, and other bodies in the universe, they saw a different pattern.
Their observations convinced them that the earth was not the center of the universe and that the
earth circled the sun and not the other way around. Instead of accepting the traditional beliefs of
their age, they reported their observations and conclusions. Other scientists, at great risk from
religious authorities who defended the traditional beliefs, repeated the studies of earlier scientists
and came to similar conclusions.
Based on the agreement among their observations, these early scientists gained confidence in
their methods and, after a struggle lasting several centuries, succeeded in establishing a new way
of inquiring about the natural world. In several centuries, the scientific method of inquiry spread
from the study of the physical things, to biology and later to the study of the human mind and
human behavior and, lastly, to the investigation of social life. Today, the scientific approach is
accepted worldwide as the best way of establishing knowledge about what we can observe. This
includes understanding how social relationships develop and are carried out.
PREV NEXT

Chapter 1 Page 4
The scientific approach to knowledge

Introduction At this point in your education, you are no doubt familiar with the "scientific method." We put
Alternatives scientific method in quotation marks because there is no single scientific method. There is no
to scientific single way of describing the scientific approach to establishing knowledge. We present one way.
inquiry In addition to our description, you may want to look at other descriptions. Go to google.com or
some other search engine and do a search on "scientific inquiry" or "scientific method" and you
will find that there are literally millions of sites with views on these topics. Or, you may wish to
Scientific
visit three sites we recommend. These are:
inquiry
Introduction to the Scientific Method describes steps in the scientific method, discusses
Limits of hypothesis testing, identifies common mistakes in applying the scientific method, and addresses
scientific other topics related to scientific inquiry.
inquiry An Introduction to Science provides a definition of science, discusses scientific and critical
thinking in terms of three central elements; empiricism, use of empirical or observational
The research evidence; rationalism, the practice of logical reasoning; and skepticism, possessing a questioning
process attitude.
Aids The Scientific Method defines the scientific method, the difference between a fact, theory, and
hypothesis, how science established truth, and other issues related to establishing scientific
knowledge.
By comparing what we say with the discussions on these sites and others you may find, you will
gain a clearer idea of how scientific inquiry is carried out.
Although scientists may describe the process of scientific inquiry in different ways, all scientific
research is based on a way of answering questions about the natural world. Scientific inquiry is
based on a set of assumptions about how knowledge is established and a set of norms for guiding
how research is conducted. First, let's examine the assumptions underlying scientific inquiry.
Assumptions underlying scientific inquiry
Scientific inquiry is based on certain assumptions about the natural world we live in and how we
learn about it. Although the following assumptions cannot be "proved" as true, scientists accept
them as reasonable bases for conducting scientific inquiry.
There is an objective reality. Science assumes there is an objective reality outside of what we
experience through our senses. Each of us develops some idea of this reality from our daily
experiences. We see the sun rise each day. We do not imagine this. It is a real event. So are the
many other things we experience daily. They are real and external to us.
Nature is orderly. Natural events and processes occur in an orderly, predictable manner. Long
before scientific research began, humans saw orderly processes in nature - in the movement of the
sun, moon, and planets; in the recurring seasons; in the cycle of life, growth, and death; and in so
many other ways. Scientists seek to discover orderly processes in nature and to describe them as
precisely as possible. In the physical sciences, description takes the form of mathematical

Chapter 1 Page 4
relations among things. Social scientists also seek to discover recurring patterns in social
behavior, although we are not able to summarize these patterns as precisely as physical scientists
do.
Nature is an interactive, closed system. To scientists, nature is not only orderly, but it is an
interactive system. Each part or each event is connected to other parts and events. Weather
systems illustrate this point. We see the changes coming, often in the form of changes in cloud
formations and wind directions. Heavy clouds generally bring rain. It doesn't rain without clouds.
The clouds form because of changes in temperature, pressure, humidity, or wind directions at
points far away. Although far from perfect, scientific understanding of what causes changes in
weather patterns helps us predict and prepare for future weather conditions.
Our description of the assumptions of science is adapted from the work of Merton (1968, pp.
607-615); Nachmias & Nachmias (2000, pp. 5-7); and Williams (2000): See the List of
References.
PREV NEXT
1 Our description of the assumptions of science is adapted from the work of Merton (1968, pp.
607-615); Nachmias & Nachmias (2000, pp. 5-7); and Williams (2000): See the List of
References.

Chapter 1 Page 5
Social scientists make similar assumptions about social behavior. We look for the regularities -
Introduction the recurring patterns in what people, say, do, and think. Repeated forms of behavior are
Alternatives expressions of the norms and values of a society. Changes in norms of a group are linked to
changes in other social conditions of the group. Changes in roles of women, for example, from
to scientific
working only in the home as mothers and wives to assuming employment outside the home, don't
inquiry just happen. These kinds of changes generally result from changes in the overall economic
system of a society followed by changes in views about women working outside the home. Social
Scientific scientists look for these kinds of relationships and seek to explain why they occur.
inquiry
Mankind can understand nature, including human behavior. Social scientists also assume
Limits of that the human mind can comprehend our external world. Although we are part of the world's
biological system, we also possess several unique abilities. Only humans, as far as we know, can
scientific
record what we experience and use these observations to construct a description of our external
inquiry world. Further, our capacity extends to knowing ourselves and understanding the minds of others.
On this basis, social scientists are able to discover regularities of social life and develop
The research explanations for social behavior.
process
Agreement about reality is possible. For years the scientific community believed it was
Aids possible to do objective research. With proper methods and careful research, scientists thought
they could provide an objective description of the natural world, devoid of human error. With
advances in understanding of how human sensory and intellectual systems work, we now
recognize that absolute objectivity is impossible. Research is conducted by human beings: None
of us, no matter how well trained and disciplined in conducting research, can completely
overcome basic human limitations in perception and ways of thinking. We all bring at least
traces of our personal interests and our own way of perceiving and thinking to whatever we
study. All observation, which includes all data collection, therefore, is subject to error. This
being the case, how do scientists agree on what is believed to be true based on the results of
scientific inquiry?
While recognizing that absolute objectivity is impossible, scientists assert that it is possible to
construct an approximation of the real conditions we seek to understand. Our nearest
approximation to reality is based on the extent of agreement among competent scientists about
what is accepted as a true description of some aspect of nature. Scientists in a given field, in
effect, agree to accept certain conclusions as true when they are supported by a large amount of
verified data. Agreement comes after a rigorous process of criticism and review by fellow
scientists. Still, the truth of statement always remains open to question, subject to change based
on new discoveries.
Box 1.1 below summarizes the assumptions of science.

Chapter 1 Page 5
Box 1.1. Assumptions of Scientific Inquiry

1. There is an objective reality apart from our perceptions of
things.
2. Natural events and processes occur in an orderly, regular
manner.
3. The natural events and processes we observe are part of larger,
closed, interactive systems. All events have natural causes
within these systems.
4. Humans can comprehend how natural systems function.
5. Scientific inquiry provides a basis for reaching agreement of
what is true about things we observe what really exists.
Norms of scientific inquiry

Scientists seek to apply the norms and values listed in Box 1.2 in all their work. These are the
ideals. In practice, individual scientists and even scientific organizations do not always live up to
these norms. When scientists fail to.
PREV NEXT
1 Our description of the norms of science is based mainly on the work of Merton (1968, pp.
607-615); and Zucherrman (1988, pp. 515-16).

Chapter 1 Page 6
Universalism. Ideally, the scientific community recognizes no advantages or distinctions among

Introduction scientists other than the merit of their research. Research is judged by its scientific quality, not by
Alternatives who did the research or where it was done. Whether the investigator was a male or female, from
one country versus another, or what their previous research record is should not matter: Only the
to scientific
quality of research being reported is considered. Science strives to be a true international effort.
inquiry
Reliance on observation. Science relies on evidence derived from observations. In doing
Scientific research, scientists record what they observe through their senses - seeing, hearing, touching,
inquiry smelling, or tasting. Relying on human senses to establish knowledge is known as empiricism.
Scientific knowledge rests on empirical discovery and logical analysis of what is observed.
Limits of Scientists accept no other basis for establishing knowledge. All conclusions or generalizations
must be based on empirical evidence.
scientific
inquiry Box 1.2. Norms of scientific inquiry
1. Science is universal. It is based on the same rules, norms, and criteria
The research for judging quality of research for scientists anywhere in the world.
process
2. Science relies on empirical observation, derived from use of the human
senses.
Aids
3. Every research report is open to critical review by other scientists.
4. Scientists accept unexpected or contrary results equally with expected
findings.
5. Scientists seek to be as objective as possible in all their work.
6. Scientists expect absolute honesty by researchers in conducting and
reporting scientific research.
7. The scientific community is committed to communicating research
results for scientific and public use.
8. Scientists are expected to be open-minded and be ready to accept new
knowledge that emerges from the worldwide scientific community.
Skeptical review of all research. Every scientific report is subject to criticism. Nothing is
accepted as self evident. Criticism focuses on how important the topic is, how the study was
designed, descriptions of protections that were taken to ensure collection of valid data, how the
data were analyzed, and the validity of the conclusions or generalizations that are offered.
Scientific criticism serves to weed out results that do not meet the stringent criteria of scientific
research and to recognize results that have scientific merit. Scientists expect and accept this
criticism as part of the way the scientific community works. They also recognize that criticism,
no matter how much they may dislike it, is not personal. When your research advisor criticizes
your research proposal, design, analysis, or conclusions, remember this is simply the way the
scientific community seeks truth.
Maintaining a disinterested attitude. Scientists are trained to accept what their data tell them.
Ideally, scientists are open to unexpected findings or even look for data that are contrary to

Chapter 1 Page 6
tentative conclusions. The great British biologist, Charles Darwin, for example, kept notebooks in
which he recorded every report or observation contrary to the theory of evolution that he was
developing. He used contrary reports to test the strength of his own observations and
interpretations of his data.
Being objective. Closely related to remaining disinterested in the results of a study is the norm of
being objective. Being objective or showing objectivity means we examine a problem or set of
data with no preconceived ideas or personal bias. Ideally, a scientist's objectivity will allow the
person to see something like it really is, as it exists in its natural form, without any distortion or
error. As we said earlier, complete objectivity is not possible, but social scientists try to be as
objective as possible. The opposite of objectivity is subjectivity. Being subjective about
something means that we show an obvious bias or unfounded personal preference for one idea or
conclusion over others. Using methods of scientific inquiry helps protect against subjectivity and
increases our capacity to be more objective in research.
PREV NEXT

Chapter 1 Page 7
Being completely honest. Scientists are not unique in stressing honesty in their profession. This
Introduction norm, however, is particularly strong in scientific research. Researchers are expected to be honest
Alternatives in what they report. A research report provides only a brief description of the many actions that
occur in the course of an investigation. Data are almost always presented in summarized form.
to scientific
Other scientists reading the report assume that what they read presents an honest, accurate
inquiry description of the methods used and how the data were collected and analyzed. Any cheating or
misrepresentation, when discovered, is viewed as a major violation of the scientific ethic.
Scientific
inquiry Commitment to communicate. Scientists are expected to share their results with other scientists
in their area of interest and for broad public use. Reporting on research is most often done by
Limits of publication of short articles in scientific and professional journals. Articles, however, are not
accepted in most journals until reviewed and accepted by other qualified scientists. This process
scientific
of peer review applies the norm of skepticism and helps assure publication of only quality
inquiry research. Today, research results are often made available over the Internet, a worldwide means
of electronic communication. Results published in this way or in books are also subject to peer
The research review by other scientists.
process
Keeping an open mind. Finally, scientists remain open-minded. As we mentioned earlier, all
Aids knowledge is considered tentative, subject to change as new findings emerge from the
international scientific community. Periodically, new results emerge that are contrary to what
scientists in some field had accepted as established knowledge. One example occurred a few
decades ago when researchers began to discover that young children had greater intellectual
abilities than was commonly accepted. When these results were first announced, they created a
controversy among researchers who study early childhood development. Additional research was
conducted; the results of various studies were compared; in time, the results clearly indicated that
young children are capable of more advanced intellectual processes than had been earlier
believed. These debates can go on for years until one side of the argument builds up a convincing
set of data. The outcome may be acceptance of a new view, as was the case of the early learning
ability of children. But when additional studies fail to produce similar results, the new findings
remain questionable or may be rejected. By this process, science produces an open, growing,
changing body of knowledge.
The scientific method of inquiry has proved to be the most powerful, reliable way of establishing
knowledge about the natural world. There are, however, limits to the kind of questions that can be
answered by scientific inquiry.
Limits of scientific inquiry
Scientific inquiry and the research process can be applied only to things we can observe.
Scientific inquiry cannot be used to answer questions for which we cannot obtain observations.
Questions regarding ultimate existence, whether there is life after death, or whether a certain
behavior is morally right or wrong, cannot be answered by scientific inquiry. The reason is that
questions like these cannot be stated in a way that observations can be made to answer them.
Persons can argue about them and offer reasons for one answer versus others, but there is no way
to offer empirical evidence that one answer is more truthful or correct than another.

Chapter 1 Page 7
Also, scientific inquiry cannot be used to answer questions based on value judgments. These are
statements that simply assert that one thing is "better" or "preferable" to something else. As the
name implies, these judgments are based on one's values - what we think is "right" or "wrong" or
"good" or "bad." There is no way, for example, to test the statement: "Men are more important
than women." This is simply a judgment made on the belief that men are more important than
women. Persons believing in such a position can argue it; persons with contrary views can argue
back, but there is no objective way of settling the issue. Whether one thing or group is more
important than another depends on how "important" is defined and this usually depends on who is
offering the definition of importance.
When this kind of issue is argued, however, one side or the other will frequently give reasons for
their position. Many times, these reasons offer a basis for scientific inquiry. If, for example, a
person says men are more important because they are more intelligent, we now have a statement
that can be tested by observation. We can subject this statement to scientific inquiry. We could
have a large number of men and women take a standardized intelligence test and compare the
results to see if, in fact, men score higher than women. Adding the "because" pointed to
something that could be measured and compared. This little change makes the statement a basis
for scientific inquiry. When the statement or question can be answered by obtaining data of some
sort, it can become the basis of scientific inquiry; if not, the statement or question remains in the
realm of philosophy, religion, or plain argument.
PREV NEXT

Chapter 1 Page 8
Introduction The research process

Alternatives Stages in the research process
to scientific
inquiry Scientific inquiry combines the set of norms just described with a method for observing events
and processes in the world about us. Figure 1.1 shows the research process in terms of eight
inter-related stages. In Chapter 2, we show how a group of researchers went through these steps
Scientific
in conducting a large research project in the Sudan. For now, we want to point out three
inquiry
important characteristics of the process.
Limits of A sequence of actions
scientific
First, research generally follows the sequence of stages shown in Figure 1.1. A researcher begins
inquiry by stating the problem or research question he or she intends to investigate and then proceeds
through each of the remaining stages. The review of the literature, generally the second stage,
The research consists of identifying and analyzing research reports bearing on the research question. After
process becoming familiar with other research on the question being investigated, the researcher may
formulate a hypothesis, but this step is not required. All research, however, requires a plan or
Aids design for the collection and analysis of data. The design is a very important part of any study. At
this stage, the researcher selects and defines the specific characteristics for which data will be
collected and decides how to measure these characteristics. Identifying who the data will be
collected from, called constructing a sample, is part of the design phase. The next step consists of
data collection, using the measuring instruments developed in the design phase. A nalysis
follows, which produces the results or findings of the study. Based on the results obtained, a
researcher then provides an interpretation of what the results mean. The interpretation of the
findings leads to presentation of the conclusions of the investigation. The final stage consists of
writing a report that describes the decisions and actions taken throughout the entire process.
An iterative process
Second, notice that the arrows connecting the stages are bi-directional: They point in both
directions. Although an investigation normally proceeds from the definition of the research
question to the review of the literature and on to the next stage, unexpected problems almost
always come up. Some of these may be serious enough to require going back to an earlier stage
for rethinking some earlier decisions. For example, a researcher may select a certain problem and
move on to the review of the literature only to discover that the planned research has already been
done. Then, the researcher has to start over.
At the design stage, researchers sometimes find that additional measures are needed for the
variables they wish to investigate. This too can lead the researcher back to the literature for new
ideas. This kind of back and forth movement occurs at all stages of research.
As a beginning researcher, you may have a false start or have to abandon your initial idea after
you get into your project. If this happens, don't despair. Consider it as a learning experience.
Also, be assured that this back and forth process is common in research. All researchers, even

Chapter 1 Page 8
those with a lot of experience, go through this iterative process. With experience, however, the
process becomes easier.
PREV NEXT

Chapter1 Page 9
Reliance of emerging knowledge

Introduction
Third, Figure 1.1 also illustrates that at each stage of the research process, researchers draw on
Alternatives the relevant scientific body of knowledge. At each stage of research, you will need to search for
to scientific and apply knowledge related to the tasks you are doing at that stage. At the beginning, you will
inquiry want to review the research literature so you can place your research in the context of previous
research; at the design stage, you will want to look at how previous researchers designed their
Scientific research. When you analyze your data, you will want to see how others did analyses similar to the
inquiry ones you are planning to do. The same point holds when you prepare to interpret your findings
and develop your conclusions. What did other researchers on topics similar to yours find? How
Limits of did they interpret their findings? What conclusions did they draw? In this way, researchers
constantly review and draw upon the accumulating body of social science information.
scientific
inquiry
The research
process
Aids
Figure 1.1. Stages in the research process

Professor Trochim presents an alternative way of viewing the research process. In the Structure
of Research, he describes the process of research as resembling the shape of an hour glass;
researchers start with a broad question, narrow the focus to a specific problem or question,
convert this to an hypothesis which further restricts the inquiry, and then develop measures and
collect data for testing the hypothesis; following analysis of the data, the focus broadens again
with the formulation of conclusions and generalizations from the investigation
Aids
Internet Resources
Previously in this chapter we referred to three Web sites containing alternative and additional
information on scientific inquiry and the application of the scientific method. Web sites
accessible through the Internet are now a valuable source of information on many social research

Chapter1 Page 9
topics. You can copy, download, or printout what you want. For researchers in developing
countries, where libraries have limited collections of books and journals, the Internet can provide
immediate, free access to valuable information that otherwise would not be available. Also,
information on millions of valuable Web sites is free.
If you don't know how to use the Internet, go to Help Internet for help in searching the Internet. If
you know how to move around the Internet, you might want to view some additional sites that
discuss scientific inquiry and the scientific method, using www.google.com of some other search
engine.
PREV NEXT

Chapter 1 Page 10
Key Termsc
Introduction
● Analysis ● Over-generalizing
Alternatives ● Conclusion ● Premature closure
to scientific ● Copy ● Print-out
inquiry ● Data collection ● Results
● Design ● Review of the literature
Scientific ● Download ● Sample
inquiry ● Empirical ● Scientific inquiry
● Empiricism ● Scientific method
Limits of
● Findings ● Search engine
scientific
● Generalization ● Selective observation
inquiry
● Hypothesis ● Subjectivity
● Internet ● Value judgement
The research
● Interpretation ● Web site
process
● Objectivity
Aids
Definitions for key terms are provided in the Glossary.
Main points
1. Relying on traditional or authoritarian beliefs or one's own experience can lead to false
conclusions.
2. Science assumes that there is an orderly, objective reality apart from our perception of
nature; that events occur within natural systems; that we can understand this reality; and
that we can agree on what is real or true about natural processes.
3. Scientists rely on observation for establishing what is true; seek to establish truth on a
universal level; subject all new research findings to intense criticism; try to be objective
and honest in all aspects of their research; remain disinterested in how research results turn
out; keep an open mind toward new knowledge; and are committed to sharing the results of
research with the scientific community and the public.
4. Scientific inquiry can only be used to address questions for which data can be collected.
Questions for which empirical answers cannot be obtained remain in the realm of
philosophy or religion.
5. The research process generally follows a sequence of steps. The typical steps are: selection
of the research question; review of the literature; development of a hypothesis (optional);
preparation of the design for the collection and analysis of data; data collection; analysis
and interpretation of the data; formulation of conclusions; and preparation of a report
describing the entire project.
PREV NEXT BEGINNING TOC

Chapter 1 Page 10


Chapter 2 Page 1
Chapter 2. The Sudan Fertility Survey: An Introduction to Research

Introduction
Introduction
Specifying In Chapter 1 you learned about the scientific approach to conducting research and the typical
the research stages in the research process. In this chapter, we show how the research process was used in
question conducting the Sudan Fertility Survey, a large scale research project designed to provide
information on an important condition affecting the future of the Sudan its birth rate. As with all
Designing research, the Sudan Fertility Survey began with the definition of the problem to be investigated.
the study Specifying the research question
Collecting In most research, the researcher decides what to investigate, as you will have to do in your initial
the data study. Sometimes, however, researchers are asked to investigate a question for some
organization, such as a government ministry. This is how the Sudan Fertility Survey occurred.
Analyzing The Department of Statistics of the government of the Sudan wanted accurate, detailed
information of the current and the estimated future fertility rate in Sudan. In population research,
the data
the fertility rate is defined as the number of live births per 1,000 women of childbearing ages.
Interpreting The resulting investigation became known as the Sudan Fertility Survey (Department of
the results Statistics, 1982). We describe this study for four reasons:
1. To show the value of social research - the survey was requested by the government of the
Generalizing Sudan to provide information for developing family planning programs;
the results 2. To illustrate the application of social research methods to an important social problem -
that of high population growth;
Aids 3. To show how research is planned and carried out in practice; and
4. To give you an idea of how the results of research can be used to understand social
conditions in a country.
We begin by examining how the study was carried out because the value of the results depends
on how information is collected and analyzed, and this depends on how well the study was
planned in the first place.
Designing the study
All research projects require a design or plan for the collection and analysis of the data. In
preparing a design for the Sudan Fertility Survey, a number of important decisions had to be
made, one of which was who to study.
References begin with the name(s) of the authors(s). In the case of the Sudan Fertility Survey, the
author is a government organization. The full reference to this report and others we cite later are
provided in the List of References
PREV NEXT
1.References begin with the name(s) of the authors(s). In the case of the Sudan Fertility Survey,

Chapter 2 Page 1
the author is a government organization. The full reference to this report and others we cite later
are provided in the list of References

Chapter 2 Page 2
Who to study?
Introduction
Given the objective the of the study, it was obvious that married women would have to be the
source of the desired information. In research terms, the women became the respondents in the
Specifying
study. Their responses to questions they were asked became the data of the investigation.
the research Incidentally, data are plural. No one would base a study on the answer of a single respondent to
question a single question, which would produce a datum or just one bit of information. In contrast,
research is based on the collection and analysis of a body of data. The Sudan Fertility Survey, for
Designing example, was based on responses by more than 3,000 women to over 200 questions. That's a lot
the study of data.
With the decision made to collect data from married women, the researchers faced a new
Collecting
decision. This was whether to collect data from all eligible women in northern Sudan or to limit
the data data collection to some smaller number of women. All eligible women, those who were ever
married and living in northern Sudan constituted the population being studied. For the Sudan
Analyzing study, the population included over three million women, far too many to try to collect data from:
the data Doing so would take too long and cost too much money. Knowing this, the researchers chose the
alternative used in most social research. They selected only part of the population as the
Interpreting respondents for the study. This smaller set of women, called a sample, was selected so that the
the results women in the sample were like the population in all important ways, such as being about the
same ages, having the same levels of education, and having the same number of children. In
Generalizing Chapter 8 you will learn how samples are selected.
the results How to collect the data?
Aids Next, the researchers had to decide how to collect the data from the sample of women. The
method chosen was to conduct a survey based on personal interviews with each woman in the
sample. With this decided, the investigators turned to developing the questions to be asked.
Stating the questions to be asked is a critical step in a planning a research project because, as in
everyday life, the answer you get to any question you ask often depends on how the question was
asked. Considerable care, therefore, was taken in framing each question. This task was made
easier in the Sudan study because many of the questions used were used in previous studies of
fertility in other countries.
Studies frequently require translation of questions into the language of the respondents. This was
the case with the Sudan survey. Questions, originally in English, were translated into Arabic, the
language of the women who would be interviewed. This translation was checked to make sure
that the meaning of each question was not changed as a result of being translated. Checking was
done by translating each question back from Arabic into English, and then by comparing the two
English forms of each question. When the back translation agrees with the original language, the
translation is considered safe to use. If the two forms differ, the process is checked to find the
cause of the difference. In this case, English and Arabic versions of the questions were compared.
Back translation, however, can be used with any set of languages.
After the researchers were certain that the translated questions asked what was intended, a small

Chapter 2 Page 2
sample of women was interviewed to make sure that the women who would be interviewed in the
main study would understand the questions and be able to answer them accurately. Following
this step, called a pretest, the questions were organized into a questionnaire. As the name
implies, a questionnaire is the final set of questions used to collect data from a sample.
Persons called interviewers were then trained to use the questionnaire to interview each woman
included in the sample. In conducting interviews, each respondent was asked each question on
the questionnaire and her answers were recorded by the interviewer.
So far we have discussed the typical elements of a social survey. A survey is one form of social
research. Generally, surveys are based on data collection from a sample using a questionnaire.
Collecting the data
The collection of data, in this case the process of interviewing the respondents, lasted from
December, 1978, to April, 1979, and resulted in completion of 3,115 questionnaires from eligible
women. Reporting the time period for data collection is expected in research reports because
reports are frequently published years after data are collected.
Therefore, it is important to tell when the data were collected. This is the only way readers can
know how old the data are.
PREV NEXT

Chapter 2 Page 3
Analyzing the data

Introduction
Current fertility
Specifying The purpose of analysis is to organize the data and see what was found. Analysis generally
the research occurs in two phases. First, investigators summarize responses to each question. The central
question question of the Sudan Fertility Survey was how many children each woman had. Each woman
was represented by a number, from zero for those who had not yet given birth to a child, to the
Designing maximum number born to any woman. These numbers represent the raw data for establishing the
the study fertility rates in northern Sudan in 1978/79. The raw data were analyzed to find the average
number of babies born to the women. Two averages, in fact, were calculated. One average was
based on all the women in the sample from whom data were obtained. This average was 4.2
Collecting
children. It summarized the number of children born to all women, regardless of their ages or
the data how long they had been married.
Analyzing Another average was calculated to find out how many children had been born to women who
the data presumably would not have any more babies. For this average, only data for women who were 45
to 49 years of age were used. This average described the completed fertility of married women in
Interpreting northern Sudan. As you might expect, the average for completed fertility (6.2 babies) was higher
than that for all married women. This result would be expected because the first average included
the results
data for younger women, some of whom had only been married for a short time, whereas the
average for completed fertility included only women who had many years to produce children.
Generalizing
the results We cite these two averages to illustrate that a single research project can be used to answer more
than one question. Chapter 18 and Chapter 19 you will give you some ideas of various ways
Aids you can analyze the data you will collect.
Estimates of future fertility
As researchers we often want to suggest how we think certain things may change in the future.
The following examples show how data from the Sudan Fertility Survey were analyzed to get an
idea of possible changes in fertility in northern Sudan.
First, the research team compared the number of babies born to younger women with the number
older women had given birth to when they were the same ages as the younger women. The
analysis showed that younger women were continuing to have about the same number of babies
as their older relatives had at the same ages.
In addition, the researchers examined the number of children the women said they would like to
have if they could have the exact number of children they wanted. For all women, the preferred
number was an average of 6.4 children, which was higher than the actual completed fertility of
the older women (6.2 children). Younger women between the ages of 15 and 24, however,
indicated they wanted an average of 5.4 children, less than the 6.4 as reported by all women.
These findings also point to continued high fertility in northern Sudan.
The researchers also looked at the extent to which family planning was being practiced. The

Chapter 2 Page 3
women were asked a number of questions about their knowledge and use of contraceptive
methods. Here are some of the results:
● Only 12% of the women had used contraceptive methods sometime in their lives:
● Of those who had tried some method, only 9% stated they intended to do so again in the
future;
● And only 16% of the women wanting no more children said they were using a reliable
means of contraception.
PREV NEXT

Chapter 2 Page 4
Seeking an explanation for fertility rates

Introduction
So far, the results suggest that fertility will remain unchanged in northern Sudan. Women wanted
and were still producing large families and few of the couples were using reliable means to limit
Specifying
family size. Still, before stating a conclusion based on these findings, we need to examine fertility
the research in light of other broad social trends in Sudan. Chief among these is the recent increase in years of
question schooling among girls.
Designing How might increased schooling be linked to fertility? To answer this question, the researchers
the study analyzed the relationship between schooling and fertility. Here are some of the things they
discovered:
Collecting ● Women with no schooling had an average of 4.2 children;
the data ● Women with 1 to 5 years of schooling had an average of 4.4 children;
● Women with 6 or more years of schooling had only 3.0 children on the average.
Analyzing
These findings indicate that completion of primary school was associated with lower fertility.
the data
The education of women was also related to use of contraceptives. As their schooling increased,
Interpreting so did the use of contraceptives:
the results
● Only 2.5% of the women with no schooling reported use of contraceptives;
● While 15.0% of those with 1 to 5 years of schooling did so:

Generalizing
the results ● And an even larger percentage, 42.0%, of women with at least 6 years of schooling
indicated use of contraceptive methods.
Aids Among women who wanted no more children, schooling was even more strongly associated with
contraceptive use:
● Only 8.5% of the women with no schooling and who wanted no more children reported use
of contraceptives.
● This was true for 29.5% of those with 1 to 5 years of schooling.
● A much larger percentage, 63.0%, of the women with 6 or more years of school and who
wanted no more children reported use of contraceptives.
Interpreting the results
From these findings, we could draw the conclusion that fertility in northern Sudan will not
change much in the immediate future. In the long run, however, as schooling for girls continues
to increase, fertility rates will probably decline. These conclusions would represent our
interpretation of the findings. In a sentence or two, we say what we think the findings mean. To
summarize: results are based on data; results are facts. Statements that give meaning to the facts
or results represent the researcher's interpretation of the results.
Generalizing the results
When a proper sample is used, researchers can extend a conclusion by saying what they think is

Chapter 2 Page 4
true for the population based on what was learned from a sample. Thus, the results from the
sample of 3,115 married women who supplied data for the Sudan Fertility Survey could be
extended to describe fertility and conditions affecting fertility among the 3 million married
women living in northern Sudan at the time the data were collected. When conclusions are
extended in this way they are referred to as empirical generalizations. Empirical is used because
the generalizations are based on data. The process of creating a generalization is called
generalizing.
PREV NEXT

Chapter 2 Page 5
Some empirical generalizations that can be drawn from the results of the Sudan Fertility Survey
Introduction are:
● Fertility in northern Sudan is high, averaging slightly over 6 children per married woman.
Specifying ● Fertility in northern Sudan will probably remain high in the coming years.
the research
● However, in the long run, fertility in northern Sudan will probably decline as females
question obtain more schooling.
Designing Notice that these generalizations sound like conclusions. Often generalizations do, but
the study remember, generalizations are offered as the broadest or most general statements one can make,
based on the findings of a study. Researchers are careful in drawing either conclusions or
Collecting generalizations. Sometimes, because of limited data, we have to limit conclusions and
corresponding generalizations. The important thing is to be honest in what you say: Be careful
the data
not to over generalize or go beyond what your data indicate. For example, the three
generalizations we stated earlier were limited to "northern Sudan." We did not try to generalize to
Analyzing all of Sudan because data were not available for other parts of the country.
the data
Aids
Interpreting Internet resources
the results
In this chapter, we have presented an analysis of only one social research report. Thousands of
Generalizing additional research reports on all kinds of topics are available on Web sites or through other
the results information services. Since this chapter dealt with a report on fertility, we did an Internet search
using Google, a popular search service. Google reported about 203,000 Web sites dealing with
Aids "fertility rates." We also looked for reports of studies of fertility rates in POPLINE, an
information service that covers population-related topics and issues. On February 9, 2005,
POPLINE, listed 3,899 items concerned with fertility rates. Some of these were Web sites with
the complete text of reports. For example, one report, Transitions in World Population,
provides a comprehensive description of population changes, examines bases for future changes,
and discusses other issues related to the changing characteristics of the world's population. Others
provided summaries of journal articles, books, and other reports related to fertility. Chapter 4
provides explains how to construct and carry out a search of POPLINE.
Either Google or POPLINE and many other information services (see Chapter 4) provide
access to thousands of social research reports on all kinds of topics.
Key terms

Chapter 2 Page 5
● Analysis ● Population
● Back translation ● Pretest
● Data ● Raw data
● Design ● Respondents
● Empirical generalization ● Questionnaire
● Generalizing ● Sample
● Interpretation ● Survey
● Over-generalizing
Main points
1. Information collected in an investigation is referred to as data. Data are plural; the singular
of data is datum.
2. Data are analyzed to produce the results of a study.
3. Social scientists use data to establish relationships between variables. Clearly established
relationships between variables provide the basis for explaining why behavior occurs as it
does.
4. Findings or results are interpreted to produce the conclusions of an investigation; to
interpret findings is to say what we think they mean.
5. Conclusions are statements based on findings.
6. An empirical generalization extends findings from a sample to a population.


Chapter 3 Page 1
Chapter 3. Learning the Language of Research
Introduction Introduction
In Chapters 1 and 2, you learned how social research is conducted. These chapters also
Understanding introduced you to key research terms. This chapter will acquaint you with additional research
concepts & terms used by researchers in their work. Understanding the terms used by researchers will help
variables you plan and carry out your own research.
Conducting research also requires the use of logic or reasoning. Two forms of reasoning,
Theory as a
inductive and deductive logic, are used in research. We explain and illustrate each form. Also, we
way of examine the logic used in cause and effect analyses and explain the conditions that have to exist
organizing to establish a cause and effect relationship.
knowledge
Two of the most frequently used terms in research are concepts and variables. We start with an
Hypothesis & explanation of concepts because this term is basic to understanding how scientists communicate
and conduct research.
research
Understanding concepts and variables
The logic of
Concepts
scientific
inquiry With the beginning of speech, you started absorbing and using concepts without knowing you
were doing so. You learned to describe things as big or small, temperatures as hot or cold, and
The logic of persons as members of your family, as friends, or as strangers. In social science courses, you have
scientific encountered many additional concepts, such as role model, traditionalism, modernity,
inquiry globalization, personality, bureaucracy, and many more.
None of these concepts refers to anything concrete things that we can see, touch, or otherwise
Cause experience directly. They exist only as mental images that we share and use in talking about
and effect things. Social scientists also refer to concepts as constructs because they are constructed or
created for the purpose of analyzing social behavior and for communicating what we learn.
Aids
Through use of constructs, we are able to link our mental images of things we observe and agree
on what we are referring to. For example, we cannot directly observe bureaucracy, but we can
share a mental image or construct that describes what we mean by this concept and then use the
concept to describe how organizations differ in this regard.
Social scientists use concepts in identifying research problems and in expressing conclusions and
generalizations. For example, most social scientists agree that urbanism tends to reduce the
cohesion of the extended family. The previous sentence contained three concepts. Like all
concepts, each term urbanism, cohesion, and extended family is an abstraction. Each has
meaning to social scientists and others, but without referring to any specific thing that can be
observed. Their meaning is abstracted or derived from the many specific or concrete examples of
urban development, varying bonds of cohesion or togetherness among families, and many
instances of large family groups. Concepts allow us to communicate easily at a general level
without having to specify each and every instance of what we mean. In this way, concepts permit

Chapter 3 Page 1
us to communicate effectively in scientific as well as everyday conversation.

Concepts also imply variation in what they refer to. Countries and regions of the world, for
example, vary in degrees of urbanism. The countries of Western Europe and North America are
highly urbanized while countries in Africa and other parts of the world have mainly rural
populations. Also, in all countries the bonds of cohesion vary among family groups, from very
strong to weak or virtually absent. Researchers look for variations in concepts selected for study.
PREV NEXT

Chapter 3 Page 2
Concepts are used at all stages of research. Research topics or questions are expressed in terms of
concepts; so are hypotheses and conclusions.Think back to the Sudan Fertility Survey. The
Introduction conclusions were expressed in concepts: one was that fertility, a concept, declined as the
education of women, another concept, increased. Generalizations are also expressed in concepts.
Understanding The conclusion we just cited from the Sudan Fertility Survey is also accepted as a broad
concepts & generalization because so many studies found the same result.
variables Concepts, however, cannot be observed directly. As we said, they are abstractions that exist only
as mental images of things we want to talk about. To do research, we have to convert concepts to
Theory as a things we can observe. We do this by defining concepts in terms of measurable variables.
way of
Variables
organizing
knowledge Variables represent concepts. Like concepts, variables are defined in words, but, as used in social
research, variables have a special characteristic. Variables have two or more observable forms or
Hypothesis & values. In short, they vary. Any condition or aspect of social behavior or social life that has at
research least two conditions or amounts of something is a variable.
In the previous chapter, we described several variables central to the Sudan Fertility Survey.
The logic of These included fertility and education. Each could and did vary from zero to some number.
scientific Variables to which we can assign numbers are known as quantitative variables. A few examples
inquiry of quantitative variables include your age, your height, your intelligence, your family's income, or
the number of students at your university. Other variables exist in terms of qualities or categories
The logic of rather than in terms of numbers. Gender is a variable with only two (the minimum) categories -
scientific male and female. Other examples of categorical or qualitative variables are social class, which is
inquiry often described as "high," "middle," or ""low;" occupation, which can be expressed in various
categories as "farmer," "merchant," "official" and so on; or marital status, commonly described as
"single," "married," "divorced," "separated," or "widowed." Once you think about it, you will see
Cause
all kinds of quantitative and qualitative variables.
and effect
Attributes of variables
Aids
The values or categories making up a variable are its attributes. Attributes are what social
scientists measure to describe a variable. It is easy to mix up the name of a variable with its
attributes. Box 3.1 illustrates the attributes for a few variables.
To summarize, attributes are the observable characteristics of variable; variables are the logical
combinations of attributes. Male and female, for example, are the attributes of the variable we
call gender. The numbers of persons in a household, from 1 to perhaps 20 or more, represent the
attributes of the variable, household.
Box 3.1 Some variables and their attributes
Variable Attributes
Gender Male, female

Chapter 3 Page 2
Nationality Sudanese, Zambian, Egyptian, etc.

Intelligence 120, 97, 111, 105
Your major Management, psychology, chemistry
Family size 4, 12, 6, 10, 2
Social status Low, middle, high
For another way of describing variables and their attributes, take a look at Variables.
PREV NEXT

Chapter 3 Page 3
Variables and relationships
Introduction In social research, we seek to learn about variables and relationships among them. Once empirical
relationships are established among variables, we can logically construct relationships among the
abstract concepts represented by the variables. Going back again to the Sudan Fertility Survey,
Understanding
the research team described the variable fertility in terms of averages for the sample used. These
concepts & averages were found by analyzing the data for the attribute of fertility, namely the number of
variables children each woman gave birth to. Level of education, another variable, was measured in terms
of years of schooling completed. The relationship between fertility and education was based on
Theory as a attributes for fertility, in terms of the number of children born, with the attributes for education,
way of namely, years of schooling completed. Establishing empirical relationships between variables
organizing allows us, as social scientists, to speak of relationships among the concepts represented by the
knowledge variables and to extend our understanding of social relationships and behavior.
Theory as a way of organizing knowledge
Hypothesis &
research So far, we have emphasized that scientific knowledge rests on empirical research. Scientists,
including social scientists, however, are not satisfied with only presenting empirical results: They
The logic of also seek to offer an explanation for why any set of results is consistently found. Scientists refer
to a broadly accepted explanation for something as a theory.
scientific
inquiry Many students have difficulty grasping the idea of theory. To most persons, theory refers to a
vague idea about why something has happened. You may have heard someone say: "I have a
The logic of theory about that," and go on to offer some poorly supported argument in favor of their "theory."
scientific For scientists, theory has a special meaning. In scientific terms, a theory is a conceptual
inquiry framework that explains existing facts and predicts new ones. Each of us accepts a large number
of events as facts. For example, the sun rises, follows its daily course across the sky, sets, and
daily repeats this pattern. A stone thrown in the air falls back to earth. Both of these common
Cause
events and many other natural events are events are explained by Newton's theory of gravity.
and effect Further, this theory allows scientists to predict new events, like to appearance and paths of
comets and other planetary bodies.
Aids
Social scientists also seek to develop theories about forms of social behavior. Although far more
limited than those in the natural sciences, our theories can explain a large number of related facts
and allow us to offer predictions for the direction of social change.
A widely used theory used for explaining the direction of change in many developing countries is
the theory of modernism. This theory is based on an extensive set of research findings on changes
occurring in developing countries throughout the world. According to the theory of modernism,
as employment shifts from agriculture to industrial and other urban jobs, other social changes
inevitably occur. These include movement of population from villages and rural areas to urban
centers; increased importance of formal education, including equal education for girls; less
emphasis on relations with the extended families; a shift to smaller, nuclear families; increases in
individual freedom for family members, including wives, who become freer to participate in
community and political affairs; and greater emphasis on the individuality of children and

Chapter 3 Page 3
cultivation of their intellectual achievements.

A large amount of research supports these generalizations. Further, each of these generalizations
appears to be related to the others urban location is associated with greater educational
achievement for females that is related to lower fertility and other social changes. Social
scientists use the theory of modernism as a way of summarizing and integrating these
generalizations. Like any other theory, the theory of modernism can be used to derive research
questions and hypotheses for guiding research. Later in this chapter we show how one
investigator used the theory of modernism as well as several other theories to develop hypotheses
for her research.
Some new researchers have difficulty dealing with the idea of theory. Part of the reason is that
theory is based entirely on the relationships among concepts, which exist only as mental images.
Also, developing a theory requires considerable knowledge about a specific topic or area of
research. There are, for example, alternative theories of social and economic development,
personality, social structure, feminism, and other forms of social interaction.
PREV NEXT

Chapter 3 Page 4
Sorting out competing theories can be difficult for a beginning researcher, especially if you have
had only a few social science courses. Nevertheless, you will have to use theory at several points
Introduction in any research you do. To start, you will have to place your research in some context, to show its
significance or value. Later, when you offer generalizations based on your findings, you will
Understanding want to place your generalizations in the context of what is known. At this point, you will be
concepts & drawing on the integrated knowledge related to your research question. This will probably be best
variables expressed as a theoretical set of relationships among a set of concepts related to your
investigation. To prepare to use theory to strengthen your own research, we suggest you carefully
read the "Theory" or "Theoretical Framework" sections of reports you find in your review of the
Theory as a
literature. Try to understand what theory the authors drew on and note how they presented the
way of theories they used.
organizing
knowledge Hypotheses and research
What is a hypothesis?
Hypothesis &
research A hypothesis is a tentative expression of an expected relationship between two variables. When a
hypothesis is used, the purpose of the study is to find out whether or not the hypothesis is
The logic of supported by the data. Several steps are involved in stating a hypothesis:
scientific 1. Deriving the hypothesis from a theoretical framework;
inquiry 2. Designating the independent and dependent variables in the hypothesis; and
3. Stating the hypothesis.
The logic of
scientific
inquiry Deriving a hypothesis
Cause In order to state a hypothesis we have to have a sound set of reasons for saying why we think the
hypothesis will be supported by some body of data. This set of reasons is based on knowledge
and effect
about relationships among the variables that make up the hypothesis. Researchers refer to this
rationale the theoretical framework or the theory underlying the hypothesis.
Aids
An illustration will help you understand how a hypothesis is derived. Fattah (1981) investigated
the role of rural Iraqi women in family decision-making. She used individual modernism as the
main theoretical framework for her investigation. The theory of individual modernism is derived
from the more general theory of modernism, which is widely accepted as a way of explaining and
predicting social changes at the societal level. Fattah developed a theory of individual modernism
by extending generalizations associated with the broader theory of modernism to the individual
level. In the course of her review of literature she found a number of variables that could be used
to measure the extent to which rural Iraqi women held modern views. Three of these variables
were preference for living in a modern, nuclear family setting versus living in an extended family
setting; a high level of achievement orientation; and a strong positive attitude for equity for
women. Women with these characteristics were considered as having a high level of individual
modernism.

Chapter 3 Page 4
Other research showed that women with these characteristics would also seek to participate in
decisions that affected them. On this basis, Fattah reasoned that women with high levels of
individual modernism would also participate more in making family decisions. Specifically, she
decided to investigate the relationship between her measures of individual modernism and the
participation of wives in decisions related to farm operations, family financial matters, social
relationships, entertainment, child training, and childcare. Fattah then developed a number of
hypotheses expressing expected relationships between each measure of individual modernism
and participation in each of the six decision-making areas. To simplify this process, we cite just
one of the many hypotheses she developed.
Taking only the concept for family orientation and participation only in family financial matters,
she stated the following hypothesis:
Preference for living in a modern nuclear family is positively associated with

participation in making family financial decisions.
PREV NEXT

Chapter 3 Page 5
This hypothesis received considerable support. So did similar hypotheses for participation in the
other five areas of decision making. Fattah's investigation illustrates the derivation of hypotheses,
Introduction using one theoretical base; namely, individual modernism. Other theoretical bases could have
been used to derive the same hypothesis. In fact, Fattah did exactly that. She also derived the
Understanding same set of hypotheses from two other theoretical frameworks. One framework was based on
concepts & "task allocation" theory, which says that persons who perform a task, in effect, have earned the
variables right to make or at least participate in decisions related to that task. Since rural Iraqi women
participate in heavy farm labor, it could be hypothesized that they would also participate in
various family decisions, particularly those related to farm operations. And this is what Fattah
Theory as a
found.
way of
organizing The third theoretical framework she used is known as "resource theory." This theory says that
knowledge persons who bring valued resources to a situation are given a greater role in decision-making. In
traditional rural Iraq, resources of women include giving birth to sons, having an extensive
Hypothesis & extended family network, being literate, achieving greater education, and having been married for
a number of years. Hypotheses based on task allocation had strong support, while those derived
research
from resource theory received only minimal support.
The logic of As you can see from Fattah's research, deriving a hypothesis requires imagination and creativity
scientific as well as knowledge about the variables being used. Different researchers use different
inquiry approaches to the same problem and may use different theoretical bases in deriving hypotheses.
Hypotheses can also be derived directly from observation. If we were to repeat the Sudan Fertility
The logic of Survey with a sample from a population in another country, for example, it would be reasonable
scientific to hypothesize that fertility will decline with increases in education of women. Empirically
inquiry derived hypotheses, however, simply extend a previous conclusion to a new population. They do
not extend knowledge as theoretically derived hypotheses can.
Cause
Independent and dependent variables
and effect
Hypotheses usually express a relationship between two variables, as illustrated by those used by
Aids Fattah. It is customary to designate one variable as the independent and the other as the
dependent variable. The independent variable is assumed to be the one responsible for changes in
the other, the dependent variable. In Fattah's research, for example, the variables identified as
representing components of individual modernism were designated as the independent variables
and participation in family financial decisions became the dependent variable. It makes more
sense to argue that the preferences or orientations of the wives could influence the degree of
participation in family decisions than the other way around.
Hypotheses generally express an expected relationship between an independent and dependent
variable. To start, you can think of a hypothesis as a single sentence with three parts:
1. The subject, which generally is the independent variable;
2. A connecting verb, which defines the relationship between the independent and dependent
variables; and
3. The object, which is the dependent variable.

Chapter 3 Page 5
The hypothesis we quoted from Fattah's investigation was in this form:

● "Preference for living in a modern nuclear family" was the subject (independent variable);
● "Is positively associated with" defines the relationship between "preference for living in a
modern nuclear family" and "participation in making family financial decisions (the
dependent variable); and
● "Participation in making family financial decisions was the object (dependent variable).
Try to express any hypothesis you write in the same way; as single sentence with a subject, a
connecting verb, and an object.
Also, since most hypotheses deal with the actions of groups of persons, plural forms are generally
used. In the hypotheses we use as examples, you will note we used plural forms - females,
children, families, incomes, etc. Also, because you want to generalize to conditions as they
currently exist, hypotheses are generally expressed in the present tense. The independent and
dependent variables, for example, "are associated." However, if your hypothesis deals with some
past situation, you would obviously express the hypothesis in past tense.
PREV NEXT

Chapter 3 Page 6
There are also some expressions you should avoid in stating a hypothesis. Researchers do not use
"prove" in a hypothesis. As you will understand more fully after you read Chapter 19, "Inferential
Introduction Statistical Analysis," research never proves anything. We can only offer degrees of support in our
conclusions. There is always the possibility that our conclusion could be wrong. Also, avoid
Understanding stating exact degrees of association, such as 65% of the respondents will or will not respond in a
concepts & certain way. When the exact amount specified in a hypothesis is not found, the hypothesis has to
variables be rejected. Yet, the data might support a more general level of association between two
variables. Using a more general expression of an expected relationship, gives you more freedom
Theory as a in drawing a conclusion from the test of a hypothesis.
way of Designation of variables as either the independent or dependent has no meaning by itself. The
organizing same variable may be treated as an independent variable in one investigation and as a dependent
knowledge variable in another. In Fattah's investigation, years of schooling were used as one of the
"resources" of the wives. In this case, she treated schooling as the independent variable. In a
Hypothesis & study whose purpose was to find out why some women went to school longer than other women,
research years of schooling would be taken as the dependent variable, with perhaps the education of the
parents as the independent variables.
The logic of Forms of hypotheses
scientific
Hypotheses can be stated in a number of ways. To help you state a hypothesis, we present some
inquiry
ways hypotheses are frequently expressed. These are:
The logic of As a negative relationship between two variables. Negative means the values for the variables
scientific move in opposite directions - as one variable increases the other decreases or vice a versa. A well
inquiry established negative relationship exists between the schooling of women and their fertility: that
is, the higher the schooling, the lower the fertility. The words inverse or indirect are also used to
Cause describe a negative relationship. A negative (or inverse or indirect) relationship can be expressed
as:
and effect
Schooling of females is negatively (or inversely or indirectly) related to their
Aids fertility.
This form of a hypothesis is based on the form outlined earlier - independent variable, connecting
verb, dependent variable. Once you get used to writing hypotheses, you might be more
comfortable stating hypotheses in a slightly different form, such as:
There is a negative (or inverse or indirect) relationship between schooling of females
and their fertility.
Schooling among females and their fertility are negatively (or inversely or
indirectly) related.
Also, the word associated can be used in place of related and "association" for relationship.
Thus, the hypotheses just given could be rewritten using association and associated.
Schooling of females is negatively (or inversely or indirectly) associated with their

Chapter 3 Page 6
fertility.
There is a negative (or inverse or indirect) association between schooling of females
and their fertility.
Schooling among females and their fertility are negatively (or inversely or
indirectly) associated.
In the form of a positive relationship or association between variables. Positive means the
variables moves in the same direction - as one increases so does the other. The word direct is also
used to describe a positive relationship. A positive or direct hypothesis can be expressed in
several ways:
Schooling of wives is positively (or directly) associated with participation in family
financial decision-making.
There is a positive (or direct) relationship (or association) between the schooling of
wives and their participation in family financial decision-making.
Schooling of wives and their participation in family financial decision-making are
positively (or directly) related (or associated).
PREV NEXT

Chapter 3 Page 7
In the form of a curvilinear relationship. A curvilinear relationship is easy to spot: First, the
relationship goes in one direction, then in the other. A well-established curvilinear relationship is
Introduction the association between the ages of mothers and the rates of deaths among infants (called infant
mortality). The highest infant mortality rates occur among mothers 16 years of age or younger.
Understanding Rates decline with increasing age until mothers are about 40 or older; after which the rates
concepts & increase sharply. Looking at the entire range of ages, we conclude that the relationship is
variables curvilinear. Nonlinear, meaning that the relationship cannot be expressed as a straight line, is also
used to describe this kind of a relationship.
Theory as a In the form of no relationship. Sometimes we wish to hypothesize that two variables are not
way of related to one another. For example, one might argue that the happiness of children is not
organizing associated with the income of their families. Another way of saying this is that the two variables
knowledge are independent of one another. Thus, a hypothesis stating no relationship between variables can
be expressed in several ways as well:
Hypothesis & The income of families is not related to (or associated with) the happiness of
research children.
The logic of There is no relationship (or association) between the incomes of their families and
the happiness of children.
scientific
inquiry The incomes of families and the happiness of children and are not related (or
associated, which ever you prefer).
The logic of
The income levels of families and the happiness of children are independent.
scientific
inquiry Some group (A) is different from another group (B). This form of hypothesis occurs when the
purpose of research is to find out if two groups are different with respect to some variable.
Cause Generally, however, we can go beyond just saying we expect to find some difference. Usually, a
and effect hypothesis specifies the direction of the difference by saying that the variable for one group is
larger or smaller than for another group. To illustrate, we could hypothesize that:
Aids School attendance rates are greater in urban than rural areas.
In the form of the direction of change for some variable. To illustrate:
The age at marriage among females has increased in the past 20 years.
Are hypotheses necessary?
Research does not require the use of a hypothesis. A large number of studies are conducted
without one. Most fact gathering surveys do not involve hypothesis testing. If you wanted to
describe the changing ratio of females to males at your university over the past ten years, a
hypothesis would not be needed. You could collect and analyze enrollment data over the years,
present your findings, and draw a conclusion about the rate and direction of change.
When a sound hypothesis can be derived, there are advantages in using one. A hypothesis gives a
study a clear, specific focus. Data are obtained, analyzed, and interpreted in terms of the

Chapter 3 Page 7
hypothesis. Also, the conclusion of the study can be easily and clearly presented by simply
stating whether or not the hypothesis was supported by the data.
PREV NEXT

Chapter 3 Page 8
The Logic of scientific inquiry
Introduction All scientific inquiry involves logic or reasoning. In research, logic is used in analyzing data: we
compare results in terms of relative sizes (how large is one average in relation to another) or we
make a judgment about how strongly one variable is related to another (like schooling and
Understanding
fertility). Logic is also used in drawing conclusions from our findings.
concepts &
variables Specifically, two systems of logic, called inductive and deductive logic, are used in research.
Inductive logic is used in reasoning from many particulars, such as many bits of data, to a more
Theory as a general level. Drawing a conclusion from various bits of data is an example of inductive logic.
way of Deductive logic works in the opposite direction. It involves formulating a statement from some
broader basis. Deriving a hypothesis, as a specific statement, from a theoretical framework,
organizing
illustrates deductive logic. Several examples should help you see how each form of logic is used.
knowledge
Inductive logic
Hypothesis &
First, let's look at how inductive logic is used. To illustrate this process we will use two now
research familiar sets of data from the Sudan Fertility Survey years of schooling among women and their
fertility. With these data we would have two observations, one for schooling and one for fertility,
The logic of for each woman in the sample.
scientific
inquiry These observations are the particulars, the bits of raw data, which we will use to illustrate the
inductive process. Our intent is to find out what relationship exists between these two variables.
One way to do this is to display the observations in graphic form as shown in Figure 3.1(A).
The logic of
scientific Each women is represented Figure 3.1(A) by a dot or point corresponding to her years of
inquiry schooling, shown on the horizontal line of the figure (the X axis), and her fertility, shown on the
vertical (Y) axis. Assume that the first woman in the sample had 6 years of schooling and had
Cause given birth to 5 babies. In the figure, she is represented by the dot above the 6 on the X-axis and
and effect across from the 5 on the Y-axis. Another woman, with 11 years of schooling and with 3 children,
is represented by the dot above the 11 on the X-axis and opposite the 3 on the Y-axis. Continuing
this process, observations for fertility and schooling for all women would be plotted. With over
Aids
3,000 women we would have a dense mass of points - one for each woman. For the purposes of
this illustration we have simplified the graph by showing only a small number of points
After showing the data in this way, we would look for the pattern running through the mass of
points. Looking at the points, it seems there is a pattern. To show the pattern, we draw a line
through the middle of the mass of points. When we did this, as indicated in Figure 3.1(B), we
find a line sloping downward from left to right. This line indicates a negative relationship
between the variables: that is, as schooling increases, fertility tends to decrease. This line
summarizes the relationship between schooling of women and fertility.
Figure 3.1(C) shows the outcome of this inductive process. The particulars, the original bits of
data represented by the many dots, are now replaced by a single line that represents the finding of
the study. We have gone from analysis of particulars to the expression of a conclusion - namely,
that there is a negative relationship between the schooling of women and their fertility. In actual

Chapter 3 Page 8
research, a statistical measure called the coefficient of correlation would be used to express the
relationship between the two variables (see Chapter 19).
PREV NEXT

Chapter 3 Page 9
A. Plot of observations B. Fitting a line to the mass of points
Introduction
Understanding
concepts &
variables
Theory as a
way of
organizing C. Conclusion
knowledge
Hypothesis &
research
The logic of
scientific
inquiry
The logic of
scientific
inquiry Figure 3.1. Illustrative use of inductive logic
Deductive logic
Cause
and effect Now, what about deductive logic? In deduction we start with some statement we believe to be
true and apply it to a new situation to see if it is true there as well. Fattah's research was based on
hypothesis testing. She derived her hypotheses from the theory of individual modernism. This
Aids
theory suggested that women with a more modern orientation will also seek a stronger role in
family decision making. One of the hypotheses she derived (and which we cited earlier) was:
Preference for living in a modern nuclear family is positively associated with
participation in making family financial decisions.
We can also visualize this hypothesis as a graph with a line sloping up from left to the right, as
shown in Figure 3.2 (A). However, in actual practice, this is not done. We have done this only so
you can see the expected pattern graphically.
PREV NEXT

Chapter 3 Page 9

Chapter 3 Page 10
A. Hypothesized relationship between B. Observations for the relationship

family living preference of wives and their between family living preference of wives
Introduction participation in family financial decisions and their participation in family financial
decisions
Understanding
concepts &
variables
Theory as a
way of
organizing
knowledge
Hypothesis &
research
C. Fitting a line to the mass of points D. Conclusion
The logic of
scientific
inquiry
The logic of
scientific
inquiry
Cause
and effect
Aids Figure 3.2. Illustrative use of deductive logic

The next step is to see if the expected or hypothesized relationship is found. This test is the same
as described earlier for inductive logic. To illustrate hypothesis testing, we have modified Fattah's
analysis a bit. We show the measures of individual modernism as varying from 0 to 14 and from
0 to 10 for the participation of wives in family financial decision-making. With the variables
expressed as numbers, we can plot the set for each wife, as shown in Figure 3.2 (B). Next, we
draw a line that comes the closest to the largest number of dots (Figure 3.2 (C)). This line
describes graphically the relationship that was observed between the two variables.
After drawing the line that best fits the pattern of dots, we can remove the dots, and have only the
line, as shown in Figure 3.2 (D). We do not expect the empirical pattern shown in Figure 3.2 (D)
to match the line in Figure 3.2 (A) exactly. Rather, we would inspect the line to see if it was in
the predicted direction. In our example, we found an empirical relationship, as shown in Figure

Chapter 3 Page 10
3.2 (D). In this illustration, the two match pretty closely: Therefore, we would conclude that the
hypothesized positive relationship was supported
Deductive and inductive logic combined
In practice, both forms of logic are used in n used together, the process is referred to as a
hypothetical-inductive operation. First deductive logic is used to derive a hypothesis from a
theoretical framework. The researcher then switches to use of inductive logic to test the
hypothesis. The results of the inductive process become the conclusion of the study. This
conclusion may then be extended as a generalization or be used to suggest new topics or
hypotheses for further research.
PREV NEXT

Chapter 3 Page 11
Cause and effect
Introduction Interpreting an empirical relationship

Earlier, we referred to the negative relationship between schooling of women and their fertility.
Understanding Discovery of a relationship, however, does not explain why the variables are related. Providing
concepts & an explanation for an association requires additional understanding of how one variable, taken as
variables the dependent variable, interacts with not only the designated independent variable, but also with
other variables related to both the dependent and independent variables. Thus, any time you
Theory as a discover a relationship between two variables you are forced to think about why the relationship
way of exists. As shown in Figure 3.3, there are three main ways to interpret a relationship between two
variables.
organizing
knowledge First, you can say that the two variables, as shown by A and B in example (1), are related and that
is all. The arrow pointing both ways between A and B shows this kind of relationship. Sometimes
Hypothesis & it is impossible to say which variable influences the other. We only know that the two variables
research are related. Without more information, we cannot say which of the variable is the cause of
changes in the other. Even so, such limited findings may be useful to other researchers.
The logic of In conducting research, however, our goal is to offer the strongest explanation we can about any
scientific relationship we find. If possible, we would like to say what we think may be the causes of
inquiry variation in some variable. This leads us to think in terms of cause and effect; how do variations
in one variable show up as observable differences in another variable? Three decisions are
The logic of required to establish one variable as the cause of changes in another variable. These are:
scientific ● Deciding on the direction of influence: which variable influences the other;
inquiry ● Establishing a strong relationship between the presumed cause and its effect; and
● Eliminating other possible causal factors

Cause
and effect
Aids
Figure 3.3. Forms of relationships among variables

Establishing the direction of influence
Efforts to establish a cause and effect or causal relationship are generally based on a theoretically
derived association between two or more variables. To illustrate this process, we return to our
hypothesis for the possible relationship between the preference for a nuclear family orientation of
wives and their participation in family decision making. We want to test whether their nuclear
family orientation could be the cause of differences in participation of wives in financial decision

Chapter 3 Page 11
making. Our first step is to decide on the possible direction of a possible causal relationship.
Which variable affects the other? Which is to be considered as the independent and which as the
dependent variable?
Establishing a possible direction of a causal relationship is based on a simple rule:
The variable thought to be the independent variable must precede the
dependent variable in time.
Let's apply this criterion to the hypothesized relationship between nuclear family orientation and
their participation in family financial decisions. Which variable generally precedes the other?
For virtually all women in developing countries, regardless of their culture, develop views
regarding family orientation occurs before marriage. Thus, it is reasonable to designate family
orientation as the independent variable for any explanation of the relationship between it and later
participation in family matters. On this basis, participation in family financial matters would be
treated as the dependent variable.
This was an easy decision. In other cases, you may have to review the research literature on the
variables you are analyzing to see which should be treated as the independent or the dependent
variable in a relationship.
PREV NEXT

Chapter 3 Page 12
Establishing a strong empirical relationship
Introduction In addition to establishing the time sequence, two other criteria have to be met before we can
claim a possible causal relationship between two variables. The second criterion is:
Understanding There must be a strong empirical relationship between the two variables.
concepts &
We must have data showing that the values of the dependent variable always or almost always
variables
change depending on the strength of the independent variable. In this regard, the world of
physical relationships is our ideal. For example, when a gas in a closed container is heated, the
Theory as a pressure exerted by the gas on the walls of the container also increases. The exact amount of
way of change in pressure can be measured by changes in the temperature of the gas in the container. We
organizing can vary the amount of heat and record each change in pressure. We know that any change in
knowledge temperature is always followed by a specific, observable change in pressure. Therefore, we can
conclude that a strong empirical relationship exists. Such data are strong evidence for causality.
Hypothesis &
But we seldom have such clear or perfect relationships among variables in social research.
research Empirical relationships among social variables are generally weaker, with numerous exceptions
to the general pattern. Consequently, researchers are faced with the question of how strong a
The logic of relationship has to be before it can be accepted as evidence for causality. Unfortunately, there is
scientific no clear answer. All we can suggest is that a stronger relationship provides more powerful
inquiry support for a possible cause and effect relationship.
Returning to our example, we need to examine the empirical relationship between nuclear family
The logic of
orientation and participation in family financial decision-making. A strong or high association is
scientific necessary for making a claim that the family living preferences of wives may be responsible for
inquiry differences in their participation in family financial decisions. Evidence of a relationship between
the two variables, as found by Fattah, meets at least the minimum requirement for the second test
Cause of possible causality. There is an empirical relationship and we have reason to believe that it is
and effect living preference, shown as variable A, in Figure 3.3 (3) may cause changes in participation in
family financial decisions, variable B in the same figure, and not the other way around. But with
Aids this evidence can we say that the family preferences of wives alone is responsible for differences
in their participation in family financial decision making? To answer this question, we still have
to meet a third and hard criterion.
Eliminating other causal factors
The third criterion is:
The observed relationship cannot be explained by the influence of any other
variable that could cause changes in either the independent or dependent
variables.
To satisfy this criterion, we have to rule out the effects of other variables on an observed
relationship. This is usually hard to do because we know most variables are related in varying
degrees to many other variables. Consequently, another variable may be the real cause of
variations in participation in family financial decisions rather than the one that we happened to

Chapter 3 Page 12
measure. Now our task becomes one of choosing between the alternative explanations
represented by examples (2) and (3) in Figure 3.3.
Example (2), Figure 3.3, illustrates a spurious or false relationship. In a spurious relationship, the
apparent relationship between two variables is really caused by the effect of a third or additional
variables on both the independent and dependent variables. In Example 2, variable C could be
responsible for changes in both A and B.
There are many examples of spurious relationships in social research. In Malawi, for example,
Mwango (1986) found that literate men were much less likely than illiterate men to brew beer for
sale from part of their maize crop. This relationship, however, vanished when he compared
literate and illiterate men with the same wealth. Wealth, in this case, was associated with both
higher education and hence, literary. Variations in wealth explained why some men purchased
rather than brewed beer. Wealthier, but illiterate men, also were less likely to brew beer. Further
analysis using the additional variable of wealth made the initial relationship spurious.
PREV NEXT

Chapter 3 Page 13
The best protection against accepting a spurious relationship is to subject any relationship you
find to additional analysis. Think about additional variables that might cause a change in either or
Introduction both of the variables in your relationship. Try using these additional variables to see whether the
initial relationship still stands or disappears when you take into account the effects of additional
Understanding variables. The chapters in Part 4 on analyzing data show how to do this.
concepts &
In our example, we could see whether the empirical relationship between nuclear family
variables orientation and participation in family financial decisions remained after testing for the influence
of other variables. Example (3) in Figure 3.3 illustrates the possible effects of variables C, D, and
Theory as a E on the initial relationship between variable A and B. To illustrate, we could test whether the
way of relationship was still observed when we tested for it among younger wives as compared with
organizing older wives, as represented by "C" in Example (3). This would rule out long term changes in
knowledge norms affecting relations between wives and husbands. We might do other tests as well; such as
seeing whether the relationship was still observed within low and high status families, as
Hypothesis & represented by "D; or among wives with different levels of schooling, shown as "E" in Example
research (3). If the relationship persisted after making these more refined tests, we would have a stronger
basis for asserting that family orientation, indeed, causes wives to be more assertive in family
financial decisions.
The logic of
scientific Perhaps you now see how difficult it is to claim you have discovered a causal relationship. We
inquiry can never collect data on all the variables that might affect some dependent variable we are
studying. Therefore, we can never know all the relationships that may exist between a dependent
The logic of variable and other variables. Recognizing this fact, researchers are extremely cautious in making
scientific cause and effect statements.
inquiry Three Web sites provide clear, short explanations of relationships and the requirements for
establishing a causal relationship. Types of Relationships, distinguishes between a correlational
Cause relationship and a causal relationship. A correlational relationship is like the one described in
and effect Example 1 in Figure 3.3. Without any additional information, this example simply shows the
interdependence between two variables. With a correlational relationship, as pointed out in Types
Aids of Relationships, all one can say is that the two variables are related and nothing more. The
relationship doesn't tell us whether one variable causes changes in another. The other two sites,
Causal Relationships, and Establishing Cause & Effect, present brief, clear explanations of the
three criteria that must be met before you can say you have evidence for a causal relationship. To
repeat, these are timing, a correlation between the presumed causal variable and the dependent
variable, and the elimination of all other possible causes.
PREV NEXT
http://www.srmdc.net/chapter3/13.htm [2/11/2008 8:45:11 AM]

Chapter 3 Page 14
Aids
Introduction Key terms

● Abstraction ● Hypothetical-inductive
Understanding ● Associated ● Independent
concepts &
● Attribute ● Independent variable
variables
● Categorical variable ● Indirect relationship
Theory as a ● Causal relationship ● Inductive logic
way of ● Cause and effect ● Inverse relationship
organizing ● Coefficient of correlation ● Negative relationship
knowledge ● Concept ● Non-linear relationship
● Concrete ● Positive relationship
Hypothesis &
● Construct ● Qualitative variable
research
● Curvilinear relationship ● Quantitative variable
The logic of ● Deductive logic ● Spurious relationship
scientific ● Dependent variable ● Theoretical framework
inquiry ● Direct relationship ● Theory
● Hypothesis ● Variable
The logic of
scientific Definitions for key terms are provided in the Glossary.
inquiry
Main points
Cause 1. Concepts are mental images we agree upon and use to describe what we want to talk about.
Social scientists also use the word construct to describe a concept.
and effect
2. Concepts are used at all stages of research - to state the question for investigation, for
Aids describing variables, and for expressing findings and conclusions. Concepts also are the
basis of communication among social scientists.
3. A theory expresses a logical and organized set of relationships among concepts. Theories
are based on research findings but go beyond any specific results. A theory integrates
existing knowledge and provides an explanation for a set of relationships.
4. Variables are composed of two or more attributes, each of which differs numerically or in
terms of qualities.
5. Attributes are the characteristics of variables. The variable gender, for example, is
composed of two attributes: male and female.
6. Hypotheses are tentative statements of expected relationships among variables. Hypotheses
are generally derived from a theoretical framework, but may also be derived from
empirical relationships among variables.
7. In formulating a hypothesis, the variable that is presumed to influence another variable is

Chapter 3 Page 14
designated as the independent variable. The variable affected by the independent variable
is designated as the dependent variable.
8. Hypotheses are usually expressed as positive or negative. A positive hypothesis states that
as the independent variable increases so does the dependent variable. A negative
hypothesis says that an increase in the independent variable is associated with a decrease in
the dependent variable. Others forms of hypotheses express curvilinear or no relationship.
A curvilinear relationship is one in which the association goes first in a positive or negative
direction and then reverses to the other direction. When two variables show no relationship
they are said to be independent of one another.
9. Two important forms of logic used in research are inductive and deductive reasoning.
Inductive logic involves reasoning from data to a conclusion. Deductive logic is based on
reasoning from a general statement to a predicted set of observations.
10. Both forms of logic are used in deriving and testing a hypothesis. Deductive logic is used
to derive the hypotheses from statements the investigator believes to be true, while
inductive logic is used to test whether or not the hypothesis is supported by the data.
11. In a causal relationship, the independent variable is presumed to cause changes in the
dependent variable.
12. To establish a cause and effect relationship, an investigator must show that the independent
variable precedes the dependent variable; that there is a strong relationship between the
two variables; and that the observed relationship cannot be explained by the influence of
any other variable that could cause changes in both the independent and dependent
variables.
13. Cause and effect relationships are extremely difficult to confirm. Researchers, therefore,
are extremely cautious in making any cause and effect statements.
PREV NEXT


Chapter 4 Page 1
Chapter 4: Selecting a Question to Investigate
Introduction Introduction
Selecting a question to investigate may be the hardest part of your initial research project. It is
Your initial also a very important decision. Every other decision you will make in planning a research project
research will be based on what you decide to study. In this chapter, we offer some suggestions for working
question through this important process. Part of this process involves learning about what is already
known about the topic you choose to investigate. In research, becoming informed about previous
Reviewing research findings is referred to as conducting a review of the literature. Therefore, we combine
the the process of selecting a question to investigate with the process of learning about previous
research. Today, with the increasing importance of the Internet as a source of information,
literature
literature reviews include Internet searches in addition to looking for information in libraries.
Preparing The usual processes involved in selecting a question to investigate are outlined in Box 4.1.
references
Box 4.1. Processes in deciding on a research question
Searching 1. Getting an initial idea: may be expressed as a "topic,"
Internet "interest" or "problem"
sources 2. Evaluating the idea, topic, interest, or problem
3. Conducting a comprehensive review of the topic or
Deciding on problem
the final 4. Making a final decision on the research question
research
question Your initial research question
Getting an initial idea
Aids
Research starts with getting an initial idea about something to investigate. We use the word
"idea" at this stage of the process to cover different ways you might start. You may begin with a
"topic" or a "problem" that interests you.
Topics, problems, or questions for research can come to you at anytime and from a variety of
sources. Course work is an obvious and frequent source of research questions. You may be
stimulated by something an instructor says or by something you have read. Some research reports
end with section entitled "Recommendations for future research." One of these recommendations
may excite you or and lead to a problem you want to investigate.
Frequently, things mentioned as "problems" by friends or relatives or something you read about
in a newspaper or magazine can be rephrased as a question for study. In addition, your own
personal experience or interests may lead you to do research on a certain problem. A student from
a religious or ethnic minority group may be motivated to investigate attitudes or behavior of the
majority group toward the student's group. A student from a rural area may want to investigate
ways of improving social services in his or her village.

Chapter 4 Page 1
You may find that recording ideas in a notebook as they occur to you is helpful. You can review
these ideas periodically, cross out ones that no longer appeal to you, and keep others for
consideration when you have to submit a topic or problem for your research requirement.
PREV NEXT

Chapter 4 Page 2
Evaluating the question
Introduction No matter how excited you may be about an initial research question you need to analyze the
question carefully before committing yourself to it. Regrettable, sometimes one's initial question
is not a good basis for research. A common flaw is that the problem or question is defined too
Your initial
broadly. To illustrate, suppose a student became concerned about crimes involving women and
research decided to investigate the topic of "women and crime." We call this initial idea a "topic" because
question it simply describes a broad area for possible research. In this form, it is not a problem or a
question, but it can serve as a starting point for specifying a question for investigation.
Reviewing
the As the student got into this topic, he or she would soon discover that this topic covers a number
of more specific topics. These include:
literature
What kinds of crimes are committed by women (theft, beer-making, prostitution,
Preparing etc.)?
references
Which kinds of crimes are committed against women (beating, rape, etc.)?
Searching What variables are associated with criminal behavior of women (age, economic
Internet level, ethnicity, etc.)?
sources What kinds of punishment are given for each kind of crime (fines, imprisonment,
etc.)?
Deciding on
the final Do punishments differ for women from different ethnic or social backgrounds?
research What effect does punishment of women criminals have on the women? Their
question children? Their families?
Aids Any of these questions could become the basis of a research project. As with most beginning
researchers, you will probably go through this kind of process, starting with a broad topic, and
then working toward a specific question that you can investigate. Some ways of doing this are
listed in Box 4.2.
Box 4.2. Ways of converting topics into researchable questions
● Discuss your ideas with fellow students.
● Ask your instructor or research advisor to review your ideas.
● Seek comments by experts; using the "women and crime"

example, you might ask police or judicial officials for their
opinions.
● Do a preliminary review of the literature on you topic.
● Try to state the topic or question in a clear, single sentence; if you

can, you probably have a good idea of what you want to
investigate.
Practical considerations

Chapter 4 Page 2
In selecting a question to investigate, two other things to consider are:

● Whether you can do the work required; and
● How excited you are about the project.
In addition to being too broad, many first-time researchers under estimate the work required to
complete a research project. You can avoid this kind of mistake by realistically estimating the
amount of time and effort that will be required to do your research. If you were conducting a
survey, some questions to ask include:
PREV NEXT

Chapter 4 Page 3
How many hours will I have to spend collecting my data?

How many respondents will I have to interview to have a sufficient number of
Introduction interviews?
How long will it take to locate the respondents?
Your initial How much travel time will be involved?
research How long will each interview last?
question Also, how much will data collection cost?
Will there be costs for transportation to get to respondents?
Will there be other costs, such as having to pay for meals while collecting data?
Reviewing
Can I pay for these costs?
the How long will it take to organize and analyze my data?
literature Can I complete the entire project in the time I have?
Preparing If you decide that the project will take too long or cost too much, you will have to reduce its
references scope or find a question you can investigate within your time and financial resources.
In addition, you should also ask yourself how excited you are about doing the project. Why am I
Searching interested in this question? How excited am I about it? Research is hard work. Even a small scale
Internet project can take many hours, weeks, and possibly months to complete. Being excited about your
sources project and wanting to find answers to the problem you have selected can provide tremendous
motivation to see the project to a successful conclusion. Your excitement, however, should not
Deciding on lead you to favor one set of results over others. Remember the norm of being disinterested in
the final results. As a person, you can be interested in a certain problem and want to do research on it, but,
as a researcher, you will want to do the research as honestly as you can and be ready to accept
research
whatever results occur.
question
As you think about what you want to study, you will probably reject some ideas and begin to
Aids focus on a topic or question that interests you more than others. At this point, you will want to
learn all you can about your area of interest. Researchers do this by conducting a review of the
literature.
Reviewing the literature
Three planning activities will produce better results and save you a lot of time in conducting your
literature review. These are:
● Defining the scope of the literature review;
● Identifying relevant information sources; and
● Preparing to record information you will find.
Finding information for a research paper is a research undertaking in itself. Several Web sites
show ways to get started. Take a look at Research Helper or Steps in Research and Writing
Process, Our site and the two just mentioned describe the typical steps involved. How you use
these steps will depend on your topic and the kinds of information sources available to you. Be
resourceful: adapt the steps described to your needs.

Chapter 4 Page 3
Defining the scope of the review

This step is necessary because a given concept or variable may be expressed in different ways in
various reports you will read. A common variable, such as gender, for example, may be referred
to as gender, as the sex of the person, or in terms of male-female or masculine-feminine
relations. If you were doing research on gender roles, you would need to look for publications
that contain any of these words in their titles. To do this, you would need to make a list of all
possible words, terms or phrases that might be used to refer to gender and the roles of males and
females. Your list of terms will define the scope of the relevant literature.
Frequently new variables are discovered during the review process. These variables and all the
related terms you can think of should be added to the list of terms used in defining the scope of
publications you will want to review. At some point, you have to decide how far you want to go
in looking for similar or related terms. A practical guide is stop looking when you no longer find
any new terms.
PREV NEXT

Chapter 4 Page 4
Identifying information sources
Introduction For students in many developing countries, finding research information is difficult. Most
libraries have limited collections of books and journals. Still, some information on a wide range
of topics can be found in most countries. Four sources you will want to explore are:
Your initial
● Your university library
research
question ● Nearby libraries
● Offices of faculty
Reviewing ● Internet-based sources of information
the
Using your university library. Your university library has two important resources: (1)
literature
materials, including books, journals, encyclopedias, and other materials; and (2) reference
specialists who can be of immense help in finding information you need.
Preparing
references On your own, you can check to see whether your library has books related to your interests. This
is a good way to start any review of literature. Today, most catalogs of university library holdings
Searching can be viewed online from a computer terminal. If your university library has an online catalog,
Internet searching for titles of books related to your research question is relatively easy - certainly easier
and faster than using a card catalog. You can enter the words or phrases from your scope of
sources
interest and see if your library has books related to these topics. If your library is not automated,
check the card catalog to identify books based on the main terms used in the catalog.
Deciding on
the final Getting help from library reference staff. The reference staff of your university library may be
research your best information resource. These helpful information specialists can lead you to useful
question information you would otherwise not know about. In the social sciences, for example, most
research is published in professional journals. Librarians will know whether your university
library has journals that may contain information related to your problem. If not, they may be
Aids
able to tell you whether a nearby library has the kind of information you are looking for. Also,
ask if your library has encyclopedias that may contain articles on topics related to your interest.
Some of these may provide useful summaries of literature in your area of interest.
Reference librarians can also suggest other sources of information, such as books, government
reports, and other research reports on file at the library. Also, ask your librarian about journals or
books that contain only reviews of literature. Many reviews on various topics are published each
year. These will be the single most valuable source of information you can find, provided the
article is related to your problem. A good review article lets you start with much of what is
already known about some topic and provides references to publications you will want to read.
Using nearby libraries. Other libraries near your university may be worth exploring. The library
of another university may have materials your library does not have. Many government ministries
and research centers maintain libraries that contain not only all the public documents of the
organization, but also reports of international organizations and those from similar foreign
organizations. For some research topics, these specialized libraries may be the best sources of
information in your country.

Chapter 4 Page 4
Offices of faculty. Faculty members often have their own collection of books in their area of
interest. Instructors you have had for various social science courses may have books that contain
reports of research related to your interest. It doesn't hurt to ask, and, if you do, you may find
information you can use.
Preparing to record information
Prior to starting your literature review, decide how you will record notes. Some students scribble
notes on scraps of paper with no consistent record of what they have read. Later, they discover
they cannot figure out what they had scribbled down. To avoid this problem, we suggest using a
single sheet of paper to record information you take from a publication. Using the same size of
paper for all notes will also help you organize your notes. In addition, we suggest using a
consistent set of categories for recording notes. This will help ensure that you get all the
information you need at one time, and save you from having to find a publication a second or
third time to get information you need for writing your research report
PREV NEXT

Chapter 4 Page 5
The most important information you will want to record on your note cards are the findings or
conclusions of the reports you read. You will also need to copy information for preparing a
Introduction reference for each publication you will cite in your final report. A reference includes the names
of the authors, the date and title of the publication, and other information we describe later in this
Your initial chapter. Other information you may need in writing your report includes the statement of the
research problem or the research question included in the publications you select, and a brief description
question of the design used (survey, experiment, observational study), the kind and size of the sample
used, and other points that may interest you.
Reviewing Box 4.3, below, illustrates use of a note card. The categories for recording information are shown
the in bold. These categories are to remind you that information on these points is important and to
literature help you get consistent information for all publications you read. To illustrate use of a note card,
we have entered notes we took for a publication we used in reviewing literature for writing this
Preparing book.
references Box 4.3. Example of a note card
Searching Reference: Rahama, A. (1997). Gender roles in crises situation: the case of the
Internet famine of 1985/85. The Ahfad Journal: Women and Change. 14:2, 4-15.
sources Problem/question: What coping strategies were used by individual households in
northern Kordafan for surviving the famine of 1984/85?
Deciding on
the final Design: A multimethod approach was used, including a survey supplemented by
information obtained from participant observation and from the analysis of
research
available records and secondary data.
question
Sample: The survey was based on a sample of 120 women selected from three
Aids ecologically different areas of northern Kordafan; women living in their villages,
women who had migrated to shanty towns, and those who had migrated to relief
camps.
Findings/conclusions: The overall conclusion was that women's roles were
crucial to the survival of their families; also, roles changed greatly for both men
and women. In the initial stages of the crisis, women used their knowledge of local
plant species to supplement the food supply for their families; women also
prepared only two meals a day, used cereals to supplement other food, and
rationed what food was available; men migrated with their animals and began
selling animals to get money for food. As the crisis continued, selling assets by
both men (animals and handicrafts) and women (gold and handicrafts) became
more common; migration continued. In the final stages of the crisis, migration was
combined with entry of females in the labour market, resulting in changed roles for
men and women; women engaged for the first time in paid labor while men stayed
home and cared for the children. A final effect of the crisis was that 63% of the
women became household heads.

Chapter 4 Page 5
Finding information and taking notes

In the process of identifying potentially useful sources of information, you will discover articles,
material in books and research reports, and possibly information on Web sites. As you start the
actual review process, your first task is to decide which sources to examine in detail. We offer the
following guidelines for getting the most useful information in the least amount of time:
● Focus on material directly related to your research question
● Learn to read critically: look for materials that tell you something new about the topic you
are investigating
● Take careful, comprehensive notes
● Prepare a complete, accurate reference for each source used
PREV NEXT

Chapter 4 Page 6
Focusing on relevant material
Introduction Some students think a good literature review is based on reading everything they get their hands
on. This can lead to a great waste of time and prevent you from finding and learning from really
useful material. Before starting, therefore, it is a good idea to set some limits on what you are
Your initial
going to try to get. The scope you defined for your literature search provides a useful set of
research limits. Look first for material that relates to the central concepts or terms in your scope of
question interest.
Reviewing Second, decide how far back in time you intend to search. In some fields, research moves rather
the quickly, making research done five years or so earlier relatively useless. Start with the most
current publications you can find and work back until you don't find anything new or particularly
literature
valuable. Then stop.
Preparing When you find a publication that looks promising, first read the abstract, if the publication has
references one. An abstract is a short summary usually found at the beginning of an article. If an article does
not have an abstract, turn to the summary at the end of the articles and read it. Then read the
Searching introduction or the review of literature section to see how the author says the report fits into the
Internet broader literature. After all, you might as well benefit from the reviews conducted by previous
writers. If these sections indicate the report has information related to your interest, then read the
sources
whole report critically and take notes on the points most valuable to you.
Deciding on Learning to read critically
the final
The key to analyzing publications and to preparing a review of literature is to read critically.
research Publications vary greatly in quality. Many publications meet the highest standards of research
question and scholarship: Unfortunately, others do not. With relatively little research experience, you may
find it hard to distinguish between good quality reports and ones of lesser quality. By the time
Aids you finish this book and your course on research methods, you will be better able to judge the
quality and value of reports. In general, if you suspect that the methods of data collection and
analysis are weak, you can probably disregard a report.
As you read, be alert to references to other publications. For new references, first examine the
title and decide whether you think it might contain information useful for your study. If you think
it does, copy down the reference on a note card for use in finding the publication. Continue this
process until you run out of new leads. Also, try to learn as much as you can from each
publication you read. The authors of each report you read dealt with the same things you will face
in planning and carrying out your study. As you read, watch for points listed in Box 4.4. Learning
how authors handled these points can help you at every stage in your own research, from stating
your research question to writing your report.

Chapter 4 Page 6
Box 4.4 Points to look for in reading a research report -

how authors:
● Defined and presented the problem they were
investigating
● Organized their reports (what was the order of the
various parts)
● Described the design they used
● Described their sample design and actual sample
● Presented and analyzed data
● Reported and interpreted their findings
● Derived and presented their conclusions
● Presented references to publications they cited
Taking notes
Using the headings (in bold type) in Box 4.3 as a guide, record your notes. How extensive
note-taking might be will depend on how valuable you think the information from any
publication is to you. Once you have a publication in hand, recording comprehensive notes for
later use is a lot better than going back to the library or other sources a second or third time to get
missing data. Sometimes, in the haste of getting through publications, students fail to write
clearly. At the time, this may seem like a timesaving action, but sloppy note taking often leads to
later frustration and a waste of time. Frequently, notes originally taken to help define the research
question, are used again when the report is being written, and this can be months after the notes
were taken. By then, sloppy notes may be difficult to read, causing considerable frustration and
even delay in completing a report on time.
As part of note-taking, you will need to create a reference for each publication you intend to cite
in your report.
PREV NEXT

Chapter 4 Page 7
Preparing references
Introduction The reference to a publication gives readers of your report the information needed to find a
publication you cited. Before recording references, you will want to check with you instructor to
see if your university has a preferred way of preparing references. If it does, you will want to
Your initial
follow the format required by your university. If your university does not have a preferred format,
research you may want to use the formats shown in Box 4.5 or another of the widely used styles for
question references.
Reviewing A reference is composed of elements. As you see in Box 4.5, the last names of authors are listed
the first for all kinds of references, followed by the initials of the authors, each of which is followed
by a period. For materials having a second or additional author, the last names of the additional
literature
authors are listed first, followed by their initials. The abbreviation for "and," shown as "&," is
used between the next to last and the last author. Next comes the year of publication, set off in
Preparing parenthesis and followed with a period. Sometimes you may have to refer to more than one
references publication by the same author in the same year. When this happens, add "a," "b," "c," etc., to the
year, as (1999a), (1999b), or (1999c), to differentiate among publications by the same author or
Searching authors in the same year. From this point on, elements of references differ, depending on whether
Internet you are citing a journal article, book, or something else.
sources
In Box 4.5, we provide examples of the most commonly used reference styles. For publications
with features we don't show, the following site may help:
Deciding on
● Formatting
the final
research
Box 4.5. Examples of commonly used references
question
Journal article
Aids Ghorayshi, P. (1996). Women in developing countries: methodological and
theoretical considerations. Women and Politics, 16, 89-109.
Grotberg, E. H., & Badri, G. (1986). The effects of early stimulation by Sudanese
mothers: an experiment. The Ahfad Journal: Women and Change, 3:2, 3-16.
Book
Lobban, C. F. (1987). Islamic law and society in the Sudan. London: Frank Cass.
Chapter in a book
Styos, J. M. (1983). Sample surveys for social science in undeveloped areas. In
M. Martin & D. P. Warwick (Eds.), Social research in developing countries:
surveys and censuses in the third world. New York: Wiley.
Dissertation
Darkoh, C. A. A. (1994). Women's roles and social change in Sudan. Unpublished
doctoral dissertation, Iowa State University, Ames.
Unpublished report

Chapter 4 Page 7
Department of Statistics, Ministry of Planning and the National Economy,

Republic of Sudan. (1982). The Sudan fertility survey. Voorburg, The
Netherlands: International Statistics Institute.
Online document
Darden, L. (2003). The nature of scientific inquiry. Retreived on February 4,
2003, University of Maryland.
PREV NEXT

Chapter 4 Page 8
For a journal article, the title of the article comes after the year of publication. Only the first letter
of the first word is capitalized: lower case is used for all the rest of the title. The exception is that
Introduction proper names (of persons, organizations, countries, etc.) are capitalized. A period is placed at the
end of the title of the article. The title of the journal that contained the article is listed next. The
Your initial first letters of main words in the title are capitalized and the journal title is placed in italics. A
research period is placed at the end of the journal title. The last element in a journal reference contains the
question volume number, issue number (when used by a journal), and the page numbers of the article
being cited. When the issue number is used, the volume number appears first, followed by a
colon, then the issue number, followed by a comma, and then the set of page numbers for the
Reviewing
article, beginning with the number of the first page of the article, followed by a dash, and then the
the number of the last page of the article, and ending with a period. Information for creating a journal
literature reference can usually be found on the on the cover of the journal; on the first page of an article; or
in the table of contents of the journal containing the article you are citing.
Preparing
references The second reference in Box 4.1 shows how to prepare a reference with two authors. Two more
authors of a book or any other publication are listed in the same way as shown for two authors of
a journal article.
Searching
Internet Reference to a book
sources
The third reference in Box 4.5 shows how to prepare a reference to a book. As with a journal
reference, the names of the authors are listed first; then comes the year the book was published,
Deciding on set off in parentheses and followed by a period. The next element is the title of the book. The title
the final is placed in italics with only the first letter of the first word in the title is capitalized. A period of
research placed at the end of the title. When punctuation, such as a comma or question mark is used in a
question title, include it exactly in the same way in your reference. The next element is the name of the
city where the book was published, followed by a colon, and then the name of the publisher of
Aids the book. The end of a reference to a book is marked by a period. The date, the city of
publication, and the name of publisher usually are listed on one of the first pages of a book.
Reference to a chapter in a book
The fourth reference in Box 4.5 describes a chapter authored by J. M. Styos that appeared in a
book that was edited by M. Martin and D. P. Warwick.. The only new elements in this reference
are the addition of the word "In" to show that the chapter is "In" a publication put together by
other persons. The names of the editors are listed next in their nature order, as M. Martin and D.
P. Warwick. Their role as editors is shown by the abbreviation of "Eds." shown in parentheses.
The rest of the reference for a chapter is the same as for any book.
Reference to a doctoral dissertation
As the fifth example, we show a reference to a dissertation submitted by Cecilia A. Adae Darkoh
to Iowa State University, where she was enrolled. For doctoral dissertations or master's theses,
the title is in normal type (not in italics). The words, "Unpublished doctoral dissertation," are
added, followed by the name of the university and the city in which it is located. A reference to a
master's thesis is the same except that "Unpublished master's thesis" would be used at the end.

Chapter 4 Page 8
Reference to a research report

Many government ministries and other organizations publish research reports. The report of the
Sudan Fertility Survey, shown as the next to last item in Box 4.5, is used to illustrate a reference
to a research report. As it happened, a team of researchers and writers prepared this report for the
Department of Statistics. The Department, therefore, is listed as the author. Librarians refer to
authorship by the organization that publishes a report as the corporate author. When you find
that an individual is not listed as the author of a publication, list the organization that published
the report as the author of the report. The title is in normal type.
PREV NEXT

Chapter 4 Page 9
The last reference is to a document found on a Web site. The format is the same as for an
unpublished report, except that the date when the information was retrieved is given, followed by
Introduction the name of the site, and its address. No punctuation is used at the end of the reference. Formats
for references to other electronic sources, such as electronic journals or newsletters, are described
Your initial on the site of the American Psychological Association.
research
Refining your list of references
question
Your set of note cards will change from day-to-day as you drop some sources and add others.
Reviewing Some references will lead to additional, useful information; others will not. As you work through
the your original and new references, you will also be coming closer to the end of the literature
literature review phase of your project. Before stopping, consider browsing in the book collection of your
university library.
Preparing Browsing is easy because books on the same subject are placed together on library shelves. When
references you find a reference to a book clearly related to your research question, go to the book's location
(given by its call number) and examine the titles of nearby books. You might find additional
Searching books that are useful as well. To save time, start by examining the indexes at the back of the
Internet book. Look for topics that relate to your interest.
sources With a slight variation, browsing can also be applied to journals. When you find that several
valuable articles came from the same journal, examine other issues of that journal. You may find
Deciding on additional valuable articles. With journals, scan the pages of contents to identify titles of articles
the final that grab your attention, and then read each article you select critically.
research
Creating a list of references
question
As you finish your review of literature, you will have a pile of note cards with references to
Aids publications and possibly Web sites from which you took information. As part of your report, you
will need to covert the references you created on your note cards to a list of references for your
report. To create this list, place the references in alphabetical order according to the last names of
the authors and create a list with the full reference to each publication or source of information
you mention in your report. For an example of how this is done, click on References and look at
the references we cite.
Searching Internet sources
We have added this section on searching the Internet because of its increasing importance to
researchers.
Understanding the Internet and Web sites
The terms Internet and Web sites are often used to describe the same thing, but, technically, they
refer to different things. The Internet is the set of connections and common means for linking
users to Web sites throughout the world. The Web sites contain the information one can access
through the communications and linkages provided by the partners who make up the Internet.
Today, Web sites are maintained by most major organizations, including universities, research

Chapter 4 Page 9
centers, government ministries, professional organization, international organizations, and

businesses. The ever-increasing numbers of Web sites offer information on almost any topic you
can think of. Because of the growing importance of Web sites as a source of research
information, researchers routinely check Web sites for information. If you have access to the
Internet, we recommend you include a Web search as part of your literature review.
Web site addresses
Each Web site has a unique address referred to as its Universal Resource Locator or URL. In the
beginning, the addresses for all Web sites began with "www," which stands for the World Wide
Web. Now, many Web sites do not include the "www" prefix; instead, they begin with some short
version or the initials of the name of the organization sponsoring the site. For example, the Web
address for the World Bank is www.worldbank.org while that for the Ahfad University for
Women, in Omdurman, Sudan, is http://ahfad.org. You will see Web addresses in both forms
with various sets of letters and numbers. Also, you may see Web address set off by with a set of
arrowheads, < and >. These are not part of the addresses; they are used to set an address off from
any text around the address. Also, Web addresses may end in a variety of ways. Some may
include the letters "html," but others may have only "htm" or some other ending. Be sure to note
the full and exact address of any site you see and wish to return to; otherwise, you will have
difficulty returning to it.
PREV NEXT

Chapter 4 Page 10
Kinds of information on Web sites
Introduction Many kinds of information are available on Web sites. We describe five kinds of information
sources that are particularly useful to researchers. These are:
Your initial ● Social science databases
research ● Sites of international organizations
question ● Other social science Web sites
● Guides to specialized Web sites

Reviewing ● Full text sites
the
literature Social science databases
A database is a computer-based index to journal articles, books, and research reports in some
Preparing field of knowledge. Some leading databases covering the social sciences are described in Box
references 4.6. Each provides quick online access to information about thousands of articles and other
research reports. Each also:
Searching ● Can be (has to be) searched by computer;
Internet ● Provides the full reference of the cited publication; and
sources
● Supplies a short summary or abstract of the publication.
Deciding on Because of the number of references they contain, databases have to be searched by computer.
the final This lets you quickly find material matching your specific interest from the thousands of other
research reports in the database. Many databases are accessible through Web sites. Some are also available
question on CD ROM (Compact Disk Read Only Memory). You might want to check to see if your library
subscribes to any database in your area of interest on CD ROM.
Aids Searching a database through a Web site, requires a connection from your location to the Internet,
and, for most databases, a subscription is required as well. You may wish to see if your library
subscribes to any of the databases we list or others in your field of interest. Two widely used
databases, POPLIN and ERIC, however, can be searched at no charge.
Illustrative database search
We will illustrate database searching using POPLINE, because it is free and covers topics of
interest to persons in developing countries. If you want to search the ERIC database, detailed help
is available at ERIC Slide Show.
Information from most databases can be downloaded to the disk on your computer or a floppy
disk or be printed out. Each also provides a short abstract of each document in the respective
database. Abstracts describe the problem that was investigated, methods used, findings, and
conclusions. Often, an abstract will provide all the information you will need. In addition,
POPLINE and ERIC can provide copies of the full text of most publications in case users want
more details than provided in abstracts. POPLINE Digital Service provides free copies of
documents for users in developing countries. For more information, go to POPLINE and click on

Chapter 4 Page 10
"Document Delivery Policy" in the list of topics on the left side of the page. A number of the
documents cited in ERIC can also be downloaded free.
PREV NEXT

Chapter 4 Page 11
Searching POPLINE or any other database is a two-step process. First, you need to decide what
you are looking for; second, you need to develop a plan for conducting the search. Your plan,
Introduction called a search strategy, will depend of what you are looking for and what you already know
about the topic you have chosen for research.
Your initial
Most database searching is based on typing in key terms. These may be the same words or
research phrases you developed for the scope of your review. Let's say that our research question concerns
question the evaluation of a family planning program in African countries. In this case, our scope of
review would include any reports on evaluations of family planning programs in any African
Reviewing county. "Africa" and "family planning program evaluation," therefore, become our key terms for
the this search. If you aren't sure what key terms to use, look to see if the site has a "Browse"
literature window. Sites with a browse function let you type in a word or phrase and then present you with
the terms the database will use to obtain information corresponding to your terms.
Preparing
Box 4.6. Some social science databases
references
Anthropology Index, published by the Royal Anthropological Institute,
Searching London, contains bibliographic citations from journal holdings of the
Internet Anthropology Library of the British Museum, with emphasis on Commonwealth
sources countries and Africa; all areas of anthropology and archeology are represented
Dissertation Abstract. provides information, including abstracts, of most PhD
Deciding on dissertations and many MA theses from almost all North American and many
the final other universities
research
question Educational Resources Information Center (ERIC), sponsored by the U.S.
Department of Education, covers educational journals, books, and unpublished
Aids educational reports, although mainly from sources in the United States
Google Scholar, covers peer- reviewed papers, theses, books, abstracts and
articles across the world of scholarly research
POPLINE, the world's largest bibliographic reproductive health database,
covering the fields of population, family planning, and related health issues
PsycINFO, produced by the American Psychological Association, this
international database covers the scholarly and professional literature in
psychology and related fields; including journal articles, chapters in books, and
complete books
USAID (United States Agency for International Development), maintains the
Development Experience System, a set of databases containing records for about
100,000 technical and program documents
To understand the rest of this illustration best, we suggest you repeat the steps using a computer
connected to the Internet. If this is not possible, you should still be able to grasp the process

Chapter 4 Page 11
involved. Click on the following Internet address, db.jhuccp.org/popinform/basic.html, and the

Internet will take you to the page for starting a search of the POPLINE database. Since POPLINE
is a database site, this page is designed to help users quickly begin their searches.
POPLINE offers a choice between a "Basic" and an "Advanced" search. The Basic search is
easier to use and is a good way to start most searches. Although Advanced search requires a bit
more skill, it is easy to learn and generally will produce results closer to your interest.
Let's first use the Basic mode for searching. Once on the Basic search page, you will see a page
with the following two boxes:
Subject:
Author:
PREV NEXT

Chapter 4 Page 12
Since we are using key terms, the next step is to type the search you want to do in the box labeled
"SUBJECT." In doing this, you have to follow POPLINE's rules for entering a search. These
Introduction rules are given under "Search Tips," the second topic in the lower list on the left side of your
screen. Since you want information about "Africa" and "evaluation of family planning
Your initial programs," you will need to link both terms in the search box. POPLINE uses the symbol "&" to
research combine the two key terms. This means you will get information about all documents dealing
question with both Africa and evaluation of family planning programs. Instead of "evaluation of family
planning," we ask you to use "family planning program evaluation." The reason is that we know
this is a term POPLINE uses. If you want to check, type in "family planning" in the "Browse"
Reviewing
box and scroll down and you will find "family planning program evaluation." Now, type in the
the two terms as follows in the Subject search box.
literature
Subject:
Preparing Africa & family planning program
evaluation
references
After entering the key terms, click on the "Search" button on the screen, and POPLINE will tell
Searching you how many records match the two search terms. When we did this on February 15, 2005, we
Internet found 849 records for reports on the two subjects. Your search will certainly produce more than
sources 849 "hits." This is because POPLINE adds hundreds of new references each day, some of which
will deal with the evaluation of family planning programs in Africa. The search results consist of
Deciding on a list of brief descriptions of the 849 records for documents that match the search terms you
the final entered, but POPLINE shows only 50 at a time. Scroll down the list and click on any record that
looks interesting to you. The descriptions are either to information on a Web site or to a printed
research
document. Web sites can be identified by their distinctive addresses. If the record is to a
question document on a Web site, click on it and you will be taken to that Web site. Once there, you can
read, copy, or download what you see. If the description is to a document, click on the number
Aids for that record and an abstract for that document appears. Also, for documents with abstracts,
there is a set of related key terms at the bottom of the page. You can select any of these terms to
modify your search or to do a new search.
Box 4.7 shows the full record for one of the references we found.

Chapter 4 Page 12
Box 4.7. Example of a record found on POPLINE

Title: Family planning programs: efforts and results, 1972-94.
Author: Ross JA; Mauldin WP
Source: Studies in family planning. 1996 May-Jun;27(3):137-47.
Year: 1996
Language: English
Abstract: The varied strengths of key elements of selected national family
planning programs were assessed. 93 developing countries were rated according
to the strength of their family planning efforts in 1972, 97 in 1982, 98 in 1989,
and 94 in 1994. Time trends and program efforts are presented for those
countries. Overall program effort for the developing world increased sharply
from 1972 to 1982, and again from 1982 to 1989, but only modestly thereafter.
Some country programs were already nearing their ceiling levels, a few with
very low fertility rates deliberately weakened their programs, and other
programs deteriorated for undetermined reasons. After 1989, the lowest rated
regions improved the most, mainly with regard to their policies, and improved
least with regard to the availability of contraceptive methods. The relationship
of program strength to socioeconomic setting has steadily weakened over the
22-year period.
Keywords: Developing Countries; Historical Survey; Family Planning
Programs [History]; Family Planning Program Evaluation; Family Planning
Evaluation; Family Planning Program Administration
Earlier, we mentioned the "Author" search box. If you know the name of an author and want to
find a report by that person, you can use the "Author" search box to do a search. To illustrate, we
used the name of "Mauldin," one of the authors listed in Box 4.7. Simply type the author's name
in the box, as show next, and click on search.
Author:
Mauldin
PREV NEXT

Chapter 4 Page 13
You could use either "Mauldin" or "Ross" and you will get the same abstract and record. Try it.
Introduction In this illustration, we examined only one of the references found in our search on February 15,
2005. Examining all 849 records would take a long time, be extremely tedious, and could result
in high telecommunication charges for your computer connect time. You can reduce the number
Your initial
of references or "hits" you get by using the "Advanced" mode of searching offered by POPLINE.
research To do this, return to the POPLINE home page and click on "Advanced Search." This opens a set
question of search entry boxes. In the Advanced mode, you type each search term in a separate entry box,
entering "Africa" in the first box and "family planning program evaluation" in the second box.
Reviewing Select AND as the way of combining the two terms. The Advanced search option also lets you
the limit the search. For example, you could limit the search to only English-language documents.
literature You can do this by selecting "English" in the options for "Language." Also, with the Advanced
search option, you can limit the years the search will cover. Let's say, you limit the search to only
Preparing documents reported in 2004. Do this by selecting only 2004 under the "Date" option With this
references more restricted search, limited to only English-language documents published in 2004, we got
only 3 references or hits. This is a much more manageable number and might provide all the
information you need on methods used in evaluating of family planning programs.
Searching
Internet If 3 records seem too few or you don't find what you want, you can broaden your search. While
sources you still might want only documents in English, you could change the time period to include all
reports from 2001 through the last year. Try experimenting and note the different number of hits
Deciding on you get.
the final The key words listed below the abstract can be very helpful. They can suggest additional ways to
research construct a search. After seeing key terms from our search, we might decide that we want to see
question documents related to the "family planning program administration," a key term that POPLINE
uses, instead of family planning program evaluation, the term we initially used. We could also
Aids change our search by looking for documents describing programs in other areas of the world.
Suppose, in addition, we decided to seek information on programs in either Africa or the Middle
East In a POPLINE search, the relation of "or' between terms is represented by a slash line (/). In
the Basic search mode, this search would be entered in the following way:
family planning program administration &
Africa/Middle East
The "&" shows the AND relationship between "family planning program administration" and
either Africa or the Middle East. The "/" indicates the OR relationship involving Africa and the
Middle East. In the Advanced mode of searching, we would type "family planning program
administration" in the first box; select AND as the method of linking "family planning program
administration" with the next term; type "Africa" as the term in the second box; select OR as the
linking term; type "middleeast" in the third box; and then click on the search buttom. Various
combinations of OR and AND can be used in either search mode. With a little trial and error or
with some help from a more experienced searcher, you will able to develop combinations of key
terms that closely match any search interest you may have.

Chapter 4 Page 13
Web sites of international organizations

All international and major nongovernmental organizations maintain Web sites. Box 4.8 lists
sites for major international organizations that focus on needs and developments in developing
countries. In addition to describing their own programs, Web sites of these organizations provide
the most current information and data in their area of interest. The home page of the Food and
Agricultural Organization site for example, lists headlines for articles on major developments
in the field of agriculture and food availability throughout the developing world and has links to
many different topics and issues. The topic of "Sustainable Development," listed on the left side
of the page (at least when we visited the site in February, 2005) caught our eye. We clicked on
this topic and were taken to a page title "What's new." A number of recent reports were listed. We
choose "Rural Households and Resources A SEGA pocket guide for extension workers" listed
under the heading of "People: Gender and Development." Clicking on the title, we then saw an
abstract of the report and then clicked on the line to view the entire document. This brief journey
through the FAO site illustrates the wealth of information that can be obtained from the Web sites
of international organizations - all brought right to you by the Internet.
PREV NEXT

Chapter 4 Page 14
Box 4.8. Web sites of major international organizations
Introduction Food and Agricultural Organization (FAO information on food and food
insecurity, agriculture, and agricultural indicators
Your initial International Labour Office (ILO), source of wage, labour force, and
research employment data
question
Organization for Economic Co-operation and Development (OECD)
Reviewing provides analytical reports on social and economic trends in member countries
the and Eastern Europe, including flows of aid to developing countries
literature United Nations (UN), source of news, information and data on programs in
developing countries
Preparing
references United Nations Children's Fund (UNICEF), provides a wide variety of
information and data on the status and well being of children throughout the
Searching world
Internet United Nations Educational, Scientific and Cultural Organization
sources (UNESCO) source of data related to education and educational attainment in
various countries
Deciding on
the final United Nations High Commissioner for Refugees (UNHCR) provides data on
research refugees worldwide
question United Nations Population Division (UNPOP) produces international data on
population trends, including demographic estimates and projections
Aids
United Nations Population Fund (UNFPA),, news and information about the
world's population and population and environmental issues
United Nations Statistical Division (UNSD) produces a wide range of
statistical data about conditions in countries
World Bank produces data on economic trends as well as broad array of
economic and social indicators
Other social science Web sites

Today, every major social science organization has a Web site. Internet addresses for some sites
that contain social science research reports and other information you might find useful are listed
in Box 4.9. The address listed for an organization will take you to the organization's home page.
On most sites, the home page provides information about the organization's mission,
organization, key personnel, and descriptions of the content found on other parts of the site. Most
sites also include links to other sites containing information in related areas.

Chapter 4 Page 14
Box 4.9. Other useful social science Web sites

American Psychological Association provides current information on many
topics related to psychology, with feature articles on important issues
American Sociological Society provides information about sociology in the
United States; notable for providing the table of contents of the issues of the
American Sociological Review, the premier sociological research journal in the
U.S.
BUBL LINK (Catalogue of Selected Internet Resources) provides links to
hundreds of social science Web sites organized under more than 300 specialized
topics, such as research ethics, sociology, women's issues, as well as links to
other major fields of knowledge
CASS (Social Science Information Gateway) provides an online newsletter on
applied social survey, links to other survey centers, and a question bank
containing items used in various surveys
Research Resources for the Social Sciences produced by McGraw-Hill
Ryerson, an American company, this site provides links to Web sites by
discipline - psychology, sociology, anthropology, economics, etc.; also permits
users to construct their own searches
SOSIG (Social Science Information Gateway) provides selected, high quality
information for students and researchers in the social sciences, business, and
law; also provides links to over 50,000 social science Web pages
PREV NEXT

Chapter 4 Page 15
Full text Web sources
Introduction A growing part of the Internet consists of sites that provide copies of the full texts of documents,
similar to printed copies, but in electronic form for viewing from a computer terminal.
Your initial Unfortunately, most of these sites require a subscription, but a few sites, such as the following,
research provide free copies of publications.
question FIND ARTICLES, a free article search service; brief articles are provided in full text; payment
is required for access to the full text of some articles
Reviewing
the HighWire provides online access to articles in over 4,500 medical journals and over 830,000 free
literature full text articles in nearly 800 other journals.
Practical Assessment, Research and Evaluation (PARE), this is a peer-reviewed electronic
Preparing journal with articles on research methodology, evaluation design and methods, and other topics of
references interest to research students
Searching Social Science Resource Network (SSRN) provides abstracts to scholarly working papers and
Internet copies of their texts
sources
Deciding on PREV NEXT

the final
research
question
Aids

Chapter 4 Page 16
Some of the widely used search engines, with their addresses are:
Introduction AltaVista
Google
Your initial
research Hotbot
question
Teoma
Reviewing Yahoo, this resource is different from the others in that it retrieves sites that have been reviewed
the and classified into subject areas by experts
literature
To conduct a search, select a search engine, enter your search terms in the search window of the
search engine, and click on the "Search" or "Go" button or just hit "Enter." Each search engine
Preparing
has a set of rules for conducting a search, but most have a number of features in common. Also,
references each search engine has a "help" section that shows how to construct and carry out searches.
Friends or a librarian can offer help as well. After initiating a search, the next screen shows a list
Searching of Web sites that match your search terms with a short description of each site. Examine the titles
Internet and select the first one you want to go to. Then, click on the title of the site. Once at the site, you
sources can read and take notes as you would with a publication. Web searching, however, has a
tremendous advantage over reading and taking notes from a printed publication. You can
Deciding on download content from most sites to a disk and then print the content out or later paste it into
the final your report, without having to take notes or copy anything by hand.
research Before leaving a site from which you have taken information, be sure to get all the information
question you need to create a reference to the site. See Box 4.5 for what is needed. Note that a reference to
a Web site should also include the day you obtained the information. This is necessary because
Aids the content of sites change from day to day.
While the Internet clearly is a valuable resource for social researchers, users also need to
recognize and be prepared to deal with its limitations. In Help, we briefly describe some of the
limitations of the Internet, particularly for users in developing countries.
Deciding on the final research question
By now you may have forgotten the purpose of reviewing the literature and that is to learn all you
can about the topic or question you selected to study. The process of reviewing the literature can
have one of several effects on the initial question selected for research. As a result of reviewing
the literature, you may decide not to pursue your initial topic or question. In this case, you would
have to start looking for a new focus for your research. Frequently, the result is to proceed but
with a slightly different focus. Or, you may decide the original statement of the topic or question
is still a suitable one and proceed to the design phase of your investigation.
For another view of the process of stating, researching, and selecting a research question, we
suggest you visit:

Chapter 4 Page 16
Before the Inquiry: Scoping the Research Terrain, this Web site provides detailed guidance
based on a six-step process, beginning with clarifying your question, conducting a literature
review, noting patterns in what you find, obtaining the assistance of others in integrating and
confirming what you have learned, and finally, synthesizing existing knowledge about the
question you intend to investigate
PREV NEXT

Chapter 4 Page 17
Aids
Key terms
Introduction
● An abstract ● Print-out
Your initial ● Browsing ● Record
research ● CD ROM ● Reference
question ● Database ● Review of the literature
● Download ● Search engine
Reviewing
● Home page ● Search Strategy
the
literature ● Internet ● Surfing
● Key term page ● Universal Resource Locator (URL)
Preparing ● Links ● Web sites
references ● List of references ● World Wide Web
● Online
Searching
● Paste
Internet
sources Main points
1. Questions for research can come from a wide range of sources, including things you read
Deciding on or hear about, lectures, comments by relatives or friends, your personal experience, or from
the final an idea that suddenly pops into your mind.
research 2. A research question is best expressed in a single, simple sentence.
question 3. Research questions should be evaluated to make sure an initial idea is not too broad; that
the research needed to answer y\our research question can be completed within your time
Aids and financial limits; and that planned research is something you are excited about doing.
4. Reviewing the literature is an important part of preparing for research, and includes
learning as much as possible about previous research on the question being investigated.
5. Preparing for a literature review begins by defining the scope of the investigation. This is
done by expressing the topic or research question in terms of all the concepts covered by
the key words in the research question.
6. While your university library and other nearby libraries may provide relevant information,
Web sites on the Internet are becoming important sources of information on all kinds of
topics
7. As part of an Internet search, it is a good idea to look at sites containing social science
databases, guides to specialized Web sites, and sites of research centers, professional
associations, and governmental and international organizations.
8. Use of note cards, prepared in advance of a literature review, will help ensure systematic
and comprehensive recording of information you will want to use.
9. Find out what style your university uses for recording references and follow these or use

Chapter 4 Page 17
the formats described in this chapter.

10. When you have a useful publication in hand or see useful information on a Web site, be
sure to record all the information you will need for creating a reference to it.
PREV NEXT


Chapter 5 Page 1
Design and Chapter 5: Creating a Research Design

purpose
Design and the purpose of the research
of the
research Your research design is your plan for providing a sound, and, if possible, conclusive answer
to your research question. Designs vary greatly, depending on the research question being
Quantitative addressed and the methods of data collection the investigator chooses to use. Creating a
and design starts with the purpose of the research.
qualitative Most research is done for one of three purposes. The most common purpose is to describe
data some set of variables or relationships among them as accurately as possible. Other purposes
of research are to explore a topic to learn more about it or to explain why certain social
Design patterns or relationships occur as they do. Some studies combine more than one purpose. For
alternatives example, while the Sudan Fertility Study was designed primarily as a descriptive
investigation, we showed how data from this study were analyzed to explain why future
Aids fertility rates may decline in northern Sudan. Nevertheless, we will describe each approach
separately to show how each purpose affects design decisions. Exploratory research generally
requires less rigorous design decisions; so we will start with it.
Exploratory research
Exploratory research, as the name suggests, is a way of gaining some initial information
about a problem or topic. You may be interested in a certain problem, but don't have enough
information to write a clear research question. Perhaps you are not sure what the critical
variables are or what methods of data collection may work best. Before beginning an
investigation, you might decide to explore the problem by doing some informal interviews or
by living for a short time with the group you want to study. These techniques are often used in
exploratory research.
Example. Julia and Ridha (2001) explored experiences of Kuwaiti women during the Iraqi
invasion and following occupation of Kuwait in 1990. Twenty women who played various
roles, including active opposition to the Iraqi invaders, told of their experiences before,
during, and after the invasion. The women reported that they actively sought their own
self-development and liberation, but once the conflict ended, their choices and rights tended
to become restricted again by traditional male prerogatives.
Exploratory studies, however, seldom provide satisfactory answers to research questions. One
reason is that most exploratory studies are based on samples too small to permit generalizing
the results to a larger population. This is certainly true of the study of Kuwaiti women. But
exploratory studies can provide valuable, even critical, information for designing larger scale
descriptive or explanatory studies.
Descriptive research
Descriptive research is more specific and focused than exploratory research. The researcher
starts with a well defined problem or research question and a clearly defined plan for
collecting and analyzing data. Descriptive research is intended to produce clear, well-founded

Chapter 5 Page 1
answers to some question or specific, factual information. Surveys are frequently used in
descriptive research.
Example. Fattah (1981) conducted interviews, using a questionnaire, with 200 randomly
selected farm families in Iraq. The questionnaire contained over 200 items about
husband-wife decision-making related to the operation of the farm, financial matters, social
life, entertainment, training of children, and childcare. She also obtained information on
whether the families had a "modern" or "traditional" view of things. These data provided a
detailed description of the degree to which wives participated in decision-making with their
husbands. Fattah concluded that factors associated with modernism will continue to increase
and will enhance the position of women in rural Iraq, which, today, remains male dominated.
PREV NEXT

Chapter 5 Page 2
Design and Explanatory research

purpose
Explanatory research goes beyond exploratory or descriptive research by trying to find the
of the reasons why certain relationships occur. It seeks to provide explanations for what has been
research observed. Explanations are based on interpretation of findings in terms of broader concepts
and accepted theory.
Quantitative
and Example. Using extensive interviews and observation, Dei (1992) developed an explanation
for how Ghanian villagers survived prolonged drought and related hardships. He found they
qualitative
got through hard times by shifting from farm production for a market economy to shared
data production with other villagers for household consumption. A strong sense of community
solidarity developed and allowed the villages to survive with what they could produce
Design together. The explanation for their survival was the successful return to earlier farming
alternatives methods.
Aids Regardless of the purpose of your research, you will also need to decide on a technique for
data collection. There are two contrasting yet complimentary techniques - quantitative and
qualitative means of collecting data.
Quantitative and qualitative data
Many fields of science, particularly the physical sciences, report observations in terms of
numbers. Observations reported in numbers are referred to as quantitative data. Social
scientists obtain and analyze quantitative data, as you have seen with various studies we have
cited. Social scientists also obtain and analyze data in qualitative form. These are observations
recorded in words - descriptions of what persons said or did, how they interacted with one
another, or what a researcher observed by watching their behavior. Each technique is useful;
each also has certain strengths and weaknesses; each also presents a different set of design
decisions.
Some researchers draw a sharp distinction between quantitative and qualitative research. A
different view is presented in Types of Data. This discussion shows how quantitative data is
based in part on qualitative judgments and that qualitative data can also be described and
analyzed numerically.
Quantitative research
A quantitative approach requires a well-developed research design. Quantitative research is
usually based on:
● Careful and precise specification of the question to be answered;
● Identification, definition, and measurement of the key variables;
● Selection and specification of one or more methods of collecting data;
● Development of a sampling plan; and
● Numerical analysis of the data, including use of appropriate statistical tests.

Chapter 5 Page 2
Numbers used in quantitative studies may be as simple as counting the number of "Yes"
versus "No" responses to some question. More complex forms of measurement, however, are
generally used. The descriptive research by Fattah (1981), for example, used a quantitative
design. The many decision-making variables were defined and measures were developed for
each. These measures, in the form of questions, were then combined in a questionnaire that
was used in interviewing a random sample of farm families.
Quantitative data has important strengths. Using numerical measures provides more precise
descriptions of variables. In addition, quantitative research permits use of larger samples. As
you will see in Chapter 19, "Performing Inferential Statistical Analyses," larger samples
provide a stronger basis for generalizations. Also, quantitative data can be combined and
analyzed using various statistical techniques. You are certainly familiar with averages and
percentages as ways of summarizing data. Later, in Part 4, you will learn about additional
ways of analyzing quantitative data.
Most of this site focuses on planning and conducting quantitative research and analyzing data
based on quantitative variables.
PREV NEXT

Chapter 5 Page 3
Design and Qualitative research

purpose
Qualitative research, in contrast to quantitative research, seeks to dig deeper into the reasons
of the for behavior we observe. Generally, researchers doing qualitative research start with a broad
research idea of what they are looking for. In qualitative studies, researchers often combine informal
interviews, in the form of long conversations with people, with systematic observation of their
Quantitative daily activities. Data are derived continuously from what the persons in the group being
and studied do and say, how they say they feel about things, and the reasons they give the
qualitative researcher for what they do. The researcher accumulates a large amount of data, generally in
data the form of long sets of notes and written descriptions of what was observed. As data are
recorded, the researcher tries to interpret what the data mean and to develop an explanation
Design for what has been observed. In this sense, analysis of data proceeds with data collection.
Qualitative research is generally a much more flexible process, allowing researcher to take
alternatives
advantage of new lines of inquiry as they develop.
Aids The research studies cited earlier by Julia and Ridha (2001) and by Dei (1992) were based on
qualitative research methods. Research by ElSayed and Ahmed (2001) on the "Socio-Cultural
Aspect of Kala-azar among the Masalit and Hansa Tribes" further illustrates use of qualitative
techniques. (Kala-azar is a severe eye disease that frequently results in blindness). The
authors used small group discussions with villagers, personal interviews with patients, and
direct observation of health and sanitation practices in the villages. They found that lack of
knowledge about the disease is the main reason for its spread.
We have just touched on qualitative research in this section. In Chapter 13, we describe some
of the more frequently used qualitative research methods.
Combining quantitative and qualitative techniques
Researchers frequently use both quantitative and qualitative techniques in the same
investigation. In this way, researchers benefit from the strengths of each approach and
minimize their respective limitations. Altareb (1997), for example, investigated attitudes
toward Muslims held by undergraduate students at an American university. He began by
conducting focus groups, a qualitative technique, to get an initial understanding of students'
attitudes. After this exploratory phase, he developed a quantitative measure of attitudes for the
main part of the investigation. Davidson (1992) also combined qualitative and quantitative
methods in his investigation of ways families in the Nuba Mountains of the Sudan were
adapting to rapidly changing socio-economic conditions in their area. He began by
interviewing village elders and then conducted a survey to identify the characteristics of each
of the 12 villages in his study area. Following the survey, he carried out in-depth interviews
with selected households. This combination of methods provided a rich mix of quantitative
data for describing adaptive techniques used by villagers and detailed and qualitative data for
understanding how they were adapting to changes.
Unit of analysis
In designing research you also have to decide on what basis you will analyze the data you will

Chapter 5 Page 3
collect. This requires a decision about the unit of analysis for the study. Units of analysis may
consist of individuals, groups, organizations, geographical places, or other entities. Generally
the way you state your research question indicates which unit will be used as the basis of
analyzing the data.
It is easy to confuse unit of analysis with the entities you collect data from. Frequently, they
are the same. In the Sudan Fertility Survey, for example, the data were collected from women
and the data were analyzed in terms of the same women. In this case, the decision was clear.
In another study, a researcher might interview factory managers to get information to compare
the efficiency of different sized factories. In this case, the unit of analysis is factories, not
managers because the purpose of the research is to compare factories and managers happened
to be the source of data. If you have any doubt, think about the question you are trying to
answer. How do the data have to be organized to answer the question?
PREV NEXT

Chapter 5 Page 4
Design and Individuals as the unit of analysis

purpose
When the purpose of research is to say something about individuals, they become the unit of
of the analysis. Individuals are the most frequent unit of analysis. They may be any age, either
research gender, or become the units of analysis because they have a common characteristic. Primary,
intermediate and secondary students were the units of analysis in a trend study by Badri and
Quantitative Burchinal (1985). Many studies are based on collecting data from university students, who
and then become the unit of analysis. Cook (2001), for example, investigated the attitudes of
qualitative Egyptian university students toward Islam and higher education; Had-Elzein and Ahmed
data (2001) obtained data on perceptions and vision of peace from Sudanese university students.
Married women were the unit of analysis in the Sudan Fertility Survey (1982). Mageed,
Design Sulima and Kawther (2000) collected data about attitudes toward female genital mutilation
from adult men and women. Agricultural technical workers were the unit of analysis in a
alternatives
study of worker satisfaction in Kenya (Mulinge and Mueller, 1998). Muneer (1989) selected
farmers, who became the unit of analysis, in his study of the role of cooperatives in western
Aids Sudan. Khalafalla (2001) selected policy-makers as her sample who also became her unit of
analysis.
Groups as the unit of analysis
Research is often conducted by collecting and analyzing data for groups of people. When the
purpose is to describe two or more persons as a unit, these groups become the unit of analysis.
Households, consisting of various numbers of persons, often are used as the unit of analysis.
Examples include the research by Julia and Ridha (2001) on Iraqi farm families and research
by Davidson (1972) on changes in household life in the Nuba Mountains. Households were
the unit of analysis in an extensive study of year-long spending patterns among farm families
in Sierra Leon (King and Byerlee, 1977). The sample used by Grotberg and Badri (1986)
consisted of intact families with a child not older than five and one-half years. The
mother-child pairs in these families became the unit of analysis in this study. Other units of
analysis at the group level include friendship groups, clubs, social groups, and groups of street
children.
Organizations as the unit of analysis
Research often is directed at learning about organizations, such as businesses, firms,
ministries of government, universities, political parties, religious bodies, or military units.
Although data about each organization may be obtained by interviewing members of an
organization, the data would be used to describe and compare features of each organization.
For example, Gimbel and associates (2002) studied job satisfaction in community-based
organizations. They collected data from employees but analyzed their data in terms of
characteristics of the organizations. Mohamed (2002) analyzed operations in a sample of 150
government divisions in the United Arab Emirates. In a study of safety programs,
Vredenburgh (2002) collected data about practices in 62 hospitals. These are only a few of the
many management studies based on organizations as the unit of analysis.

Chapter 5 Page 4
Geographical places as the unit of analysis

Places, such as where people live, can also become the focus of study and therefore be used as
the unit of analysis. Fishing villages in Uganda were the unit of analysis in a socio-economic
investigation by Namara (1997). Cities, provinces, and rural versus urban areas are used as
the unit of analysis in most reports issued by international organizations. Countries
frequently are used as the unit of analysis. Bulluck (1986) compared the extent to which
developing countries meet the social needs of citizens. Countries were also used in comparing
infant mortality rates throughout sub-Saharan Africa (Frey and Field, 2000). Wang (1996)
examined the extent to which women's reproductive rights were protected in 101 countries.
PREV NEXT

Chapter 5 Page 5
Design and Other units of analysis

purpose
Although most social research is based on the kinds of units of analysis just mentioned, other
of the things may be selected for analysis. These include the contents of newspapers and magazines
research (Badri and Osama, 1995; Swanjord, 1989); rural development projects (Rao,1981); and folk
tales (Mathews, 1985).
Quantitative
and Box 5.1 summarizes frequently used units of analysis.
qualitative Box 5.1. Some frequently used units of analysis
data 1. Individuals - by far the most frequently used
2. Groups- husband-wife pairs, social groups
Design
alternatives 3. Organizationsbusinesses, offices, factories, government
ministries
Aids 4. Classrooms, schools, university student bodies or groups
5. Geographical areas- villages, communities, cities, regions of
a country, entire countries
6. Mass media materials- newspapers, magazines, television
shows
Deciding on the unit of analysis

Most likely the unit of analysis you will use in a study will be clear from the way you defined
your problem. If you have any doubts about the unit you should use, you will need to resolve
this confusion before going on. Otherwise, you may not analyze your data properly and your
entire study could be jeopardized. Ask yourself what unit you will base your analysis on. This
is, as the term implies, your unit of analysis. Also, in the process of analyzing your data, you
may shift from one unit to another. In the study by Gimbel and associates (2002) mentioned
under organizations as the unit of analysis, the authors used organizations as the unit of
analysis. However, since they collected data from employees, they could have undertaken
other analyses as well. For example, they might have become interested in the level of morale
among employees and the length of time each worked for one of the organizations. In this
case, they would have shifted to employees as the unit of analysis. Our point is simple: just be
sure of your unit of analysis and that you have data appropriate to that unit.
Collecting data at the lowest possible unit
Also, remember an important rule: Regardless of what you are studying, always obtain data in
terms of the lowest unit of analysis possible. There is a simple reason for this. You can
usually combine data collected at a lower level into a higher level for analysis, but it won't
work the other way around. For example, if you intended to analyze data about the production
of things by households, collect data about the production activities for each member of the
household. Then you can combine the data from each household member for analysis at the
level of the entire household. This way you can still do other analyses at the level of members

Chapter 5 Page 5
of the household, such as what wives and daughters produced in contrast to husbands and
sons, as well as for the entire household. But if you asked only for production of the
household as a whole, you would not be able to describe production by separate members of
the household.
Design alternatives
Designing research is a creative process. Designs can be put together in a number of ways. By
describing some of
the more frequently used designs, we hope you will get an idea of the various kinds of
research questions you can ask and designs you can use in answering them. To start, designs
can be developed for: (1) describing variables as they currently exist (present time); (2) for
describing changes that have occurred (looking at the past); and (3) for describing changes as
they occur (present into the future). These time dimensions are illustrated in Figure 5.1. We
start with the typical survey design, shown by Illustration A in Figure 5.1.
PREV NEXT

Chapter 5 Page 6
Design and Cross-sectional designs

purpose
Surveys are designed to describe some set of variables as they exist at the time of data
of the collection. Data are collected from a sample over a relatively short period of time, shown as
research T1 in Figure 5.1. The design used for a survey is also called a cross-sectional design because
the sample is carefully selected to represent the cross section of some population. Either
Quantitative quantitative or qualitative techniques can be used to collect survey data, although, in general,
and quantitative techniques are used. The frequently cited Sudan Fertility Survey was based on a
qualitative survey or cross-sectional design.
data
The survey design can be expanded to include more than one sample, as we will show shortly
as part of the discussion of seeking differences between groups with respect to some variable.
Design
alternatives Trend designs
Designs for trend studies are used to measure changes that may have occurred in one or more
Aids variables. The common design for trend studies is illustrated by B in Figure 5.1. Data for the
same variable are obtained on two or more occasions, as illustrated by T1 for the time of the
first set of data, T2 for the time of the second set, and so on to Tn, where the subscript n
represents the time of the last set of measurements for the variable. Trend studies also can be
based on use of data collected in earlier surveys with data from a current survey, as
represented by C in Figure 5.1. T1, T2 and T1 represent data collected at three points in the
past while Tn, in this case, would represent data collected in a survey, which would provide a
measurement of a variable at the present time. All four measurements could be used to
establish a trend in a variable from points in the past to the present time.
For a variable to show a change from one time to the next, new conditions have to arise that
can have some effect on the variable. The length of time required for any change to occur will
vary, depending on the variables being investigated and the prevailing social conditions. Time
intervals may be as short as weeks or months or, more often, extend over many years.
Knowledge of the variables under investigation will help in deciding whether sufficient time
has passed for changes to occur. This knowledge has to be matched against the time periods
for which data are available. When using official statistics, the researcher has no choice: One
has to use whatever is reported.

Chapter 5 Page 6
T1 represents the time of the first data collection; T2 represents the second; and T3 the third.
The dots (...) represent additional possible measurements. Tn stands for the last time of data
collection.
Figure 5.1. Illustration of time as an element in research designs
PREV NEXT

Chapter 5 Page 7
Design and Badri and Burchinal (1985) were fortunate in finding data over a period of 27 years for
purpose enrollment of boys and girls at the elementary, intermediate, and secondary levels for most of
of the northern Sudan. Enrollment data were available for 1956, 1967, 1977, and 1982/83 from the
research Ministry of Education. They constructed ratios showing the proportion of girls enrolled
relative to boys at each level of schooling for each time period. A low ratio indicated that few
females were enrolled relative to males; a higher number indicated female enrollment was
Quantitative
moving closer to that for males. A ratio of 1.00 would indicate that the same percentages of
and girls as boys were enrolled at each level. For simplicity, ratios are presented only for the
qualitative secondary level. In 1956, the first year for which data were available, the ratio was 0.18; it
data rose to 0.23 in 1967; further to 0.47 in 1977; and still further to 0.61 in 1982/83. With four
points spread out over 27 years, Badri and Burchinal could confidently conclude that
Design enrollment for females relative to males had increased substantially over the 27 years for
alternatives which data were available. A clear linear or straight-line trend was established.
But what could have been concluded if data were available for only 1956 and 1982/83? With
Aids just these data, the authors could have concluded that a significant rise in female/male
enrollment ratios had occurred over the 27 year period. But with only two points, the question
of year-to-year changes could not have been addressed. Enrollment ratios could have
declined over some years within the 27-year span To avoid these kinds of problems,
researchers try to get data for as many points in time as possible. This strengthens the basis of
analysis and gives greater support to any conclusions that are drawn.
Longitudinal designs
Social scientists also study changes in variables as they occur. These investigations, called
longitudinal studies, are based on data collection at a given point in time, followed by
successive additional waves of data collection at future times. The right side of Figure 5.1
illustrates this design. Time intervals between data collection may vary from weeks, to
months or even years, depending on the purpose of the study and the money or staff available
for data collection.
In Figure 5.1, illustration D represents the general pattern for a longitudinal investigation.
The investigator starts with data collected at some current or present time (T1) and then
collects additional data at successive times in the future, represented by T2) through Tn).
Three widely used longitudinal designs are described next.
Panel designs. A powerful way of conducting a longitudinal study is to collect data at
successive times from the same sample. The time periods involved can range from weeks to
many years. Kadri and others (2000) studied the irritability of Muslim men during Ramadan.
Their study extended over six weeks, starting with data collection a week before Ramadan,
followed with repeated data collection each week during Ramadan, and ending with final data
collection one week after Ramadan. King and Byerlee (1977) wanted to know how the
spending patterns of rural families in Sierra Leone varied throughout the year. To find out,
they arranged for a sample of farmers to be interviewed twice a week for a year. Some panel
studies extend over years. A study of the effects of malnutrition on children in a Mexican

Chapter 5 Page 7
village, for example, extended over 24 years (Chavez, Martinez and Soberanes, 2000).
Panel designs are particularly useful for detecting changes in variables, but they have the
disadvantage of the loss of respondents over time. As the initial sample gets smaller, it
becomes less like the original sample, and hence comparisons among the waves of data
collection can become less meaningful. When the time period is relatively short, over several
months for example, losses generally are not too great. Over longer periods of time losses in
respondents inevitably occur. Causes include death, illness, moving away, lose interest, or
refuse to cooperate after several periods of data collection. Careful researchers report losses in
samples used in panel designs and discuss the possible effects of the losses on the findings.
Designs based on independent samples. A weaker form of the longitudinal design is based
on comparisons among two or more successive, but new, independent samples from the
population being studied. Researchers are forced to use this design when the population being
investigated is undergoing rapid change. Investigators collecting data on trends in household
composition in camps for displaced persons, for example, face this problem. They know the
population changes daily: new arrivals come, some previous residents leave, and some die. A
panel approach simply would not work: the loss of respondents, even in a month, would be
very large. The alternative is to select a random sample of households at one time and record
the composition of the households in the sample and repeat the same survey with a new
random sample some time later. The comparison of results from the two samples would show
what, if any changes had occurred in the composition of the households.
PREV NEXT

Chapter 5 Page 8
Design and Approximating longitudinal designs. Although longitudinal studies provide the most
purpose trustworthy data about changes in variables, they also have some serious limitations.
of the Longitudinal studies can be expensive and time consuming. Each wave of data collection
research adds to the cost and duration of the study. It is not surprising, therefore, that researchers have
sought ways to measure changes in variables without having to wait and pay for repeated
waves of data collection.
Quantitative
and One method is based on analysis of data from a single survey. A common technique is to
qualitative classify respondents on the basis of some time-related variable, such as their ages and then to
data compare the several age groups on some other variable. The investigators of the Sudan
Fertility Survey used this technique. They compared the fertility rates of younger and older
Design mothers to see if the younger women were having the same approximate number of babies as
their older relatives had when they were the ages of the younger mothers. The results showed
alternatives
no change.
Aids The same procedure can be used to study the effects of any time-related variable, such as
length of residence in some area, length of employment, or years of marriage, on some other
variable.
Hevi-Yobee, Fanslow and Cowan (1986) used a different approach in creating an
approximation of a longitudinal investigation. They compared attitudes of first and fourth
year students at the Ahfad University for Women with regard to working in rural areas,
combining family and professional roles, and their interest in getting involved in political
activities. In general, the fourth-year students expressed more positive views with regard to all
three "modern" roles. The authors concluded that the experience at the Ahfad University for
Women tended to move students toward a more modern perspective. This conclusion,
however, was based on the assumption that three years later, the first year students would
express attitudes similar to those of the current fourth-year students. This kind of assumption
is common to studies based on the approximation to a longitudinal study. Researchers using
this technique should explicitly acknowledge any assumptions they are making and provide
sound reasons for any conclusions they offer.
Comparison designs
Most studies are based on one sample, but some problems call for the use of more than one
sample. To illustrate, Hassan and colleagues (1988) compared attitudes toward male/female
equality for males and females at each of two levels of education. They selected four
independent samples for their investigation. The samples consisted of adult women who had a
minimum of a secondary school education and adult women who had not finished secondary
school. Two similar samples of adult men were also selected. The four samples were
compared for attitudes toward male/female equality. As hypothesized, at either level of
education, females had more positive attitudes; also, both males and females with more
education had more positive attitudes than those with less education. An interesting result was
that level of schooling appeared to have a greater effect on attitudes than gender. In another
study, Darkoh (1994) compared attitudes in gender roles and changes in these roles for

Chapter 5 Page 8
various groups of women from areas in northern Sudan. Comparisons were based on women
from higher-income urban areas, lower-income urban areas, and women from a low-income
rural area. Urban women from high-income areas had the most "modern" versus "traditional"
views; women from rural low-income areas had the most traditional views.
Designs using available data
Designs also differ in the sources and kinds of data used. In most research, the investigator
collects new or original data. Research also can be conducted using data that already exists.
One way to do this is to find and re-analyze data that were collected in a previous survey.
This is what Nouri (1983) did when he re-analyzed the data originally collected in 1978-79
for the Sudan Fertility Survey. Darkoh (1994) also did this when she did additional analyses
of data collected earlier as part of a study by Grotberg and Badri (1986).
Data provided by government ministries are the most frequent source of available data for
studies in developing countries. Examples of these kinds of investigations are:
Analysis of school enrollments by girls and boys (Badri and Burchinal, 1985)
Crime rates in African countries (Arthur, 1998)
Fulfillment of basic social needs of citizens of developing countries (Bulluck,
1986)
Children's health and death rates (Farah, 1981; El-Tigani, 1989)
Factors affecting fertility rates (Osman, 1995)
The contents of newspapers, magazines, or television shows, and other media can be
examined as well to describe current social views or changes in views over time.
PREV NEXT

Chapter 5 Page 9
Design and Experiments

purpose
Experiments are a powerful way of testing hypotheses. In its simplest form, the researcher
of the seeks to learn whether, under carefully controlled conditions, a change in one variable
research produces a hypothesized effect on some other variable. Careful procedures are followed to
rule out, to the extent possible, the influence of anything that could affect the outcome of the
Quantitative experiment.
and
Experiments can be conducted under carefully controlled conditions in a laboratory or in
qualitative
natural social settings. Grotberg and Badri (1986), for example, conducted an experiment to
data test whether the developmental level of young children could be accelerated by teaching their
mothers how to provide additional stimulation for their children. The authors reported
Design evidence of greater development among children whose mothers participated in the
alternatives experiment compared with children whose mothers were not given the special instruction.
Aids Designs based on qualitative data

Qualitative data can be obtained under almost any design, but we will describe briefly only
three designs frequently used with qualitative data. These are field or observational studies,
case studies, and focus groups.
Field or observational studies. Field study methods were developed by anthropologists who
studied isolated and generally preliterate societies. Their techniques have spread to
investigations of behavior in many kinds of social settings, including business organizations,
informal organizations such as clubs, and many other groups with distinct membership and
activities. Data collection is based on a combination of data gathering techniques. The
investigator observes, listens to what people talk about, asks questions, and records what he or
she sees and hears. A detailed record is kept of everything that is observed and discussed. Any
available data are analyzed as well. The investigator continuously reviews his or her data and
seeks to develop an understanding of why the behavior being studied occurs in the way it
does. Analysis is based on the investigator's skill and creativity in organizing and drawing
meaning from the mass of data that is collected.
Case studies. In a case study, the investigator selects a group for intensive study. The case
study method can be applied to a single individual, a community, a region of a country, or to
any organization. A combination of qualitative and quantitative techniques is used to collect
data. These might include observation, informal interviewing, surveys, and analysis of
existing data and records. The objective is to provide a comprehensive, integrated description
of the essential features of the case being studied. While only one or a few cases are studied in
any single investigation, the case study method often produces greater richness and greater
depth of understanding than can be obtained by surveys.
Focus groups. A focus group is a group of persons brought together by an investigator to
learn how the persons feel about some issue or topic. A group of 8 to 10 persons is assembled
by the investigator and asked to discuss a selected topic. The investigator or a trained
facilitator conducts the discussion, which really is a kind of group interview. The topic is

Chapter 5 Page 9
introduced and members of the group are encouraged to voice their views and opinions. The
discussion is recorded, often by a tape recorder, or by hand, and then analyzed. Focus groups
are an excellent way for exploring a topic, testing the meaning of terms or concepts, and for
refining hypotheses.
Other design decisions
We have provided only brief descriptions of some common designs and research techniques.
For further discussion of various kinds of research designs and their features, go to Designing
Designs for Research. Although this site focuses on evaluation of programs, it covers basic
elements of design and illustrates features of many commonly used designs. It also includes
some designs we have not discussed in this chapter.
Two other sets of decisions have to be made in creating a research design. These involve
constructing ways of measuring the variables you want to learn about and selecting the
sample from which data will be obtained. These points are covered in the following two
chapters.
PREV NEXT

Chapter 5 Page 10
Design and Aids

purpose
Internet resources
of the
research Here are just a few of the many sites that provide information and guidance on the design of
research. You may want to examine some of these to get additional ideas on ways to design
Quantitative your research project.
and Research Methods, provides brief descriptions of and links to online books, manuals, and
qualitative guides about research methods
data
Research Methods and Statistics, Centre for Psychology Resources: Athabasca
Design University, presents links to a wide variety of sites dealing with research designs and
alternatives methods, including library research, qualitative and quantitative methods and research
tutorials
Aids Time in Research, distinguishes between cross-sectional and longitudinal studies; notes two
ways of collecting longitudinal data through repeated measurement and as part of time series
collection
Types of Designs, defines and discusses three types of designs randomized experiments,
quasi-experiments, and non-experiments
Types of Questions, presents research questions as one of three kinds: descriptive in purpose,
which is to describe a variable as it currently exist); relational, to measure the relationship
between two variables; or causal (to determine whether one variable, a new focus for a family
planning program, for example, resulted in changes in use of contraceptives)
Key terms
● Case study ● Focus group
● Cross-sectional design ● Longitudinal design
● Descriptive research ● Observational design
● Design ● Qualitative research
● Experimental design ● Quantitative research
● Explanatory research ● Panel design
● Exploratory research ● Trend design
● Field research ● Unit of analysis
Main points
1. A design is the plan you develop for conducting an investigation and for analyzing the
resulting data.
2. Designs have to be appropriate for the purpose of the research. Social research is
conducted to: (1) explore a topic or problem; (2) to describe one or more variables as

Chapter 5 Page 10
accurately as possible; and (3) to explain why certain patterns or relationships occur as
they do.
3. Social research may be quantitative, where observations are represented by numbers; or
qualitative, where observations and relations are described in words. Each approach is
useful for different research purposes.
4. Quantitative designs tend to be fixed and once set are not easily changed, while
qualitative designs allow a flexible approach to data collection.
5. In designing research, investigators have to be clear about the unit or basis for
analyzing the data. Units of analysis may be individuals, groups, geographical areas, or
other entities. The unit of analysis sometimes differs from the unit from whom data are
obtained.
6. Research is a creative process. While many designs have common features, each is
created to answer a specific research question.
7. There are many research designs. Some of the more frequently used ones are:
cross-sectional designs for surveys, designs for observing trends in a variable over
time, longitudinal designs for observing changes in variables as they occur, designs for
using available data, experimental designs, qualitative designs based on unstructured or
informal observation and interviewing, and designs based on the case study approach or
use of focus groups.
8. Many studies use a combination of quantitative and qualitative approaches.
PREV NEXT


Chapter 6 Page 1
Chapter 6: Developing Measures

Overview
Overview
Steps in In quantitative research, developing measures for variables is an important part of preparing a
measurement research design. This chapter describes the steps involved. You will also learn about different
levels of measurement and the requirements for developing measures that the scientific
An illustration community will accept as sound. In addition, this chapter discusses some ethical issues that
may arise during the course of research.
Level of Ordinarily, when we think of measuring something, we think of its length, height, weight, or
measurement speed. Social scientists also use measurement in describing the social characteristics they
investigate. To mention just a few studies, measurements in the form of numbers was
Validity used to describe:
● Changes in ages at marriage (Abdelrahman, 1989)
Reliability
● Differences in fertility rates among groups of women (Abdelrahman, 1995)
● Expressions of attitudes toward female genital mutilation (Mageed, Sulima and

Relationship
Kawther, 2000)
between
● Views on gender roles and professionalism (Hevi-Yoboe and others, 1986)
validity and
reliability Measuring a variable numerically has several advantages. First of all, with numerically
measured variables, the variable can be expressed precisely. An investigation into ages at
Reactivity marriage among males and females in northern Sudan (Abdelrahman, 1989), illustrates this
point. He compared ages at marriage for males and females for the year 1973 with those in
Ethical 1979. Instead of saying that "an increase" had occurred, Abdelrahman indicated precisely how
considerations much older persons were at marriage in 1978/79 compared with 1973. To be precise, the
average age for females increased from 18.6 in 1973 to 21.3 in 1978/79. The corresponding
Aids change for males was from 25.5 to 27.9.
Second, with numerical data, summary measures, such as averages, percentages or other
statistics can be used to summarize a large set of data. This greatly simplifies analysis and
interpretation of results. Thus, Abdelrahman could confidently conclude that ages at marriage
had increased substantially in the time period observed. A third advantage in using numerical
data is that we can use statistical measures to express the strength of relationships among
variables. This is a powerful tool in analyzing data, as we show in Part 4 of this book.
Steps in measurement
Two main steps are involved in developing measures for any concept. First, the investigator
has to specify clearly how a concept is defined. This process is known as conceptualization.
Second, ways of measuring concepts and variables have to be developed. This related process
is called operationalization.
Conceptualization

Chapter 6 Page 1
Concepts frequently have more than one meaning or definition. Personality, for example, is
defined in dozens of ways. Bureaucracy and many other social science concepts also have
multiple meanings. Therefore, when concepts are used, researchers have to define concepts
and show how they are being used in a particular research setting.
You might think that the meaning of the concept of "job satisfaction" is clear: Are workers
satisfied with their work or not? In conceptualizing job satisfaction, Mulinge and Mueller
(1998), however, identified two components or dimensions of job satisfaction. Based on data
from agricultural workers in Kenya, they differentiated between intrinsic and extrinsic
rewards of work. Intrinsic rewards included the satisfaction gained from learning new skills,
doing challenging work, having an opportunity to be creative, and other human satisfactions
of the job. Extrinsic rewards were associated with the conditions of work, such as how easy it
is to get to the place of work, having good hours for work, the pay received, and relative
comfort during working hours. Mulinge and Mueller found different levels of satisfaction,
depending on how workers felt about the relative balance of intrinsic and extrinsic rewards of
their jobs.
PREV NEXT

Chapter 6 Page 2
A second example shows how a seemingly simple concept can really be more complex.
Overview Hassan and colleagues (1988) set out to investigate the concept of male/female equality.
During the conceptualization phase of their research, the researchers identified seven ways
Steps in this concept could be expressed. These were male/female equality with regard to: (1)
measurement education; (2) marital roles; (3) home and child care roles; (4) work roles; (5) social activities;
(6) political participation; and (7) management of money. Researchers refer to the components
An illustration or different aspects of a concept, like the seven for male/female equality, as the dimensions of
the concept.
Level of In conceptualization, investigators draw on definitions and conceptualizations reported in the
measurement literature and then present how they intend to use a concept or one or more of its dimensions.
The next step is to operationalize the concept or one of its dimensions.
Validity
Operationalization
Reliability Operationalization is the process of creating a way of measuring a concept or its dimensions.
This includes selecting a technique of data collection, such as conducting a survey or using
Relationship observation, and developing specific ways of obtaining measures, called indicators, for each
between concept or dimension.
validity and An indicator is a single, observable measure for a concept or one of its dimensions. Any
reliability variable can be used as an indicator. After deciding upon what indicator to use for a concept,
we develop a measure or a measuring instrument for obtaining data for that indicator. Going
Reactivity back to the investigation of attitudes toward male/female equality, Hassan and colleagues
(1988) developed sets of questions for measuring the extent to which respondents expressed
Ethical views in favor of or opposed to each of the seven dimensions of male/female equality. The
considerations combined responses to each set of questions became the measuring instrument for each
dimension. Thus, there was an indicator for attitudes toward equality with regard to education,
Aids based on a set of questions, another for attitudes with respect to marital roles, and so on for the
other five dimensions. The questions used for each dimension of equality became the
operational definition of the concept or dimension. In other words, each dimension was
defined by the way it was measured.
An illustration
Let's move from the abstract description of the process of conceptualization and
operationalization to a concrete example. Measures for a concept or any of its dimensions
may be based on a single indicator or on a set of indicators. Also, more than one indicator can
be used for measuring a variable or one of its dimensions. The following illustration shows
use of single and multiple indicators.
Suppose we asked: "What is the relationship between the social status of men and their
attitudes toward equality between men and women?" We start with two abstract concepts -
social status and attitudes toward male-female equality. Our next step would be to define each
concept so readers of our report will know what we mean when we use each concept.

Chapter 6 Page 2
Conceptualizing social status

Starting with social status, we could use an accepted definition or create one that we think is
better suited for our study. Social status is generally understood to refer to a person's ranking,
from high to low, in terms of status within their community. But this definition contains
several aspects or dimensions of status in relation to one's neighbors. Do we want to refer to
some form of achievement, such as their levels of schooling or the kind of work the men do?
Would the socially accorded status of the men's extended families be a better measure of their
social status? Or, should we think in terms of the power the individual or his family has in the
community? Each of these and perhaps other dimensions might be used.
PREV NEXT

Chapter 6 Page 3
In deciding how to conceptualize social status, we would certainly review the literature on
Overview how social status has been defined and measured in studies similar to the one we are planning.
In reviewing dimensions to measure, we would also consider how practical it would be to
Steps in collect data for any dimension we are considering. Of the dimensions mentioned above,
measurement getting data on the men's levels of schooling or their occupations would be easier than trying
to measure how extended families are ranked or how members of the community view the
An illustration power of various families. Practical considerations in obtaining data frequently decide which
dimensions are selected and how they are measured.
Level of In our case, we decided to focus on social status as unique to each man and one not dependent
measurement on the views given to his extended family or the power of the family in the community. The
process of conceptualizing social status is shown on the left side of Figure 6.1.
Validity
Conceptualizing attitudes
Reliability Identifying dimensions for the concept of male/female equality is a more complex task. No
matter how we define this concept, we would find that it embraces a number of dimensions.
Relationship Hassan and colleagues (1988), you will recall, identified and measured seven dimensions of
between attitudes toward male/female equality. Julia and Ridha (2001) used six dimensions for the
validity and closely related concept of husband-wife decision-making.
reliability The question of how many or which dimensions to measure, illustrates again how practical
considerations enter into research design and the selection of indicators. We have to decide
Reactivity whether the research question we are investigating requires measurement of all possible
dimensions or whether selection of just a few dimensions would be sufficient for our
Ethical purposes. For many investigations, a few well chosen dimensions and their indicators are
considerations enough to demonstrate any association the investigator is seeking. We could, therefore, select
only a few dimensions of male/female equality for the purposes of our study. The choice is
Aids ours, but we would have to provide a reason for choosing whatever dimensions we decided to
measure. Let's say we decided to limit measurement to attitudes with regard to equity in the
right of women to schooling, in marriage relationships, and to work outside the house because
we think these are the most critical ones.
The right side of Figure 6.1 shows the results of conceptualization for the concept of attitudes
toward male/female equity.
Operationalizing concepts
Following conceptualization, our next step is to select indicators for each of the dimensions of
concepts we choose to measure. As shown in Figure 6.1, we decided to use two indicators for
measuring our selected dimension of social status; namely, years of schooling and the status of
the occupations of the respondents. Each is widely recognized as an indicator of social status.
Operationally, we defined each by a single item on a questionnaire; one item asked how many
years of schooling each respondent had completed; the other asked for his main occupation.
For the dimensions of gender equality in schooling and in work roles, we also decided to use

Chapter 6 Page 3
only one indicator, operationalized as a single questionnaire item. Two indicators were
selected for measuring equality in marriage, one based on observation of how much equality is
reflected in the everyday interaction between husbands and wives and the other based on
responses to a set of questions asking for the men's views on how much equality there should
be between husbands and wives in various daily activities, such as raising children, deciding
what to buy, who to visit, etc.
Establishing relationships
The six indicators would become the measures used in the investigation of the relationship
between men's social status and their attitudes toward male/female equity. Responses to the
indicators become the data for the study. These data would be analyzed to establish the degree
of association between each indicator of social status with each indicator of attitudes. With
two indicators for social status and four for measuring different dimensions of attitudes toward
female equity, the analysis would involve eight relationships, as shown in Figure 6.2. Part 4
of this book describes ways these relationships are measured.
PREV NEXT

Chapter 6 Page 4
Overview
Steps in
measurement
An illustration
Level of
measurement
Validity
Reliability
Relationship
between
validity and
reliability
Reactivity
Ethical
considerations
Aids
Figure 6.1 Conceptualization and operationalization

PREV NEXT

Chapter 6 Page 5
Assessing dimensions and indicators

Overview
In selecting indicators, we could use items that had been used in similar studies in North
America, Europe, or elsewhere. While most indicators developed for research in these regions
Steps in
may also be appropriate for research in developing countries, some may not. If the dimensions
measurement and indicators for a concept are not relevant in local terms, any resulting data will be
meaningless. Also, although a concept may be equally valid across a wide range of cultures,
An illustration the operations used to measure it in one cultural setting may be quite different from those used
somewhere else. Social status, a frequently used concept, is a good example. Results from
Level of many studies shows that social differences exist in virtually all societies. The bases used to
measurement assign social rank to persons in any society, however, vary greatly from one society to
another. Any operational definition used for this concept should be appropriate in terms of
Validity definitions of social status in the cultural setting of an investigation. If you aren't sure what
indicators to use for measuring a variable, in this case, social status, conduct an exploratory
Reliability study to find out how individuals are ranked in the cultural setting of your investigation.
Relationship
between
validity and
reliability
Reactivity
Ethical
considerations
Aids
Figure 6.2. Illustrative tests for empirical relationships between two sets of indicators
In our example, we used two indicators for social status and four for measuring attitudes
toward male-female equality. Analysis of data for these indicators would provide independent
tests for relations between each operational definition of social status and each operational
definition of attitudes toward female equality, as shown in Figure 6.2. These data would
provide a much stronger basis for establishing a relationship between the concepts of social
status and attitudes toward male/female equality than would just a single test based on one
indicator for each concept.
In selecting indicators and in developing operational definitions of them, we also have to

Chapter 6 Page 5
consider the level of measurement each indicator provides.

PREV NEXT

Chapter 6 Page 6
Level of measurement
Overview
The attributes comprising indicators and variables differ in levels of measurement.
Specifically, there are four levels of measurement. The lowest level is called nominal
Steps in
measurement; the others in increasing order of the amount of information provided are
measurement ordinal, interval, and ratio measurement. The characteristics of each level of measurement are
shown in Figure 6.3. This figure can also help you decide correctly what level is represented
An illustration by any set of data you may have to analyze.
Level of
measurement
Validity
Reliability
Relationship
between
validity and
reliability
Reactivity
Ethical
considerations
Aids
Figure 6.3. Identifying levels of measurement
Nominal measurement
In deciding about the level of measurement represented by some set of data, first ask: Are the
data in the form of words or are they represented by numbers? Data recorded in words, such
as descriptions of actions, are considered as qualitative: Data in the form of numbers represent
quantitative measurement. Qualitative data may be either in nominal or ordinal form. To
separate the two, one needs to ask a second question: Can the data be ordered on some
criteria? If the answer is "no," the data were measured at the nominal level. Nominal
measurement is shown at the left in Figure 6.3. Nominal comes from the Latin word "name;"
with this level of measurement, the attributes of an indicator are known only in terms of the
names of the categories they represent. Examples are gender, marital status, and nationality.
For this kind of indicator, one can say that each of the characteristics is different and that is
all.
Ordinal measurement

Chapter 6 Page 6
If the data are qualitative but the categories making up the variable can be ordered according
to some criteria, the data are measured at the ordinal level. For example, suppose you asked
respondents to indicate the level of schooling they had completed, using categories such as
"none," "primary," "intermediate," and "secondary or higher." These levels represent ordinal
measurement because respondents can be placed in some kind of rank order, from low to high.
Responses for many questionnaire items are obtained at the ordinal level. Respondents, for
instance, may be asked whether they "strongly agree," "agree," are "uncertain," "disagree" or
"strongly disagree" with some statement. Clearly, there is an order in these categories from
strong positive to strong negative views. In ordinal measurement, each category can be
compared to the others in relative terms. Differences can be expressed in terms of "more" or
"less," which, though still limited, is an improvement over nominal measurement.
PREV NEXT

Chapter 6 Page 7
Interval measurement
Overview
There are two levels of quantitative measurement, depending on the nature of the zero point
on which the numbers are based Interval measurement is based an artificial or arbitrary zero
Steps in
point invented by humans for our convenience. Measurement of temperature, for example, is
measurement based on an interval scale. Although a thermometer shows a zero point, it is artificial. Zero on
either a Celsius or Fahrenheit thermometer does not mean the absence of heat. It can get a lot
An illustration colder than zero as people in the far north know so well. Zero in either case is simply a point
we have all agreed to use to measure what we call temperature.
Level of
measurement Interval measurements are used widely in social research. The intelligence or IQ test is
probably one of the best known of these. When psychologists developed these tests, they fixed
the normal or average intelligence at 100 with scores below 100 indicating progressively less
Validity
intelligence and those above 100 indicating progressively more intelligence. Using IQ scores,
psychologists can estimate a person's chance of doing certain things. A score of 60 would not
Reliability be a good basis for getting through college, while a score of 120 would be, but the two scores
cannot be interpreted to mean that a person with a score of 120 is twice as intelligent as a
Relationship person with a score of 60. The reason is that intelligence is not based on an absolute measure
between of zero intelligence. We can only say that the person with the higher score measured 60 points
validity and higher than the other person in terms of the way intelligence was measured.
reliability
But this statement is a further advance in providing information over that obtained with
ordinal measurement. With interval measurement, a number can be used to describe
Reactivity differences, but the difference has to be described in terms of the intervals or numbers used in
the measurement; hence the name interval for this level of measurement. Many measurements
Ethical in the social sciences are at the interval level. Measurement of attitudes, values, preferences,
considerations prestige, and various abilities, for example, are frequently measured at the interval level.
Aids Ratio measurement

In addition to the features of interval measurement, ratio measurement is based on the unique
feature of starting from an absolute zero point. Many indicators we use have an absolute zero
point. Examples include the size of any population, age, years of schooling, time spent
somewhere, amount of money, and the frequency of events. For each of these and many other
variables, we can compare the size of any measurement with some other measurement of the
same variable and say how much they differ in absolute terms, such as one is twice or four
times larger than the other. A student, for example, may be 20 years old and his or her father
may be 40 years old. We can accurately say that the father is twice as old as the student. In
this instance, we calculated the ratio of the ages (40 divided by 20) and came up with a ratio
of 2. This is possible because we started with an absolute zero for measuring age, hence the
name ratio for this level of measurement.
Level of measurement and research design
The level of measurement used can limit the way data are analyzed. Ratio or interval
measurements can be analyzed by a number of powerful statistical tests. Analyses of nominal

Chapter 6 Page 7
or ordinal data are far more limited. Researchers, therefore, seek to obtain measurement at the
highest possible level for any variable. For indicators in nominal form, there is no choice.
Gender, for example, can only be recorded as male or female. For many variables, researchers
can choose a level of measurement. Data for schooling, for example, can be obtained at the
ordinal or ratio level. If the question about schooling was stated as: "What is the highest level
of schooling you completed" and the response categories were "none," "primary,"
"intermediate," "secondary," "beyond secondary," the resulting measurement would be at the
ordinal level. Analysis of the data would be limited to what could be done with ordinal data.
Instead of asking for the level of schooling completed, respondents could be asked for the last
year completed, from 0 for no schooling to some number representing the total for years of
schooling. This would produce ratio data, which could be analyzed with more precise and
powerful statistical methods.
When a choice exists for levels of measurement and you believe you can get accurate
responses, use interval or ratio over ordinal measurement. Doing so will give you more
freedom in choosing methods of analyzing your data and allow you to use more powerful
analysis techniques.
PREV NEXT

Chapter 6 Page 8
Continuous and discrete variables

Overview
Variables can also be viewed as continuous or discrete. Continuous variables are measured in
terms of smaller and smaller amounts. Age is an example of a continuous variable. We start
Steps in
by measuring age in hours, then days, followed by months, and finally in terms of years. But,
measurement if you wanted to be absolutely precise you could measure age in terms of seconds or tenths or
even small factions of a second. We think you get the point of continuous variation. In
An illustration contrast, a discrete variable changes from one category or level to another without any
in-between steps or increments. Your class level at the university is an example of a discrete
Level of variable. You go from being a first year to a second year student in one jump: university
measurement officials, for example, do not recognize a first year and one-half.
As you will see in Part 4, different analysis techniques are used for continuous versus discrete
Validity
data.
Reliability For an additional view of measuring social characteristics, visit: Measurement, this site
discusses the fundamental ideas involved with measurement in social research
Relationship
between In developing measures for indicators, you also have to be concerned about their validity and
reliability.
validity and
reliability Validity
Validity is the extent to which an indicator or set of indicators actually measure the concept it
Reactivity
represents. When students argue with their professor whether an exam really measured their
knowledge of a subject, they are questioning the validity of the questions making up the exam.
Ethical Did the questions provide a valid measure of the students' knowledge of the course content?
considerations This is the idea behind validity of indicators. For social scientists, an indicator is valid to the
extent that it measures the intended concept or one of its dimensions and not something else.
Aids Schooling, for example, could be a valid indicator of several concepts, such as social status or
achievement, but it probably would not be a valid indicator of happiness. Different indicators
would be needed for measures of happiness.
Validity is never absolutely established. The best researchers can do is to offer evidence for
the validity of measurements they use. Three ways are frequently used to test the validity of
indicators.
Content validity
In making the case for content validity, the investigator shows that the wording of the items
used to measure a concept are clearly related to that concept or one of its dimensions. To
illustrate, in the next chapter we list seven items used to measure attitudes toward wives
working outside the home. (See Box 7.3). One of these items is: "A woman's place is in the
home." Another item is: "Children can be assured of love and guidance even if the mother is
employed." One could reasonably argue that these items can be considered measures of
attitudes toward wives working outside the home.

Chapter 6 Page 8
Some researchers draw a distinction between content and f ace validity. In practice, however,
it is hard to distinguish between the two and the terms are often used interchangeably. Like
content validity, face validity is based on the "face value" of the items or measures. This
assertion, however, is considered weak because it is based only on the judgment of the
researcher. To strengthen claims for content or face validity, other knowledgeable persons can
be asked to judge whether the items represent the concept being measured. Strong agreement
among the ratings of qualified judges can increase validity claims, but stronger evidence for
the validity of a measuring instrument is usually expected.
For many variables, face or content validity is assumed. There is little doubt that most simple
questions, like asking for the gender of persons, their ages, their occupations, and many other
commonly known facts about persons, are valid ways of getting the needed data.
Requirements for establishing validity apply mainly to development of measures of less easily
established variables. These include attitudes, perceptions, beliefs, and many other things
researchers measure.
In presenting data for validity you should, as a minimum, establish the basis for asserting the
content validity of any indicator you use. If a measuring instrument lacks obvious validity,
based on the content of the indicators used, there is no point in proceeding to stronger bases
for establishing the validity of the instrument. Careful researchers, however, do not stop with
just claims for content validity. They seek more convincing evidence for the validity of any
measure they use.
PREV NEXT

Chapter 6 Page 9
Concurrent validity
Overview
A stronger way of testing validity of an indicator is to establish a strong relationship between
the indicator and some external criterion. This approach is known as establishing concurrent
Steps in
validity. It is also referred to as criterion validity because it depends on use of an external
measurement criterion. The criterion is selected because the persons or other units of analysis are known to
differ greatly in terms of the concept being measured. For example, Kridli (1997) wished to
An illustration establish the validity of a set of questions for measuring the intention of Jordanian Muslim
women to use oral contraceptives. One of the validity tests she used was to compare
Level of differences in responses for women who were known to have used oral contraceptives with
measurement those who had not. Kridli reasoned that past users would answer more positively than
non-users. And this is what she found. Kridli properly accepted these results as evidence for
Validity the validity of the indicator for intended use of oral contraceptives.
Predictive validity
Reliability
In testing for predictive validity, the investigator uses a future criterion measure in place of a
Relationship concurrent one. Suppose we wanted to develop a written test for estimating the success in
between college of secondary students based on their performance as secondary students. First, we
validity and would have to decide on indicators of the students' performance at the secondary level. Let's
say we decide to use a combination of the students' knowledge of secondary school subjects
reliability
and their ability to think clearly and logically as indicators of their likely success in college.
We have the students take a test that includes measures of both indicators. To test the validity
Reactivity of our measuring instrument for predicting success in college, we decide to use the students'
accumulated grade point average at the end of their first year in college. These averages
Ethical become the criterion for testing the validity of the "college success" measure based on
considerations secondary school performance. A statistical test would then be used to measure the
relationship between the results of the tests given at the secondary school level and the
Aids accumulated grade point averages of the students at the end of their first year of college. A
high relationship between the two sets of scores would provide evidence that the measure of
secondary school performance is a valid predictor of success in college, as measured by one
criterion - the students' accumulated grade point averages.
Interpreting validity tests
When the expected results are obtained, an indicator can be considered as a valid measure.
But how should the data be interpreted when the expected results are not found? What if
Kridli (1997) had found that the past users and non-users of oral contraceptives had given
about the same responses to the test she was developing? What if the accumulated grade point
averages of college students were about the same for those who had low as opposed to those
who had high accumulated grade point averages a the end of their first year in college? When
validity fails to produce the expected results, investigators have to sort out the reason why.
Negative results can occur for four reasons. These include:
1. selection of a poor criterion for establishing validity;

Chapter 6 Page 9
2. poor measurement of the indicator;

3. poor measurement of the criterion; or
4. any combination of these errors.
When negative results occur, the researcher has to seek new criteria for validating the
indicator, seek improved measurement of indicator and the criterion, or do all these things.
There is a lot more to establishing claims for the validity of a measure. If you want to dig
deeper, take a look at Measurement Validity Type. This site provides an alternative way of
thinking about validity. Validity is presented in terms of how well a concept is translated into
an operational measure. If you speak another language, you know there are good, bad, and
very bad translations: so too with establishing claims for validity. In effect, how well did the
researcher achieve a translation of a concept or construct into an operational measure?
PREV NEXT

Chapter 6 Page 10
Reliability
Overview
Reliability is the extent to which any measuring procedure gives similar results with repeated
use with the same respondents. In actual practice, we do not expect to get exactly the same
Steps in
results with repeated use, but we do expect to see a high degree of consistency from one use of
measurement a measuring instrument to the next. Persons scoring high one time would be expected to score
high again, give or take a little. Those scoring low or in between would be expected to score
An illustration about the same way on repeated measurement. Generally, a coefficient of correlation is used
to establish the reliability of a measuring instrument (see Chapter 19). Several methods can
Level of be used to test for reliability.
measurement
Test-retest method
Validity An obvious way to test for consistency is to get responses for an indicator on one occasion and
then at a second time. The same sample and the same measure are used each time. A test of
Reliability the relationship between the two sets of scores is used to establish the degree of consistency or
reliability of the measuring instrument. Returning to our earlier example about attitudes
Relationship toward male/female equality, we could ask a sample of couples to answer a set of questions
between about equality in marriage and then return several weeks later and have them respond to the
validity and same questions again.
reliability While easy to use, the test-retest method has several limitations. Many respondents may think
answering the same questions a second time is foolish. Some may even refuse. For those who
Reactivity do cooperate a second time, some may consciously try to respond the way they did the first
time. This could create greater consistency in responses than is actually true. Another problem
Ethical is that some event may occur between the two times of measurement that could influence how
considerations they respond the second time. In our illustration of attitudes toward male/female equity in
marriage, for example, some crisis in a family could change how a husband or wife would
Aids answer. This danger increases as the time interval between measurements increases. To get
around this limitation, another method is sometimes used.
Equivalent forms method
In this method, the investigator develops two equivalent forms of the measuring instrument.
The items are combined in one set and respondents respond to all the items at the same time.
A statistical test is generally used to estimate the degree of consistency in answering items
making up each form of the measure.
The equivalent forms method, also known as the split-half method, has the advantage of ruling
out the effects of the first measurement, such as memory or the influence of external events,
both of which affect the validity of the test-retest method. This method, however, has three
disadvantages. First, the researcher has to develop twice as many carefully constructed and
tested items. Most of us have difficulty developing one set of useable items, let alone two.
Second, there is always some doubt about how equivalent any two forms are. To protect
against their obvious interest in claiming equivalence, investigators sometimes ask a panel of
judges to rate the equivalence of items. Using judges certainly strengthens claims for this

Chapter 6 Page 10
approach, but still relies on judgment instead of any external empirical evidence. Third,
research has shown that different reliability results are obtained depending on how the two
halves are created. Another method, described next, overcomes these limitations.
Chronbach's alpha
Many researchers use a method introduced by Chronbach (1951) in calculating the reliability
of a set of items. This method requires understanding how to use and calculate correlation
coefficients (see Chapter 19). Chronbach's alpha is based on calculating the correlation
coefficient for relationships between each item and every other item in the set making up a
measuring instrument. Even with a small number of items, this method involves a number of
calculations. For instance, for a scale with 10 items, 45 inter-item correlations would have to
be calculated. If you don't have access to a statistical package on a personal computer,
calculations of the inter-item relations will require a large amount of work. With a statistical
package, the work is greatly reduced.
PREV NEXT

Chapter 6 Page 11
Once the inter-item correlation coefficients are found, the remaining steps are easy. The mean
Overview for all the correlation coefficients is found by adding the values for all the coefficients and
dividing by the number of coefficients. The result, called alpha (?, a Greek letter) is put into
Steps in the following formula for the calculation of alpha, the Chronbach measure of reliability.
measurement Assume that our calculation of the mean coefficient of correlation was .55, which would
represent a moderate degree of relationship, on the average, between each item and every
An illustration other item in the scale. Putting the value of .55 for ? and the value of 10 for the number of
items into Chronbach's formula, we get:
Level of
measurement n(?) 10(.55) 5.50 5.50 5.50
alpha = ________ = ___________ = ________ = _______ = ______ = 0.92
1+?(n-1) 1+ .55(10-1) 1+ .55(9) 1 + 4.95 5.95
Validity
Like a positive correlation coefficient, alpha varies between 0 and 1.00. Higher values indicate
Reliability greater reliability. In this example, an alpha of 0.92 would indicate a high estimate of
reliability.
Relationship
For an additional discussion of reliability, visit Types of Reliability. This discussion covers
between
all ways of establishing reliability and provides a technical and precise definition of reliability
validity and based on statistical operations.
reliability
Relationship between validity and reliability
Reactivity We have presented validity and reliability as separate concepts so that you understand each by
itself. In research, we want measuring instruments that are high on both validity and
Ethical reliability. But in practice, we may find any of four relationships between validity and
considerations reliability:
● High reliability, but low validity the indicators measure something consistently but not
Aids the intended concept;
● High validity, but low reliability the indicator represents the concept well but does not
produce consistent measurements;
● Low validity and low reliability the worst case; the indicators neither measure the
concept nor produce consistent results of whatever they measure; or
● High validity and high reliability what we hope for; the indicators consistently
measure what they were intended to measure.
When poor validity or reliability results occur, a researcher has to seek the source of error,
revise the measuring instrument as appropriate, and test again for validity and reliability.
Reactivity
Reactivity arises when the method used to obtain data affects the measurement of some
variable. This problem is particularly acute in the social sciences because humans can and do
react to being observed or being asked questions. Knowing they are being observed, some

Chapter 6 Page 11
people will react by becoming self conscious and even change their behavior to look good in
the eyes of the observer. In a survey, for example, respondents may give socially approved
answers instead of truthful ones for certain questions. In both instances, invalid and probably
unreliable data would be obtained which would put any findings from the investigation in
doubt. Any conclusions from the study would probably be invalid as well.
PREV NEXT
The number of pairs is found by using the formula: pairs=n(n-1)/2. With 10 items, the
calculation is: 10(10-1)/2 = 10(9)/2 =90/2 = 45.

Chapter 6 Page 12
Data collection methods vary in reactivity. Some can be highly reactive, while others are
Overview nonreactive. Data collection methods that are nonreactive are also called unobtrusive
measures because they do not interfere with or intrude into the process of obtaining data. As
Steps in we discuss each of the main techniques for collecting data, we will note how reactive each is.
measurement For more extensive discussion of this useful and generally inexpensive way of collecting data,
go to:
An illustration
Unobtrusive" Data Collection, Unobtrusive Measurement
Level of
You want to develop measures that meet all three criteria for a good measurement. Any
measurement measure you use or develop should:
● Accurately represent the concept you want to measure (validity);
Validity
● Provide consistent data (reliability); and
Reliability ● Does not affect the data obtained (reactivity).
Ethical considerations
Relationship
between In this concluding section, we will look at some of the ethical issues that can occur in
validity and research. Guidelines for conducting research ethically are listed in Box 6.1. One is whether the
reliability researcher is ever justified in collecting data without the consent of those being studied.
Box 6.1. Important ethical guidelines
Reactivity
1. Researchers are expected to obtain informed consent from
respondents or subjects. Collecting data in public places
Ethical without informing respondents remains an issue.
considerations
2. Participation in any study should be voluntary and based on
the consent of the participants.
Aids
3. Investigators should avoid doing any harm to participants in
the research. This includes causing undue embarrassment,
humiliation, or stress.
4. Investigators are responsible for protecting the privacy of
information obtained during research.
5. Investigators are honor bound to report all aspects of a
research project fully and accurately.
6. Many ethical issues can be avoided by anticipating them and
by explicitly discussing them with participants, sponsors, and
others who have an interest in the investigation.
Obtaining informed consent

Is it ethical to collect data without telling the persons involved what you are doing so? Some
researchers say that no research should be conducted in secret. Respondents, they say, should

Chapter 6 Page 12
always be informed about the nature of the study and give their consent before any data are
collected. This viewpoint is based on the individual's right to privacy, which, in this view, is
more important than the need to conduct research regardless of how important the results
might be. According to this view, research should only be done when the respondents freely
surrender their right to privacy and agree to participate in the research. Other researchers take
a somewhat different view. They agree that reporting on many kinds of behavior, such as the
private lives of people in their homes without their knowledge and consent, is unethical.
But according to the second viewpoint, observation without knowledge and consent does not
violate persons' rights if several conditions exist.
Three conditions are cited by some researches as legitimate bases for conducting research
without obtaining informed consent. These are:
● That the behavior is public when persons know they are being seen or heard by others
and know the behavior is public knowledge;
● That no demands are made on the people and their behavior occurs naturally in a public
setting; and
● That the observation in no way alters the well being of the people.
Examples where these criteria might apply include observation of behavior in streets, at
market places, in classrooms, or in public meetings. However, remember that collection of
data without informing persons about the study and getting their consent remains an issue.
PREV NEXT

Chapter 6 Page 13
Voluntary participation only

Overview
Researchers agree that participation in a study should be voluntary: no one should be forced to
participate in a study against his or her will. Trying to force persons to participate in a study is
Steps in
not only unethical; it is also shortsighted and dumb. Uncooperative respondents are also more
measurement likely to give invalid and unreliable responses.
An illustration Avoiding harm
Another point researchers agree on is that no harm should come to any participants in a study.
Level of How, you might ask, could participation in social research hurt anyone? Physical harm is
measurement unlikely, but psychological damage in the form of lowered self respect or embarrassment can
occur.
Validity
Studies that focus on unpleasant experiences, such as death, circumcision, divorce, or on
Reliability sensitive areas, such as family planning, sometimes cause embarrassment or unpleasantness.
In these situations, the researcher should have a strong justification for any study that can
cause these kinds of reactions and should seek to minimize possible harm.
Relationship
between Protecting participants
validity and
As a researcher you also have an ethical responsibility to protect the privacy of persons who
reliability supply data. First, you should guarantee either confidentiality or anonymity to respondents;
second, you are responsible for seeing that whatever level of privacy is promised is actually
Reactivity carried out. Confidentially means that while the researcher may know persons' responses, he
or she promises to hold these in confidence and not make them available to anyone else. This
Ethical is the usual case when a survey is conducted. Questionnaires can be matched to individual
considerations respondents, but the researcher guarantees that no one else will be able to do this. Anonymity
means that not even the researcher is able to match questionnaires or other data with persons
Aids who provided the data. Once all the questionnaires or other data are collected, anonymity can
be achieved by removing all identification numbers, names, addresses, or other information
that could identify respondents. Assuring respondents of anonymity or confidentiality at the
beginning of an interview helps gain their cooperation and helps ensure obtaining valid and
reliable data.
Accuracy of reporting
Another set of ethical issues sometimes arises at the end of a study. These concern what is
reported about a group that was studied. When results do not please sponsors or participants in
a study, they may try to get the researcher to present the results more to their liking. If caught
in this kind of a situation, you have three choices:
1. You can stick to your findings and present them clearly and honestly and be prepared to
accept whatever consequences may occur;
2. You can decide not to publish anything rather than falsify the results in some way; or
3. You can try to negotiate the differences while preserving the basic scientific integrity of

Chapter 6 Page 13
your research.
A better course of action is to think ahead and try to avoid this kind of situation.
PREV NEXT

Chapter 6 Page 14
Avoiding ethical problems

Overview
You can do at least two things to head off or minimize ethical problems. First, you should
identify possible ethical problems before they arise and then be prepared for them and face
Steps in
them honestly when they occur. Examine your plans for data collection. What ethical issues
measurement could arise? For any issue you see, think about the alternative courses of action you could take
to avoid any conflict that might occur. Have a course of action ready rather just letting things
An illustration happen.
Level of Second, have a frank discussion about issues you see with anyone who has claim on your
measurement research, such as your advisor, persons who helped you gain entry to the group you are
studying, or local leaders whose help you will need to do the study. Try to get clear, explicit
agreement about the freedom you will have in collecting and analyzing data and in reporting
Validity
what was found. By bringing an issue out in the open, you can prepare people for possibly
different outcomes than they might expect and help them accept these findings. This course of
Reliability action can also save you and others a lot of grief.
Relationship A number of professional and governmental organizations have issued ethical guidelines for
between research involving humans. We mention only the following examples of codes of conduct for
validity and social research:
reliability Ethical Principles of Psychologists and Code of Conduct, American Psychological
Association, provides the code of ethics of the American Psychological Association
Reactivity
American Sociological Association Code of Ethics, provides the code of conduce of the
Ethical American Sociological Association
considerations The Canadian Code of Conduct for Research Involving Humans, provides the Canadian
code of conduce for research involving humans
Aids
Ethics in Research, discusses ethical issues that arise as researchers seek to establish the
validity of evaluation results
Examining these ethical codes of conduct may help you in thinking about ethical issues you
might face in your research.
Aids
Internet resources
In addition to Web sites mentioned earlier in this chapter, we call your attention to several that
provide alternative explanations of topics in this chapter.
Measurement, Reliability, and Validity, discusses conceptualization, operationalization,
unit of analysis, and reliability and validity
Overview: Reliability and Validity, presents an alternative description of reliability and

Chapter 6 Page 14
validity and ways to estimate each. This site also contains commentary on the difficulty of
achieving reliability, defines key terms related to reliability and validity, and provides an
annotated bibliography on reliability and validity issues.
PREV NEXT

Chapter 6 Page 15
Key terms
Overview
● Anonymity ● Measure
Steps in ● Chronbach's alpha ● Measuring instrument
measurement ● Concptualization ● Nominal measurement
● Concurrent validity ● Nonreactive measurement
An illustration ● Confidentiality ● Operational definition
● Content validity ● Operationalization
Level of
● Continuous variable ● Ordinal measurement
measurement
● Criterion validity ● Predictive validity
Validity ● Dimension ● Ratio measurement
● Discrete Variable ● Reactivity
Reliability ● Equivalent forms ● Reliability
● Method of Reliability ● Split-half reliability
Relationship
● Face validity ● Test-retest reliability
between
● Indicator ● Unobtrusive measurement
validity and
reliability ● Interval Measurement ● Validity
● Level of Measurement
Reactivity
Main points
Ethical 1. Measurement is based on the inter-related processes of conceptualization and
considerations operationalization. Conceptualization is the process of defining concepts and specifying
and defining any selected components or dimensions of concepts. Operationalization is
Aids the process of developing measurements for concepts or their dimensions.
2. Indicators are variables used to measure concepts or their dimensions. An indicator
must have two or more attributes.
3. All indicators should be carefully analyzed to be sure they are appropriate in the
cultural setting in which they will be used.
4. When possible, it is a good idea to use two or more indicators for measuring any
concept or dimension.
5. There are four levels of measurement: nominal measurement indicates only that
attributes differ in some way, as males vs. females; ordinal measurement indicates some
order or ranking of attributes (high, middle and low social status, for example); interval
measurement is based on numbers, but without an absolute zero point (temperature,
intelligence); and ratio measurement is also numeric, but with a real or absolute zero
point (age, size of a population).
6. Always use the highest level of measurement possible for any indicator. This will
permit more powerful analyses of the resulting data.

Chapter 6 Page 15
7. A valid indicator measures the intended concept and not something else. Ways of
assessing the validity of an indicator include face or content validity, concurrent or
criterion validity, and predictive validity.
8. A reliable measure provides consistent results from one time to another. Ways of testing
the reliability of a measuring instrument include the test-retest method, the split-half or
equivalent forms method, and use of Chronbach's alpha.
9. Reactivity refers to the extent to which the way data are collected affects the data
obtained. Some methods of data collection are more reactive than others. Researchers
seek to minimize reactivity effects in the collection of data.
10. Investigators are responsible for adhering to ethical standards in collecting and
analyzing data and in how results are reported.
11. Two basic ethical guidelines are: (1) participation in research must be voluntary; and (2)
participants should not be harmed.


Chapter 7 Page 1
Chapter 7: Constructing Composite Measures: Scales and Indexes

Introduction
Introduction
Scales In previous chapters you became familiar with the language of research; particularly, the use of
indicators and their attributes for measuring the dimensions of concepts. These discussions were
Likert Scales based on the use of individual items, as in questionnaires. In this chapter, we expand the ways
researchers measure indicators by showing how responses to sets of items can be combined to
Bogardus form composite measures of indicators.
Social Composite measures are based on a simple idea: A number of indicators provide a better measure
Distance of a concept than a single one. In composite measurement, responses to two or more items are
Scales combined to provide a single score or measure for the indicator.
Guttman Every exam you have taken is a kind of composite measure. On your last exam, suppose you
answered 18 of 20 questions correctly. Your instructor gave you a grade by assigning 1 point for
Scales
each correct answer and 0 for incorrect answers and then added the correct points to get your
grade of 18 on the exam. The 20 questions formed a composite measure of your knowledge of the
Indices material covered in that course. Your grade of 18 was your score on the exam. Throughout this
chapter, you will learn how this logic is applied in the development of more complex composite
Additional measures.
Considerations
Kinds of composite measures
Typologies Scales and indexes are the main forms of composite measures. Each is based on assigning
numbers to responses to the items making up the index or scale. These numbers are added
Measurement together to generate the score for the composite measure. The score is then used in place of the
Error individual responses.
The terms scales and indexes are frequently used interchangeably. This causes some confusion.
Aids
In general, scales are used for measuring the intensity of feelings, attitudes, or beliefs of persons.
Indexes are used to measure variables such as socio-economic status or various health or other
social indicators.
Essential features
Scales and indexes are alike in several important ways. Each provides measurements at the
ordinal, interval, or ratio levels, depending on the level of measurement used for the items.
Construction of indexes and scales is also based on similar requirements. First, all the items
making up the index or scale should measure the same thing. In technical terms, the items should
be unidimensional or one-dimensional in content. Going back to the exam illustration, you would
expect all the test items to be related to material covered in your course. Questions about the
content of other courses would violate the unidimensional requirement, and you would rightly
object to this error. In addition, responses to the items used in a scale or index have to be
mutually exclusive and exhaustive. Mutually exclusive means that respondents can give one and
only answer; exhaustive means that response categories must allow for all possible answers.

Chapter 7 Page 1
PREV NEXT

Chapter 7 Page 2
Why use composite measures?

Introduction The answer to this question requires a little foundation. Social scientists do research to establish
valid, reliable relationships among concepts. These relationships form the basis of social science
Scales knowledge. Establishing a relationship between two concepts, however, depends on establishing
an empirical relationship among the indicators used to represent the concepts. Establishing valid,
Likert Scales reliable empirical relationships depends upon obtaining a valid, reliable, and precise measure for
each indicator. Years of research experience have shown that composite measures generally
Bogardus provide the best way to get good measurements of indicators.
Social We can use the exam illustration again to show how composite measures improve measurement
Distance of indicators. The 20 questions were selected by your instructor to measure your knowledge of
Scales the course material. Instead of 20 questions, would you want your grade to be based on one
question: Pass or fail? You would certainly answer "No" to this question. Your instructor might
Guttman ask the one question that you could not answer. You would much prefer having a number of
Scales questions. Increasing the number of questions provides a better measure of all the questions you
were prepared to answer.
Indices This applies as well to the construction of measures of attitudes, beliefs, behavior, and other
variables. Attitudes toward various religious, ethnic, racial, or other groups, for example, can
Additional range from extremely positive or highly negative. A single question or item seldom captures a
Considerations person's attitude toward some group. Using a number of items, each of which probes a different
aspect of relations with members of that group, gives a more valid and precise measure of where
Typologies a person's attitude falls on the continuum from one extreme to another.
Increasing the number of items also improves measurement by allowing for greater variability in
Measurement
the scores obtained. With additional items, scores will have a broader range, from favorable to
Error unfavorable, for example, or from high to low numerically. Using the exam illustration again,
scores on the exam could range from 0 to 20 instead of just from 0 to 1. Greater variability also
Aids improves measurement of validity and reliability. In the case of the exam, both you and your
instructor have a more valid estimate of your knowledge based on 20 questions instead of just 1
question. The same is true for measurement of variables.
Box 7.1 lists the advantages of using composite measures.
Box 7.1. Advantages of using composite measures
1. Provides more accurate measures of indicators
2. Increases the validity and reliability of indicators
3. Increases the variability of measurements for
indicators
Composite measures aid in analysis of data in other ways. Your grade point average illustrates
this advantage of composite measures. A grade point average or its equivalent is the mean (the
sum of all grades divided by the number of grades) of all grades received so far. Each final grade
for each course is the score for that course. Having the scores in numerical form allows your

Chapter 7 Page 2
university to calculate and adjust grade point average as each new grade is added. Similarly, in
social research composite measures can be used in a variety of statistical analyses, giving clearer
and more powerful results.
Scales
Scales are used widely in social research, mainly to measure variables that vary greatly in degree
or intensity. A few of the thousands of recent studies based on composite measures include:
Alrareb's (1997) investigation of attitudes toward Muslims; Kridli's (1997) investigation of plans
to use oral contraceptives; Fattah's (1981) study of husband-wife decision-making among Iraqi
farm families; and the investigation by Hassan and colleagues (1988) of attitudes toward
male/female equality.
These studies were based on the use of scales, consisting of sets of statements with responses
ranging from one extreme, such as "strongly agree" to the other extreme, such as "strongly
disagree." This technique was developed by Renis Likert in the 1930s and is widely used in social
research.
PREV NEXT

Chapter 7 Page 3
Likert scales
Introduction
Using existing scales. With thousands of Likert scales (Likert, 1932) reported in the research
literature, it is tempting to see a scale and decide to use it for measuring some indicator instead of
Scales
developing a new one. Your decision depends on several factors. First, as the investigator, you
will need to show that the scale, including the statements it contains and the response categories,
Likert Scales are appropriate in terms of the culture of the group being studied. Will the respondents you intend
to collect data from understand the statements? Do the statements have meaning in terms of the
Bogardus local culture? Would any of the statements raise serious social, political, religious, or other
Social issues? Would a statement be seen as foolish?
Distance
In research terms, you would be testing of content validity of the items. Applying these kinds of
Scales
criteria may lead you to abandon use of an existing scale, but in the process, you may see ways of
adapting at least some the statements and response categories for use in your setting. This is what
Guttman Tessler (1987) did in a study of religious conservatism and opinions about the effectiveness of the
Scales government in serving citizens in Tunisia. Scales for both measures existed in the research
literature but were not suitable for use with the population he was investigating. He modified the
Indices scales and tested them for validity and reliability with Tunisian samples before using the scales in
his investigation (Tessler, 1987, pp. 141-144).
Additional
Second, in using an existing scale, a researcher has to demonstrate convincingly that the scale is a
Considerations
valid and reliable measure for the variable being studied. Evidence offered for validity and
reliability by the developer of the scale is not sufficient. A scale developed in Europe or North
Typologies America, for example, may be valid and reliable for use there, but not in other regions of the
world. Therefore, the first requirement is to establish that the scale is valid and reliable for its
Measurement intended use. Any of the commonly used methods described in Chapter 6 can be used for this
Error purpose.
Aids Whether you modify an existing Likert scale or construct a new one, the steps are the same. As
with every measuring instrument, the first step is to analyze the concept being studied to
determine if it has one or more dimensions and to identify and describe the dimensions you
intend to measure. Construction of a scale starts with the selection of statements to serve as
indicators of a concept or one of its dimensions.
Selecting statements. Scales are based on a number of items, usually in the form of statements
that express on opinion or view regarding the variable being measured. Statements can be taken
from a variety of sources. As mentioned before, existing scales can be a source of statements.
You may discover these in the course of reviewing the literature. Brief exploratory research,
based on interviews with persons who have different views on the topic you are researching, can
produce a range of useful statements. Focus groups are a good source of items (Altareb, 1997).
Statements may also be based on comments you hear or opinions expressed in newspaper or
magazine articles. You can also draw on your own experience in preparing statements.
It is a good idea to assemble as many statements as you can. Only clearly favorable or
unfavorable statements about the issue in question should be used. Statements should also

Chapter 7 Page 3
represent different levels of feeling about the issues. Some questions should express mild positive
or negative views, while other should express increasingly stronger views in each direction. Do
not use neutral statements. They will not differentiate between persons with different views on
whatever you are measuring. A few other guidelines for writing items are listed in Box 7.2.
Using these guidelines, examine the statements you have assembled. Eliminate questions that are
not clear, do not express a clear positive or negative opinion, or that are too much alike. At this
point, it is also a good idea to check to make sure each statement reflects some aspect of the
dimension you want to measure. As a further check, ask several fellow students or informed
persons to evaluate the items for unidmensionality and for face or content validity. Each reviewer
could be asked to rate each question as "very appropriate," "appropriate," "inappropriate" or
"completely inappropriate." Depending on the kind of items being tested, another set of judging
categories could be used. Keep only the items that a majority rated as clearly appropriate.
PREV NEXT

Chapter 7 Page 4
Box 7.2. Some guidelines for writing Likert items

Introduction
● Write the statements as simply as possible.
● Use language and words that your respondents will understand.

Scales
● Express the statements in everyday language of the
Likert Scales respondents.

● Reverse the direction of questions, with some expressing a
Bogardus positive or favorable view and others a negative or unfavorable

Social opinion.
Distance ● Include statements expressing a range of opinions from
Scales relatively mild to strong positive and negative views.

● Mix up statements with positive and negative views in any list
Guttman of statements.
Scales
Also, you will need to include statements that show positive as well negative views. This is done
Indices to minimize errors arising from the tendency of respondents to answer a set of items in pretty
much the same way. If all the statements are stated in a positive fashion, like items 1, 4, and 5 in
Additional Box 7.3, showing only positive views with regard to women working outside the home, many
respondents tend to give the same response after a few items. To avoid this tendency, known as
Considerations
response set, positive and negative items are mixed up in any list of statements, forcing
respondents to think about their responses to each item. The best way to see how this is done is to
Typologies follow through an illustration.
Measurement Illustration of Likert scale construction
Error To illustrate the process so far, let's say we wanted to create a scale for measuring attitudes
toward employment of married women outside the home. After reviewing a number of items, we
Aids selected those items listed in Box 7.3
Box 7.3. Initial items for measuring attitudes toward employment

of wives outside the home
1. A working wife feels useful.
X
(4)SA (3)A (2)Un (1)D (0)SD
3
3. An employed wife cannot bring children up properly.
X
(0)SA (1)A (2) Un (3)D (4) SD
4
4. Today, wives have to work to help support the family
. X

Chapter 7 Page 4
(4)SA (3)A (2) Un (1)D (0)SD

3
5. Children can be assured of love and guidance even if the mother is
employed
X
(4)SA (3)A (2)Un (1)D (0)SD
4
6. Taking care of the home and children is all a wife needs to feel
important.
X
(0)SA (1)A (2)Un (3)D (4)SD
3
7. It is better if the husband works and the wife stays home.
X
(0)SA (1)A (2)Un (3)D (4)SD
3
Tentative score
23
PREV NEXT

Chapter 7 Page 5
Items and responses. Likert scales require a minimum of two response categories, such as
Introduction "agree" or "disagree." Generally, though, three or five categories are used. Using five categories
gives a greater range of choices for respondents and increases the range or variability of total
Scales scores for the scale. But there is a limit to the number of categories one might use. Research has
shown that five or seven categories will produce the best data and that increasing the number of
Likert Scales categories above seven confuses some respondents. As you see in Box 7.3, we decided to use five
commonly used response categories. We used SA for strongly agree, A for agree, Un for
Bogardus uncertain, D for disagree, and SD for strongly disagree.
Social Scoring. Each category is given a numerical value, which becomes the score for that statement.
Distance The process of assigning values to attributes of indicators is called scoring. For items with five
Scales response categories, the categories are given assigned values from 0 to 4 or from 1 to 5. Either set
of numbers can be used. Values assigned to the responses, however, have to be adjusted
Guttman depending on whether the item expresses a favorable or unfavorable viewpoint. The set of items
Scales in Box 7.3 includes both favorable and unfavorable items for measuring attitudes toward wives
working outside the home. Item 1 gives a favorable view and the response, "strong agree," is
given a value of 4 (or 5). Item 2 presents an unfavorable view. To be consistent, "strongly
Indices
disagree" is given a value of 4 (or 5). And so on with the remaining items. Switching scoring,
depending on whether the item is favorable or unfavorable, allows us to add scores for each item
Additional to get a total score for all items.
Considerations
In scoring, a decision has to be made whether to give the most favorable responses the high or
Typologies low item score. For the set of items in Box 7.3, we gave the highest score (4) to the most
favorable or positive responses and 0 to the most negative ones. By scoring this way, high total
Measurement scores indicate high positive views. In the illustration (Box 7.3) the total or composite scale was
23, rather high in relation to a possible high score of 28 (7 times 4 for the most favorable
Error
response to each item). Results then can be easily interpreted: Higher scores will represent higher
levels of whatever is being measured. Presenting results in this way will also help readers of your
Aids report quickly understand what you are reporting.
Deriving initial composite scores. In Box 7.3, we show the values assigned to each response
category so you see how scores are assigned. When used in a questionnaire, however, scores for
items would not be shown. Seeing the numbers, some respondents may try to answer to get what
they think is a high or low score. The interviewer or respondent would see only empty brackets in
front of each response category.
Item analysis. Initial total scores are only tentative scores because another step, called an item
analysis, is necessary to select the items to be included in the final index. Either of two ways can
be used. One way relies on calculating the coefficient of correlation between the scores for each
item and the composite scores. Unless you have access to a computer with a statistical analysis
package this method will require doing a large number of tedious calculations. The other method
is based on comparing the mean (the average) or median responses for each item for respondents
who had the lowest and highest composite scores. These two extreme groups are usually defined
as the lowest and highest 25% of the composite scores.

Chapter 7 Page 5
As a way of illustration, imagine that 125 persons answered the 7 items listed in Box 7.3 and
their total scores assumed the distribution shown in Table 7.1. In this figure, "X" stands for the
values of the total scores, which range from a low of 5 to a high of 24. The letter "f" stands for
the frequency or how many times each score occurred. With this distribution, we want to find the
lowest 25% and the highest 25% of the scores.
Twenty five percent of 125 is 31.25 which can be rounded to 31. So, we want to select the lowest
and highest 31 scores. To find the lowest 31 scores, we add frequencies for the scores beginning
with the lowest one, which was 5, and continue until we reach or come as close to 31 as we can.
Adding down the frequency column we get an accumulated frequency of 22 at the score of 15.
When we add the frequency of 11 for the score of 16 we get an accumulated frequency of 33.
This is more than we want, but is as close as we can get to 31. Repeating this process from the
high end of the distribution, we keep adding frequencies, beginning with the frequency of 2 for
the score of 24, the highest score, to the frequencies for each of the next lower scores. At the
score of 21, we have an accumulated frequency of 29. When we add the frequency for the next
lower score of 20, we would go way over the desired number of 31. Therefore, we use the score
of 21 as the cut off point for the high scoring group. Thus, the lowest scoring group includes
scores from 5 through 16 and the highest scoring group scores from 21 through 24.
PREV NEXT

Chapter 7 Page 6
Table 7.1. Frequency distribution of total scores for 125 respondents

Introduction
X f
Scales 5 1
7 2
Likert Scales 10 2
13 3
Bogardus 14 6
Social 15 8
16 11 9- adding down from the low end,
Distance we get 33
17 13
Scales 18 15
19 17
Guttman 20 18 9- adding up from the high end,
Scales 21 13 we get 29
22 10
Indices 23 4
24 2
Additional Total 125
Considerations
After these groups are identified, means or medians are calculated for the responses for each item
Typologies for each group. The mean (or what is commonly known as the average) is the sum of scores
divided by the number of scores added. The median is the middle score in any set of numbers;
Measurement half of the numbers or scores are above the median score, half are below it. Calculation of the
Error mean and median are described in detail in Chapter 17.
For our item analysis, let's use the mean for responses to the 7 items for highest and lowest
Aids scoring groups. Illustrative mean scores for each item for the two groups are shown in Table 7.2
Next, we compare the means (or it could be medians) between the low and high groups for each
item. A larger difference between the means indicates that the responses to that item
differentiate between the low and high scorers. Little or no difference indicates that the items
failed to discriminate between the two groups. Using these guidelines we would select items
having the largest differences between the means for low and high scoring groups.
In our example, item 5, shows little difference between the two means. Respondents having low
or high composite scores gave about the same response to this item. Also, the difference for item
4 is a lot smaller than differences for other items. Since neither would add much to the total
scores, both are eliminated.
Table 7.2. Means for the lowest 25% and highest 25%
Groups

Chapter 7 Page 6
Items High Difference

Low 25% 25%
1 2.15 3.50 1.35
2 0.95 3.17 1.22
3 1.00 3.58 2.58
4 1.44 2.25 0.81
5 2.53 2.91 0.38
6 1.63 3.25 1.62
7 0.87 3.47 2.60
Larger mean differences are found for the remaining items. Deciding on what is "small" or
"large" rests on judgment, unless you want to do a statistical test for the significance of the
difference between the means. (This test is explained in Chapter 19). Statistical tests, however,
are not necessary and require a considerable amount of tedious calculating. Judgment, based on
the sizes of differences between sets of means, generally is sufficient. In making decisions,
remember that the means are based on small numbers. In our example the raw data for
calculating the means ranged from 0 to 4. Thus, the difference of 1.22 for item 2, the smallest
difference we accepted, may seem small, but it is quite large when considered in terms of the 0 to
4 range. The mean of 0.95 for the low scoring group was just under the score for disagree, which
was scored as 1, while the mean for the high scoring group of 3.17 was just over the score of
agree, which was scored as 3. Item 2, therefore, would be considered a strong item.
PREV NEXT

Chapter 7 Page 7
Final Composite Scores. After deciding which items to keep, the scores for the remaining items
Introduction are added to get the final composite score for each respondent. In our example we would have
retained items 1, 2, 3, 6 and 7. For the illustrative respondent, whose scores were shown in Box
Scales 7.3, item scores would be 3+3+4+3+3 and the person's total score would be 16. Each
questionnaire would be scored in the same way for each respondent. Based on these five items,
Likert Scales composite scores could range from 0 to 20.
For another illustration of Likert scaling, go to Likert Scaling.
Bogardus
Social Bogardus social distance scales
Distance Imagine that you wanted to find out the attitudes of people in your city toward Europeans or
Scales some other group of foreigners. Instead of developing a Likert scale, you could ask persons a
series of questions about how closely they are willing to associate with Europeans, from the least
Guttman possible contact to more intimate interaction. One set of questions could be:
Scales 1. Are you willing to let Europeans live in our country?
Yes • No
Indices 2. Are you willing to let Europeans live in your village, town or city?
Yes • No
Additional
3. Are you willing to let Europeans live in your neighborhood?
Considerations Yes • No
4. Are you willing to let Europeans live next to you?
Typologies
Yes • No
Measurement 5. Are you willing to let your children play with European children?
Yes • No
Error
6. Would be willing to let a child of yours marry a European?
Aids Yes • No
This form of scaling is named after Emory Bogardus who developed this form of measurement in
the 1930s (Bogardus, 1933). Items are selected and ordered to measure the social distance
respondents want to keep from members of some designated group. Social distance, in effect,
becomes a measure of how willing persons are to associate with members of the group in
question. Social distance is measured by presenting a series of statements, as illustrated with
attitudes towards Europeans. The reasoning behind the approach is that persons who are
uncomfortable or refuse contact at a specified social distance will also refuse closer contact.
Constructing a social distance scale follows the same process as described for Likert scales, but
with several modifications. If you decide to develop a social distance scale, you have to do the
proper conceptual analysis, identify what you intend to measure, and write a series of culturally
appropriate and clear questions. Usually 5 to 10 items are sufficient for measuring social
distance. Once written, the questions are presented to each respondent in order of increasing
contact. Questions are asked until the respondent answers "no." The last "yes" is taken as the
respondent's willingness to interact with members of the group in question. It is assumed that

Chapter 7 Page 7
once a "no" is given, the respondent would answer no to all following items.
Generally social distance scales are scored as 1 for each "yes" response and 0 for each "no"
response. Scale scores, therefore, vary from 0 for those who would not have even the least
association with member of some group to a number equal to the number of items answered as
"yes." Progressively higher scores indicate greater acceptance and presumably more favorable
attitudes toward the group in question.
PREV NEXT

Chapter 7 Page 8
Guttman scales
Introduction
Guttman (Guttman, 1980) scaling is also based on the principle that the composite score indicates
how the items making up the scale were answered. The following example will show what we
Scales
mean. To keep our example simple, we will use only three items for measuring attitudes toward
wives working outside the home and use only "agree" or "disagree" as responses. The items are:
Likert Scales
1. It is all right for married women without children to work outside the home.
• Agree • Disagree
Bogardus
Social 2. It is all right for a mother with older children to work outside the home.
Distance
Scales 3. It is all right for a mother with young children to work outside the home.
Guttman These items were selected because the responses were expected to form a predicted pattern. The
Scales greatest amount of agreement was expected for item 1, the least for item 3, with item 2 being in
between. Scores of 1 for agreement and 0 for disagreement are assigned to each item. Composite
Indices scores, therefore, could vary from 0, when respondents disagree with all three items, to 3, when
they agree with all three items. The top half of Table 7.3 shows the responses that would fit the
Additional expected pattern.
Considerations The first four patterns, which represent the predicted patterns of responses, are known as scale
types. For each scale type a respondent's composite score says exactly which items the
Typologies respondent agreed with. The bottom half of the table shows patterns of responses that deviate
from the predicted patterns. Each of these four mixed types involves a degree of error. No matter
Measurement how the items are scored, the composite score will not faithfully describe which items the
Error respondent agreed with. For mixed types, we first have to assign a number, called an index score,
which comes the closest to describing the response pattern to the items used. Notice that an index
Aids score of 1 for mixed types could be obtained under two different response patterns. You can see
this for the first and third mixed types. The same is true for an index score of 2 for the second
and fourth mixed types, but with scale types, the score of 1 and 2 are just as clear as the score of
0 or 3.
Table 7.3. Illustrative scaling response patterns
Scale types Responses Score
Agree with all 3 items 111 3
Agree with items 1 and 2 110 2
Agree with only item 1 100 1
Disagree with all 3 items 000 0

Chapter 7 Page 8
Index Index
Responses score score
010 1 2
Mixed types,
101 2 3
do not fit the
expected
pattern 001 1 0
011 2 3
In developing a composite score, the scores for the scale types are simply added, but scoring the
mixed types presents some problems. No matter how they are scored, some error will result. Our
objective is to minimize the amount of error that will occur. To achieve this goal, mixed types are
first scored by adding the "yes" responses. Scores assigned in this way are shown under the
heading "index score" in Table 7.3. The index scores are then converted to scale scores, but in
such a way as to minimize the amount of error that is involved. For the mixed type of 0-1-0, a
scale score of 1 would imply an actual response pattern of 1-0-0. Giving a score of 1 for the first
mixed type would involve 2 errors, 0 to 1 for the first item and 1 and 0 for the second item. But if
we assume the 1 response for the second item really meant a 1 response for the first item also,
then we could assign a score of 2 to this mixed type. In making this assumption we are creating
an error of 1, but this is lower than the error of 2 that would have occurred if both items had been
scored as 1.
Using the same reasoning, we would assign a score of 3 to the mixed type of 1-0-1. From the
pattern of the three responses we could assume that the middle response should have been 1 also.
The next mixed type, 0-0-1, is assigned a score of 0. We assume from the pattern that the last 1
response was in error: So, we maintain an error of 1, which is the smallest amount possible. The
last mixed type is scored as 3 for the same reason as given for the second type.
After scoring all the mixed types, we determine the percentage of all responses that fit the
predicted scale types. The number for all responses is found by multiplying the number of
respondents by the number of items used. Assuming that we had a sample of 150 respondents, the
number for all responses would be 150(3) or 450 responses. Now we need to find how many of
these responses fit the expected pattern by being a scale type. This percentage is called the
coefficient of reproducibility and is found by the following formula:
PREV NEXT

Chapter 7 Page 9
1-number of errors
Coefficient of
Introduction =
reproducibility
(number of respondents times the (number of items)
Scales
We know the denominator in the fraction, 450, but we need to determine the number of errors
that occurred. To get this number, we count how many times each mixed type occurred and then
Likert Scales
add these frequencies to get the total number of errors that occurred. To complete our calculation,
we will use the illustrative frequencies for mixed types shown in Table 7.4, which we created for
Bogardus this illustration of scoring. The frequency of the mixed types of 1-1-0 was 10. Frequencies for
Social other mixed types were 5, 4, and 2.
Distance
Scales Table 7.4 Frequencies for mixed types
Mixed Types Frequency
Guttman
110 10
Scales
101 5
Indices 001 4
Additional 011 2
Considerations Total 21
Typologies A perfect scale is represented by a coefficient of reproducibility of 100%. But such scales seldom
occur: almost all sets of items result in some mixed types and coefficients of less than 100%.
Measurement Researchers accept this fact and have set a standard of 90% as the minimum level of acceptable
Error reproducibility: In other words, about 10% of mixed types is considered acceptable.
Three final comments are necessary in concluding this discussion of Guttman scaling. First, scale
Aids types exist only in terms of a specific set of responses from a specific group of respondents. This
means that you have to be careful about using items that formed a scale for some other sample.
The same items may or may not form a scale with your sample. You will have to collect the
necessary data and test the items to see whether the responses meet the criteria we have just
described. Second, Guttman scaling becomes more difficult as the number of items and the
number of responses used increases. Both can contribute to a larger number of mixed types which
lowers the coefficient of reproducibility. If you plan to try to form Guttman scales, we suggest
using a limited number of items and the fewest possible response categories. Third, because of
the way Guttman scales are constructed, researchers generally do not report reliability data for
them. Sometimes validity data are not reported as well. We recommend, however, that validity
data be obtained and reported to strengthen any conclusions you will present. Finally, we
probably do not have to tell you that scoring mixed types can be difficult.
For another discussion of Guttman scaling, go to: Guttman Scaling.
Social scientists have developed all kinds of scales. It is not surprising, therefore, that hundreds
of Web sites on scaling and scales exist. We refer you to only one of these sites. For an

Chapter 7 Page 9
alternative and, in some ways different approach to the development of scales, visit Scaling, a
link from this page will take you to General Issues in Scaling.
Looking at these sites should help you grasp the basic idea behind scaling and perhaps give you
some ideas for use of scaling in your research.
PREV NEXT

Chapter 7 Page 10
Indexes
Introduction
Definition
Scales Indexes are based on multiple items, each of which taps a dimension of some variable. Unlike
scales, which are based on items of varying intensity, items used in indexes are treated as having
Likert Scales the same intensity. Each taps a different aspect or component of a variable.
Development of an index follows the same process described earlier for construction of scales,
Bogardus
beginning with the definition of the concept to be measured. As with scale construction, the next
Social step is to select indicators for use in constructing the index.
Distance
Scales Selecting indicators
An example should help in understanding how an index is constructed. Imagine that we wanted
Guttman to develop an index for socio-economic status of families. We define the concept as it is
Scales commonly used; as the generally perceived ranking of families from high to low within the
community. We further decide that we will use three indicators of socio-economic status: (1)
Indices years of schooling of the male heads of households; (2) the occupations of the heads; and (3) the
kind of material used in the construction of their houses.
Additional
Other indicators could have been used. We might, for example, have decided to use the square
Considerations meters of floor space in houses or the wealth of the heads of households. For practical reasons,
we decide to use the kind of building material instead of floor space. An interviewer can easily
Typologies see and record what kind of material the house is made of, whereas getting floor space would
require a lot of questions or some form of measurement. By using what the house is made of, we
Measurement also avoid imposing further on respondents. Also, we realize that getting accurate data on wealth
Error will be difficult. Respondents will resent being asked about their wealth and probably will refuse
to answer or will give false replies. Asking about schooling and occupation is less threatening
Aids and are facts that most persons will provide.
Also, in selecting items, we also try to select those that will produce the greatest amount of
variation in scores. As we said earlier, variation is necessary, for without variation there is no
point in using an indicator. In building an index, therefore, look for indicators for which there is
considerable variation. Our three indicators, we think, will provide wide variation.
We also have to demonstrate that the selected indicators are valid measures of socio-economic
status. Unidmensionality and validity can be established in several ways. One approach is to
argue that the items are logical or reasonable measures of the concept and that concept only. Such
claims of face or logical validity can be made by the researcher, but it is better to produce
stronger support by having other competent persons review the items. Agreement by a panel of
reviewers strengthens claims for face validity. (Other ways of estimating validity are described in
Chapter 6). Let's say we did this and found that all judges agreed that the three indicators were,
in fact, reasonable indicators of socio-economic status. We would then be ready to develop the
index. This begins with writing the three indicators we have chosen.

Chapter 7 Page 10
Writing items
We describe how to write items in Chapter 10, as part of questionnaire development, so we
won't go into detail here. Let's say we are using a questionnaire to collect our data and decided to
use the following three items:
1. What is the main occupation of the head of this household?
2. How many years of schooling did the head of this household complete?
3. The house is made of:
❍ combination of cardboard, cloth, tin, etc.
❍ mud bricks
❍ concrete blocks
❍ fired bricks
❍ other, describe: _______________________
The first two items are open-ended; the interviewer would record what the respondents gave as
their answers. The third item includes four response categories with an open-ended response.
Together, these categories meet the criteria of being mutually exclusive, but we added the
category of "other" to ensure that the attributes of house construction would also meet the
criterion of exhaustiveness. We might be fairly sure the four categories cover all the kinds of
house construction, but we added the "other" just in case some other material might be found. On
the blank line following "other," the interviewer would record a description of any additional
form of house construction. The three items would then be included in our questionnaire to
collect data for constructing the index.
Identifying attributes
Our next decision would be to assign numerical values to each attribute or category making up
each indicator. To achieve greater variability, researchers generally use three to five categories
for an indicator. In our illustration, however, for simplicity, we use only two categories for each
variable. These categories are simply described as "low" or "high" for each item. We would
examine each occupation and place each in a high or low category, based on ratings of social
prestige of occupations in the community where we are doing our study. A panel of judges could
be used to establish the breaking point between low and high prestige occupations. A dividing
point would be established for years of schooling as well. We might combine the first two
attributes for house construction to form the low category and the other two to form the high
attribute for this indicator. "Other" responses would be placed in either category: depending on
how similar they are to one category or the other.
PREV NEXT

Chapter 7 Page 11
Testing responses to items

Introduction
After the data are collected, we need to conduct an item analysis to see the extent to which
responses to each item are related to responses to the other items. If each is a valid indicator of
Scales
socio-economic status, each item will show at least a moderate relationship with the other items.
If any item was not related to the other items, this would suggest that the item did not measure
Likert Scales the same thing as the rest of the items. On the other hand, a very high relationship between any
two items indicates that one item would do as well as the other and that both are not needed. In
Bogardus that case, we can delete one of the two items. A moderate relationship - not too high or not too
Social low - suggests that the items are tapping a common dimension and that each will contribute
Distance independently to the total index score.
Scales
As a beginning researcher, you may be confused by the vague descriptions of "too high," "too
low" or "moderate." After you read Chapter 19, particularly our description of correlation
Guttman
analysis, you will have a better idea of the relative degrees of association between two variables.
Scales Right now, we want you to get the idea of testing items to see if they form an index.
Indices The results of our item analysis are shown in Tables 7.5, 7.6, and 7.7. Before examining the
results in the tables, we need to explain how they are constructed. (Chapter 18 provides further
Additional details on constructing tables). Table 7.5 shows the relationship between schooling and
Considerations occupation; Table 7.6 reports results for the association of schooling with house type; and Table
7.7 provides the relationship between house type and occupation. Each table has parts, called
Typologies cells. Using Table 7.5, for example, the four cells represent the logical combinations of low and
high levels on each of the two variables. The cell in the upper left shows the number (N) of men
with both a low level of schooling and a low occupational level. The cell in the lower left
Measurement
indicates how many men had a low level of schooling but who were classified as having a high
Error occupational level. In Table 7.5, 112 men were classified as having a low level of schooling and
a low level of occupation. The other 38 men with a low level of schooling were classified as
Aids having high-level occupations. The right set of cells show frequencies for men with a higher level
of schooling, but who differed in terms of occupational levels.
Table 7.5. Item analysis for schooling and occupation
Schooling
Low High Total
Occupation N % N % N %
Low 112 75 18 35 130 65
High 38 25 32 65 70 35
Total 150 100 50 100 200 100
In addition, each cell includes a percentage column. Table 7.5 shows that the 112 men in the
low-low cell represented 75% of all men with a low level of schooling. Totals for cells are shown

Chapter 7 Page 11
at the bottom and on the right side of each table. Going down the columns, the Ns add to the total
number for the indicator shown on the top line. In Table 7.5, the total for all men with a low level
of schooling was 150. Another 50 men had a high level of schooling. The corresponding
percentages add to 100. The total column to the right shows the total for occupation, the other
indicator in the relationship. Of the 200 men in the total sample, 130 were classified as having a
low-level occupation and 70 were placed in the high-level. The other two tables are constructed
and "read" in the same way.
Now, we turn to the results of the item analyses. Table 7.5 shows a clear relationship between
levels of schooling and occupations. Seventy five percent (75%) of the men with a low level of
schooling also were classified as having a low-level occupation. A mismatch on these two
indicators occurred for 25 % of the men. Sixty five persons (65%) of the men with a high level of
schooling also were classified as having a high level occupation. The mismatch in this case was
35%. Clearly, the two indicators are related. For a large majority of men, a low level of schooling
was associated with a low level of occupation. Most men with a high level of schooling also had
high occupational levels.
As shown in Table 7.6, schooling and house type were also associated, although not as strongly
as for occupation and schooling. Sixty three percent (63%) of the men with a low level of
schooling also lived in what were defined as low-level houses. For men with a high level of
education, 60% lived in residences classified as high level.
PREV NEXT

Chapter 7 Page 12
Table 7.6. Item analysis for schooling and house type

Introduction
Schooling
Scales House Low High Total
type N % N % N %
Likert Scales
Low 95 63 20 40 115 58
Bogardus High 55 37 30 60 85 42
Social
Total 150 100 50 100 200 100
Distance
Scales House type and occupation, Table 7.7, also showed a substantial relationship. Seventy six
percent (76%) of the men with a low level of schooling lived in houses defined as low. A
Guttman majority, 64% of the men in the high occupational category lived in houses defined as high.
Scales
Table 7.7. Item analysis for house type and occupation
Indices Schooling
Low High Total
Additional Occupation
Considerations N % N % N %
Low 110 76 20 36 130 65
Typologies
High 35 24 35 64 70 35
Measurement Total 145 100 55 100 200 100
Error
These results tell us that each item is related to the other items. We can safely assume, therefore,
Aids that all three items are independent but related measures of socio-economic status. We can also
assume that each item will contribute to the composite score in a consistent manner. If an item is
not associated with other items, it should be deleted because it would not add to the composite
score.
Scoring items
After you have selected the best items for an index, the next step is to assign a score or number to
each response for each item. In our example, we would need to assign scores for the low and high
categories for each of the three items. The usual practice is to assign a 0 to the lowest category, 1
for the next higher category and so on for the remaining categories. Because we had only two
categories, the low category for each item was assigned a score of 0 and the high category a score
of 1. The composite score for socio-economic status, therefore, could vary from 0 for respondents
in the low categories for all three items to 3 for those in the high categories for all three items.
Scores of 1 and 2 would occur for any mixture of low and high categories. The resulting
composite scores then would be used in subsequent analyses as the measure for social status.
Additional considerations

Chapter 7 Page 12
Index construction also involves a decision about whether or not to give added importance, called
weighting, to certain indicators. Decisions also have to made about missing data and how to test
for validity and reliability. These considerations affect development of scales as well as indexes.
Weighting indicators
An unweighted index is one in which the indicators are of equal importance or, in research terms,
are given the same weight. Scores for the indicators are added to get the index score. Baring a
strong reason for assigning different weights to indicators, we suggest you use equal weights in
any index you construct.
A weighted index, on the other hand, is based on giving some indicators greater importance or
value relative to others. This is done by multiplying the score of items selected for weighting by
some number. The size of the number may be based on knowledge about the importance of one
indicator over others or on basis of statistical analysis. Going back to the construction of a
socio-economic index, if we had reason to believe that occupation was far more important than
years of schooling or house construction, we might decide to give occupation greater weight by
multiplying scores for occupation by 2 or perhaps 3. Then the total range of scores for the
weighted index would vary from 0 to 4 (if the weight of 2 was used) or 5 (assuming a weight of
3).
PREV NEXT

Chapter 7 Page 13
Handling missing data

Introduction
Missing data is a problem in all quantitative investigations. In any survey, for example, some
respondents fail to answer some items. The best defense against this problem is to take all
Scales
possible steps to get valid responses to all items from all respondents, but some omissions are
inevitable. What do you do in these situations?
Likert Scales
There is no satisfactory answer to this problem, but there are several ways of dealing with it. One
Bogardus solution is simply to eliminate any cases for which data are missing for some indicator. If you
Social have to drop too many cases, however, this should be a warning that your data are an insufficient
Distance basis for constructing an index. Another solution is to make an "educated" estimate about what
the missing value should be. In our example, knowledge of the community might suggest a basis
Scales
for supplying a value for the missing data. Still another solution is to create a substitute for the
missing data. You can find the average of the values for the indicator for which there is missing
Guttman data and substitute this value for any missing response. Using the average will have the least
Scales distorting effect on the remaining data.
Indices The problem of missing data becomes more serious when you are using only a few indicators and
have 100 or fewer cases. When you have less data, any loss affects the final results much more
Additional than when you have a number of indicators and a large number of cases.
Considerations Validation of composite measures
Typologies The item analysis we described earlier also provides a form of internal validity. When the items
show moderate association with each other, we have some evidence that they are valid measures
of the dimension or variable being measured. External validity tests provide an additional and
Measurement
stronger basis for demonstrating the validity of an index. One of the techniques described in
Error Chapter 6 should be used to estimate the validity of a composite measure.
Aids Typologies
A typology is a form of composite measurement, but typologies differ from scales and indexes in
two important ways. First, while scales or indexes are unidimensional, meaning they measure
only one dimension of a variable, typologies are frequently multidimensional: They are based on
a combination of different characteristics or variables. Researchers use typologies when they
want to describe persons, groups, or other units of analysis in more than one dimension at the
same time.
Swanjord (1988) created four categories for describing how Kuwaiti women were portrayed in
publications over a sixty year period, from the early 1920s through the early 1980s. Each
category was defined in terms of a number of attributes writers used in describing Kuwaiti
women. The four categories or types were: traditional, fundamentalist, secular, and modern. Each
represented an abstract type of woman as described in various publications. Together, the four
types formed a typology for analyzing the contents of the publications. Swanjord found that
portrayals of the women were predominately fundamentalist or traditional from 1920 through
1980 and shifted to secular portrayals in the 1980s. The shift reflected the social changes in

Chapter 7 Page 13
Kuwait following the large increase in revenues from the sale of oil.
Noting that the value of women's work in developing countries is often overlooked and under
valued, Donahoe (1999) set out to more fully measure the value of their work. In the course of
her research with a sample of Egyptian households, she developed a five-point typology of
women's work. She classified women as: (1) employees, employed in some work not in the
family or with relatives; (2) income generators, employed in some familial arrangement; (3)
providers, working at least 14 hours per week in subsistence production; (4) part time providers,
same as providers, but working less than 14 hours per week; and (5) housewives, doing none of
the above. Donahoe argued that her typology accounted for a larger amount of the productive
work done by women than is generally recognized.
Typologies are useful for measuring independent variables. In Swanjord's investigation, the
typology for images of Kuwaiti women was used to analyze the trend in images used to describe
women and other variables. Donahoe's typology can be used as a nominal variable for the study
of women's work in any developing country. Because they are usually multidimensional,
typologies generally are not be used as dependent variables.
PREV NEXT

Chapter 7 Page 14
Measurement error
Introduction
The true value is what we would like to obtain for any variable we measure: What we get,
however, is the observed value. The difference between the two is caused by some combination
Scales
of possible systematic error and some amount of random error. Systematic error refers to any
bias or errors affecting all the data obtained. A few of many sources of systematic error are:
Likert Scales having invalid or unreliable measures, poor interviewing skills, personal bias of the researcher, or
consistent errors in scoring items in a scale or index. Random error includes any error that occurs
Bogardus at a specific time, such an interruption or misunderstanding during a particular interview but not
Social in others or an occasional mistake in scoring an item or in adding item scores. The relation
Distance among these three components is shown by the following formula:
Scales Observed value = a score's true value + systematic error + random error
Guttman Obviously, we want to avoid any systematic errors and minimize the inevitable random errors.
We can avoid or mimimize possible systematic errors by developing a detailed, sound research
Scales
design and having it reviewed by others to identify any bias or potential systematic error. Clear
evidence for validity and reliability of measures is a further safeguard against systematic errors.
Indices Careful attention to each research task and rechecking every action will protect against random
errors.
Additional
Considerations For a more extensive discussion of measurement error, go to:
Measurement Error
Typologies
The Role of Measurement Error
Measurement
Aids
Error
Key terms
Aids
● Cell ● Response set
● Coefficient of reproducibility ● Scale
● Composite measure ● Scale types
● External validity ● Scoring
● Index ● Social distance scales
● Index score ● Systematic error
● Internal validity ● True value
● Item ● Typology
● Item analysis ● Unweighted index
● Likert scale ● Weighted index
● Mixed types ● Weighting indicators
● Observed value

Chapter 7 Page 14
Main points
1. Composite measures are based on a set of indicators. The main forms of composite scores
are scales and indexes.
2. Scales and indexes are composed of sets of indicators, each of which is given a score or
value. The total score for the scale or index is the sum of the item scores.
3. Composite measures provide more valid, reliable, and precise measurement for many
variables than can be obtained from use of single items.
4. Scales provide measures of the intensity of responses to a set of items. Frequently used
forms of scales are the Likert, social distance, and Guttman scales.
5. Indexes are based on multiple indicators, each of which provides a separate measure of a
dimension of the variable being measured.
6. Indicators in scales and indexes are represented by items or statements. Criteria for
selection of items includes: demonstrating their face or content validity and their
unidimensionality and providing evidence that each item contributes significantly to the
total score.
7. Items in a scale or index are scored by assigning numbers to each of the response
categories for the items.
8. Items making up a scale or index should be empirically related to one another. This shows
that the items are indicators of the same variable or one of its specific dimensions.
9. Scoring items includes making a decision whether to give greater importance or weight to
certain items or to leave them all unweighted.
10. Scoring items also requires deciding how to handle missing data. This can be done by
dropping cases with missing data; making an "educated" estimate of what the missing
value should be; or by assigning the average value in place of the missing data.
11. A typology is a classification system based on two or more characteristics. Typologies
produce nominal measurement and should be used only as independent variables.
12. Any measurement involves two kinds of errors: systematic error, which affects every
measurement taken for some indicator (poorly phrased questions or bad interviewing
style); and random error (something that occurs in a particular setting only, like a mistake
in recording a single response). Investigators seek to minimize both forms of errors so that
any measurement for any indicator is as close to its true value as possible.


Chapter 8 Page 1
Chapter 8: Selecting a Sample

Introduction
Introduction
Probability The purpose of most research is to learn something about some large group, called the
sampling population. In the Sudan Fertility Survey, for example, the objective was to learn about the
fertility for the population of all married women living in northern Sudan. One way to learn
Simple something about a population is to collect data from all the members of the population. This is
random known as taking a census or enumerating the population. Generally, because of the time and
sample cost involved, it is impractical to enumerate a population. The only practical alternative is to
obtain data from a part of the population, called a sample.
Systematic In designing your research you will have to decide whether to enumerate the population you
random are studying or to use a sample.
sample
Enumeration versus sampling
Stratified Enumeration has one great advantage over sampling. For smaller populations, enumeration
random produces accurate descriptions of variables.If you collected data on absence from work for all
sample workers in a government ministry for a certain period of time and every worker gave valid,
accurate responses, the results would accurately describe absenteeism for that population of
Cluster workers. However, if you used a sample, even with valid and reliable responses, the results
for absenteeism would describe the rate of absence for the part of the population from whom
sampling
you obtained data. Another sample would produce a different rate of absenteeism because
every sample contains a certain error based on the variations that occur from one sample to
Creativity in the next. In Chapter 19 we explain why this occurs. With this apparent disadvantage, why is
sampling most social research based on samples?
Weighted Why sample?

samples The main reason for sampling is that many populations are simply too large to enumerate.
Collecting data from a population of thousands or millions requires a huge planning effort and
Problems to a large staff of interviewers, supervisors, and data processing personnel. Months or even years
watch for in would be required and would be expensive. With a sample, data can be obtained in a short
sampling period of time and at a reasonable cost. All the planning and administrative requirements are
greatly reduced as well.
Nonprobability
Similar practical considerations will probably determine whether you enumerate or use a
sampling sample. If the population you are studying is rather small and you can collect data from all its
members relatively easily, by all means do so. Then, you will not have to deal with the
Sample size problem of sampling error, which we discuss, in Chapter 19. However, if your population is
too large to enumerate, you will have to use a sample.
Aids
Sampling is not limited to selection of individuals to become respondents in a survey.
Sampling can be used to select organizations, villages, other geographical or political entities,
the content of newspapers for analysis, or time periods for observing certain behaviors. In

Chapter 8 Page 1
fact, sampling can be applied to almost any unit of analysis you might think of. Assuming you
decide to sample, you are then faced with a second choice. As suggested by the "decision
tree" in Figure 8.1, you have to decide:
First whether to enumerate a population or to select a sample; and
Second, if you decide to sample, whether to use a probability or a nonprobability
method of sampling.
PREV NEXT

Chapter 8 Page 2
Figure 8.1 also lists various kinds of probability and nonprobability sampling. We discuss
Introduction each kind of sampling later in this chapter.
Probability versus nonprobability sampling
Probability
sampling Probability sampling methods rely only on random or chance selection. Only a carefully
selected probability sample allows a researcher to generalize from sample results to the
Simple population from which the sample was selected. One of the four sampling methods we
random describe should meet about any sampling requirement you will face. Each method is based
sample on chance and chance alone. No other consideration is allowed to affect who is included in
the sample. Human choice or judgment is specifically ruled out. Once the chance process is
started, the researcher accepts the resulting sample and makes no adjustments or other
Systematic
changes to make it come out any differently.
random
sample
Stratified
random
sample
Cluster
sampling
Creativity in
sampling
Weighted
samples
Problems to
watch for in
Figure 8.1. Decisions regarding sampling
sampling
In contrast, nonprobability samples are selected by means other than chance, typically on
Nonprobability some from of human choice or judgment. Respondents, for example, might be selected
sampling because they were friends or relatives or lived in easily accessible places. Any nonprobability
sample has a serious limitation. There is no way to show that a nonprobability sample
Sample size represents any population. A nonprobability sample is just a collection of some number of
persons or other units and nothing more. This means that results from nonprobability samples
Aids cannot be safely generalized beyond the particular collection of persons from whom the data
were collected.
Use of probability samples avoids this limitation. By using chance, we eliminate any possible
bias in the selection of respondents. When proper procedures are used, probability sampling

Chapter 8 Page 2
gives the best assurance that the resulting sample represents the population from which it was
selected.
PREV NEXT

Chapter 8 Page 3
Probability sampling
Introduction
Sampling terms
Probability Shortly, we will describe the four main methods of selecting probability samples. To
sampling understand and apply these methods, you first need to understand certain concepts and terms
used in sampling.
Simple
Population. Your research question defines the group or population you want to learn about.
random Many small populations can be defined precisely. All students who graduated last year from
sample your university would form a concrete, easily defined population. So would all staff of a
certain business or all heads of households in a village. Many populations we choose to study,
Systematic however, are harder to define with any accuracy. The population of a city, for example, exists
random only as an abstract concept. We may have an estimate of the population size, but at any
sample moment the population changes: Some persons and families leave, others arrive. The same is
true for many organizations and other groups we want to study. Because of constant changes
Stratified in populations, researchers have to define a population as clearly as possible in concrete terms
random before preparing to select a sample. This is generally done by placing specific geographic,
time, membership, or other limits on the abstract population. The specifically defined
sample
population is referred to as the target population. The target population is the population we
want our sample to represent. Box 8.1 illustrates some target populations.
Cluster
sampling Box 8.1. Illustrative target populations
1. All persons who have lived throughout the past six months in
Creativity in households in the city of Omdurman, Sudan, and were present
sampling during the month of March, 2005.
2. All government ministries in Egypt with 500 or more full time
Weighted staff, as reported on June 1, 2005.
samples 3. All patients admitted to the private hospitals with at least 25 beds
in Nairobi, Kenya, during the month of February, 2005.
Problems to 4. All editorials published on the editorial pages of the Times of
watch for in London from September 12 through December 31, 2001.
sampling 5. All households headed by women who lived full time in the Dar
Elsalam camp for internally displaced persons during the week of
Nonprobability May 23, 2005.
sampling
Notice that each definition creates a concrete, measurable population. The population for
Sample size Omdurman, for example, is limited to persons who have lived in a household for the past six
months and were present during the month of March, 2005. This rules out persons who live in
Aids hotels or who are temporary guests of persons living in Omdurman. The Egyptian example
includes only full time staff of ministries with at least 500 employees and who were employed
as of June 1, 2005. Each of the other populations has specific limits.
Sampling element. A sampling element or sampling unit is a single member or unit of the

Chapter 8 Page 3
target population. This is the unit about which information will be obtained. Frequently, the
sampling element is an individual, but as illustrated in Box 8.1, sampling elements may be
organizations, editorials, or groups, such as households.
Sample frame. This is the list of all sampling elements from which the sample is selected.The
sample frame is the practical, operational definition of the target population. Ideally, we begin
by defining the target population for a given study and then search for a sampling frame that
matches the target population as we have defined it.In practice, however, the process is often
reversed. We sometimes begin with a general definition of the population and using this as a
guide, search for and evaluate sampling frames until we find one that comes close to or is as
close as possible to the definition of the target population and then use it. Finding an accurate,
up-to-date sample frame is crucial to good sampling. Unfortunately, valid sample frames are
difficult to find in many developing countries. Lack of a valid sample frame is a major source
of error in sampling.
Statistic. A statistic is a finding based on a sample. All the results described for the Sudan
Fertility Survey are statistics. Statistics are generally reported as percentages, averages, and
measures of variation among scores.
PREV NEXT

Chapter 8 Page 4
Parameter. A parameter is a result based on enumeration of a population. Because

Introduction enumeration is seldom undertaken, parameters generally are not known. Instead, using
methods we describe in Part 4, statistics from a sample are used to estimate the values of
Probability parameters.
sampling
Sample design. This is the plan prepared in advance for selecting a sample, using a
probability or nonprobability method. A properly described sample design includes:
Simple
● A precise definition of target population;
random
● Definition of the sampling element (defined a bit later);
sample
● Description of the sampling frame used (also defined later);
Systematic ● Description of the method of sample selection;

random ● The planned size of the sample;
sample ● The time period during which data were obtained; and
● The size and composition of the actual sample that was obtained.
Stratified
random Other social scientists judge the adequacy of a sample using these criteria. You should include
sample all the elements listed above in the description of any sample design you use.
Selecting or drawing a sample. This is the process of selecting sampling elements from a
Cluster sample frame using a probability or nonprobability method.
sampling
Sampling error. This is a mathematical term that describes how well a statistic for a given
Creativity in sample provides an estimate of the corresponding parameter. Chapter 19 describes how to
sampling calculate the sampling error for a mean (average).
A number of Web sites define and illustrate the terms we just defined. One site you might find
Weighted helpful is: Sampling Terminology.
samples
With this background, we now turn to four ways for selecting probability samples. These are
Problems to the:
watch for in ● Simple random sample;
sampling ● Systematic random sample;
● Stratified random sample and;

Nonprobability ● Cluster or multiple stage sample.
sampling
Each has its specific uses. We begin with the basic and simplest of these the simple random
Sample size sample.
Simple random sample
Aids
An illustration should help you grasp the concept of a simple random sample. Imagine we
wanted a probability sample of 100 faculty members out of a population of 500. One way to
get the sample would be to write the name of each faculty member on a slip of paper, put the

Chapter 8 Page 4
slips in a box, shake the box until the slips are thoroughly mixed, and then reach in and draw
out a slip of paper. This would result in the random or chance selection of the first faculty
member of our sample of 100. The selected name would be written down and that person
would become a member of the sample. This process would be repeated 99 times. Each time a
slip of paper is drawn, all the remaining pieces or names have had an equal chance of being
selected. This is the basis of a simple random sample.
Selecting
Steps in selecting a simple random sample are:
1. Define the target population and sampling element;
2. Select a sampling frame; and
3. Select the sample.
PREV NEXT

Chapter 8 Page 5
Define the target population and sampling element. Definitions for the target population
Introduction and the sampling element are generally made together. Definition of one pretty well defines
the other. With the sampling element defined, the target population becomes the aggregation
Probability of all elements or units with the characteristics defined for the sampling element. Starting the
sampling other way, a clear, specific definition of the target population also defines the elements
comprising this population. Either way, however, we would need a sampling frame.
Simple Select a sampling frame. This is a crucial step in developing a simple random sample design.
random Any potential sample frame you plan to use should be carefully examined to determine to
sample what extent it matches the target population.
Returning to the faculty example, we would need to obtain a list of the members of the
Systematic
faculty. This should be easy to get, but, even with the list, we still have to define our target
random population. Should we include only full time staff or should part time faculty be included as
sample well? What about administrative and support staff: Should they be included or are we limiting
the target population to teaching staff only? Do we include visiting faculty from other
Stratified universities? Answers to these and other questions will depend on the purpose of our research.
random If our purpose is to investigate teaching experience, we obviously would limit the target
sample population to only teaching faculty and exclude administrative and support staff. If we were
investigating morale of all university staff members, teaching and administrative staff as well,
Cluster we would want to include all staff.
sampling Regardless of how the target population is defined, any faculty list would be checked to make
sure it corresponds to the target population as we defined it. We would want to make sure
Creativity in that the list is up to date, that names of all eligible members are included, that no names are
sampling listed more than once, and that no other errors occurred. Any of these errors could invalidate
of sample.
Weighted
Finding a suitable, valid sampling frame is frequently difficult. In many developing countries,
samples there is limited census data to draw on. Further, with civil unrest, wars, drought, and other
problems, large population changes frequently occur, making previous census data inaccurate.
Problems to It is always a good idea, therefore, to carefully check the accuracy of any sampling frame
watch for in you intend to use. Lists of residents in a village, for example, can be checked for accuracy
sampling with village elders. Names of dead persons or those who have moved away can be removed
and names of new residents can be added. With these changes, a local list of residents could
Nonprobability become a suitable sampling frame for designing a simple random sample of households in a
sampling village. Similar checking can be used to convert a questionable list of employees of a factory
or other organization into a useable sampling frame. When a trustworthy sampling frame
Sample size cannot be found or created, other forms of sampling should be considered.
Osman's research (1995) illustrates extreme care in testing the adequacy of a sampling frame.
Aids He wanted a random sample of farmers in a rural area of the Sudan. Available lists of
households in each village could have served as the sampling frame. But he made sure the
lists were accurate and up-to-date before he used them. In consultation with local leaders,
Osman eliminated duplicate names, deleted households that had left the area, and added new

Chapter 8 Page 5
households. As a further step, he checked the revised lists for each village against lists
maintained for those eligible to receive government rations. After further revisions were
made, Osman used the lists as his sampling frame. He wrote the names of each household
head on a slip of paper, put the papers in a box, and drew out the number of slips he needed
for his sample size. His sample was as random as possible under the conditions he had to
work with.
Select the sample. Once you are satisfied that the sampling frame represents the target
population reasonably well, you can select your sample. You can use the "box" procedure as
Osman did or you can use an alternative. Instead of writing the names on slips of paper,
putting them in a box, and drawing out names, you could use a safe shortcut. The following
illustration shows how this can be done using a table of random numbers.
Using a table of random numbers
There are two ways to find a table of random numbers. One way is to find a statistics textbook
with a table of random numbers. Your library or instructor probably has a statistic book with a
table for random numbers. The other way is to use one of the sets of random numbers that are
available online free. Following are some suggestions:
PREV NEXT

Chapter 8 Page 6
SurfStat 500 Random Digits, which provides 100 5-digit sets of randomly generated
Introduction numbers
Probability The Research Randomizer, includes an online form for inputting instructions for designing a
sampling simple random sample; following the instructions you provide, the Randomizer will produce a
set of random numbers; also includes a tutorial that walks you through the process of selecting
a sample
Simple
random Regardless of the method you select, any random table or randomizer is used in the same way.
sample A table of random numbers consists of rows and columns of numbers, usually in sets of 5
numbers. Each number in each set is a computer generated random number. Consequently,
Systematic any single number or any combination of numbers, such as the first 2 or last 3 or all 5, also
random comprises a random number. With this explanation of the numbers in a table of random
numbers, we turn to using the table to select a sample.
sample
1. Assign a number to each sample element in the sample frame. In our faculty case we
Stratified would number the names on the sample frame from 1 to 500.
random
2. Select a start point in the table. In using a table of random numbers, the first step is to
sample select a starting point in the table. Since all the numbers in the table are randomly generated, it
doesn't matter where one starts, but it is important that the start point is selected randomly
Cluster entirely by chance. This can be done by simply closing your eyes and lightly touching a page
sampling of the table with a pencil. The number the pencil rests on is your start point.
Creativity in To continue with the faculty example, we want numbers between 1 and 500 or, in terms of a
table of random numbers, between 001 and 500. But numbers between 001 and 500 are not
sampling
listed as such in tables of random numbers. The numbers generally are listed in sets of 5
digits. You can select three-digit numbers by reading the sets of five digits from either the left
Weighted side or from the right side of the column. It doesn't matter which way is used since all
samples numbers or any combination are random. Let's say we decide to read numbers from the right
side of the column. Then, reading left, we would accept as our starting point, any three
Problems to numbers between 001 and 500. Any number larger than 500 is ignored because it is beyond
watch for in the size of the sample we are selecting. From the initial number our pencil rested on, we can
sampling move either up or down, it doesn't matter, until we come to a set of three numbers that are
between 001 and 500.
Nonprobability When we prepared this example, our pencil came to rest on the set of numbers consisting of
sampling 85205. Since we decided to read from the right side of the column, then reading right to left
we find the number 205. This was in the range we were using (001 to 500), so it became our
Sample size starting point. The number 205 would also be the first number selected for our sample.
Aids 3. Select the remaining numbers needed to complete the sample. Continuing the faculty
illustration, we need to select another 99 three-digit numbers. From our starting point, we
decided to move down each column, looking for any set of 3 digit numbers between 001 and
500 (and always reading from the right side of the column to the left). As we moved down the
column, the next number was 71899, which we skipped over because 899 was beyond the size

Chapter 8 Page 6
of our population of 500. Going down further, we came to 96214. Since this was within our
range, we recorded the number 214 as the second selection for our sample. Going down
further, we found 47348 and noted sample element 348 as our third selection. We continued
this process, going down the column and then moving to the top of the next column, to the
right or left it doesn't matter and then going down each column until we had selected the
remaining 99 numbers.
In the process of selecting numbers, a number that already has been selected is bound to
reoccur. When this happens, ignore the second or any additional occurrences of that number.
The same procedure can be used with sampling frames of any size. If the sampling frame
contained up to 9,999 elements, entire sets of 5 digits would be used. For sampling frames
with numbers in excess of 9,999, combinations can be created by including numbers from a
row opposite the one first selected.
4. Record the numbers as they are identified. Each number is recorded as it is selected The
list of 100 selected numbers constitute the sample for the study of faculty members. Using the
numbers assigned to each faculty member, we would then identify the names of faculty
members represented by these numbers and seek data from them.
PREV NEXT

Chapter 8 Page 7
Systematic random sample

Introduction
Selecting
Probability Systematic or interval sampling, as it is sometimes called, is based on random sampling, but
sampling often is easier to complete than simple random sampling. As with all random sampling, the
investigator must define the target population, find or create a suitable sampling frame, and
Simple number each element in the frame. Instead of drawing each element at random, the
random investigator calculates a sampling interval and uses this interval in selecting elements to be
sample included in the sample.
We can use our faculty illustration to show how a systemic sample, is selected. Calculation of
Systematic the interval is based on the ratio of the sample size to the size of the target population. Using
random the faculty illustration, we want a sample of 100 faculty members from a target population of
sample 500. The ratio then becomes 100/500 or 1 in 5. To get a sample of 100, we will need to
establish an interval of 5. Then, selecting 1 name out of every 5 names in the sample frame
Stratified will give us a sample of 100 names. Using a faculty list as the sampling frame, we select
random every fifth name. But, we have to do this in a random way. To create a random systemic
sample, we:
sample
1. Number each element (name) on the sampling frame. In our example, the names, as
Cluster before, would be numbered from 1 to 500.
sampling
2. Establish the sampling interval that is needed. As already explained, we will use an
interval of 5.
Creativity in
sampling 3. Using the box method or a table of random numbers, select a random start point. In
this case, we are looking for a number between 1 and 5. Using the procedures defined
Weighted previously, we would select a starting point. Let's say the number chosen was 4. Sample
samples element 4 would therefore become the first member of the sample.
4. Select elements from the sampling frame that occur at the stated interval. Starting with
Problems to the fourth name on the list, we would select every fifth name. Thus, the sample would consist
watch for in of names represented by numbers 4, 9, 14, 19, etc., up to 499 on the list of faculty members.
sampling The resulting 100 numbers and corresponding names is a systematic random sample.
Caution in using systematic samples
Nonprobability
sampling Systematic sampling can introduce a special kind of bias. To illustrate how this could
happen, suppose you were studying the levels of job satisfaction among staff of a government
Sample size ministry. Since the ministry has a complete list of all staff, you decide to use systematic
sampling. Also, you decide to use a sampling interval of 20. Now, suppose that the first name
Aids you selected was a supervisor and that a supervisor's name appeared as every 20th name
thereafter. If this were the case, you would have selected a supervisor for the first sample
member. Then, by using the interval of 20 you would have selected only supervisors for the
rest of the sample. Thus, the entire sample would be made up of supervisors. The problem
with this is that a sample based on supervisors only would not be representative of the

Chapter 8 Page 7
population of all staff of the ministry. With their higher positions, supervisors have greater
responsibilities, are better paid, and have more privileges. Consequently, their job satisfaction
levels certainly would be different from the rest of the other staff. Obviously, any results
based on responses of only supervisors would be biased and could not be used to describe the
morale of the staff in the ministry.
This kind of problem can occur in using an interval sample in any formal organization, such
as a government ministry, university, or military unit. When you plan to use a systematic
sample, check to see if the names on the sampling frame are arranged in any systematic order.
If some kind of order exists, then a simple random sample should be used or another method,
called a stratified sample, which we describe next, should be used.
PREV NEXT

Chapter 8 Page 8
Stratified random sample

Introduction
A properly drawn simple random sample or a systematic random sample is a trustworthy way
to select a sample, but for some investigations a stratified random sample is more appropriate.
Probability
A stratified random sample can be easily understood by defining its two key terms stratified
sampling and random. You already know that random means selection by chance. Stratified simply
refers to selection within subgroups that make up some population. The student population of
Simple a university, for example, can be stratified in several ways by gender, into males and
random females; into class level, first, second, third and fourth years; by the college in which they are
sample enrolled; or by undergraduate versus graduate levels. Most populations can be stratified in
one or more ways.
Systematic
Selecting
random
sample In selecting a stratified random sample, the same steps are followed as with a simple random
sample, except that an independent sample is selected from each of the strata of the
Stratified population. The specific steps are:
random 1. Define the target population and strata to be used. As with all sampling, it is
sample necessary to clearly define the target population and sampling element. In addition, for
a stratified random sample, each strata to be sampled has to be defined. Definition of
Cluster strata is a matter of choice and depends on the nature of the problem being investigated.
A suitable sampling frame must be available for each strata.
sampling
2. Obtain or develop a suitable sampling frame for each strata. As described for a
Creativity in systematic sample, care must be taken to avoid conditions that could bias sample
selection.
sampling
3. Decide on the sampling rate to be used for each strata. With stratified sampling,
Weighted different rates can be used for each strata. The value of this feature will become clear
shortly.
samples
4. Select a simple random or systematic sample within each strata. Either a simple
Problems to random or a systemic sampling method can be used.
watch for in Strengths and limitations
sampling
A great advantage of stratified sampling is that different rates can be used for sampling
different strata. To illustrate, imagine you were studying differences on some issue between
Nonprobability
male and female students in a university with 2,500 male and 500 female students. And
sampling suppose you wanted a sample of 300 students or 10% of this population. If you used a simple
or systematic sample you would get approximately 250 males and 50 females, give or take a
Sample size certain number due to chance variations that occur with every sample. But do you need 250
males to get enough data and would having data from only 50 females be enough? Why
Aids make a comparison based on 250 versus 50? Would it not be more sensible to base the
comparison on an equal number of males and females?
Yes, it would; and this illustrates the great feature of stratified sampling. Instead of selecting

Chapter 8 Page 8
10% of all the students, you could select separate samples of males and females of equal size.
Let's say you decide that the comparisons should be based on 100 students in each strata
(male versus female). To get 100 males you would select 100 out of the 2,500 males or a
sample of about 4% while for females the sampling rate would be 100 out of 500 or about
20%. Although different rates were used, each sample would be a legitimate sample for its
strata. Results from each strata could be used to describe characteristics of males versus
females, provided, of course, that proper procedures were followed in selecting each of the
samples. In addition, results from the samples could be safely generalized to the population
represented by each strata; from the sample of males to the population of males, and the same
for the females.
Stratified sampling, however, has some drawbacks First, to use it properly you have to know
the proportion of each stratum in the population. In our example, we did, but often in
research in developing countries the proportions for strata are now known. Some strata,
therefore, may not be properly represented in the final sample. Also, use of different
sampling rates for various strata requires an adjustment called weighting when data from the
strata are combined to describe some variable for the population as a whole. A simple
example of weighting is described later in this chapter. Weighting procedures, however, can
become complicated.
PREV NEXT

Chapter 8 Page 9
Simple random or systematic random sampling does not have these disadvantages. Either
Introduction method generally is safer to use when the proportions of the strata of some population are not
known.
Probability
Stratified sampling designs are frequently used when the investigator wishes to compare
sampling various groups on some set of variables. Hassan and others (1988) selected separate samples
of males and females from each of two levels of schooling. Gender represented one strata;
Simple level of schooling another. Mageed, Sulima and Kawther (2000) also used a stratified
random sampling design. They selected separate samples from two contrasting socio-economic areas
sample of Omdurman for their study of male attitudes toward female genital mutilation. Muneer's
(1989) sample was based on whether or not farmers were members of an agricultural
Systematic cooperative. Khamis and Alsuni (1998) also based their study on a stratified sample. They
random selected separate samples within each of three urban and three rural areas for measuring the
sample prevalence of tuberculosis. In each of these investigations, data analysis consisted of
comparing the results from the separate samples used in each study.
Stratified Cluster sampling
random
Each of the three sampling methods described so far requires use of a sampling frame. These
sample
methods also work best when the population is in the low thousands or smaller, and when the
population is reasonably accessible or concentrated in small areas. But what can be done
Cluster when there is no sampling frame or when the population is very large and scattered over a
sampling wide area? Then, the methods described previously are impractical. To get around this
problem, researchers have developed another sampling method, called the cluster, multistage
Creativity in or area sampling.
sampling
The idea behind this method is quite simple. Instead of selecting the ultimate sampling
elements (usually households or individuals) right off, a several-step process is used. First, the
Weighted
investigator defines a set of large clusters that together cover the target population. Clusters
samples often are geographic areas, such as villages, regions of a rural area, sections of a town or city,
or physical units such as schools or hospitals. Each cluster defines some part of the target
Problems to population. Depending on the size of the ultimate sample wanted, some number of these
watch for in clusters is selected at random.
sampling
Each of these first order clusters is subdivided into small clusters and some number of these
secondary clusters is randomly selected. Generally, only first and second order clusters are
Nonprobability
used. In rare circumstances, a third set of clusters might be needed. These choices depend on
sampling how large a sample is wanted and how widely the population is spread over an area. With a
larger sample and greater population dispersion, more clusters might be required. Using a
Sample size simple random or systematic method to randomly select sampling elements from the last set
of clusters completes the process.
Aids
Selecting
A map of the area to be sampled is usually necessary for designing a cluster sample. If
available, aerial maps provide a good basis for creating clusters. Lacking any map,

Chapter 8 Page 9
investigators often draw their own maps, based on driving and walking around the areas
where clusters are to be defined. If the purpose were to design a sample of households in a
certain section of a city, the area could be inspected and houses noted on a map, which could
then be used in selecting clusters. This approach, however, will work only for relatively small
areas. For large areas, such as provinces or large cities, maps are essential.
Cluster sampling requires making decisions about the number and sizes of clusters to use at
each stage of sampling. A good way to start is to decide on the size of the final sample.
Suppose that you think you have enough time to interview 150 persons in a city of
approximately 15,000 persons. To get this sample, you will need to make three decisions:
● How many primary clusters to select;
● How many secondary clusters to select within each primary cluster; and
● How many households to select randomly within each secondary cluster.
PREV NEXT

Chapter 8 Page 10
The answers to these questions depend on two conditions. One is the arithmetic involved.
Introduction Obviously, the final number of 150 is going to be a product of the number of primary and
secondary clusters chosen and the number of households selected within each secondary
Probability cluster. In any real situation, many combinations of these numbers will produce the desired
sampling number for the final sample. Fortunately, several guidelines help in making these decisions.
First, select as many clusters at each level - primary, secondary, and, if used, tertiary (third
Simple level down) as your resources will allow. We will make this point clear shortly. This will
spread the sample over the largest possible area. The reason behind this rule is simple: As
random
smaller areas are selected, populations generally become more similar. Households or
sample whatever is being sampled in the last set of clusters generally will be more alike and differ
somewhat from households in more distant clusters.To ensure that the final sample best
Systematic represents the population from which it was selected, we want to get as much variation or
random heterogeneity as possible. Therefore, it is advisable to spread the sample as widely as possible
sample at the beginning, by selecting as many primary clusters as possible and minimizing the
number of sampling units selected in the final clusters. Figure 8.2 shows how this can be
Stratified done.
random
sample
Cluster
sampling
Creativity in
sampling
Weighted
samples
Problems to
watch for in
sampling
Nonprobability
sampling
Sample size
Aids

Chapter 8 Page 10
Figure 8.2. Illustrative cluster sample design

PREV NEXT

Chapter 8 Page 11
Practical considerations, however, often limit application of the guidelines just described.
Introduction The intended effect of these guidelines is to spread the sample as widely as possible over the
area representing the population. While increasing the chances of getting a representative
Probability sample, the wider spread also increases the time, effort, and cost required to conduct the
sampling interviews. Selection of a smaller number of primary clusters concentrates interviewing in a
smaller number of areas, making it easier to complete the required fieldwork. This approach
Simple increases the possibility that the final sample will be less representative of the population
being studied. In practice, researchers balance these two competing conditions. To the
random
extent practical, larger numbers of primary and secondary clusters and fewer ultimate
sample sampling units are used.
Systematic A final word: anytime you cannot find a suitable sampling frame, consider using a cluster
random sample. Some form of clustering, whether in terms of geographical areas or institutions, is
sample almost always possible.
Creativity in sampling
Stratified
random Different sampling techniques can be creatively combined to meet the requirements of an
investigation. For example, ElTigani (1989) used a multistage sampling design, but with an
sample
adaptation. He first selected two primary sampling units within the Geriza area of the Sudan.
In addition, he selected another primary unit on purpose because it contained a health clinic.
Cluster Since his study focused on child health, he wanted to see if the clinic made a difference in the
sampling health of children in the area it served. In another variation, Khamis and Alsumi (1988) first
stratified their sample by identifying rural and urban areas for sampling and then used
Creativity in systematic sampling to select households within each of six selected areas.
sampling
These variations illustrate how sampling methods can be creatively combined and adapted to
meet requirements of research.
Weighted
samples Weighted samples
Probability sampling is based on the fact that each sampling element has an equal chance of
Problems to
being selected. In our example of stratified random sampling, however, females had a much
watch for in great chance of being selected than males. Disproportionate sampling of females versus males
sampling raises no problems as long as the data from each of the samples are used to generate estimates
of the parameters for the corresponding population. Results from each of the samples could
Nonprobability also be used in comparisons between the two groups. We could do either of these things
sampling because each sample was a valid probability sample of the sub-population of males and
females.
Sample size
The separate data for each sample, however, cannot be combined as they stand to generalize
to the entire student population. While we had probability samples for each sub-population
Aids based on gender, we did not have a valid probability sample for the entire student body
population: Females had a much greater chance of being selected than males. There is a way,
however, to correct for the disproportionate selection of females. The sample has to be
corrected to reflect the greater representation of females relative to males. Here is one way

Chapter 8 Page 11
this can be done.

In our illustration, the chance of selecting a man was one-fifth that for selecting a women. If
we wish to combine data for both samples to form one sample to represent the entire student
population, we have to correct this imbalance. Since we know males are under represented by
a factor of 5, we could give any data for males a weight of 5 compared to data for females.
Accordingly, any measurement for males would be multiplied by 5, while those for females
would be left as obtained. As an alternative, we could weight the statistics, such as mean, for
the male sample before combining it with the mean for the female sample. A mean for the
male sample would be multiplied by 5 and then added to the mean for the female sample, after
which the sum would be divided by 6, the number of means involved in the summation. For
example, suppose the mean for some variable for the male sample was 20 and that female
sample was 17. The mean for the total sample would be 20 times 5 for the male sample plus
17 for the female sample, giving a weighted mean of 100 plus 17 or 117 divided by 6 or 19.5
for the entire sample.
Weighting can become complicated and involve some difficult calculations. This is
particularly true when area sampling is used. When the clusters at any stage contain different
numbers of units (households, etc.), weighting is necessary to correct for the disproportionate
selection of various clusters. This problem can be avoided by creating clusters with
approximately the same numbers in each before selecting the clusters at any stage in the
process. When this is done, the sample becomes self weighted. Then, data from the final
clusters can be safely combined to make estimates of characteristics of the population.
PREV NEXT

Chapter 8 Page 12
The sample used by Osman (1995), described earlier, also illustrates self-weighting sampling.
Introduction His sample was scattered over a number of villages. To avoid a weighting problem, he
selected samples in each village proportional to the size of the number of households in each
Probability village. Larger samples were selected from villages with more households and smaller
sampling samples from villages with fewer households. As a result, the sample became self weighted.
Data from each household were combined for the total sample without having to make
Simple complicated mathematical adjustments.
random We also want to point out how Osman handled a practical problem in selecting his sample. He
sample did not know the exact population of each village. However, he was able to get a list of
households for each village and used these lists for drawing his sample. Households, in effect,
Systematic became an indicator for population. This approach was based on the assumption that the
random average household size in each village was about the same, which probably was a safe
sample assumption to make.
Problems to watch for in sampling
Stratified
random In selecting a probability sample, researchers strive to avoid or at least minimize problems
that can bias the sample. Three kinds of problems often occur:
sample
The target population is not clearly defined. This error occurs most often when the
Cluster population is left at a general or abstract level instead of being defined in concrete, operational
sampling terms. An error like this led to an international issue in February, 2002, involving high
government officials in Kuwait and the United States. Stone (2002), an American reporter,
Creativity in wrote a story based on surveys in Kuwait and eight other Muslim countries. The Gallup Poll,
sampling a highly respected survey organization, conducted the surveys. The results showed that 36%
of the Kuwaiti respondents said the attacks on the World Trade Centers in New York City on
September 11, 2001, were morally justified. This was the highest percent found in any of the
Weighted
countries included in the survey. Further, only 17% of the Kuwaiti respondents approved of
samples the activities of the United States in Afghanistan. Americans were outraged that the citizens of
Kuwait, who United States had rescued in the Gulf War, had such negative attitudes toward
Problems to the American government. The Kuwaiti ambassador to the United States, however, correctly
watch for in pointed out that the responses did not represent the views of Kuwaiti citizens.
sampling
The Gallup Poll, the organization that conducted the surveys, had selected a sample of persons
who lived in Kuwait and not a sample of Kuwaiti citizens. Workers from other countries make
Nonprobability
up 60% of the population of Kuwait, with many drawn from Pakistan, Egypt, and other Arab
sampling countries. The Gallup organization failed to distinguish between Kuwaiti citizens and persons
who lived in Kuwait. They did not define the target population and limit their sampling to
Sample size Kuwaiti citizens. The lesson of this episode is clear: Base your sample on a clearly defined
target population and make sure this is the population you want to describe something about.
Aids
Poor sample frame. Bias at this step can be eliminated by making sure the frame is
up-to-date, complete, does not contain duplications, and in all other ways matches the target
population or comes reasonably close to doing so.

Chapter 8 Page 12
Mistakes are made in sample selection. Mistakes can occur, even when one is careful.
Errors can occur in numbering elements, in using a table of random numbers, in copying the
numbers selected, and in other ways. The only way to avoid such errors is to check and
double check each step in sampling and to correct each error that is found.
Low response rate. This problem arises because data are not obtained from some of the
persons or other units selected as members of the sample. Some sample members cannot be
located, some are never at home when an interviewer tries to contact them, and some refuse to
be interviewed. Whatever the reason, the actual number of persons interviewed generally is
less than the number selected to form the sample. As the response rate declines, the resulting
sample becomes less representative of the target population. This reduces the value of the
results for generalizing to that population.
At this point, we suggest you look at other discussions of probability sampling. We
recommend any of the following sites for further discussion on probability sampling:
PREV NEXT

Chapter 8 Page 13
Probability Sampling, defines and describes how to select commonly used samples
Introduction
Sampling, presents key sampling concepts and terms; discusses types of random sampling;
Probability describes how to deal with sampling problems; and provides answers to some frequently
sampling asked questions about sampling
Sampling Guide, provides a guide to selection of samples; includes how to define
Simple measurement objectives for a survey; examines how sample size can be estimated; looks at
random ways of selecting samples; and discusses how to analyze the data collected
sample
Sampling in Research, discusses general issues related to sampling for research purposes;
Systematic identifies dangers in sampling and to minimize them, and provides guides for deciding on the
size of a sample
random
sample Nonprobability sampling
Although we earlier advised using only probability sampling, sometimes nonprobability
Stratified
methods may be warranted. If the purpose of the research is to explore some topic, for
random example, to identify key concepts or to test respondents' understanding of items to be used in
sample a scale, nonprobability methods are appropriate. In such cases, the purpose is not to
generalize, which requires a random sample, but to learn something for later use. Also, when a
Cluster sampling frame does not exist and cluster sampling is impractical, nonprobability methods are
sampling the only way to obtain a sample.
If any of these conditions apply, you may want to consider using some variation of the
Creativity in
following nonprobability samples. But remember, lacking random selection of the sampling
sampling elements, nonprobability samples have two serious limitations:
● There is no control or protection against bias introduced by the investigator in the
Weighted
selection of the sample, whether this is by conscious or unconscious action.
samples
● Since the variation that inevitably occurs in sampling cannot be estimated, results from
a nonprobability sample cannot be safely generalized beyond the specific sample that
Problems to
was used.
watch for in
sampling Still, there are occasions, when nonprobability sampling methods may have to be used. Four
frequently used nonprobability sampling methods are:
Nonprobability ● Convenience sampling;
sampling ● Quota sampling;
● Purposive sampling; and

Sample size
● Network sampling.
Aids Convenience samples

In convenience sampling, also called haphazard sampling, the investigator selects some
number of persons or other sampling units because they are easily accessible. Some
illustrations of convenience samples are shown in Box 8.2.

Chapter 8 Page 13
Box 8.2. Illustrations of convenience samples

1. Students found on the street near your university
2. Persons selected by the investigator from those in a market
on Monday morning
3. The first 20 persons encountered at a bus stop
4. Persons who respond to an invitation to comment on a
television show
5. All the mothers at the local well baby clinic on a Thursday
morning
PREV NEXT

Chapter 8 Page 14
Students frequently confuse convenience sampling with random sampling. As we have

Introduction stressed repeatedly, random selection is based on a conscious, deliberate plan that removes
any choice on the part of the investigator. In contrast, a convenience sample is based on
Probability whatever criteria the investigator happens to use. On the positive side, convenience sampling
sampling is quick, easy, and inexpensive. It may be appropriate for exploratory research or for testing
scales, for example, but never for developing estimates of parameters.
Simple Quota samples
random
sample A quota is a share or an allocation of something. Some universities, for example, have quotas
for the number of students from various parts of the country. Students from each region are
accepted until a certain number, their quota or ratio to the total enrollment, is reached.
Systematic
Quota sampling works on the same idea. A certain number of persons is selected to represent
random subgroups that make up the population. The number or quota for each subgroup is set in
sample advance and persons having the right characteristics are selected until that number is met.
Stratified An example may help explain quota sampling. Imagine that we wanted a sample of male
random heads of households in a village and we decided to use a quota sample design. Further, let's
say we discovered that social status is an important variable for this study. Therefore, we want
sample
to be sure that our sample represents the population of male heads a well as possible with
respect to social status. In talking with village elders, we learn that about 25% of the heads
Cluster are considered poor, about 60% are in the middle range, and 15% are thought to be well off by
sampling local standards. We want a sample, let's say, of 50 men: so we find the quota we need for
each status group by multiplying the percentage in each group by the total. Thus, for the low
Creativity in status group the quota would be 25% times 50 or 12.5, which we could round to 12 men; the
sampling quota for the middle group would be 60(50) or 30 men; and the quota for the high group
would be 15% times 50 or 7.5 men, which could be rounded up to 8 men. Using these
Weighted quotas you could walk around the village, talk with men, and then select and interview enough
samples men to the meet the quota for each group.
Quota sampling can be based on more than one selection factor, but this can get pretty
Problems to complicated. If you want to use more than one factor, you will need to consult a book on
watch for in sampling.
sampling
Quota sampling represents an improvement over convenience sampling. It is based on some
definition of the intended population. Also, sub-groups of a population are consciously
Nonprobability
included in the sample. In our illustration, the quota sample would at least include male
sampling household heads for all three important social strata. If we had used a convenience sample, we
might have included few poor heads because they probably would be less convenient to locate
Sample size than middle or high status heads. But once the quotas are set, operationally, quota sampling
becomes another instance of convenience sampling. Within each quota, the investigator uses
Aids personal judgment in selecting sampling elements, resulting in all the biases that occur with
convenience sampling.
Quota sampling has another important limitation. The investigator has to know how the

Chapter 8 Page 14
variable used for estimating quotas is distributed in the target population. In our illustration,
we took the word of local elders for the proportions of low, middle, and high status heads in
the village population. If reasonably accurate information is not available for establishing
quotas, the sample becomes another form of convenience sampling, with all it limitations.
The idea behind quota sampling can be adapted to limit some of the disadvantages of
nonprobability samples. Hassan and others (1988), for example, wanted to analyze possible
differences toward male/female equality between men and women with different levels of
education. The sample design for this study was based on seeking 30 adults of each gender
who met the criteria of either having less than a secondary school certificate (the low
education level for this study) or at least having graduated from secondary school (defined as
the high education level). In effect, Hassan and her colleagues set a quota of 30 males with
low education and similar number with high education and the same for females. The first 30
persons they found in each of the four defined sets became the sample for their investigation.
PREV NEXT

Chapter 8 Page 15
Sample size
Introduction
How large should a sample be? There are several ways to answer this question? One answer
depends on the purpose of the investigation. If the purpose is to explore some topic, sample
Probability
size is not critical. Cases are selected until the investigator learns enough about the topic to
sampling meet his or her needs. For descriptive or explanatory studies, sample size is important because
the size of a sample affects our ability to generalize results of the research.
Simple
random Several factors have to be considered in deciding on the size of a sample. One is the
sample heterogeneity of the population - how much the population varies with respect to any of the
key variables being measured. A larger sample is needed for a population with a great deal of
variation in contrast to a population in which persons are more alike. When there is relatively
Systematic
little variation in a population, say, by ethnic background or socio-economic status, a smaller
random sample can be used. When large differences exist within a population, a larger sample is
sample needed to adequately reflect the variations in the population.
Stratified Another factor is the degree of accuracy desired for estimating the parameters in the
random population. Generally, larger samples will provide more accurate estimates than a smaller
one. In Chapter 19, you will see why this is true.
sample
Sample size should also take into account the number of variables to be analyzed
Cluster simultaneously. If variables are going to be analyzed one at a time, like describing means for
sampling ages of respondents, their years of schooling, or for measures of some attitude, smaller
samples will do. When two or three variables are analyzed together, larger samples are
Creativity in necessary to ensure there are enough cases in the cells of the tables that will be involved. (See
sampling Chapter 17 and Chapter 18 foror more on this point).
There are precise ways to estimate how a sample should be to achieve a certain degree of
Weighted accuracy for estimating population parameters, but these methods require detailed information
samples about the population from which a sample is to be selected. The necessary data often are not
available. To overcome this problem, some commonly accepted guidelines can help in
Problems to deciding about the size of a sample. The size of the population, which we generally know with
watch for in reasonable accuracy, influences how these guidelines are applied. Strangely, the required
sampling sample size is inverse to the size of the population. For populations under 1,000, a sampling
ratio of about 30% of the cases is recommended. For a population of 10,000 or so, a 10%
Nonprobability ratio is sufficient. As the population further increases, to 15,000 or larger, 1% of the cases can
sampling produce very accurate results. Random samples of 2,000 to 2,500 are usually accurate for
populations in the millions.
Sample size The numbers mentioned may not be practical for individuals doing their own interviewing.
Then, compromises have to be made. Statistical analyses, which we discuss in Part 4 of this
Aids book, should be based on a minimum of 30 cases, but at least 100 cases is recommended. The
final size of a sample often is based on a compromise between larger ideal size and the largest
practical size a researcher can manage. Researchers take into account the purpose of their
study, the heterogeneity of the population being investigated, how precise they want the

Chapter 8 Page 15
results to be (this requires some statistical knowledge which we present in Part 4 of this book),
and make a decision.
PREV NEXT

Chapter 8 Page 16
Aids
Introduction
Key terms
Probability ● Area Sample ● Response rate
sampling ● Census ● Sample
● Chain sampling ● Sample frame
Simple
● Chance selection ● Sample design
random
sample ● Custer sample ● Sampling element
● Convenience sample ● Sampling interval
Systematic ● Enumeration ● Sampling unit
random ● Haphazard sample ● Self weighted sample
sample ● Interval sample ● Simple random sample
● Multistage sample ● Snowball sample
Stratified
● Network sample ● Stratified random sample
random
● Nonprobability ● Statistic
sample
● Parameter ● Systematic random sample
Cluster ● Population ● Target population
sampling ● Probability sampling ● Weighted sample
● Purposive sample
Creativity in
● Quota sample
sampling
● Random selection
Weighted Main points
samples
1. Samples are used because it is frequently impractical to collect data from the large
groups of persons or other entities, called the population, that we want to learn about.
Problems to
2. Probability sampling is based on selection of a sample entirely by chance, without any
watch for in
human judgment or preference. Probability sampling is also called random sampling.
sampling
3. Only probability sampling allows an investigator to generalize findings from a sample
to the population from which it was drawn.
Nonprobability
sampling 4. Methods of probability sampling are: the simple random sample; systematic or interval
random sampling; stratified random sampling; and cluster or multistage sampling.
Sample size 5. Necessary steps in sampling include: (1) defining the target population, which is the
specific, concrete definition of the population based on elements that will be sampled;
Aids (2) identification of a suitable sampling frame; and (3) random selection of the desired
number of sample elements to compose the sample.
6. The most frequent problems in sampling are: (1) the target population is not clearly
defined and delimited; (2) the sample frame does not match the target population; and

Chapter 8 Page 16
(3) mistakes or errors are made in the process of selecting the sample.
7. Nonprobability samples sometimes have to be used. Kinds of nonprobability samples
are: (1) the convenience or haphazard sample; (2) quota sample; purposive sample; and
(4) the network sample, also called a chain or snowball sample.
8. Factors to be considered in deciding on sample size are: (1) how heterogeneous the
population is larger samples are needed for populations with greater variation; (2) how
accurately one wishes to estimate parameters in the population larger samples provide
more accurate estimates; and (3) the number of variables that will be analyzed
simultaneously for seeking relationships among two or more variables, larger sample
sizes are desirable. This said, samples of at least 100 cases are recommended, but
generally larger sample sizes are preferable. Practical limitations, however, often limit
the number of interviews that can be completed and, hence, also limit the size of the
sample.


Chapter 9 Page 1
Chapter 9: Designing Experiments

Introduction
Introduction
Seeking In its simplest form, an experiment consists of making changes in a variable and, under
causal carefully controlled conditions, observing the effects of the change on some other variable.
relationships Experiments are especially valuable for testing hypotheses that suggest a cause and effect
relationship. Clear understanding of a presumed causal relationship is essential for
Alternative understanding the logic and design of experiments. Therefore, we begin by reviewing these
explanations conditions. Chapter 3 describes these conditions in more detail.
Seeking causal relationships
Internal validity
Necessary conditions
The classical Three conditions are necessary for arguing that one variable is the cause of changes in another
experiment variable. These are:
1. The variable believed to be the cause must always precede its effects;
Quasi-
2. The two variables must be consistently and preferably strongly associated; and
experimental designs
3. No other explanation provides a more satisfactory explanation of the observed changes
in the dependent variable than the one put forth as its cause.
Quasi-
experimental designs In designing an experiment, the researcher has to create conditions that meet or that come as
close as possible to meeting these requirements.
External validity
Application of conditions
Further An illustration will show the difficulty of meeting the conditions for establishing causality.
variations Some persons have negative or hostile attitudes, often unfounded, about a particular
in experimental group of persons. Social psychologists refer to such views as prejudice. Obviously, there
design are many causes of prejudice, and many conditions that could increase or reduce attitudes of
prejudice toward some group. Suppose we wanted to conduct an experiment to see if we could
reduce attitudes of prejudice among members of some group toward another group. Our first
Strengths &
step would be to discover factors that are known to be related to feelings of prejudice.
limitations of
experimental Through a review of the literature we might discover that persons with high prejudice towards
research members of some group tend to report that they had heard only negative things about that
group. Using this as our theoretical basis, we could hypothesize that providing positive
Aids information about the group in question will reduce prejudice toward that group. In this
hypothesis, the providing positive information would be designated as the experimental or
independent variable and prejudice would become the dependent variable.
Our next step would be to design an experiment to test this hypothesis. We would start by
getting some people, called subjects, to agree to participate in the experiment. Let's say we got
50 subjects to agree. We also have to create the conditions under which the subjects
experience the experimental variable. Here is where creativity in design comes in. We could

Chapter 9 Page 1
create whatever kind of experience we think would be most effective in reducing prejudice.
PREV NEXT

Chapter 9 Page 2
To keep the illustration simple, imagine that we developed an educational program consisting
Introduction of two-hour sessions held once a week for three weeks. During these sessions subjects would
be provided positive information about the group in question, such as descriptions of the
Seeking accomplishments of members of the group, how members of the group are like one's own
causal group, or ways members of that group and one's own group have worked together and helped
relationships one another. This information might be provided in the form of videos, talks by members of
the designated group, and by participation in small group discussions.
Alternative We also have to decide how to measure prejudice. We could use a Bogardus social distance
explanations or some other scale for measuring prejudice. Whatever measure we use, we have to use it
twice - once before the educational program is started and again after it is over. The first is
Internal validity called the pretest measurement and the second is referred to as the posttest measurement. By
comparing the two levels of prejudice we could see if there was a change in the degree of
The classical prejudice among the subjects.
experiment Imagine that we actually carried out this study as described, and, further, that the results
supported our hypothesis. We found that the posttest measurement for prejudice was lower
Quasi- than the pretest measurement. Are we justified in concluding that the educational program
experimental designs caused the decline in prejudice? No, we are not! Our inability to draw that conclusion goes
to the heart of experimental design.
Quasi-
The design we just described is referred to as a single group design. In this design, you collect
experimental designs data from a group of subjects on one or more selected variables before beginning of the
experiment, apply some kind of experimental treatment, and collect data again on the variable
External validity or variables measured before. Single group designs do not rule out conditions other than the
experimental treatment that could affect the outcome. With a single group design, one
Further cannot meet the three requirements for establishing a cause and effect relationship. In the
variations experiment we just described, we safely met the first requirement. We know that the
in experimental experimental variable preceded posttest measurement of the dependent variable. Also, we
design know that there was some association between the two variables, as indicated by the decline
measurement of prejudice following the educational program. But can we say that nothing
Strengths & else could account for the decline other than the educational program? To answer this
question, let's look at some alternative explanations for why a decline in prejudice could have
limitations of
occurred.
experimental
research Alternative explanations
The following discussion is based mainly upon the ideas of Donald Campbell and his
Aids associates, as reported in their careful analyses of experimental designs (see Campbell, 1985;
Campbell and Stanley, 1966; and Cook and Campbell, 1979). Their analyses show that there
are at least seven other reasons why prejudice could have declined among the subjects in our
example. These alternative explanations, which have to be eliminated before one can say
that the data support a hypothesis, are known as:
● History
● Maturation

Chapter 9 Page 2
● Testing effect
● Instrumentation
● Experimental mortality
● Subject reactivity
● Experimenter bias
Each of these factors or any combination of them could have affected our results.
History
History refers to any event outside the experiment that could affect the results. Suppose, for
example, that during the three-week period of the experiment a rich person from the group in
question gave an exceptionally large and widely announced donation to a local hospital.
Could knowledge of this act have made subjects feel less prejudiced and therefore have been
responsible for the decline, rather than the educational program they were in? Possibly: Our
experimental design is flawed in this respect and would prevent us from drawing the
conclusion that the hypothesis was supported.
PREV NEXT

Chapter 9 Page 3
Maturation
Introduction
Natural changes occur to persons over time. In experimental design, any natural change that
could affect the experiment is known as maturation. So, in addition to protecting against the
Seeking
misleading effects of history, an experimenter has to make sure that changes in the dependent
causal variable are not due to naturally occurring changes among the subjects. In all truthfulness,
relationships the results in our illustrative experiment would probably not be influenced by maturation.
The time period would be too short for naturally occurring changes to show up. Maturation,
Alternative however, can be a problem in experiments lasting months or longer.
explanations
Testing effect
Internal validity Sometimes simply giving a pretest can influence the attitudes and behavior of subjects.
Questions included in the pretest, for example, could cause some people to begin thinking
The classical about prejudice in ways they had not done before. As a result, when they were asked about
experiment the group in the posttest they might give less prejudiced views. Consequently, the decline
could have been at least partly due the pretest experience and not wholly because of the
Quasi- educational program. Anytime a pretest is used there is danger of creating a testing effect.
experimental designs Our experimental design also failed to protect against this threat to drawing a valid conclusion
from the decline in prejudice.
Quasi- Instrumentation
To get around the problem of testing effects, an experimenter can use different measuring
instruments for the pretest and posttest measurements. Changing measuring instruments,
External validity
however, raises the question of the comparability of the instruments, or what is known as
instrumentation. If the instruments differ in their validity, reliability or degree of precision,
Further differences in pretest and posttest measurements could be due to this fact and not because of
variations changes in characteristics of the subjects.
in experimental
design Subject reactivity
In our experiment subject reactivity could have occurred. The persons selected to participate
Strengths & in the experiment could have been pleased to be singled out for special attention and, as a
limitations of result, express less prejudiced views. Also, some might have guessed what the purpose of the
experimental educational program was and expressed views in keeping with this objective, even though
research their views had not changed that much. Thus, subject reactivity and not the educational
program could have at least partially accounted for the decline in the posttest measure of
Aids prejudice.
Experimental mortality
We do not mean to imply that experimenters are prone to early death or that subjects run some
risk to their lives. Research use of this term is far less menacing. Experimental mortality
refers to the loss of subjects during the life of an experiment. As experiments extend over
longer periods of time, some subjects are usually lost: Some move away, some become ill and
cannot participate, and some just quit. A high loss of subjects can distort the results of an

Chapter 9 Page 3
experiment.
Experimenter bias
The investigator can be the cause of invalid conclusions as well. After all, the experimenter
develops the hypothesis, creates the conditions under which the experiment is carried out, and
collects and analyses the data. The investigator, therefore, is in a strong position to influence
the results obtained from an experiment. Experimenters have to guard against this temptation.
Because of the powerful position of the experimenter, other researchers expect to see a
detailed description of the experimental design used and a full display of the data obtained in
the report of an experiment.
Returning to the example involving prejudice, you may now see some of the reasons why we
could not say that the decline in prejudice was caused by the independent variable. A number
of other factors could have caused the decline. In experimental terms, we failed to meet the
criterion of internal validity.
PREV NEXT

Chapter 9 Page 4
Internal validity
Introduction
This term refers to the degree to which the conclusions of an investigation are in fact based
solely on the effect of the experimental variable. For an experiment to be internally valid we
Seeking
have to be able to eliminate all alternative explanations, leaving the effect of the experimental
causal variable as the only plausible explanation for the observed changes in the dependent variable.
relationships
How can we achieve internal validity for an experiment? Simple: our best protection against
Alternative threats to internal validity is to use the model of the classical or true experiment.
explanations We have barely introduced the concept of internal validity. For additional discussion of this
important feature of any experiment, go to Internal Validity. When you view this site, you
Internal validity will see how difficult it is to separate true, causal effects of an experiment from alternative
causes that could affect the outcome and, hence, the validity of an experiment. (You may want
The classical to review the first section of this chapter to review the requirements for advancing a causal
experiment relationship). Earlier, we mentioned the threats to internal validity of the single group design.
These are described briefly in Single Group Threats.
Quasi-
The classical experiment
The classical experiment is based on simple but powerful logic.
Quasi-
Logic of the classical experiment
In designing an experiment we want to eliminate the influence of any variable other than the
External validity experimental one on the dependent variable. This is done by using the model of the classical
experiment in which we:
Further 1. Create two equivalent groups at the beginning of the experiment;
variations 2. Administer the experimental variable to one group, the experimental group, but not to
in experimental the other which is retained as the control group; and
design 3. Measure the dependent variable in both groups before and after the application of the
experimental variable. These are the pretest and the posttest measurements.
Strengths &
limitations of The logic just described is shown graphically in Figure 9.1. Using this logic, let's redesign the
study of prejudice we presented earlier. Previously we had used only an experimental group.
experimental
Following the logic shown in Figure 9.1, we will use an experimental and control group.
research (In the next section we will show how to create these groups). Next, we would measure and
compare prejudice levels in both groups before the experimental variable is administered. If
Aids the groups are equivalent, there should be no difference or only a small difference in the
results obtained for each group. Actually, we would expect some difference in average
scores between the two groups. The difference could be in favor of one group or the other
because of random variations among members in each group. Any difference, however,
should not be statistically significant. (For a fuller discussion of this point, go to Chapter
19). If a significant difference is found, the experiment should be stopped and not continued
until the groups are found to be equivalent.

Chapter 9 Page 4
After equivalence between the experimental and control groups is demonstrated, the
experiment can begin. The experimental variable is administered to the subjects in the
experimental group, which in our example means they would participate in the educational
program. Those in the control group would continue with their ordinary daily lives. After
the education program is over a second set of measurements of prejudice would be obtained
from all subjects in both groups. The pretest and posttest data would then be used to test the
hypothesis for the study.
For a brief, clear description of the two-group experimental design, visit: Two-Group
Experimental Designs. This site also describes how the two-group design protects against
threats to internal validity.
PREV NEXT

Chapter 9 Page 5
Introduction
Seeking
causal
relationships
Alternative
explanations
Internal validity
The classical
experiment
Quasi-
Quasi-
experimental designs Figure 9.1. Design of the classical experiment
External validity Testing hypotheses with a control group

Generally a hypothesis calls for a larger difference between pretest and posttest results for the
Further experimental than for the control group. The basis for this reasoning is that:
variations 1. The two groups were alike with respect to some dependent variable at the beginning of the
in experimental experiment.
design 2. Subjects in both groups were exposed to the same kinds and amounts of extraneous or
outside influences during the course of the experiment.
Strengths &
3. Subjects in the experimental group, however, were exposed to an additional and
limitations of deliberately applied influence, the education program, which was designed to cause a
experimental reduction in prejudice.
research 4. The experimental group, therefore, would be expected to show a greater decline in
prejudice.
Aids
Use of the control group allows us to detect the effects of the experimental variable on levels of
prejudice. If the posttest measure shows about the same reduction in both the experimental and
control groups, then the change must be due to some external factor. But, if prejudice levels
decline substantially only in the experimental group, it is reasonable to conclude that the decline
was caused by the treatment we administered. Even if prejudice scores in both groups declined,
but more so in the experimental group, we could take this as evidence of the effect of the
educational program. In more precise terms, the difference in the pretest and posttest
measurements for the experimental versus the control group is known as the experimental effect.

Chapter 9 Page 5
This comparison, however, assumes that the subjects in two groups were equivalent at the
beginning of the experiment. And this depends on how the subjects were assigned to each group.
PREV NEXT

Chapter 9 Page 6
Assigning subjects to experimental and control groups

Introduction
Subjects can be assigned to either the experimental or control groups by:
Seeking ● Randomization
causal ● Precision matching
relationships ● Frequency matching
Randomization. Random assignment to either the experimental or control group is based on

Alternative
the same principle used in selecting a random sample, but is applied a bit differently in the
explanations case of assigning subjects to groups. The basic principle behind a random sample is that each
individual has an equal chance of being selected for inclusion in a sample. Similarly, in
Internal validity assigning persons to either the experimental or control groups for an experiment, each person
should have an equal chance of being assigned to one group versus the other.
The classical
experiment To show how this could be done, let's say that for the experiment described earlier we wanted
to create experimental and control groups with 30 subjects in each and that we have a large
number of persons to choose from. We could simply take the first 60 persons, and beginning
Quasi-
with the first person, randomly assign that person to either the experimental or control group.
experimental designs We could flip a coin for this purpose. If the coin lands as a "head," the person goes into the
experimental group. The opposite decision could have been made instead. We could have
Quasi- said that if the coin lands as a head the person goes into the control group. It does
experimental designs not matter which decision is made because the method of selection is random. Once we decide
how to assign the first person the same rule would be followed until 30 persons were assigned
External validity to each group.
Precision matching. This technique of assignment is a little more complicated and difficult to
Further
carry out. To use this method, we would first identify critical variables that could undermine
variations our experiment. After these variables were identified, we would look for pairs of persons
in experimental having the specified characteristics. Imagine that gender, schooling, age, and tribal affiliation
design were the critical variables that had to be taken into account. We would try to eliminate their
influence by making sure that each subject in one group is matched in terms of these four
Strengths & variables with a subject in the other group.
limitations of
Figure 9.2 shows how this can be done. From some group of potential subjects we might find
experimental 10 males from the same tribe (A) who were 24 year of age or younger and had graduated from
research secondary school. Five of these persons would be randomly assigned to one group and the
others would be placed in the other group. We might also find 6 females from another tribe
Aids who were over 24 years of age and who had an intermediate level of schooling. Three of these
persons would be assigned to one group and the other three to the other group. Other sets of
person would be assigned in the same way: half to the experimental group and half to the
control group.
In this way two equivalent groups, matched on four critical variables would be created .If
there is an odd number of persons in some set, one person can be chosen at random and be
removed, after which the others would be randomly assigned to the two groups.

Chapter 9 Page 6
Precision matching becomes difficult as the number of variables used in matching increases.
Matching on more than three variables usually requires a large number of potential subjects.
Frequency matching.Although less demanding, frequency matching is not as precise as
precision matching. In frequency matching, the researcher decides which variables to control
and then taking each variable one at a time assigns person to one group or the other to get
about the same proportion of persons with that characteristic in each group. Thus, using
gender, schooling, age, and tribe as matching variables, we would assign persons so that we
had the same proportion of females and males in each group. Then as a separate step we
would assign persons randomly to get similar percentages of subjects within each age
category used. This process would then be repeated for tribal affiliation and level of
schooling.
PREV NEXT

Chapter 9 Page 7
Introduction
Seeking
causal
relationships
Alternative
explanations
Internal validity
The classical
experiment
Figure 9.2. Illustration of precision matching
Quasi- In comparison with precision matching, frequency matching is easier to do and requires fewer
experimental designs subjects. The major weakness of this method is that only one variable is controlled at a time.
For instance, the experimental and control groups could have the same percentage of males or
Quasi- females and have similar percentages for each level of schooling, but be quite different in that
experimental designs most females had only a primary level of schooling while most males had at least an
intermediate level of schooling. As a result the two groups could be quite different, which
would destroy the purpose of matching. This is an extreme example, but it shows the main
External validity
weakness of frequency matching.
Further Both precision and frequency matching suffer from an additional limitation, and this is that
variations neither is based on strict random assignment. Both methods are based on the same idea as
in experimental quota sampling (see Chapter 8), which does not result in a representative sample. We
design mention these matching methods, despite their weaknesses, because sometimes they may be
the best one can use.
Strengths & Use of two groups greatly reduces the threats to the internal validity of an experiment. But
limitations of other threats still remain. Go to Multiple-GroupThreats for a description of threats that
experimental might remain.
research
Settings for classical experiments
Aids Generally, classical experiments are carried out in controlled as opposed to natural settings. A
controlled setting is any location created specifically for the purpose of the experiment. It may
be as simple as an ordinary classroom or office taken over temporarily for introducing the
experimental variable. Or, the setting could be a complex laboratory with special equipment
and means for observing and recording the behavior of subjects. Regardless of the
arrangements used, experiments are notable for the degree of control the experimenter has
over all the features of the experiment. Subjects are randomly assigned to the experimental or

Chapter 9 Page 7
control group. In addition, the experimental variable is applied under conditions that are
created and controlled by the experimenter
Dangers of contamination
An additional threat to drawing valid conclusions from experiments may arise from the way
the experiment is conducted. Called contamination, this threat refers to the possible spill over
effects from the experimental to the control group. In a well-conducted experiment only the
experimental subjects are exposed to the experimental variable. In addition, the experiment is
arranged so that experimental and control subjects do not interact with one another during the
course of the experiment. If interaction occurs, the control subjects could indirectly be
exposed to the experimental variable and therefore no longer be valid controls.
PREV NEXT

Chapter 9 Page 8
Prevention of contamination requires special care in the design of experiments. In most

Introduction developing countries, close communication is common among relatives, friends, and
co-workers. Consequently, it is difficult to carry out valid experiments when subjects are
Seeking selected from the same village or other settings, such as a single office or factory, where the
causal potential subjects know each other well. Experimenters avoid this problem by selecting
relationships experimental subjects from one village or community and control subjects from another
village or community similar to that of the experimental group. The experimental variable is
then applied in the setting selected as the experimental site but not in the setting being used as
Alternative
the control site. In each location subjects can be randomly selected for pretesting and
explanations posttesting.
Internal validity This approach, however, has several dangers. First, the experimenter has to be sure that the
members of the two villages or sites used are alike in all relevant ways. Second, there is still
The classical the danger of contamination as a result of communication between members of the two
experiment groups. During the experiment, the experimenter has to maintain observation of the members
of the two groups to see if communication does occur. If it does, the control group may be
contaminated which would destroy the conditions of the experiment.
Quasi-
experimental designs In addition to contamination, other threats to the internal validity of an experiment can arise.
Social Interaction Threats describes some of these. If you are planning to conduct an
Quasi- experiment, you will want to be aware of how human interaction among the participants in an
experimental designs experiment, such as program administrations and staff, the experimenter (yourself), and
subjects, can interact in ways to destroy or greatly limit the internal validity of your
External validity experiment.
Another site, Experimental Design presents the basic ideas behind experimental design; it
Further also includes links to descriptions of various experimental designs not covered in this chapter.
variations
in experimental
design Quasi-experimental designs
Quasi-experimental designs are based on some, but not all, of the characteristics of the
Strengths & classical experimental design. Some quasi-experimental designs are based on randomization,
limitations of but lack a pretest measurement; others may be based on observations for only one group; all
experimental involve at least a posttest measurement. Also, in most quasi-experimental designs the
research investigator has relatively little control over the independent variable. Instead, the investigator
creatively seeks situations where the at least some of the principles of experimental design can
Aids be applied.
The distinguishing feature of most quasi-experiments is that they are carried out in the natural
conditions of everyday life. Examples include the study of the effects of family planning
programs, effects of supplemental food on the intellectual development of young children, and
whether agricultural practices change as a result of educational programs. Three more
frequently used quasi- experimental designs are described next.
Nonequivalent control group design

Chapter 9 Page 8
This design is frequently necessary because subjects cannot be randomly assigned to the
experimental and control groups. Under these conditions the best that can be done is to create
a nonequivalent control group.
Researchers in Sudan wanted to see if they could teach mothers ways of stimulating the
development of young children (Grotberg and Badri, 1986). Following a survey of families in
several areas of Omdurman, Grotberg and Badri assigned families to either an experimental or
a control group. Frequency matching was used to create roughly equivalent groups in terms of
socio-economic status, which was known to be associated with the kind of child-rearing
practices used by mothers. This eliminated the influence of this powerful variable. Before the
experiment began, all children in both the experimental and control groups were extensively
tested for developmental status.
The goal of the experiment was to teach mothers how to provide greater stimulation for their
children, and, thereby, enhance the psychological and social development of their children.
The experimental treatment, another term for the experimental variable, consisted of teaching
sessions with mothers, based on the developmental status of each child. Sessions lasting one
to two hours were conducted twice a week for nine weeks. Mothers and children in the control
continued their interaction as before, with no training sessions. At the end of the experiment,
all children in both groups were retested with the same set of psychological and social
development tests.
PREV NEXT

Chapter 9 Page 9
This design met the criteria for the nonequivalent control group design. Pretest and posttests
Introduction were conducted, which allowed for comparison of the relative gains in development between
the children in the experimental and control groups. In addition, students, who conducted the
Seeking teaching sessions with mothers, also observed and recorded how mothers interacted with their
causal children.
relationships The students observed that the mothers showed improved skill in stimulating their children in
beneficial ways. Measured changes in development were greater among the experimental
Alternative children in comparison to the control children. Boys showed greater improvement than girls,
explanations but this was because girls scored so high on the pretest that there was less room for further
improvement.
Internal validity
Interrupted time series design
The classical In a time series design, there is no control group to compare with the experimental group.
experiment There is just one group for which there are multiple measures for some variable before and
after some event that could affect the variable being studied. This design is a variation on the
Quasi- trend study described in Chapter 5. A researcher would look for a situation where there are
experimental designs multiple measures for some variable and where some condition or event occurs that might
influence that variable. The intervening condition might be a law, such as a large increase in
the tax on cigarettes. A researcher interested in the effects of this law on the sale of cigarettes
Quasi-
could get data on tax revenues from the sale of cigarettes for six months preceding and
experimental designs following implementation of the law and then see if there was any change in sale revenues
after the law went into effect.
External validity
The general pattern of observations for this design is:
Further 0000X0000
variations
in experimental where the measurements or points of data collection are represented by the 0s and where the
design X represents the event - in this case the increased tax on cigarettes.
Multiple measures before and after design
Strengths &
limitations of This design combines features from the nonequivalent control group design with the
interrupted time series design. An experimental and control group are used and multiple
experimental
measurements have to be available for the variable of interest both before and after some
research event occurs that could cause a change in the variable. This design can be applied in school,
organizational, and other social settings. To illustrate, let's say we learned that a change was
Aids planned in the methods of teaching reading in a local elementary school. We also are given
access to the reading grades for children from this school for some years before and following
the change. We also obtain access to reading grades for children from one or several other
elementary schools with student populations similar in all major respects to the children in the
school using the new method.
With these conditions, we could design a quasi- experiment. We could compare the trend in
reading grades prior to the change in the school that changed methods - the experimental

Chapter 9 Page 9
school - with the trend for the same period in the control schools. In making this comparison,
we would expect to see approximately the same trend in reading grades in both groups in the
period before the new reading instruction was started. The trend might show little change
from year to year or perhaps a small increase or decrease, but it should be about the same for
each group. (If there is a large difference, we would have to abandon the experiment because
the two groups are not equivalent at the outset of the experiment).
Then, we note when the change was made and see if the trend in reading grades is different
between the two groups following the change. If the new teaching method was effective - as
the school administrators and teachers hoped it would be - then the grades for the children in
the experimental school should show a greater increase than those in the control group. If this
in fact happened, then it would suggest that the new teaching methods contributed to an
increase in reading grades. But because other factors cannot be ruled out, as a true experiment
with randomized assignment would allow, we cannot say for certain that the new teaching
method alone is responsible for the increase. On the other hand, if no difference was found
between the grades of the two groups, we are left with a problem. Was the failure to find a
difference because the new method was not that effective; was the lack of difference due to
poor measurement of the children's reading skills; or were other factors involved? With this
design, we would not be able to sort out the reasons for not finding the expected result.
PREV NEXT

Chapter 9 Page 10
Two group, posttest only design

Introduction
The logic behind the quasi-experimental design can also be applied to investigate factors that
may have influenced an event that has already occurred. Osman (1995) used this design in
Seeking
his investigation of the impact of the Rahad rural development project on fertility. He
causal compared fertility rates in eleven villages that were resettled on irrigated farmland with
relationships fertility rates in six villages from the same region, but that did not receive irrigation for their
lands. The two sets of villages were matched on a number of characteristics, based on data
Alternative from a previous survey. Data on fertility were collected later from 853 women living in the
explanations villages with irrigation and from 356 women in the villages with dry lands. Fertility rates
remained high among both groups of women, but recent rates had increased slightly more in
Internal validity the villages that benefited from irrigation. Another result was a greater decline in child deaths
in villages with irrigated land.
The classical One group, pretest and posttest design
experiment
This design is based on one group, a pretest measurement of a variable, the application of an
Quasi- experimental variable, and a posttest measure of the variable measured in the pretest. The
experimental designs example involving reduction of prejudice, used at the beginning of this chapter, was based on
a one group design. Because no control group was used, we cannot say with certainty that any
observed decline in levels of prejudice in the group studies was due to the program they
Quasi-
experienced.
Sometimes, however, the one group design is the only one that can be used. Let's say we
External validity wanted to see if a workshop could make managers more sensitive to the needs of employees
they supervise. We could develop two measures of an employee sensitivity scale, administer
Further one at the beginning of the workshop, conduct the workshop, which would be our
variations experimental variable, and then a week or so later, have the participants respond to the
equivalent sensitivity scale.
in experimental
design With this design, at least we would have before and after data. The short time period involved
would reduce, if not eliminate any threats associated with history, maturation or experimental
Strengths & mortality. Delaying the posttest for a about a week will help reduce reactivity and testing
limitations of effect, although instrumentation and other threats would remain. These limitations would have
experimental to be acknowledged in a report describing this study.
research For additional discussion of quasi-experimental designs, we suggest you visit:
Aids Quasi-Experimental Design. This site presents information on quasi-experimental designs

not covered in this chapter and has links to additional pages with information on
quasi-experimental designs
True and Quasi-Experimental Designs. While this site focuses on designs used in
evaluation, it has a lot of good, general information of both kinds of designs.
Overview: Experimental and Quasi-Experimental Research, an overview of basic

Chapter 9 Page 10
concepts and an introduction to experimental research; described steps involved

PREV NEXT

Chapter 9 Page 11
External validity
Introduction
External validity refers to the degree that the results of an experiment can be extended beyond
the conditions of the experiment to conditions of everyday life. The purpose of research,
Seeking
including experiments, is to learn something about the behavior of people. Obviously, this
causal knowledge is useful to the extent that it describes normal, everyday behavior. But this
relationships objective is often undercut by efforts to achieve internal validity. As conditions from
everyday life are changed to reduce threats to internal validity, subjects in the experimental
Alternative and control groups can become different from those in the general population. Consequently,
explanations while results may have high internal validity, they may lack external validity, meaning that
they cannot be generalized beyond the particular groups used in the experiment. In designing
Internal validity an experiment, each experimenter has to decide which requirement is more important, internal
or external validity, and seek a balance between them.
The classical Further variations in experimental design
experiment
So far we have discussed experiments in which an experimental variable is applied in the
Quasi- same way to all subjects in one experimental group. These represent "all or none"
experimental designs applications of the experimental variable. There are other ways to administer the
experimental variable. For example, instead of the all or none application, different levels or
amounts of the experimental variable can be applied to two or more experimental groups.
Quasi-
experimental designs As a way of illustration let's return to the example of reducing prejudice. Instead of using
only one experimental treatment and group, we could test the effects of three different ways of
External validity reducing prejudice. In this case, we would create three experimental groups, while still
retaining one control group. Subjects would be randomly assigned to one of the three
Further experimental or to the single control group. The three treatments could consist of the
variations following:
in experimental 1. Experimental group 1 in which subjects see a specially prepared film, are given
design material to read, and participate in discussion groups.
2. Experimental group 2 in which subjects are given the same material to read as given to
Strengths & members of group 1 and participate in discussion groups, just the same as those in
limitations of group one, but do nott see the film.
experimental 3. Experimental group 3 in which subjects receive only the reading material.
research In addition, we would create a randomly selected control group of about the size as each of
the experimental groups. Pretest data would be obtained from subjects in all four groups. The
Aids experiment would then be conducted and posttest measurements of prejudice would be
obtained from subjects in the three experimental groups and in the control group. With these
data we could determine:
1. Whether each of the treatments was effective; that is, were prejudice measurements
significantly lower among subjects in each experimental group as compared with those
in the control group.
2. If the most extensive treatment, represented by group 1, was more effective than the

Chapter 9 Page 11
less extensive treatments, as represented by groups 2 and 3.

3. If the combination of treatments, as applied to group 2 was more effective than just the
single treatment, as applied to group 3.
The additional data obtained from groups with multiple treatments are frequently useful in
making decisions involving development of programs. For example, suppose we found that
the treatment with the movies was not more effective in reducing prejudice than the
combination of reading material and participating in discussion groups. This information
could help in developing programs to reduce prejudice by showing that use of films, which
are more expensive to create and use, do not need to be used.
PREV NEXT

Chapter 9 Page 12
Strengths and limitations of experimental research

Introduction
Strengths
Seeking A properly designed and executed experiment is the most effective way of testing for the
causal effects of one variable upon another. Two powerful strengths of this method of research are:
relationships 1. Conditions other than the experimental variable that could affect the dependent variable
are minimized, if not eliminated completely; and
Alternative 2. The experimental variable can be manipulated or controlled by the experimenter.
explanations
Together, these conditions allow rigorous testing of causal hypotheses. These are hypotheses
Internal validity stating a cause and effect relationship. Experimental research also has other positive features.
Many laboratory experiments are brief, inexpensive, require little special equipment, and can
be easily carried out by one person. Laboratory-based experiments also can be easily
The classical
replicated, which allows for the accumulation of knowledge. Experiments can also be used to
experiment test the practical value of a new program, such as teaching techniques of family planning or
for improving knowledge of nutrition.
Quasi-
experimental designs Limitations
One of the main limitations of the experimental method is the difficulty of meeting the
Quasi- requirements for both internal and external validity. Carefully controlled experiments, which
experimental designs are often high in terms of internal validity, frequently have low external validity, while quasi-
experiments conducted in natural settings can be high on external validity, but often lack
External validity internal validity. In addition, experiments generally are not designed to provide empirical
generalizations to a larger population. Experimental and control groups generally are not
Further randomly selected from a population: Hence, anything learned about these subjects cannot be
variations used to describe parameters for any population. The small size of experimental and control
in experimental groups also limits generalizing from the experimental results. Finally, experiments in natural
settings can be time consuming, require considerable coordination, and can be costly.
design
Aids
Strengths &
Internet resources
limitations of
experimental There is a large body of information on the Web about the design and conduct of experiments
research and quasi-experiments. Throughout this chapter, we have cited a few sites. Far more
information can be found by entering "experimental design" or "quasi-experimental design" or
Aids some variation of these words in any search engine such as http://www.google.com or
http://www.yahoo.com.
PREV NEXT

Chapter 9 Page 12

Chapter 9 Page 13
Key terms
Introduction
● Causal hypothesis testing ● Multiple measures, before and after
Seeking ● Classical experiment design group
causal ● Non equivalent control design
● Contamination
relationships Control group ● One group, pretest and posttest design
● Experiment ● Posttest measurement
Alternative
explanations ● Experimental effect ● Precision matching
● Experimental group ● Pretest measurement
Internal validity ● Experimental mortality ● Quasi experimental designs
● Experimental variable ● Randomization
The classical ● Experiementer bias ● Single group design
experiment
● External validity ● Subject reactivity
Quasi- ● Frequency matching ● Subjects
experimental designs ● History effect ● Testing effect
● Instrumentation effect ● Time series design
Quasi- ● Internal validity ● True experiment
experimental designs ● Interrupted time series ● Two group, posttest design only
● Maturation effect
External validity
Main points
Further 1. The model for the classical or true experiment is based on two equivalent groups,
variations referred to as the experimental and control groups, each of which is measured on the
in experimental dependent variable before the experiment begins (the pretest measurement); the
design experimental variable, also called the experimental treatment, is administered to the
experimental but not to the control group; the dependent variable is measured again (the
Strengths & posttest) at the end of the experiment, and the results for the experimental and control
limitations of groups are compared. It is expected that change in the dependent variable in the
experimental experimental group will be greater than that for the control group. This change is then
attributed to the effect of the experimental variable.
research
2. Equivalent experimental and control groups can be created by randomization or by
Aids precision or frequency matching. Randomization is the preferred method for creating
equivalent experimental and control groups.
3. Experiments are the strongest way to test hypotheses.
4. In conducting experiments, investigators have to guard against contamination. This is
the accidental or uncontrolled exposure of the control group to the experimental
treatment.
5. Quasi-experiments are based on some but not all the features of the classical

Chapter 9 Page 13
experiment. Sometimes quasi- experiments are the closest approximation to conditions

of the classical experiment that researchers can attain.
6. Internal validity refers to the extent that influences external to the experiment are
minimized. External validity is the degree to which results of an experiment can be
generalization to conditions of real life.
7. Quasi-experiments frequently do not meet the criteria for internal validity, but generally
are higher than true experiments in terms of external validity. True experiments are
high on internal validity, but often lack external validity.
8. Experimental designs can be varied by having several levels of experimental treatment
with two or more experimental groups and an equivalent control group.
9. Strengths of the experimental method include control over conditions that can affect the
dependent variable and the ability to apply the independent or experimental variable in
controlled ways. Many experiments can also be carried out easily, quickly, and
inexpensively by a single investigator. The main limitation of the experimental method
is that conditions necessary to ensure internal validity can make it difficult to generalize
results to the conditions of everyday life.


Chapter 10 Page 1
Chapter 10. Conducting Surveys: Questionnaire Construction

Introduction
Introduction
Characteristics It is quite possible that your first research project will be a survey. If not, and you continue as
of surveys a researcher, you will certainly use this method at a later time. Therefore, we discuss survey
research in some detail.
Planning a
Our discussion of survey techniques extends over two chapters. This chapter focuses on
survey development of questionnaires, the instrument used in conducting a survey. Researchers who
like to be precise about research terminology often refer to a questionnaire used for
Questionnaire interviewing persons as an interview schedule and reserve the word questionnaire for an
construction instrument that respondents complete on their own. We do not make this distinction: we call
either a questionnaire.
The Final
The following chapter discusses the related process of using a questionnaire in conducting
questionnaire
interviews. Together, the two chapters will prepare you to plan and conduct a survey.
Qualitative Characteristics of surveys
surveys
Surveys share three general characteristics:
● Based on probability sampling;
Internet-based
surveys ● Use of a questionnaire to collect data; and
● Data are analyzed using statistical methods.

Response rate
Probability sampling
Strengths and Surveys can be conducted by asking all members of the entire target population to respond.
limitations of This would result in a census and any calculations, such as percentages or means, would
the survey represent the parameters or values for variables for that population. Population surveys,
method however, are practical only when populations are quite small, about 1,000 or fewer members.
For larger populations, sampling is the only practical way to collect data. Details of sampling
methods are discussed in Chapter 8.
Improving
survey results Use of a questionnaire
Aids A questionnaire is a carefully prepared set of questions designed to produce the data needed
to answer a research question. The term item is generally used in place of question because
questionnaires frequently contain statements that respondents are asked to respond to in
addition to questions
Statistical analysis
Questionnaires frequently contain many items. Samples include hundreds, even thousands of
respondents, each of whom, if all goes well, answers every question. As a result, most
surveys produce a large amount of data. The only practical way to analyze data from most
surveys, therefore, is by means of statistical analysis. This may be done by hand, although

Chapter 10 Page 1
today computer-based software packages speed up analysis and removes a large source of
errors that can occur with hand analysis. Also, use of a statistical software permits more
extensive and thorough analyses of data. Statistical analysis procedures are described in Part
4 of this book.
PREV NEXT

Chapter 10 Page 2
Planning a survey
Introduction
Figure 10.1 displays the typical steps researchers follow in planning and conducting a
survey. As with every research effort, the researcher begins by specifying the problem or the
Characteristics
question to be answered by the investigation. The next step shown in Figure 10.1 is to review
of surveys the literature. Figure 10.1 shows an additional step - conducting exploratory research. Often,
even limited exploration of a topic or problem helps in defining the research question and in
Planning a selecting wording for items. Exploratory research may be based on informal interviews with
survey persons similar to the respondents who will later be questioned, analysis of comments from
members of focus groups, or results from informal, everyday observation of the group being
Questionnaire studied.
construction
As shown by the arrows leading back to the research question in Figure 10.1, results from
the review of literature or exploratory research may lead to modification or even
The Final abandonment of the original research question. Following the final decision on the research
questionnaire question, the researcher engages in conceptualization and operationalization of the variables
being measured. These processes are described in Chapter 6 and need no further comment.
Qualitative In survey research, operationalization of variables or indicators is carried out in the way
surveys items in the questionnaire are written. The bi-directional arrows in Figure 10.1 show the
interaction that goes on between conceptualization and operationalization. Initial concepts
Internet-based may be modified as operations are defined
surveys
Selection of a sampling method and development of a sample design are shown in Figure
10.1, but need no additional comment. These operations are described in Chapter 8. We only
Response rate
wish to remind you that development of a sample design is an important part of conducting a
survey.
Strengths and
limitations of The remaining steps are:
the survey ● Planning for all aspects of the survey;
method ● Constructing the questionnaire;
● Conducting the survey;

Improving
● Preparing data for analysis;
survey results
● Analyzing and interpreting the data;
Aids ● Drawing conclusions; and
● Writing the report.
This chapter discusses planning a survey and constructing a questionnaire. Conducting a

survey by means of interviewing, the most common method used in developing countries, is
the subject of the following chapter. Preparing and analyzing data and drawing conclusions
are treated in Part 4. Chapter 20 describes how to prepare a research report.
In addition to our discussion of the survey research method, you might want to look at some
Web sites that discuss how to plan and conduct surveys. Here are three you might find

Chapter 10 Page 2
helpful:
Overview: Survey Research, describes different types of surveys, methods used, advantages
and disadvantages, reliability and validity, ethics, response rates, and defines key terms
associated with survey research
Selecting the Survey Method, presents some rules for making decisions about using the
survey method, including decisions whether to enumerate or use a sample; discusses
sampling issues, construction of a questionnaire, and ethical issues and practices
Survey Research, examines issues involved in selecting the survey method; discusses types
of surveys, including the mail questionnaire, group administered questionnaire, and surveys
based on interviewing; and provides detailed guidance on constructing a questionnaire.
PREV NEXT

Chapter 10 Page 3
Introduction
Characteristics
of surveys
Planning a
survey
Questionnaire
construction
The Final
questionnaire
Qualitative
surveys
Internet-based
surveys
Response rate
Strengths and
limitations of
the survey
method
Improving
survey results
Figure 10.1. Steps in planning and conducting a survey
Aids Conducting a survey requires careful planning and making a number of critical decisions.
Three immediate decisions are:
● What method to use to collect data - personal interviews or some other method;
● Whether to use a structured or unstructured method of interviewing; and
● How to obtain permission or approval that may be needed to conduct the survey.
PREV NEXT

Chapter 10 Page 3

Chapter 10 Page 4
Choosing a data collection method

Introduction
Personal interviews. In developing countries, almost all surveys are based on personal
interviews with respondents. Interviews have to be used when respondents are illiterate or
Characteristics
are unable or unlikely to complete a questionnaire on their own.
of surveys
Telephone interviewing. In industrialized countries, where almost every household has a
Planning a telephone, telephone interviews are a preferred method of conducting a survey. Telephone
survey interviews require far less time than personal interviews; however, are more impersonal than
face-to-face interviews. It is much easier for respondents to say no to an interviewer over the
Questionnaire telephone than to a person standing before them. Also, telephone interviews generally have to
be kept brief; respondents are less likely to continue with interviews lasting over ten or
construction
twenty minutes. Consequently, only a few, very carefully worded questions can be asked.
Still, for some investigations with a target population that has telephones, this method may be
The Final appropriate and practical.
questionnaire
Self administered questionnaires. Another way to conduct a survey is to mail or deliver the
Qualitative questionnaire to respondents and ask them to complete the questionnaire on their own. This
surveys method can be used with respondents who are literate and who can be trusted to take the time
to complete the questionnaire by themselves.
Internet-based Mulinge and Mueller (1998) used a self-administered questionnaire in their study of job
surveys satisfaction among agricultural technicians in Kenya. Self-administered questionnaires are
often used with samples of secondary students (Cook, 2001) or with university students
Response rate (Hevi-Yoboe, Fanslow and Cowan, 1986). A commonly used method is to visit with each
respondent, explain the questionnaire, leave it with the respondent to be completed, and then
Strengths and return later to pick up the completed questionnaire and review it with the respondent.
limitations of When respondents are able to complete a self-administered questionnaire on their own, this
the survey technique offers a number of advantages. A large amount of data can be obtained quickly and
method inexpensively from a far larger sample than by interviewing. Conclusions based on larger
samples offer a stronger basis for generalizing to the target population. The down side is that
Improving response rates are generally much lower with self-administered questionnaires than for
survey results personal interviews.
A Web site, Plus and Minus of Survey Methods, compares the advantages and
Aids disadvantages of five ways of conducting a survey. These methods are: collecting date from a
group of respondents; sending questionnaires to respondents by mail, called a mail
questionnaire; giving questionnaires to respondents for them to complete by themselves,
referred to as a self-administered questionnaire; using a questionnaire in a personal interview
situation; and collecting date by means of a telephone interview. Each method is compared
on 16 characteristics. In addition to our discussion, looking at this site may help you select
the method you will want to use.
Internet surveys. The survey method has also been adapted to the Internet age. With the
development of Web sites and use of email, made possible by the global spread of the

Chapter 10 Page 4
Internet, the number of email and Web surveys is increasing. These methods are discussed
briefly later in this chapter.
Since almost all surveys in developing countries use some form of personal interviewing, our
emphasis will be on questionnaire development for use in personal interview settings.
Interviews can be conducted using either structured or unstructured interviews or some
combination of both methods.
Method of interviewing
The choice between these two methods of interviewing usually depends on the purpose of the
research. For exploratory and qualitative investigations, researchers usually use unstructured
interviewing. These interviews are based more on the style of casual, everyday conversation.
The researcher starts with a few initial questions and then, depending on the responses
obtained, asks additional questions. Responses are recorded or summarized in the words of
the respondents. In the course of an unstructured interview, the interviewer may also ask
specific questions prepared in advance of the interview. There is no predetermined order in
asking questions or in the exact form of questions. Instead, based on what the investigator
wants to learn about and responses already obtained, the investigator decides on the order and
nature of questions appropriate at any stage of the interview.
Unstructured interviews can take many forms. Since unstructured interviewing is almost
always used in field or observational studies, we discuss this approach more fully in Chapter
13, which deals with these topics, and in Chapter 15, Rapid Rural Appraisal, which is
widely used as a research strategy in developing countries.
When the purpose of research is to describe a set of variables precisely, particularly
quantitatively, structured interviewing is almost always used. This choice leads to
construction of a detailed set of questions commonly referred to as a questionnaire. Each
respondent is asked each question in exactly the same way, in so far as possible. For most
items, responses are recorded by checking one of several response categories.
PREV NEXT

Chapter 10 Page 5
Obtaining permission
Introduction
Early in your project planning you will have to find out if you will need permission from
local authorities or leaders to conduct your survey. Permission or approval is frequently
Characteristics
needed before a survey can be conducted in rural areas or in institutions such as schools,
of surveys hospitals, offices, or factories. If you are know local leaders or other influential persons in the
population you intend to survey, they may be able to assist you in obtaining any needed
Planning a permission. Even in these favorable circumstances, however, you may need to show them
survey how the survey will be of value to the population and could be of benefit to the leaders as
well. You will want to prepare a short, clear statement that describes the purpose of the
Questionnaire survey, its importance, your sponsorship (your university, for example), the size of the
construction sample, and what will be required of the respondents. If you do not know persons who can
speak on your behalf, try to find a leader or influential person to act as your sponsor. Your
The Final prepared statement should answer the questions you expect to be asked. When permission of
questionnaire local leaders or authorities is not granted, you will have to revise your plans.
Questionnaire construction
Qualitative
surveys Constructing a questionnaire requires a large amount of time and careful attention to details.
The quality of data obtained depends on how well the items comprising the questionnaire are
developed. Poor questionnaire development will almost always produce poor data and useless
Internet-based
conclusions. Careful questionnaire development is essential for the collection of valid and
surveys reliable data, which are the only basis for sound conclusions.
Response rate Steps in questionnaire construction
Box 10.1 lists the typical steps involved in constructing a questionnaire. The preparatory
Strengths and steps are included to remind you that use of questionnaire depends on certain prior decisions.
limitations of These include clearly stating the question you wish to answer, completing the necessary steps
the survey of conceptualization and operationalization, and other topics covered in previous chapters.
method The remainder of this chapter focuses on the steps listed in the lower part of Box 10.1,
beginning with the development of a candidate set of items for measuring the variables you
Improving intend to investigate.
survey results
Aids

Chapter 10 Page 5
Box 10.1. Steps in constructing a questionnaire

Preparation
1. Express the research question clearly and precisely.
2. Decide whether to use a personal interview, self-administered
questionnaire, telephone interview, or Internet survey.
3. Decide whether to use structured or unstructured interviewing.
4. Conceptualize to identify specific dimensions of variables you will
measure.
5. Select the indicators you plan to use.
Questionnaire construction
1. Develop a candidate set of items for use as indicators.
2. Evaluate each item see Box 10.2.
3. Select the format for each item - closed, open-ended, chart,
contingency, etc.
4. Organize the items into a draft questionnaire.
5. Add an introduction and necessary instructions.
6. Ask others to critique your questionnaire.
7. Thoroughly pretest the questionnaire.
8. Based on each pretest, revise the questionnaire.
9. Do a final, careful review of the questionnaire.
10. Reproduce the copies you will need (with a few extra).
PREV NEXT

Chapter 10 Page 6
Writing and evaluating the questions you will ask is the very heart of questionnaire design. A
Introduction number of Web sites provide valuable suggestions for constructing questions. Many provide
examples of questions that produced scientifically valid data in the cultural context in which
Characteristics they were used. For further guidance on writing questions and designing questionnaires, you
of surveys want might to visit any of the following sites
Constructing the Survey, provides clear, useful guidance for constructing a questionnaire;
Planning a includes links to pages hat will help you make decisions about content, types, wording, and
survey placement of questions; forms of response formats and all the other details that must be
managed in developing an effective questionnaire
Questionnaire
construction Hints for Designing Effective Questionnaires, provides specific, detailed guidance for
designing quality questionnaires and how to avoid common mistakes in constructing a
The Final questionnaire
questionnaire Household Sample Surveys in Developing and Transition Countries, a comprehensive
discussion dealing with survey research; consists of 21 chapters that cover survey design,
Qualitative sample design, non-sampling errors (meaning any errors other than due to the expected
surveys sampling error), survey costs, and issues related to the analysis of data; also presents some
case studies of sample surveys used to collect household data
Internet-based
Methods - Survey Methods, provides extensive links to sites with information related to
surveys
surveys; includes sets of links to academic, government, commercial and private sources;
also lists surveys on the Internet and names and Web address of organizations with
Response rate information about conducting surveys
Strengths and Overview: Survey Research, describes different types of surveys, methods used, and key
limitations of terms associated with survey research
the survey Questionnaire Design, describes the steps in designing and administering a questionnaire,
method including: defining the objectives of the survey, specifying the sample group, writing the
questionnaire, administering the questionnaire, and interpreting the results
Improving
survey results Questionnaire Design and Analysis, presented as a workbook; covers all aspects of
designing a survey and constructing and testing a questionnaire; also describes ways of
Aids analyzing survey data
Developing a candidate set of items
Potential items can come from any sources. Three frequent sources of items are:
● The research literature;
● Previously used items; and
● Discoveries from exploratory research.
In reviewing the literature, you may become interested in certain concepts. The next step is to

Chapter 10 Page 6
see how these concepts have been operationalized. Frequently, concepts are operationalized
in the form of questions or as statements the respondent is asked to react to. (Go back to
Chapter 7 for examples of statements used as items). These may be single items or a set of
items making up a scale or index. Exploratory research is another effective way to develop
items. Unstructured interviews can suggest topics that can lead to the development of items.
Focus groups, described in Chapter 13, can be another source of concepts and items.
Regardless of their source, candidate items should be carefully evaluated in terms of two
criteria. First, is the item culturally appropriate for the group you intend to survey? Items
borrowed from original use in surveys in Western or industrialized countries should be tested
to be sure they are understood and acceptable in your cultural setting. Second, candidate
items should be evaluated to make sure they meet the criteria for sound questionnaire
construction.
PREV NEXT

Chapter 10 Page 7
Writing and evaluating items

Introduction
In writing and evaluating items, we face a dilemma. We want each item to be clear and
understood in the same way by every respondent. Yet, we know this will not happen every
Characteristics
time an item is used. Because of differences among individuals, various respondents interpret
of surveys items differently. It is impossible to tailor each item exactly to the mental framework of each
respondent.
Planning a
survey Some guidelines for writing items that minimize this problem are summarized in Box 10.2.
There are, however, no foolproof guidelines for writing items. Our advice is to plunge in and
Questionnaire try writing items, using the following guidelines.
construction Box 10.2. Guidelines for writing and evaluating items for a questionnaire
1. Use your research question as the first and most important criteria for
The Final selecting items
questionnaire
2. Be sure the item is culturally appropriate and absolutely necessary for
measuring a concept.
Qualitative
3. Ask only items you know respondents can and will answer.
surveys
4. Express items clearly and simply; shorter is better.
Internet-based 5. Use neutral language.
surveys 6. State items in specific, concrete form.
7. Do not use double-barreled questions.
Response rate 8. Do not use leading or biased questions.
9. Avoid stating questions in negative form.
Strengths and
limitations of 10. In sets of items on the same topic, mix favorable and unfavorable items.
the survey
Use only relevant and necessary items. Each item should ask for responses necessary for
method answering your research question. Unnecessary and irrelevant items put an necessary burden
on respondents, waste your time, and produce data that you will not use. A good way to test
Improving the relevance of an item is to anticipate exactly how the data from each item will be analyzed.
survey results After you complete the chapters on data analysis, you will be able to do this. For now, we
urge you to think ahead and include plans for data analysis at the time you are preparing a
Aids questionnaire. If the item is not essential for measuring a variable or you don't know how
you will analyze the responses to the item, drop it.
Only ask items respondents can and are willing to answer. There is no point in asking
questions that respondents have little knowledge about or would be unwilling to answer.
Information from the review of literature, your own knowledge of the group you are
surveying, and results of a pretest should help you judge whether an item should be used. A
pretest is a test of the usefulness of the draft questionnaire. Pretests are conducted like mini
surveys, generally with a small sample of respondents similar to the respondents whom you
will be collecting data from later. A pretest provides an excellent way of testing whether

Chapter 10 Page 7
respondents are able and willing to respond to items. We describe the important sep of
pretesting later in this chapter.
Express each item clearly and simply. In writing items use simple, everyday words like
those used by the group who will answer the questions. Do not use social science
terminology, slang, abbreviations, or the language of the educated elite. Also, state questions
in short, simple sentences, but make sure each sentence is clear.
The following is certainly a short question: "How long have you lived here?" But is it clear?
No, it is not. What does "here" mean? Does "here" refer to this house or this area (village,
town, part of a city,) or the whole city? Various respondents might interpret the question
differently and therefore give different answers than you might expect. Instead, a series of
questions might be asked to establish the respondents' residential histories, beginning with:
● "When did you begin living in this house?"
● "Where did you live before you began living in this house?"
● "When did you begin living at the previous place?"
Questions would be repeated to cover the time period you are interested in. These kinds of
questions provide clear, specific responses and data.
PREV NEXT

Chapter 10 Page 8
Use neutral language. Words carry emotional significance and can influence how persons
Introduction respond to questions. For example, the question, "Do you think the lazy immigrants from
Xuanda (a mythical country used in place of a real one) should be allowed to get food that
Characteristics belongs to our hardworking citizens?" We grant that this is an extreme item, but it illustrates
of surveys the use of emotionally charged words "lazy," "immigrants," our "hardworking citizens."
Items with emotionally laden words arouse fears and introduce bias by influencing one form
Planning a of response over others.
survey State items in specific, concrete form. Items can be made concrete by referring to the
specific activities rather than using broad, general descriptions for something. Suppose, for
Questionnaire example, you wanted to know how frequently persons participated in local social activities.
construction You might be tempted to ask a broad question such as: "How regularly do you participate in
local social activities?" If you did, you might find that many respondents would be confused
The Final by what is meant by "regularly" and by "local social activities." To some, regular might mean
questionnaire every day; others might think their participation once a week is pretty regular; while still
others might have a different concept of what regular means. The same could be true for the
meaning of local social activities. Does this include informal gatherings or only official
Qualitative
events? Are religious activities included or not? With all the possible confusion over the
surveys meaning of the item, the data could mean almost anything.
Internet-based Both kinds of problems can be avoided by expressing the item in specific, concrete terms.
surveys Instead of using a vague word like "regular," the item could be rephrased by asking how
many times in the past week the respondent participated in a list of specific activities that
Response rate make up the social life of the group. Rephrased, the item could read, "During this past week,
how many times did you participate in each of the following activities?" which would be
followed by a list of specific activities. Week was used in this item, but month or some other
Strengths and
time period could be used if this were more appropriate.
limitations of
the survey To further illustrate the value of writing concrete items, consider this simple item: "What is
method your income?"While it meets the criterion of being simple, it is neither clear nor concrete.
Respondents, in fact, are presented with an ambiguous, abstract question. Some might think
Improving in terms of weekly income; others in terms of monthly or even yearly income. Further, some
might think in terms of only cash income while other might include income from trade or
survey results
bartering. The researcher would have to rethink what he or she really wants, whether the
question should ask about cash income only or for income from other sources as well and for
Aids what time period.
Do not use "double-barreled" questions. These are items that contain two or more
questions in one sentence. For example, an item might read: "Do you think more scholarships
should be provided for women and members of minority groups?" How would you answer
this question? Would a "yes" answer mean yes to both parts or only to one part? A "no"
answer would be equally ambiguous. To avoid such confusion, it would be better to write
two items, one for women and one for minorities. Then a "yes" or "no" responses would be
clear for each item.

Chapter 10 Page 8
Do not use leading questions. These are biased questions: by their wording they suggest one
response over other possible responses. Suppose you were asked: "The President of our
country believes women should have fewer babies, what do think?" This is a leading
question. By mentioning the President's opinion, the question might "lead" respondents to
you to give an answer that they think is the correct one, not necessary their own. The same
item could be expressed in a more neutral form as: "Some people say too many babies are
being born; others say this is not happening. What do you think?"
Do not use negative sentences. Sentences with a "not" or other negative expression are not
clearly understood by many respondents. Many will not hear the "not" and answer as if the
statement were a positive sentence entirely opposite from what you expect. The risk of
misunderstanding is even greater when a negative item is used in a self administered
questionnaire. It is very easily to read over the "not' and answer the opposite of what was
asked.
PREV NEXT

Chapter 10 Page 9
Closed and open-ended items

Introduction
The wording of items also depends on their format. Two main formats are used. Closed or
fixed choice items provide a certain number of alternative responses from which respondents
Characteristics
select the one response that best matches what they think. Responses to closed items are
of surveys created by the researcher when items are written. Open-ended or free response items are ones
for which responses are not provided.. These consist of a question, followed by space where
Planning a the interviewer records the answer provided by the respondent or the respondent writes his or
survey her own response. To illustrate, following is an example of the same item written first as a
closed item, then in open-end form.
Questionnaire
Closed: "How would you rate instruction in your social research methods course?"
construction
[ ] worst than most courses
The Final [ ] about the same as most courses
questionnaire [ ] better than most courses
Open: "How would you rate instruction in your social research methods course?" (This
Qualitative question would be followed by a blank space on the questionnaire for recording responses).
surveys
Each form has certain strengths and limitations. Closed items make the least demands on
Internet-based respondents. Respondents only have to select one answer from those provided. This
surveys advantage allows for quick collection of data in a standard way and also permits fast and easy
analysis of responses by quantitative methods. But closed items also have certain limitations.
For one thing it is difficult to check the validity of responses. the respondent simply picks a
Response rate
response without giving any more information. Any selection may be an accurate, honest
reply or it may be partially or entirely inaccurate. Another weakness is that data are limited to
Strengths and what the researcher provides in the way of response categories. there is no opportunity to
limitations of capture exactly how a respondent feels about something.
the survey
method The strengths and limitations of open-ended items are just about the opposite of those given
for closed items. Because respondents can say what is important to them, open-ended items
frequently provide rich, detailed, and often unexpected information. Opportunities are greater
Improving
for checking the validity of responses and respondents. In addition, open-ended items can be
survey results used when you have little previous knowledge about some area of inquiry. They are
particularly useful for conducting exploratory research. Open-ended items are also useful for
Aids understanding complex social relations because they allow respondents to say how they feel
about things in their own words.
Open-ended items also have several drawbacks.one is the greater burden placed on
respondents and interviewers. Respondents are required to give an answer in their own
words, which may create some embarrassment or difficulty for some respondents. Also,
interviewers have to try to copy down what the respondent says as close as possible in their
own words. For respondents who like to talk a lot, copying can become quite a task. As a
result, interviewing based on open-ended items is slower and, for interviewers, a more
demanding task. But the greatest problem in using open-ended items lies in the difficulty of

Chapter 10 Page 9
analyzing the responses obtained. Content analysis has to be applied to the responses for
each open-ended item. This method of data analysis, which we describe in Chapter 12, is
always a lengthy and many times a very demanding process.
Closed items, in general, are used more frequently than open-end ones. Closed items are
better for getting specific, concrete responses. Open-ended items, on the other hand, are
superior when you want detailed, unique responses rather than standardized ones as produced
by closed items. In practice, researchers frequently use both formats in developing
questionnaires.
Response formats for closed items
The value of data obtained from closed items depends in large part on the set of response
categories used. Application of the following guidelines should help in constructing useful
response categories:
Use only a limited number of response categories. There must be a minimum of two
categories; frequently three to five are used; but seldom are more than ten used. If you find
that you cannot specify or limit the number of categories, consider using an open-ended
question.
Be sure categories are mutually exclusive. We have made this point before, but it bears
repeating. Each category must be independent of the others; there should be no overlap. The
following are mutually exclusive with respect to age: 0-14, 15-19, 20-24 and so on. The
following are not exclusive: 0-15, 15-20, 20-24 and so on. If the latter set were used, a
person aged 15 or 20 could be placed in more than one category, which would defeat the
purpose of analysis.
PREV NEXT

Chapter 10 Page 10
Be sure the categories are exhaustive. This means that all possible answers can be
Introduction recorded. The most common way of meeting this requirement is to add a category called
"other" to each response list, as shown below. If a respondent cannot find a suitable answer,
Characteristics he or she can give an answer which the interviewer would record on the line following the
of surveys category labeled "other." The "other" response category usually appears as the last response
in a list, as:
Planning a [ ] Other (specify) _______________________
survey
The word "specify" is usually added as a reminder to try to get a specific answer which will
Questionnaire help in analyzing responses.
construction Another way to ensure exhaustiveness is to leave the end categories for responses in
numerical form open. Thus, in constructing age categories, one could use:
The Final ● 19 or less
questionnaire ● 20-29
● etc.
Qualitative
● 60 or older
surveys
For variables measured at the ordinal level, use simple, easy to understand categories.
Internet-based Generally three to five categories, as shown in Box 10.3, are used. The formats shown in Box
surveys 10.3 are frequently used in constructing Likert scales (See Chapter 7 for more information
on Likert scaling).
Response rate
Box 10.3. Examples of response formats for closed items
Strengths and Frequency of doing something:
limitations of Frequently/seldom/never; or Always/frequently/sometimes/seldom/never
the survey For how important something is:
method Important/unimportant; or Very important/important/uncertain/somewhat
Improving unimportant/not at all important

survey results For how well something is working:
Good/inadequate/poor; or Excellent/good/adequate/poor/very bad
For attitudes toward something:
Aids
In favor of/not sure/opposed to; or Approve/not sure /disapprove; or
Strongly approve/approve/not sure/disapprove/strongly disapprove; or
Strongly agree/agree/uncertain/disagree/strongly disagree
Responses as numerical ratings. Responses can also be obtained in the form of numerical
ratings. Instead of selecting a word or phrase that best describe their responses, respondents
are asked to pick a number. The numbers might range from a low of 1 to a high of 5, such as:
1 (low) 2 3 4 5 (high)

Chapter 10 Page 10
For numerical ratings, a range up to 10 numbers is often used. Having a larger range allows
respondents to express a wider range of views or opinions, giving greater variation in
responses. With a wider range of responses, you also have greater freedom in analyzing the
resulting data. If there are few responses at the extreme ends of the range, down near 1 or at
the upper end of 10, responses in these categories can be combined with those in the next
lower categories. Even when you combine categories, you will still have a number to work
with. But, if you use only 5 categories, for example, that is the largest number of categories
available for analysis. Pretesting should help you decide how many categories to use.
For another view of response formats go to Response Format.
PREV NEXT

Chapter 10 Page 11
Combining closed and open-ended items

Introduction
When both formats are used to get data on the same variable, the investigator has to make a
decision on which format to present first. An open-ended item can be used as the first question on
Characteristics
some topic to get the respondent's own, original thoughts on that topic, before closed items
of surveys suggest any specific ideas to the respondent. Following the initial open-ended item, a series of
closed items can then be used to get specific data on that topic. The alternative is to ask a set of
Planning a closed items, followed by an open-ended question to get additional thoughts on the topic. In
survey general, starting with the open-ended item runs less risk of influencing how respondents answer
an open-ended question. When an open-ended item follows a set of closed items, the respondent
Questionnaire is fully aware of the nature of the questions and might be influenced by them. The degree of risk
construction involved in either approach depends in part on how sensitive the questions are. Pretesting, as we
explain shortly, can help you decide on the order of open-ended and closed items used to measure
The Final the same variable.
questionnaire Chart formats
Qualitative Another way to present items is to combine a number of questions and responses. This works
surveys particularly well when the same questions are repeated. Many household surveys, for example,
begin with questions about the number of persons living in the household and their
characteristics. A chart or matrix format, as shown in Table 10.1. is generally used to obtain
Internet-based
these kinds of data.
surveys
Table 10.1. Illustration of a chart format: same set of questions repeated for each household
member
Response rate
Strengths and Number Name of Gender Marital Highest Working

limitations of of household Relationship 1 (male) status year of 1 (yes)
the survey persons member to head [a] 2 (female) Age [b] schooling 2 (no)
method 1
Improving 2
survey results 3
Aids etc.
10
[a] Code for relationship to head: 1=wife, 2=son, 3=daughter, 4=other relative, 5=other person
[b] Code for marital status: 1=single, 2=married, 3=separated, 4=divorced, 5=widow or widower
This format has several important features. First, a lot of information can be recorded in a small
amount of space. This is one of the great advantages of the chart format. Any time you are going
to repeat the same questions consider using the chart format. Second, instructions can be easily
located along with the item to which they refer. This can be a big help during interviewing and
later in preparation of data for analysis.

Chapter 10 Page 11
Two of the response categories in Table 10.1 use precoded responses. Instead of writing out
words for the relationship of each household members to the head of the household, numbers are
used. Five ways persons can be related to the head are coded or represented by a number ranging
from 1 to 5. Any set of numbers could be used for these codes. They simply represent nominal
designation for categories. The same is true for the codes used for the marital status of household
members.
PREV NEXT

Chapter 10 Page 12
Precoding has a number of uses. It saves space on a questionnaire and allows the interviewer
Introduction to record responses quickly. The interviewer simply records a number and moves on to the
next person or item. Later, codes for responses can be quickly tabulated by hand or entered
Characteristics into a computer for more extensive analyses.
of surveys When the content of items varies but each has the same response categories, a related format
is an efficient way to present them. Table 10.2 illustrates this format.
Planning a
survey All responses shown in Table 10.2 are precoded. In this case, the same codes are used for
each question. Table 10.2 includes only 3 questions - just enough to illustrate use of this
Questionnaire format and coding. In an actual questionnaire, more items would be included. In this case,
coding has numerical meaning. The codes used form the basis for ordinal measurement.
construction
Responses could be added to develop a score for the degree to which one spouse or the other
has the most to say with regard to family decisions. In our example, the category of "wife
The Final almost always decides" is given a code of 5, the highest score. For these items, the direction
questionnaire of coding would have to be kept in mind when the data are analyzed and the results are
interpreted.
Qualitative
surveys Table 10.2. Illustration of a chart format: same response category for different questions
(based on pre-coded responses)
Internet-based Questions (1) (2) (3) (4) (5)
surveys
For instance, who usually has the most to
say about:
Response rate
Which couples you see most often?
Strengths and Which relatives you see most often
limitations of
Purchase of major household items
the survey
method
(1) husband almost always; (2) husband mainly;
Improving (3) husband and wife about equally; (4) wife mainly;
survey results (5) wife almost always
The example in Table 10.2 is based on use of the item in a personal interviewing situation.
Aids The interviewer, who, we assume is thoroughly familiar with the questionnaire, would ask
the questions and, knowing the codes, would check the box most closely matching the
respondent's answer to each question.
When self administered questionnaires are used, we do not recommend using codes to
identify response categories. Having to connect a code, such as "3" for "husband and wife
about equally" or any other code with its response puts an extra effort on the respondent. This
may lead to errors and result in invalid data. Also, seeing the numbers may give some
respondents the idea that some answers are "more right" or "better" than others, with
resulting invalid data. For self administered questionnaires, it is better to put the responses in

Chapter 10 Page 12
words at the top of the columns in any set of questions in a char format.
Other ways of asking questions are described on Web sites we mentioned earlier. In addition,
we recommend looking at:
Types of Questions, describes various forms of structured and unstructured questions,
including dichotomous questions (ones with only two answers), questions based on levels of
measurement, and contingency questions. You may also find the following two sites useful as
you think about the content and wording of questions:
Question Wording, raises questions about wording questions in a questionnaire
Question Content, provides detailed guidance for writing questions
PREV NEXT

Chapter 10 Page 13
Contingency questions
Introduction
Still another form of combining items is the contingency or filter question. This form is
used when you know that a certain question will be relevant only to some of the respondents.
Characteristics
There is no point in asking respondents to answer a question that does not apply to them.
of surveys Doing so will annoy most respondents and give useless data. The way to avoid this problem
is to ask first a more general question that will be relevant to all respondents and then
Planning a follow-up with more specific questions, depending on how the first and perhaps later
survey questions were answered. To illustrate, suppose you were investigating participation in
political activities.
Questionnaire
As shown in Figure 10.2, you could begin by asking each respondent whether he or she
construction
currently belongs to any political party. If the answer is "No," the next question
could be whether the respondent had ever belonged to a political party. Again, you
The Final would get either a "Yes" or a "No." Depending on the answer, you would ask a different set
questionnaire of questions. For those who say "Yes" to the second question, the next question might ask
for the name of the party, when the person had joined it, why and perhaps additional
Qualitative questions. For those answering "Yes" to the original question, "Do you belong to a political
surveys party now," the next question might ask for the name of the party, followed by when the
respondent had joined this party and perhaps other questions.
Internet-based
Contingency questions can get pretty complex. To make sure all contingencies are covered,
surveys we suggest building a contingency flow diagram like that shown in Figure 10.2.
Response rate
Strengths and
limitations of
the survey
method
Improving
survey results
Aids
Figure 10.2 Illustrative flowchart for contingency questions on membership in a political

Chapter 10 Page 13
party
PREV NEXT

Chapter 10 Page 14
Sensitive questions
Introduction
Some questions can cause respondents to feel anxiety, guilt, fear, or other strong emotions.
These may include questions about sexual behavior, death and grieving, or illegal activities.
Characteristics
Investigators have to decide whether this kind of topic should be investigated and, if so,
of surveys whether questions that might be considered sensitive should be included in a questionnaire.
There are no rules for deciding this kind of issue. Investigators have to rely on their
Planning a knowledge of the group being studied, their own judgment of the ethical issues involved, and
survey the views of other knowledgeable persons.
Questionnaire When a sound reason exists for asking sensitive questions, the researcher has to decide where
to place such questions in a questionnaire. Our suggestion is to place sensitive questions
construction
somewhere in the middle of the questionnaire. Beginning with routine or non-sensitive
questions allows time for trust or rapport to be established with the respondent. Still, it is a
The Final good idea to provide a brief introduction to sensitive questions before you ask them. The
questionnaire explanation can describe why the questions are being asked and why answers to them are
important. This will help prepare the respondent for the questions. Even so, there is no
Qualitative assurance that you will get valid, reliable responses. With sensitive questions, respondents
surveys are more likely to give false or misleading responses
The final questionnaire
Internet-based
surveys Organizing the draft questionnaire
After developing your questionnaire items, you will be ready to organize a draft
Response rate
questionnaire. This involves deciding on the order of items and adding an introduction.
Strengths and Decide on the order of items. For questionnaires with relatively few items this amounts to
limitations of deciding the order of the various items. For longer questionnaires, items are grouped into
the survey sections by putting items that ask about the same thing together. Items asking about the
method background characteristics of the respondents, for example, are combined, as are those
related to other topics.
Improving Whether or not sections are used, items should be placed in some kind of logical order. Time,
survey results for example, frequently provides a logical basis for ordering items. Other bases, such as life
histories or the natural cycle of events in a village, can be used as well, but whatever basis is
Aids used should make sense to respondents. For additional thoughts on placement of questions,
visit: Question Placement.
Add an introduction. To complete the draft questionnaire you will need to add a section for
use in introducing yourself to respondents. The introduction can also describe in general
terms the purpose of the survey, why the respondent is being asked to provide information,
and cover other points needed to get the interview started. Points to include in an introductory
statement are summarized in Box 10.4.

Chapter 10 Page 14
Box. 10.4. Points to include in an introductory statement on a questionnaire

1. Identify who or what organization is behind the survey. If you are
doing the survey as part of a class assignment tell the respondent and
give the name of the university (university-related research is usually
regarded positively and should help gain the cooperation of
respondents).
2. Explain the purpose of the research in broad, general terms, preferably
in terms that will appeal to the interests or values of respondents
(pretesting will help reveal an appealing explanation).
3. Explain how the respondent was selected; most respondents will
accept the idea of being chosen by chance to represent the population
of which they are part.
4. Indicate that information the respondent provides is important; explain
how the information will be used.
5. Assure the respondent that all information will be held in confidence;
that his or her privacy will be completely protected.
PREV NEXT

Chapter 10 Page 15
You may also want to include instructions to remind yourself about how to record answers
Introduction for certain items; and you will want to add a note at the end of the questionnaire as a
reminder to thank the respondent for his or her help.
Characteristics
We have referred to the questionnaire at this stage as a draft questionnaire. It should be just
of surveys that your first draft for use in pretesting, Actually, you may go through several successive
draft questionnaires as you conduct additional pretests. For each pretest, we suggest you
Planning a reproduce only enough copies of your draft questionnaire for use in the pretest, plus a few
survey extra to give to friends or colleagues and your advisor for review. Making only a limited
number of copies leaves you free to make changes, whereas, if you prepare a large number of
Questionnaire copies for the survey you may be reluctant to make needed changes.
construction
Length of the questionnaire
The Final Many beginning researchers make their initial questionnaires too long. We repeat, therefore,
questionnaire the earlier guideline: Include only items that are absolutely necessary to provide data for
answering your research question. Remember that both you and your respondents will
Qualitative become tired as interviews go on. As either or both of you become tired, errors are bound to
surveys increase. Almost always, a shorter questionnaire is better than a longer one.
Translation and back translation
Internet-based
surveys Sometimes a questionnaire has to be translated from the author's native language to the
language of the respondents. When this is required, the best method is to write the
questionnaire in your native language, using the guidelines in this chapter and further
Response rate
guidance you find in your review of the literature. Then have a person who knows both
languages well translate your questionnaire into the language of the respondents. You will
Strengths and have to discuss the translation of each item with the translator to make sure that the exact
limitations of meaning of each item is preserved. After this is done, have another person who knows both
the survey languages independently translate the questionnaire back from the new language to the one
method you used to create the original questionnaire. This back translation should match closely the
original questionnaire you wrote. If it does not, something was changed or lost in one or both
Improving translations. You will need to check for differences and then correct them.
survey results Pretesting
Aids A draft questionnaire is only that a first attempt to construct a scientifically reliable and
valid measuring instrument. The next step is to test the draft questionnaire thoroughly to
discover and correct any flaws that can be discovered. This is done by using the questionnaire
in interviews with a small sample of respondents who are similar to those who will be
interviewed in the later survey. Pretesting with 10 or so interviews is usually adequate to
discover any flaws in the questionnaire.
When conducting pretest interviews watch for any sign of difficulty or distress shown by
respondents. Obvious signs of trouble include failure to understand a question, refusing to
answer a question, difficulty in selecting a response, or giving confusing or irrelevant

Chapter 10 Page 15
responses. When signs of difficulty appear, ask the respondent about the problem. Talk about
it until you know what the problem is. Try alternative wording for troublesome items. Make
notes on the questionnaire for use later in correcting items.
Pretests also provide answers to questions that affect the success of your survey. For
example: Does putting an open-ended item before or after some closed items on the same
topic make a difference in responses? To answer this question, try using two versions of your
questions; some with the open-ended items first, and some with the open-ended items
following closed items. See if respondents answer differently, and then decide which way to
present either form for items in the final questionnaire. Other questions you might want to
pursue in pretesting might include the following:
● Do respondents understand the items?
● Are they willing to answer sensitive questions?
● How long is the typical interview?
● Are respondents comfortable with the length of the interview?
PREV NEXT

Chapter 10 Page 16
As a rule, interviews of an hour or so are acceptable to most respondents: Beyond an hour or

Introduction so, many respondents can become less cooperative, more guarded, and be less likely to
provide full, honest, and accurate responses. As this happens, the quality and value of your
Characteristics data will steadily decline. If your interview runs longer than an hour or if you see signs of
of surveys resistance from respondents, review your items and see what you can cut.
As part of your pretesting, time how long your interviews last - from the time you appear at a
Planning a place to the time you make your exit. Getting the length of an average interview is
survey important. This information will let you estimate how many interviews you can do in a day, a
week, or a month. With this information, you can determine if you will be able to interview
Questionnaire all the members of your sample in the time you have available.
construction
In addition, pretest interviews allow you to check how well you can conduct interviews. As
we point out in greater detail in the following chapter, interviewing is a social process and
The Final involves certain social skills. Interviewing also requires skill in asking questions and in
questionnaire recording responses. Some people are better at this than others, but we all can learn how to
improve our interviewing skills. Pretesting gives you an opportunity to test your interviewing
Qualitative skills, discover areas in which you feel uncomfortable or do not perform well, and to learn
surveys how to improve your interviewing skills. Pretest interviews also provide an opportunity to
test ways of introducing yourself and the survey to respondents. By the end of your pretest,
Internet-based you should be confidant that respondents receive your introduction positively and that you
surveys are comfortable using your questionnaire in interviewing.
Don't feel badly if your first draft questionnaire is not perfect. It is very hard to anticipate all
Response rate the ways persons will respond to questions. Experienced researchers expect difficulties with
questionnaires. They prepare for this by conducting a thorough pretest of a preliminary
Strengths and version, revise this, and, if necessary, conduct additional pretests before preparing the final
limitations of questionnaire. You should plan to do the same. If you have to make many revisions in the
the survey first or even second drafts of your questionnaire, another pretest is in order. Your goal should
method be to create a questionnaire that you know contains items respondents are able and willing to
answer honestly and one you are comfortable in using.
Improving Preparing the final questionnaire
survey results
At some point you will decide that no further improvements can be made in your
Aids questionnaire. At this time you will be ready to put the questionnaire in its final form. Our
final advice is that you make the questionnaire as neat as possible; allow plenty of space
between items; show all response categories in a clear and consistent manner; and allow
enough space for writing responses to open-ended items. It is a good idea to make about
10% more copies of the final questionnaire than you think you will need for interviewing.
You will want some extra copies to use for tabulating results and to include in an appendix of
your report.
Qualitative surveys
Most surveys are based on a quantitative approach, with use of a questionnaire and structured

Chapter 10 Page 16
interviewing. Surveys also can be based on a qualitative method, using unstructured

interviewing. With unstructured interviewing, the interviewer uses a conversational style to
obtain information from respondents. This approach allows an investigator to explore all
possible thoughts or views of the respondent and to get these in the respondent's own words
with all their emotional content. Unstructured interviewing often produces ideas that would
never have been found using a questionnaire.
An investigator using unstructured interviewing starts with a general idea of what he or she
wants to know and draws up a set of broad questions for getting relevant information. The
investigator starts an interview with a few initial questions, and lets the respondent take the
lead to a large extent. Respondents are encouraged to talk at length. The investigator selects
portions of responses that are particularly revealing and follows up with additional questions.
When the conversation wanders too far off the topic, the investigator brings it back to the
topic with a question or two. New questions can be introduced at any time as well, but there
is no predetermined order or specific working set in advance for questions.
PREV NEXT

Chapter 10 Page 17
Unstructured interviewing is an excellent way to get deeper into a subject and see it from the
Introduction respondent's point of view. This is done by asking more detailed or narrower questions. To
illustrate, let's say we were investigating how students felt about having to complete an
Characteristics independent research study as a graduation requirement. We might start off a broad question,
of surveys such as:
"Does your university require completion of an independent research project as a
Planning a graduation requirement?"
survey
After the conversation runs out on this question, we might ask:
Questionnaire "What do you see as the advantages or benefits of this requirement?" And then
construction follow with specific questions, such as:
The Final "What are the advantages for you?"

"For other students?"
questionnaire
"For your university?"
Qualitative After there are no new responses to these questions, we could shift the focus of the
surveys conversation again by asking:
"What do you see as the greatest problems with this requirement?"
Internet-based "What are the greatest problems for you?"
surveys "For other students?"
"For your university?"
Response rate
We could continue to probe for the respondent's thoughts by asking:
Strengths and "How would you feel if the research requirement was changed?"
limitations of "Why would you feel this way?"
the survey "How do you think other students would feel?"
method
The questioning could continue with new questions, depending on how the respondent
replied to the first set. In qualitative interviewing, responses are often recorded, either on
Improving audiotape or as a set of notes. In taking notes, try to record the most important part of each
survey results reply. To the extent possible, your notes should allow you to reconstruct key responses in the
respondent's exact words. If you plan to use audio recording, be sure to get the respondent's
Aids permission to record the session.
After getting agreement to record the interview, it is a good idea to engage in some "small
talk" about the neighborhood, the weather, or some other commonly discussed topic. This
will help the respondent get used to being recorded. When the respondent shows signs of
being comfortable with the process, you can begin the interview proper. We have more to say
about unstructured interviewing in Chapter 13.
For another discussion of qualitative interviewing, we suggest you visit Qualitative
Interviewing. This site illustrates how questions are presented and interaction is carried out

Chapter 10 Page 17
with respondents. It also offers guidance on how to prepare and conduct qualitative
interviews.
Internet-based surveys
Survey methods have been adapted for use with email and Web sites. Web surveys are based
on placing a short questionnaire on a Web site and asking persons who visit the site to
complete the questionnaire. The investigator then tabulates and posts the results on the site or
publishes the results in printed form. Email surveys are a form of self-administered
questionnaires. The investigator prepares a questionnaire and sends it by email to a set of
addresses. Responses are tabulated and the results are disseminated by email or other means.
Although becoming more popular, Internet-based surveys have relatively limited scientific
value. In most cases, it is difficult to generalize results beyond the groups of persons who
responded. Email surveys, however, have potential in situations where all the members of a
population have email addresses. Such populations might include the faculty of a university,
all government officials in a ministry, or members of professional organizations. In these
cases, enumeration of the population or selection of a sample for an email survey is a quick,
low cost way of conducting a survey. All the requirements for constructing a sample and
questionnaire apply to conducting an email survey. Also, the questionnaire used should be
brief and related to the interest of members of the email group.
PREV NEXT

Chapter 10 Page 18
For more information on conducting an Internet or Web survey, you might begin by looking at:
Introduction
Casting the Net: Surveying an Internet Population. Chapter 10, contrasts email and Web-based
Characteristics survey techniques and examines practical issues of Web surveying
of surveys Conducting Web-Based Surveys, discusses problems of data quality with Web surveys, ways to
increase low response rate (a common problem), and describes software tools available for use in
Planning a conducting Web surveys
survey
Feasibility of Computer-Assisted Survey Interviewing In Africa, explores use of audio
Questionnaire computer-assisted self-interviewing in several areas of Kenya, discusses one survey and explores the
technical challenges faced by data collecting teams
construction
Using the Internet for Survey Research, an article in an online journal, Sociological Research Online;
The Final describes survey research using the Internet to reach an international audience of drug dealers, who are
questionnaire difficult to reach under normal circumstances; examines sampling issues; argues that an Internet survey
can produce valuable data although issues of interpretation and generalization remain
Qualitative
Response rate
surveys
We have referred to response rate previously: Now we shall define it more precisely and show its
Internet-based importance in describing the outcome of sampling. The response rate is the percentage of completed
surveys interviews over all the interviews you expected to have completed. The formula for calculating a
response rate is:
Response rate the number of completed interviews
Response rate = (100)
the number of expected interviews
Strengths and
limitations of The number of expected interviews generally is equal to the size of the sample. The ideal response rate
the survey is 100%, but this is seldom achieved. Despite one's best efforts, some interviews cannot be completed.
method Some respondents cannot be located or, if located, are not home when the interviewer comes or fail to be
there at a scheduled time. Occasionally, a respondent refuses to be interviewed. Sometimes, the
Improving investigator has to discard an interview because the responses are clearly unusable. As the number of
interviews not completed increases, there is an increasing danger that the sample for which interviews
survey results
are completed no longer is a random sample of the target population. Consequently, the sample of
persons who were interviewed may no longer represent the target population from which the initial
Aids sample was drawn. When the response rate the response rate falls below 50% the basis for generalizing
is considered weak. Response rates above 50% are generally considered acceptable, but response rates in
the 80% or higher range are far more desirable.
In most areas in developing countries, surveys based on interviewing result in high response rates,
frequently over 90% (Stycos, 1983). If you conduct interviews in some area of your country, your
response rate probably will be high as well. If it is not, you can almost be sure that something serious
was wrong. The most frequent reasons for a low response rate are poor questionnaire construction and
poor interviewing techniques.
Lower rates of response are generally obtained when questionnaires are mailed or given to persons for
them to complete and return. For these surveys, responses rates are often 30% or lower. Data from such
poor returns have limited value for drawing conclusions.
PREV NEXT

Chapter 10 Page 18

Chapter 10 Page 19
Box 11.4. Points to include in a description of the response rate

Introduction 1. The definition and size of the population. Readers will want to know the size of
the population from which the sample was selected.
Characteristics
2. The sample frame used to select the sample elements from the population.
of surveys This description allows readers to judge whether the sample frame adequately
matches the population.
Planning a
3. The method of sampling used.
survey
4. The size of the sample.
Questionnaire 5. The number of completed interviews.
construction 6. The response rate.
The Final When writing the design section of your report you should provide all the information readers
questionnaire will need to evaluate your sample selection and your success in completing interviews with
the expected sample of respondents. Points that should be reported are listed in Box 11.4.
Qualitative With this information readers will be able to judge how well the sample represents the target
surveys population from which the sample was selected.
Strengths and limitations of the survey method
Internet-based
surveys Like all other methods, survey research has special strengths and limitations. Knowledge of
these can help in deciding whether the survey method is appropriate for research you might
Response rate conduct.
Strengths
Strengths and
limitations of The great strength of survey research is that for relatively little cost you can collect a lot of
the survey data about a number of variables from a large number of persons. This is particularly true for
method measurement of quantitative variables using closed responses. With standardized questions,
data can be easily aggregated and analyzed using quantitative methods. Moreover, when
combined with sampling, results can be generalized to large populations of people.
Improving
survey results Limitations
Surveys also have a number of limitations. The most serious weakness concerns the validity
Aids
and reliability of responses obtained to questions. Surveys provide only verbal descriptions
of what respondents say they do or how they feel about something. Responses cannot always
be taken as accurate descriptions of what the respondents actually do or really feel about
something. This is particularly true for behavior contrary to generally accepted norms of
society. Persons are unwilling many times to indicate they have engaged in behavior not
accepted by their group. Researchers do well to remind themselves of this serious limitation
as they prepare items and interpret their results.
Improving survey results

Chapter 10 Page 19
There are several ways of dealing with the unpleasant, but stubborn fact of possibly getting
false responses. First, try to apply all the guidelines given for writing questions and for
organizing a questionnaire. Second, become thoroughly familiar with the group you are
studying and use knowledge about the norms and values of this group in preparing the
questionnaire. Third, when possible, use multiple methods of collecting data about key
variables. Instead of depending only verbal responses for these variables, try to get
observational data or look for additional data in census or other reports as well.
For example, if you were asking about how much meat is eaten in a household, try to observe
what is actually served at meals over several days for a small sub-sample of the larger sample
in addition to asking about this variable in question form. The same method could be used to
check the validity of responses to questions about many other variables (frequency of
attendance of religious services, whether children are inoculated, what kinds of farming
practices are used, absenteeism from work, etc.)
Some further suggestions for reducing errors in the collection and analysis of survey data are
provided in: Reducing Measurement Error in Informal Sector Surveys.
PREV NEXT

Chapter 10 Page 20
Aids
Introduction
Key terms
Characteristics ● Back translation ● Matrix format
of surveys ● Chart format ● Open-ended item
● Closed item ● Personal interviewing
Planning a
● Contingency question ● Precoding responses
survey
● Draft questionnaire ● Pretest
Questionnaire ● Email survey ● Self-administered questionnaire
construction ● Internet survey ● Structured interviewing
● Interview schedule ● Telephone interviews
The Final ● Item ● Unstructured interviewing
questionnaire
● Web survey
Qualitative Main points
surveys 1. The essential features of a survey are: data collection by means of a questionnaire from
a randomly selected sample, followed by statistical analysis of the data.
Internet-based
2. Preparing for and conducting a survey includes: defining the research question; gaining
surveys necessary permission for data collection; conceptualizing, to identify and define the
key concepts; constructing a questionnaire, consisting of the indicators to be used for
Response rate measuring the concepts; extensive pretesting of the questionnaire; preparing the final
questionnaire; locating members of the sample; and conducting interviews using the
Strengths and questionnaire.
limitations of 3. Most surveys are conducted by means of personal interviews. Other methods are use of
the survey self-administered questionnaires, telephone interviews, and surveys conducted by
method email or through a Web site.
4. Writing or evaluating items used in previous research is important in constructing a
Improving questionnaire. Items borrowed from other questionnaires should be tested to be sure
survey results they are culturally relevant to respondents.
5. Closed items provide response categories from which respondents choose the response
Aids that best describes their views or situation.
6. Closed items are useful for collecting a large amount of data quickly, are easily coded,
and lend themselves to statistical analysis.
7. Open-ended items allow respondents to answer in their own words and to provide far
richer and more complex responses, but are more difficult to analyze.
8. Items used in a questionnaire should be: relevant and absolutely necessary for
answering the research question; ones respondents are able and willing to answer;
stated clearly, simply, and briefly in language used by respondents; and ones that ask
for specific, concrete responses.

Chapter 10 Page 20
9. Do not use double-barreled questions, which are really two questions in one; leading or
biased questions, which can lead respondents to answer one way over other ways; or
questions expressed in negative form.
10. A combination of open-ended and closed items are frequently used to benefit from the
strengths of each kind of question.
11. The final questionnaire should be organized to make sense from the point of view of
respondents. It should include an introduction covering all the points needed to
introduce the interviewer and the purpose of the survey to respondents and to establish
trust with them.
12. It is absolutely essential to thoroughly pretest a draft questionnaire to make sure
respondents are able and willing to answer all items; that the items are clear; and to
discover any defects in the questionnaire or problems in conducting interviews.
Pretests also give you an opportunity to test and develop interviewing skills.
13. More than one pretest may be needed for the preparation of a questionnaire.
14. Interviewers should be thoroughly familiar with any questionnaire they use.
15. The survey is best for collecting a lot of data from large samples. Survey data can also
be easily quantified and analyzed statistically. When based on a probability sample,
conclusions from a survey can be safely generalized to the population represented by
the sample. The chief limitation of surveys is that responses may not reflect what
respondents actually feel or what they actual do in contrast to what they say in
response to a question.


Chapter 11 Page 1
Introduction Chapter 11. Conducting Interviews

Introduction
Steps in
interviewing In the previous chapter we described how to construct a questionnaire: Now we turn to the
process of using a questionnaire to interview respondents. We focus on interviewing because this
Conducting technique is used most often in developing countries.
interviews Steps in interviewing
Completing The main steps in conducting a survey interview are:

post- ● Locating and contacting respondents;
interviewing ● Introducing yourself to the respondents;
tasks ● Conducting the interview, using your questionnaire;
● Reviewing the completed questionnaire for completeness and accuracy;

Telephone
● Leaving the respondent; and
interviews
● Doing a final edit of responses.
Validity & In addition to our discussion of interviewing in this chapter, we suggest you view at least some of
reliability the following Web sites.
issues
Guidelines for Survey Interviewing, identifies sources of error in surveys, describes standard
Sources interviewing techniques, presents guidelines for recording answers for different kinds of
questions, and provides suggestions for ensuring collection of valid and reliable data
of error
Interviews, a thorough and readable description of the entire process of preparing for and
Aids conducting interviews for research purposes; includes discussion of the role of the interviewer,
suggestions for successful interviewing, and provides guidance on how to ask questions and
record responses
Internet Sites Related to Survey Research, provides links to sites related to many aspects of
survey research, including links to journals and books, organizations, ethics, data editing aids,
sampling, and other matters related to survey research
Designing Structured Interviews for Educational Research, although intended for educational
researchers, this site provides valuable guidance for all social researchers; the article covers
basic steps and operations involved in interviewing, identifies problems that often arise and
suggests ways to avoid them
Viewing the discussion on some of these sites should give you a clearer and deeper understanding
for planning and conducting successful interviews.
PREV NEXT

Chapter 11 Page 1

Chapter 11 Page 2
Introduction Locating respondents

As part of the design phase of your project, you will have designed and selected a sample. Before
Steps in
beginning to interview members of this sample, you will need to decide on the order in which
interviewing you will contact them and to physically locate of at least the first ten or so persons you will
interview. How this is done will depend on the nature of the sample. If the sample consists of
Conducting households, you will need to locate where the families live. In many developing countries, this is
interviews not a simple task. Maps or other aids generally are not available. Another complication is that
many persons have similar names. Unless you know the area and the people well, it is often
Completing advisable to use local persons as guides. This is another reason for getting local leaders or a
post- person you trust involved in a survey. You may need their help in locating respondents and in
interviewing getting the respondents to agree to be interviewed.
tasks When maps are available, locations of sample households can be marked on the map. When maps
are not available, you may need to drive or walk around the area and develop your own map.
Telephone With the help of local persons, you can then locate the households you plan to visit, mark them
interviews on your map, and decide on the order in which you will contact them.
Different procedures would have to be used for other kinds of samples. In some cases, permission
Validity &
may have to be obtained before beginning a survey. Approval is almost always required when
reliability conducting a survey in a school, business establishment, government office, health center, or
issues other formal organization. Getting permission from local leaders is also essential for surveys in
villages and rural areas. If not properly approached and involved, village leaders or elders can
Sources easily stop a survey.
of error
Contacting respondents
Aids Even when the respondent's location is known, contacting the person may not be easy. In most
developing countries, it is difficult to schedule a time for an interview. One has to go to the
location and hope the respondent is there. If this is the case, the interview might be completed;
if it is not a convenient time, another time will have to be arranged. When the respondent is not
present, one has to make a call back or return visit. In scheduling a callback, it is a good idea to
return at a different time of the day or to find out from a neighbor when the respondent is likely
to be home.
Generally two callbacks are made before one gives up in getting an interview. When contact
cannot be made, you can either drop the respondent or you can select a substitute to replace the
lost respondent. There are two main ways of selecting substitutes. One way is to attempt to get
an interview at the house or other sampling element nearest to the one where it was not possible
to get an interview. The other way is to draw a larger sample than you intend to use and then use
these extra sample elements as substitutes for those who cannot be contacted. Generally a 10%
over sample is usually sufficient for this purpose.
Introducing yourself
When you first approach a person to obtain an interview, you are a stranger to that person. Under

Chapter 11 Page 2
these circumstances, many respondents may be suspicious of you. Even greater suspicion can be
aroused when this stranger tells the respondent that he or she wants to ask the respondent a lot of
questions, some of which the respondent may think is none of the interviewer's right to know.
Your first task, therefore, is to overcome these natural suspicions and to gain the trust and
confidence of the respondent. This begins with how you introduce yourself and what you tell the
respondent about why you want to talk with the respondent. The introduction you prepared for
your questionnaire should cover all these points. In your first interviews, you may have to read
the introduction as it appears on your questionnaire. After a few interviews, you will be able to
provide the introduction easily from memory. It is very important to assure the respondent that all
information will be protected from others. This assurance generally helps respondents relax and
encourages them to respond more freely.
Following the introduction, you should try to find a place where the interview can proceed with
as much privacy as possible. Most of the time, the respondent will offer you a place to sit and, by
their manner, show that he or she is ready to talk with you. If the respondent indicates that the
time you arrived is not a good time for the interview, schedule another time.
PREV NEXT

Chapter 11 Page 3
Introduction Establishing rapport

Before starting the interview itself, it is frequently a good idea to engage in some "small talk"
Steps in
about the weather or some recent local event. Small talk is often the way we start conversations
interviewing with strangers. Engaging in small talk can help establish a feeling of trust and confidence
between the respondent and you. In research language, this is known as establishing rapport with
Conducting the respondent. If rapport is not quickly established, some respondents may refuse to be
interviews interviewed, or, if they agree, they may not give complete, honest responses.
Completing One's personality is important in establishing rapport with strangers. Interviewers who are warm,
friendly, sincere and understanding persons gain trust easily and quickly with most respondents.
post-
In contrast, persons who appear withdrawn, negative, cynical, or cool in their relations with
interviewing others usually have difficulty getting others to trust them. If you plan to do interviewing you
tasks should assess your personality characteristics against these criteria and take whatever steps you
can to become a caring, sincere interviewer.
Telephone
interviews Ways of establishing rapport are listed in Box 11.1. A few of these points deserve further
comment.
Validity & Box 11.1. Guidelines for establishing rapport
reliability 1. Approach the respondent in a warm, sincere, friendly manner.
issues
2. Immediately introduce yourself and briefly explain the purpose of
the survey in a way that will interest the respondent.
Sources
3. Tell the respondent why he or she was chosen.
of error
4. Answer any questions the respondent may have.
Aids 5. Assure the respondent that all information provided will be held
in confidence and that the respondent's privacy will be
completely protected.
6. Tell the respondent approximately how long the interview will
last.
7. Be flexible; adjust to how the respondent reacts to your
introduction.
Conducting interviews
After rapport is established, you can begin the interview. Following are some guidelines for
asking questions. These guidelines are summarized in Box 11.2.
Ask each question exactly as stated on the questionnaire. Interviewing is based on several
assumptions. These are that each question is asked exactly in the same way in all interview
sessions; that all respondents understand each question in exactly the same way; and that
differences in replies to each questions are due, therefore, to differences among the respondents
and to nothing else. These assumptions are unrealistic, but in research interviewing we strive to
come as close as possible to this ideal of perfect communication.

Chapter 11 Page 3
The best way to do this is to ask each question exactly as stated in the questionnaire with the
same normal, conversational tone of voice for each question. Sometimes, however, you may
have to depart from this rule. Occasionally a respondent may not understand a question. When
this occurs, slowly read the question again and give the respondent time to think about it. Often
this is sufficient, but if the respondent still has trouble you might change the wording slightly to
help the respondent understand the question. There is a danger, however, in this procedure.
Changing the wording can also change the meaning of the item. Changes, therefore, should
remain as close as possible to the original meaning of the item.
In so far as possible ask all questions exactly in their order on the questionnaire. There may
be exceptions to this rule as well. Occasionally a respondent may not be able to answer a certain
question: Either he or she cannot or does not want to provide an answer. Rather than stop the
interview at this point, you can say something like: "Let's go on and come back to this question
later." This flexibility keeps the interview moving in a natural, interactive fashion and is better
than insisting on an answer when the respondent is not ready to give one. Later in the interview
the respondent may be ready to return to the unanswered question.
PREV NEXT

Chapter 11 Page 4
Introduction Be patient and show understanding at all times with respondents. Respondents vary in how
quickly they answer questions. Some may take what seems like a long time in giving an answer.
Steps in As a result, there may be periods of silence. Most of us are uncomfortable when silence occurs
interviewing in a conversation: We want to say something to get the conversation moving again.
Consequently, silences in interviewing can be upsetting; but, as an interviewer, you should be
Conducting prepared to accept them and not react too quickly. When a respondent fails to answer a question,
appear sympathetic, smile, relax a bit like leaning back or otherwise demonstrate that you
interviews
understand the respondent's situation. If no response comes after a minute or so, the safest action
is to offer to repeat the question. If this does not work, you can suggest going on and coming
Completing back to the question later.
post-
interviewing In so far as possible, record all responses exactly as given. This is easy to do with closed
tasks items: You simply listen to the response and place a check mark or select the right code for the
appropriate response. Recording exact responses to open-ended questions can be difficult.
Telephone If possible, try to write down exactly what the respondent said. When a long response is given,
interviews the usual practice is to jot down a summary of what was said and then write a full record of the
reply immediately after the interview. Recording the interview is an alternative, provided the
Validity & respondent feels comfortable and agrees with this.
reliability Evaluate each response as it is given. As the interviewer, you have to judge whether each
issues response is satisfactory. Does the reply appear valid, reliable and complete? Responses given to
other questions can be used for this purpose.
Sources
of error Use probes to clarify or expand responses. When a response does not appear satisfactory, you
can follow up with probes, such as: "Could you tell me more about that;" "I do not quite
understand, could you tell me more about your ideas on that;" "Why do you say that;" "Anything
Aids
else you would like to add;" or "Why do you feel that way." Such questions often encourage
respondents to give more complete and accurate responses.
Avoid expecting certain responses. After hearing many respondents give about the same
responses to a certain question, interviewers sometimes begin to expect this response in a
subsequent interview. This is called the halo effect. When it occurs, there is danger that the
interviewer will put down what he or she expected the respondent to say rather than what the
respondent actually said. To avoid this danger, treat each interview as a new, unique experience,
one that might produce data different from any past interview.
At the end of the interview check over the questionnaire to make sure that every item was
asked and that a satisfactory response was obtained for each. This is a further way of making
sure that the data are as valid and reliable as possible. Also, at this point you should go back to
any question that has been omitted and try again to get an answer.
Be relaxed and conduct the interview in an easygoing manner. Maintain a serious,
professional, but relaxed relationship with the respondent. Do not try to rush through the
interview as if you were trying to get it over as quickly as possible. Your behavior will
communicate your feelings to the respondent. This will show in how you ask questions, consider

Chapter 11 Page 4
the respondent's answers, how you use probes, and how you record responses. By being relaxed
and confident during the interview, you will help the respondent relax and open up to you.
Perhaps both you and the respondent will even enjoy your interaction!
Box. 11.2. Guidelines for conducting interviews
1. Ask questions in exactly in their order on the questionnaire.
2. Ask each question exactly as stated in the questionnaire.
3. Be patient and show understanding with the respondent.
4. Record responses exactly as given by the respondent.
5. Evaluate each response as it is given; clarify any apparent
inconsistencies in responses.
6. Use probes to clarify responses or to help respondents expand on
their responses.
7. Avoid expecting certain responses because you got certain
responses before.
8. Be relaxed; conduct the interview in an easy going manner.
9. At the end of the interview, carefully review all responses; go
back to any questions that were not answered or only partially
answered and try to get a complete response.
10. Be sure to thank the respondent for his or her cooperation.
PREV NEXT

Chapter 11 Page 5
Introduction Thank the respondent. As a last action you should thank the respondent for his or her
assistance and leave the respondent with a feeling that he or she contributed to a worthwhile
Steps in activity.
interviewing Leaving the respondent. After thanking the respondent, take your leave graciously. Of course,
you will want to respect whatever leave-taking courtesies are customary in the culture of the
Conducting respondent.
interviews
Completing post-interviewing tasks
Completing Several activities should be completed after each interview. First, it is good practice to go over
post- the questionnaire again to make sure that a satisfactory answer was obtained for each item, even
interviewing though this was done at the end of the interview. A further check under less hurried
tasks circumstances may identify omissions. When important data are missing, it may be necessary to
return to the respondent to obtain them. Second, right after the interview is the best time to
Telephone reconstruct responses to open-ended questions. Creating a full reconstruction, in the respondent's
interviews own words will help greatly when you later analyze responses to open-ended questions.
Telephone interviews
Validity &
reliability Although opportunities are limited, telephone interviews can be conducted with specific target
populations known to have telephones. These include government officials, professional, staff of
issues
non-governmental organizations, and many business establishments. Before committing to a
telephone survey, you should be sure you have phone numbers for all the members of your
Sources sample. Also, be prepared to make a number of calls to reach persons in your sample. Even with
of error numbers in hand, making contact may require a number of calls. Once contact is made, your
introduction is critically important. Your contact with the respondent is only semi-personal, based
Aids on your voice and your powers of persuasion over the telephone. In the first minute on the phone
with a respondent, you have to introduce yourself and persuade the respondent to remain on the
line with you.
The following Web site provides the full text of a research paper as a guide in writing a research
paper. We refer you to: Telephone Methods for Social Surveys
Although written for use in industrialized countries with extensive and reliable phone systems,
the content on this site may help you if you are thinking about conducting a telephone survey.
Validity and reliability issues
At various points in this and the preceding two chapters we have mentioned conditions that affect
the quality of survey data. To help you conduct surveys well, we shall discuss these points now
in greater detail. Four sets of conditions that affect the validity and reliability of survey data are:
● The interviewer's interaction with respondents;
● The setting in which the interview occurs;
● The reaction of respondents to the process of being interviewed; and

Chapter 11 Page 5
● The social characteristics of interviewer in relation to those of the respondents.

Effects of interviewer interaction with respondents
Figure 11.1 summarizes important ways, you as an interviewer, can affect the data obtained in an
interview. How you approach the respondent and the kind of relationship you establish at the
beginning of the interview will go far in determining the kind of responses you will get. As
shown at the extreme left in Figure 11.1, establishing a feeling of trust with the respondent
generally results in complete, valid and reliable data, whereas arousing feelings of suspicion or
doubt on the part of the respondent can cause the respondent to refuse to be interviewed or to give
less
PREV NEXT

Chapter 11 Page 6
Introduction
Steps in
interviewing
Conducting
interviews
Completing
post-
interviewing
tasks
Telephone
interviews
Figure 11.1. Illustration of ways interviewers can influence responses
Validity &
The best protection against simple, occasional errors in recording responses is to create a neat,
reliability
well organized questionnaire; to train yourself to listen carefully, to record each response
issues accurately; to check each response as you record it; to allow enough time to complete each
interview without rushing; and to double check all responses. You can protect against systematic
Sources errors by establishing rapport; remaining open to receive any possible answer (avoiding the halo
of error effect); listening carefully to each response; and recording responses fully and accurately.
Aids Influence of the interview setting

The setting in which the interview is conducted can also affect the quality of the data obtained.
When you want the personal views of respondents, it is best to interview them in private, where
just you and the respondent are present. In some societies and social groups, however, individual
privacy is neither common nor considered important. Reporting on interviewing in rural Egypt,
Anker and Anker (1995) reported that others were present in nearly 90% of the interviews
conducted with wives. The additional persons included their mothers, mothers-in-law, spouses,
other adults, and children. Additional persons were less likely to be present when men were
interviewed. Even if the others say little their presence can make respondents answer differently
than if they were alone with the interviewer.
Several things can be done to overcome the influence of others. First, you can try to arrange for
the interview at a time when the respondent is alone. If this is not possible, try to get the
respondent to answer without paying much attention to others. Doing so requires tact and humor,
but can payoff in getting more useful data.
PREV NEXT

Chapter 11 Page 6

Chapter 11 Page 7
Introduction Influence of characteristics of respondents

Figure 11.2 suggests how certain characteristics of respondents can influence how they answer
Steps in
questions. To begin, respondents may or may not know the answer to a question. Careful
interviewing construction of each questionnaire item and extensive pretesting can minimize the possibility of
asking questions that respondents are unable to answer. When a respondent knows the answer to
Conducting a question, shown on the left in Figure 11.1, he or she may decide not to reply. When the
interviews respondent is willing to answer a question, any of three kinds of answers may be given. The
respondent may make an unintentional mistake such as giving the wrong birth date. The best way
Completing to protect against unintentional errors is to word the questions and response categories as simply
post- and clearly as possible. Also, during the interview you may identify an error by noting that an
interviewing answer is inconsistent with other information supplied by the respondent. When this occurs, you
tasks can politely seek clarification and record the corrected response.
Telephone
interviews
Validity &
reliability
issues
Sources
of error
Aids
Figure 11.2. Possible responses to a question

Respondents may also give intentionally false answers. They may be less than fully truthful for
any number of reasons. Where moral issues are involved or strong norms exist for or against
certain behaviors, respondents may deny doing certain things that are against the norms.
Pretesting should tell you whether these kinds of sensitive questions will be answered easily and
truthfully. If you don't believe respondents are answering an item truthfully in pretests, it is better
to delete the item. During survey interviews, there is no foolproof defense against lying on the
part of respondents. All you can do is to be alert to any indication of lying and try to get the
respondent to give an accurate answer.
When a respondent knows the answer to a question, the desired answer is an accurate, complete

Chapter 11 Page 7
response. Having a carefully constructed questionnaire and using sound interviewing practices
will help get such responses. Even after careful pretesting of items, however, some respondents
may be unable to answer some questions. When asked a question they are unable to answer,
respondents can either: (1) honestly admit they have no idea of what to say which, while not
useful data, at least is a valid reply; or (2) give a false and, therefore, invalid, response. You can
reduce the possibility of obtaining false replies by getting respondents to trust you and by
assuring respondents that any answer, even a "don't know" is acceptable.
PREV NEXT

Chapter 11 Page 8
Introduction Influence of the social characteristics of the interviewer

In many societies, the social characteristics of the interviewer have to be matched to those of the
Steps in
respondents. This is particularly true for gender and may apply in certain situations to age as
interviewing well. In some societies, interviews will be limited by local norms to persons of the same gender
as respondents: male interviewers with male respondents and female interviewers with female
Conducting respondents. The age of the interviewer may also become a limiting condition. In some villages,
interviews for example, village leaders and some older men may not be willing to talk freely with younger
interviewers. Requirements to match age and gender between respondents and interviewers is
Completing even greater when interviews get into sensitive, personal topics, such as family planning, use of
post- contraceptives, or other personal matters.
interviewing As you consider interviewing, you will have to decide whether your gender and age may be in
tasks factors in the cultural setting in which you will be interviewing.
Telephone Sources of error

interviews Box 11.3 summarizes sources of error that can occur in conducting surveys based on
interviewing. Careful attention during interviewing can prevent or minimize these errors.
Validity &
reliability Box 11.3. Main sources of error in survey research
issues 1. A poorly constructed questionnaire: for example, respondents are confused
by the order of questions.
Sources 2. Poorly worded items - items are not clear, respondents do not understand
of error them or are unwilling to answer them.
3. Questions are asked in different ways with different respondents.
Aids 4. The interviewer fails to identify and correct inappropriate or inconsistent
responses.
5. The interviewer makes mistakes or omits crucial parts of open-ended
responses.
6. The interviewer makes mistakes in recording responses to closed items.
7. Poor post review of responses at the end of each interview and during the
post-interview editing of responses.
Personal interviews are well suited for use in developing countries. This technique can be used
with persons who are not literate or who have limited reading skills. Personal interviewing also
fits well with the preference for verbal communication among people in many developing
countries. Interviewers are able to ask a wide range of questions; make sure that all questions are
answered; clarify responses that seem contradictory or are not clear; and use probes to get more
complete responses. Being on the scene, interviewers can also record additional data
unobtrusively. For example, information about the construction of the home or other indicators of
socio-economic status can be noted without asking any questions.
Limitations

Chapter 11 Page 8
Personal interviewing also has some serious limitations. Conducting face-to-face interviews is
costly and time consuming. Beginning researchers who plan to do their own interviewing have to
be prepared for weeks or months of daily interviewing. Completing a one hour interview, with
even a small sample of 50 to 60 respondents, can take weeks or even months of work. Travel
costs may be involved as well, which can limit the size of the sample used.
The greatest limitation, however, is the ever-present possibility of bias. Bias can be introduced
inadvertently by the interviewer's tone of voice, body language, way of asking questions, poor
use of probes, and in many other ways. Also, even though the respondent is assured of
confidentiality, knowing that the interviewer is hearing and recording what the respondents says
can cause some respondents to give replies they think are more socially correct than accurate.
PREV NEXT

Chapter 11 Page 9
Introduction Aids
Key terms
Steps in
interviewing ● Callback ● Response rate
● Halo effect ● Structured interviewing
Conducting ● Probe ● Telephone interviews
interviews
● Rapport ● Unstructured
Completing Main points
post- 1. Gaining the trust and confidence of respondents is called establishing rapport. Establishing
interviewing rapport quickly with respondents is essential for interviewing.
tasks 2. Interviewers should be thoroughly familiar with the questionnaire being used before
beginning interviews.
Telephone
3. Structured interviewing requires skill, tact, patience, maintenance of a neutral role, care in
interviews asking all questions in the same way and in the same order, evaluation of responses, use of
probes to clarify or expand responses, and careful and complete recording of responses.
Validity &
4. Surveys can also be conducted using a qualitative approach, based on unstructured
reliability interviewing. With this technique, the interviewer uses broad questions to get the
issues respondent talking and then focuses the conversation on topics of interest to the
interviewer. Reponses are recorded either on audio tape or in the form of notes that are
Sources later developed into a more detail record of the respondent's responses.
of error 5. Immediately after each interview, interviewers should check the questionnaire to make
sure a clear, useable response has been obtained for each item and to write out responses as
Aids completely as possible from notes taken for responses to open-ended items.
6. The validity, reliability, and accuracy of responses depend on the interaction of the
interviewer with respondents; the settings in which interviews take place; reactions of
respondents to the process of being interviewed; and, depending on local norms, the social
characteristics of the interviewer relative to the respondents.
7. The response rate is the percentage of successful, completed interviews over the expected
number. With personal interviewing, return rates of 80% or higher are expected. Rates are
lower, often below 30%, for surveys based on self-administered questionnaires.
8. When the response rate is less than about 50%, the sample no longer can be considered
representative of the target population from which it was selected. Conclusions from the
sample, therefore, have limited value, and cannot be safely generalized to the target
population.

Chapter 11 Page 9

Chapter 12 Page 1
Chapter 12. Using Available Data

Introduction
Introduction
Sources of The previous three chapters described how to conduct an experiment and how to prepare for
available and conduct a survey. Both methods depend on interaction with people. In this chapter we show
data how research can be done without contact with anyone, simply by using data that is already
available. Many kinds of available data exist in every country. These include:
Analyzing ● Social statistics published by government ministries, bodies of the United Nations, the
available World Bank, and other international organizations;
records ● Data collected in previous surveys and other studies; and
● Information that can be extracted from newspaper or magazine articles, books, diaries
Secondary television shows, folk tales, and other sources.
analyses
Some form of available data is used as background information in nearly every study.
Available data frequently are used in selecting a problem and in establishing its importance.
Content
Data on the extent of drought in western Kordafan in the Sudan, for example, led Rahama
analysis (1997) to investigate survival strategies used by households and the effects of changes on the
roles of males and females that occurred. In the design of a sample, census or other statistical
Strengths & data are needed. Available data are also used in interpreting the results of an investigation.
limitations Thus, almost any study you do will involve the use of available data.
of available
In addition, many studies are based solely on further analysis of data collected and made
data
available by various ministries of government. Some examples follow:
● A study of changes in ages at marriage in northern Sudan, based on data from official
Social
marriage records (Omer, 1991).
indicators
● Investigation of differences in fertility levels among women of different ethnic groups in
Aids the Gezira region of the Sudan (Abdelrahman, 1995).

● Examination of the changing rates of female enrollment relative to males in Sudan; based
on data from the Ministry of Education (Badri and Burchinal, 1985).
Researchers also base investigations on data collected and published by international
organizations. Following are three examples of research based on data from international
sources:
● How the spread of mass primary education in 23 countries of sub-Saharan Africa is
contributing to declining fertility rates (Lloyd, Kauffman and Hewett, 2000).
● How fulfillment of social needs is lagging behind substantial gains in economic
development in 83 developing counties (Bulluck, 1986).
● Whether social change is contributing to increased crime in African countries (Arthur,
1988); the author found little support for this hypothesis.
PREV NEXT

Chapter 12 Page 1

Chapter 12 Page 2
Sources of available data

Introduction
Using these and other sources that may occur to you, you will probably be able to find a great
amount of available data on a large number of topics or problems in your country. Three
Sources of
valuable sources are:
available
● National sources of data;
data
● Data made available by international organizations; and
Analyzing ● Data available on web sites throughout the world.
available National sources

records
A great variety and quantity of data, including the kinds listed in Table 12.1, are available in
Secondary most developing countries. Table 12.1 is organized to help you think about and find specific
analyses kinds of data. The column on the left lists broad topics such as "Population characteristics,"
etc., and names of ministries that collect data in the indicated area; the right-hand column
identifies the kinds of variables for which data are frequently available. Browsing through the
Content
table may give you an idea for a research project.
analysis
Table 12.1. Kinds of Statistical Data Available from Government Sources in Most Developing
Strengths & Countries
limitations Kind of data and sources
of available (From the Ministry of) Typical variables
data
Population Births: gender, legitimacy; gender, age distribution,
characteristics literacy, language spoken; deaths, causes of deaths; kinds
Social
Planning or statistics or of occupations; place of residence; marriages, ages of
indicators Economic Development, spouses, previous marital
status; divorces, ages at divorce
Aids
Migration Numbers of foreign visitors, country of origin;
Planning or Statistics or immigrants, origin, age, gender, place and date of arrival,
Security nationality; displaced person, origins, locations,
numbers, ages, gender; displaced persons, internal
migration
Housing or Statistics Types of dwelling, number of rooms per dwelling, water
availability, kind of or lighting, sewage
Education Enrollments, age and gender by grade level, graduates,
Education or Higher degrees, certificates; school personnel, numbers,
Education training, teachers, administrators, other personnel;
degrees, certificates; facilities, buildings, rooms,
equipment, laboratories

Chapter 12 Page 2
Health Inoculations, illnesses, deaths, accidents; personnel,

Health or Statistics number, training, kind of personnel (doctors, nurses,
others), locations (cities, towns, rural areas), degrees,
certificates
Taxation Taxes paid by individuals, amounts paid, by gender, age,
Finance or National occupations, income; taxes paid by kinds of industrial
Economy and commercial organizations, kinds and amounts paid
by organizations; other taxes paid
Data described in Table 12.1 may be available at various places in your country. Your
university library is a good place to start. Most university libraries receive copies of
publications, including statistical reports from government ministries and international
organizations. Discussion with a librarian may lead you to what you want. If your library
cannot supply the kind of statistical data you are looking for, there are other sources in most
countries. One is the ministry responsible for producing data in a given area education, health,
social services, etc. Ministries that collect and organize social data are also listed in Table 12.1.
PREV NEXT

Chapter 12 Page 3
If you visit a ministry, try to talk with a statistical specialist. These specialists are responsible
Introduction for collecting and analyzing data in a particular area such as health, education, manpower and
employment, social services, etc. Because of their experience, they know the strengths and
Sources of weaknesses of the data and can help you find and analyze data in their area of expertise.
available Generally these specialists are happy to share their knowledge and data, especially with a
data student planning a research project.
Another possible source of data is the office of the national archives. The national archive
Analyzing office in each country maintains a collection of all government publications and official reports
available and papers. Another good source of information is the national library or the documentation
records center of the national council of research. These organizations receive publications and reports
from government bodies and many foreign sources as well.
Secondary The national news service in your country can be a rich source of information. In many
analyses countries the national news service publishes an annual digest or yearbook containing a variety
of statistical data about the country, its people, economic activities, education, health, and other
Content characteristics. Many national new agencies also maintain libraries that contain not only reports
analysis from sources within the country, but also statistical reports from international organizations and
other countries. Sometimes the library of the national news agency is the best source of data in
Strengths & the country.
limitations If you are thinking about analyzing existing records, we suggest you go to Using Existing
of available Records in Evaluation. Although this site focuses on evaluation, the subject of the next
data chapter, the information on this site can help in analyzing existing records for any purpose.
Social International publications

indicators Each of the specialized agencies of the United Nations, such as the Food and Agricultural
Organization, collect data from national government ministries, organize and analyze the data,
Aids and publish statistics for all nations. These data are made available in printed publications, on
CD ROM discs, or on the Web sites of the organizations. Reports with extensive amounts of
statistical data for all countries of the world are listed in Table 12.5, located at the end of this
chapter. Most of these publications or Web sites contain data from previous years as well.
These time-series sets of data can be used to do trend studies by country, by regions of the
world (Africa, Middle East, etc.), or to make comparisons among countries for a wide range of
variables (literacy, fertility, death rates, rights of women, economic development, food
availability, etc.).
International Web sites
Government ministries and international organizations provide a wide range of detailed
information about the social and economic development on their Web sites.
Table 12.5 includes these sites and describes the kinds of data available on them. These sites
provide data about your country, your region of the world, and for other countries and regions.
With these data, you can establish trends in social and economic development in your country,

Chapter 12 Page 3
for other countries, or for various regions of the world. Examples of this kind of research were
provided earlier.
When using data from international sources, one has to keep in mind that the data were
obtained from participating countries. Errors that occurred during data collection and analysis
in these countries are carried through and appear in the international compilations. Marenin
(1997), for example, points out the many sources of errors that occur with crime statistics
reported by government ministries in African countries. These include the failure of victims to
report crimes, failure of the police to record reports, and errors in compiling statistical records.
Many of these kinds of errors, no doubt, occur in other sets of data reported by government
ministries in many countries. Despite this problem, data provided by international organizations
provide the best data for making comparisons among countries and for conducting trend
studies.
PREV NEXT

Chapter 12 Page 4
Analyzing available records

Introduction
Getting an idea
Sources of An important first step in research, as we have said so often, is to define the question to be
available investigated. In studies based on the analysis of available data, this decision is obviously
data influenced by knowledge of what kinds of data are available. For example, you might begin
with a vague idea about some question you want to investigate, discover some relevant data
Analyzing and, in light of these data, modify your question so that it can be answered with the data you
available discovered. Research questions also can emerge the other way around. You might discover a
body of data and, in thinking about it, formulate a question that you choose to answer.
records
Conceptualization and operationalization
Secondary
These processes are handled differently when one uses available data. When deciding on using
analyses
available data, the investigator has to judge whether the data can be used as valid indicators for
the concepts being investigated. Badri and Burchinal (1985), for example, used the ratio of girls
Content in relation to boys in school as an indicator of social change. Given the previous norm against
analysis equal education for girls relative to boys in rural areas of the Sudan, this ratio would appear to
be a valid indicator of changes in social values regarding the education of girls.
Strengths &
limitations When using available data, one also needs to check the level at which the raw data are
aggregated. Raw data, which generally consist of the responses from informants, is seldom
of available
reported because, without analysis, the data have little meaning. Instead, the raw data are
data combined or aggregated in some way. Sometimes the data are aggregated for comparisons
between rural and urban areas of a country; sometimes by provinces within the country; and
Social finally for the country as whole. Before deciding to use certain data, you will have to be sure
indicators that the data are reported at the level of aggregation you want to use. If you wanted to
compare school attendance rates for certain towns, but the lowest level of aggregation was at
Aids the province level the study could not be done. If data are reported at a lower level of
aggregation, say at administrative levels that make up towns, you could combine the data for
each town and continue with your analysis.
Methods used to analyze data from available records may be as simple as calculating rates of
change in variables to more complex measures of association. After you have studied Chapters
16 through 19, you will have a better idea of the kinds of analyses that can be done.
Caution in interpreting results
The level at which data are aggregated also has to be taken into account when results are
interpreted. An example will show why this is important.
Suppose we obtain the number of votes cast in each of 200 voting districts for the candidates
representing the socialist and conservative parties in some country. We could then calculate
the percentage of voters who voted for each candidate in each of the 200 districts. Further,
let's say we obtained data on the per capita income for these same districts. Suppose also that
we were testing the hypothesis that voting for conservative candidates was positively related to

Chapter 12 Page 4
per capita income: That is, the percentage voting for conservative candidates increased as per
capita income of the districts increased. Imagine also that the hypothesis was supported: As
per capita income of districts increased so did the percentage of votes cast for the conservative
candidates. So far so good. We could conclude that the hypothesis was supported at the level
of the voting district.
But suppose we wanted to make a generalization at the level of the individuals, who, after all
were the ones who cast the votes. Now we have a problem: We have no data at the individual
level. The data were aggregated at the level of the voting district. Each district might contain
thousands of individuals. And we know nothing about these individuals. We do not know
who voted; and if they voted, whether they voted for the conservative or socialist candidate or
what their income level is. Yet we want to generalize about individual behavior.
In situations like this, you might be tempted to infer that what was found at the aggregated level
is also true at the individual level. But there is a danger, you could be in danger of committing
an ecological fallacy: That is, assuming without sufficient evidence that what is true at an
aggregated level is also true at the individual level. Though probably unlikely, there remains
a possibility that a large proportion of low income persons voted for the conservative candidate
or that large proportion of the high income persons voted for the socialist candidate. With the
data in hand, we could not exclude these possibilities. When aggregated data are used, one has
to be careful in making generalizations at levels below that used in the analysis.
PREV NEXT

Chapter 12 Page 5
Secondary analyses
Introduction
Data collected by other researcher can frequently be re-analyzed to answer a new problem.
This kind of research, referred to as secondary analysis, can be undertaken with almost any
Sources of
kind of data, but is usually done using quantitative data from previous surveys or from reports
available from government ministries.
data
Investigators frequently keep the original questionnaires from a survey or the computer disks
Analyzing with raw data from the questionnaires. These can be a rich source of data for secondary
available analysis. You might also find data held by professors at your university, at a social research
institute, or in the files of a government ministry. Several of the studies we mentioned earlier
records
were based on secondary analyses. These include:
● Re-analysis of data originally collected for the Sudan Fertility Survey (Nouri, 1983)
Secondary
analyses ● Factors affecting differences in fertility in the Gezira and Managil areas of the Sudan
(Abdelrahman, 1995), based on data from a household survey conducted in 1987
Content ● Investigation of child mortality in the Khartoum metropolitan area, based on census data
analysis and a household survey (Farah, 1981)

Available data are particularly useful for conducting trend studies. This is what Badri and
Strengths & Burchinal (1985) did, using data from the Sudanese Ministry of Education to investigate trends
limitations in the school enrollment of females relative to males. Re-analysis of available data is also a
of available quick and low cost way of doing research. There is no data collection cost: All the investigator
data has to do is reorganize existing data with a new plan of analysis.
There are disadvantages as well. The main limitation is whether or not the data are valid for use
Social as indicators of the variables a researcher wants to measure. Frequently, researchers find that
indicators the questions asked in a previous survey may be close to, but don't exactly match, what they
want to measure. Too often, data from previous studies are only approximations of what we
Aids would like to have. Still, with some redefinition of concepts, secondary data offers ways of
doing research that a single investigator could not do by his- or herself. Secondary analysis is
done in the same way as described for the analysis of any other quantitative or qualitative data.
Another form of secondary data consists of mass media and other materials, which we turn to
next.
Content analysis
Content analysis is a technique for analyzing the content of all kinds of written or recorded
material or the spoken word. Written materials include the contents of newspapers, magazines,
books, official documents, advertisements - anything that is written. Recorded materials include
films, videotapes, photographs, and anything else in audio or video form. The content of
spoken words can be analyzed as well, including poems, folk tales, speeches by religious or
political leaders, or the content of everyday conversations.
These kinds of mass media materials have caught the attention of social scientists because they
contain expressions of values, opinions, and attitudes that reflect changes occurring in a

Chapter 12 Page 5
society. In conducting content analysis, the investigator uses objective and systematic analysis
techniques to generate a description of the meaning of the materials selected for analysis.
Meaning is expressed in the symbols or images contained in the materials. Symbols or images
may be expressed in references to a country's flag, by use of certain phrases, such as
"imperialistic," "terrorists," or in descriptions of the "right" or "proper" way to view some
issue or group of people. A few examples will help show how content analysis can be used to
analyze mass media materials.
PREV NEXT

Chapter 12 Page 6
Illustrations of content analysis

Introduction
How were women portrayed in Sudanese newspapers and magazine between 1984 and 1993
(Badri & Osama,1995) sought to answer this question by examining the content of articles in
Sources of
daily newspapers and women's magazines. They found little coverage of women in work roles
available or articles devoted to the needs or interests of women. Women's magazines, as one would
data expect, presented more positive images of women than other mass media.
Analyzing Another investigator, Swanjord (1988) studied trends in images of Kuwaiti women contained in
available English and Arabic publications from about 1900 through to the 1990s. Swanjord carefully
defined her population of publications. Her interest was in images of Kuwaiti women and not
records
about women in Kuwait. The latter definition would have included women from other
countries, which was outside her interest. For the purpose of analysis, Swanjord created four
Secondary role definitions for women: modern, secular, traditional, and fundamentalist. Each was
analyses carefully defined in terms of words and phrases that would capture each image. She found that
Kuwaiti women were portrayed in fundamentalist or traditional roles in the 1920 to 1971
Content period; fundamentalist in the 1975 to 1981 period; while more secular roles dominated since
analysis 1982.
How much consideration do three prominent health journals give to health issues in developing
Strengths &
countries? This question was clearly answered by Nandy (1995). He examined how frequently
limitations three major, international medical journals - the American Journal of Public Health, Prevention
of available and Education, and World Health - included articles on health issues in developing countries.
data Coverage increased from 1975 to 1989, but declined slightly from 1990 through 1994, the last 5
years studied. Over all, the author concluded that health issues in developing countries do not
Social get major attention in the three health journals.
indicators
Madani (1999) analyzed how Arabs and Muslims are depicted as compared to western
Europeans and Israelis in the headlines of three major newspapers in the United States. Panels
Aids of students were used to judge whether the headlines expressed favorable or unfavorable views.
Results showed that headlines for western Europeans were viewed most favorably, followed by
those for Israelis. Headlines depicting Arabs and Muslims were viewed least favorably.
The previous analyses were all based on printed materials. Content analysis, however, can be
applied equally well to poems, songs, folk tales, or any audio or video materials as well.
Regardless of the medium being examined, the process of content analysis is basically the
same.
Steps in content analysis
Most content analysis proceeds in the following manner:
1. Formulating a research question and getting started;
2. Conceptualizing the problem and operationalizing variables;
3. Deciding on the unit of analysis;
4. Developing a set of categories for coding the content of the material being analyzed;

Chapter 12 Page 6
5. Coding the materials;

6. Analyzing the coded content; and
7. Interpreting the results, drawing conclusions, and writing the report.
Getting started
Getting starting requires making a number of tentative decisions regarding the research
question, the selection and operationalization of concepts, and the unit of analysis to use. Initial
decisions for each of these influence the final decisions on the other points. There is no set,
proscribed way of making these decisions. Using their own individualistic styles of
decision-making, researchers work through the process of beginning content analysis. A certain
amount of trial and error is involved. You can define a question and then search for relevant
materials or you might find an interesting set of materials that suggests a question for an
investigation. In practice, the two often occuogether and influence one another. In addition,
your initial research question might have to be modified as you examine material for analysis
more closely. Decisions are made and then adjusted as additional decisions are made.
PREV NEXT

Chapter 12 Page 7
To start, you will need to express your research question in concepts that both express the
Introduction question you want to investigate and that can be operationalized in the content of the material
to be analyzed. To make this decision, you will need to examine the content of the material
Sources of selected for analysis. Is the material appropriate for answering the question, as expressed in a
available specific set of concepts? The full answer to this question will not be clear until you have
data developed a set of categories for coding the content. The categories represent the way you
operationalize your concepts. Thus, the interaction of decisions extends well into
operationalization and analysis stages of content.
Analyzing
available Your university or other libraries is a good source of materials for analysis. Libraries frequently
records have copies of newspapers and magazines, speeches by government officials, policies of
ministries, laws, and other public documents. If not, the national archives library should have
Secondary official government publications. Newspaper offices may be another source of documents for
analyses content analysis. Most newspapers keep old issues of their papers, going back many years.
These issues can provide the information for conducting a trend analysis, somewhat like that
done by Swansjord (1988).
Content
analysis Deciding on the unit of analysis
Selection of material for analysis depends on the question being investigated. So does the
Strengths &
selection of the unit of analysis. The unit of analysis may be the content of the entire
limitations publication (Badri and Osama, 1995; Swanjord, 1988; and Nandy, 1995). For Badri and
of available Osama, the unit of analysis was the entire article that appeared in a newspaper of women's
data magazine; for Swanjord, it was any publication, whether a book, article, report, dissertation, or
other publication that contained an image of Kuwaiti women; for Nandy, the unit of analysis
Social was the entire issue of the selected journals. Depending on the question being researched, other
indicators units might be used, such as only editorials or advertisements in newspapers or letters to the
editors. Still other units could be segments of films, complete songs, or folk tales.
Aids Developing a coding system
In content analysis, concepts are operationalized by developing a set of categories for
describing the attributes of the variables you are measuring. In Swanjord's (1988) investigation,
her concept was the "image of Kuwaiti women." She operationalized this concept by creating
four categories, each of which described different images of Kuwati women. If you choose to
conduct analysis of some material, you will need to develop a set of categories that describe the
variable or variables you wish to measure. Your categories will be used for coding the materials
you have selected. The simplest set of codes is based on whether or not any of the categories
appears in an item being examined. Nandy (1995) used this approach: Did any article in an
issue of one of the three journals contain some discussion of health issues in developing
countries? The research by Swanjord (1988) was based on a more complex coding system. She
developed an elaborate set of definitions for each of the four sets of roles that could be depicted
as images of Kuwaiti women.
Developing a suitable set of categories is a systematic process. Your research question will

Chapter 12 Page 7
suggest possible categories, but you have to examine your selected material carefully to see if
the tentatively established categories are appropriate. This process begins by quickly examining
the material to identify major themes, images, phrases, or whatever you are looking for. Several
passes through the selected material are frequently needed to identify possible themes or
images for defining the categories for coding the content. Swansjord looked for phrases
describing Kuwati women; Badri and Osama examined articles that reported on activities of
women. As categories emerge, you will need to develop clear, operational definitions for each.
Definitions are generally based on recurring words or phrases that capture the meaning of any
of the categories developed for the analysis. In Swanjord's research, sets of phrases were used
to define what distinguished each of four roles for Kuwaiti women.
Initial categories are tested against material to make sure they cover all the possible themes or
images that relate to the topic being investigated. In constructing categories, researchers
frequently find categories overlap. Then, redefinition is in order to ensure that categories are
mutually exclusive. This process is continued until no new categories are found and the
researcher is convinced that the set of categories will allow clear coding of the material.
PREV NEXT

Chapter 12 Page 8
Each category used also has to be operationalized. Some of the ways this can be done are
Introduction summarized in Box 12.1.
Sources of Box 12.1. Some ways to operationalize categories for use in content
analysis
available
data 1. Appearance - whether or not the theme expressed by a category
occurs
Analyzing 2. Location - how prominent is the category in the presentation
available 3. Spatial measurement - how many centimeters, for example, are
records devoted to the topic in a newspaper column
4. Time measurement - how long is the presentation of content for a
Secondary category in a film, radio or television broadcast
analyses 5. Intensity - how strongly is the point made
Content Appearance of a theme or category. This is the form of operationalization used by Nandy
analysis (1995) in deciding whether articles in health journals dealt with issues affecting developing
countries.
Strengths & Location. Where the content is located can indicate how important the topic is viewed. Madani
limitations (1999) used headlines, which suggest strong interest in a topic. In analyzing newspaper stories,
of available Badri and Osama (1995) noted whether the stories on women were on the front pages or buried
data somewhere inside the newspapers. Front page treatment would suggest stronger interest than a
similar article back farther in the newspaper.
Social Spatial measurement of the content. In the analysis of newspaper or magazine content,
indicators investigators frequently measure how many centimeters are devoted to topics under
investigation. Longer columns are taken as a measure of greater interest in the topic.
Aids
Time measurement. This form of operationalization is used for non-written content, such as
the content of films or radio or television programming. It is the equivalent of spatial
measurement in printed material.
Intensity or other qualitative measurement. Coding of headlines (Madani, 1999) was based
on qualitative judgment. The author, however, was careful to use a panel of students to do the
judging rather than simply expressing his own opinion.
The categories become the indicators for the variable the investigator is searching for. After the
categories are set, a coding sheet is prepared for coding the content under analysis. Before
using a coding sheet, however, you should conduct a pilot test, similar to a pretest of a
questionnaire. Using your tentative coding sheet, select a sample of material, preferably from
content similar to what you will analyze later, and see how well your set of categories can be
used in coding it. Each time one of the defined images, themes, or sets of words comprising a
category appears, record it by placing a check after the appropriate category to mark its
occurrence. The pilot test may lead to further changes in categories, but several test runs

Chapter 12 Page 8
usually result in a final set of coding categories.

Investigators using content analysis also have to decide whether to use all the material that
could qualify for analysis or to use of a sample of them. When the amount of material is small,
all the material can be examined (Badri and Osama, 1995; Nandy, 1995). Swanjord (1998),
however, used a systematic sample of materials on Kuwaiti women, taking every third
publication for analysis.
Finally, in creating categories you have to show that the categories you created validly measure
the concepts in your research question and that the content has been coded reliably. As a
minimum, you will need to demonstrate the content or face validity of your categories. In some
cases, it is easy to establish content or face validity. Few researchers, for example, would
dispute Nandy's (1995) classification of articles as either containing or not containing
references to health issues in developing countries. Swanjord's (1998) categories would
probably also be accepted as valid measures of images of Kuwati women. You can strengthen
any claim for validity by presenting evidence of agreement among competent judges as to how
well the categories represent intended concepts.
In content analysis, demonstrating reliability involves two operations. First, the investigator
tests for inter-analyst reliability. This is the measure of agreement between two trained
analysts. Each codes a sample of material independently. Then, the percentage of agreement
between the number of times they noted each category is found. Inter-analyst reliability is a
measure of the reliability of a set of categories. This analysis shows whether the coding system
is reliable; whether two trained persons will use it in approximately the same way.
Second, the reliability of the major analyst, usually the investigator, has to be established. This
is done by having the principle analyst code a specific set or part of the content and then code
the same material a second time after about a week or so. Known as intra-analyst reliability,
this measure is an application of the test-retest method of establishing reliability. Percentage
of agreement is also used for either calculation. For both kinds of coding reliability, perfect
agreement is not expected, but the percentage should be in the 80 % range or higher.
PREV NEXT

Chapter 12 Page 9
Second, the reliability of the major analyst, usually the investigator, has to be established. This
Introduction is done by having the principle analyst code a specific set or part of the content and then code
the same material a second time after about a week or so. Known as intra-analyst reliability,
Sources of this measure is an application of the test-retest method of establishing reliability. Percentage
available of agreement is also used for either calculation. For both kinds of coding reliability, perfect
data agreement is not expected, but the percentage should be in the 80 % range or higher.
Coding the material
Analyzing
available Coding is best illustrated with an example. Let's say we decided to conduct a content analysis
of editorials related to the United Nations that appeared in five major newspapers in the past
records
three years in some country. We start by developing a set of categories for coding the editorials.
We might start with only two categories: Did the editorials express a positive or negative
Secondary opinion of the UN? In skimming some articles, however, we find some editorials present a
analyses mixed set of views, combining some positive as well as negative opinions. Also, we become
aware that most editorials are on topics that have nothing to do with the UN. Based on this new
Content information, we revise our coding system to include four categories - positive, negative, mixed,
analysis and not relevant. Further, instead of writing out these four descriptions each time we use them,
we can code them as 1 for positive, 2 for negative, 3 for mixed, and 4 for not relevant and save
Strengths & a lot of time.
limitations We then test these categories with a sample of editorials to see if they cover all the kinds of
of available editorials we might find in the newspapers over the three-year period. Assuming that the
data categories allowed us to code all the editorials we tested, we would then construct a coding
sheet to use in coding the content of the editorials in the selected newspapers. Our coding sheet
Social might look like the one shown in Table 12.2.
indicators
Instead of writing the full name of each newspaper each time we use it, we assigned codes to
each newspaper, from 1 through 5. As with the codes for the content, any set of numbers could
Aids be used because the numbers simply represent a way of expressing a classification system and
do not have any numerical significance. One newspaper is given the code of 1, the second a
code of 2, and so on for the other newspapers, which would be coded as 3, 4, and 5.
Table 12.2. Coding sheet for one newspaper
Code Frequencies Total
1 (Positive) //// 5
2 (Mixed) //// / 6
3
0
(Negative)
4 (Not
//// //// //// 15
relevant)
Total 26

Chapter 12 Page 9
Since we are analyzing five newspapers, we would need five coding sheets. Table 12.2 shows a
coding sheet and the results of coding the editorials in the newspaper coded as 1 over the
three-year period. Each slash (/) mark represents a coding judgment we made regarding the
editorials in this newspaper. Using our coding sheet, we began with the first issue of newspaper
number 1, read it, and decided whether or not it contained views about the United Nations.
If it did not, it was coded 4 (not relevant), and a slash mark was entered for "not relevant." If it
dealt with the United Nations in some fashion, we examined it more carefully and decided
whether it expressed a positive view, in which case it would be coded 1; whether it presented a
mixed view, in which it would be coded 2; or whether it expressed a negative view, in which
case it would be coded as 3. A slash mark is shown for each time a code was recorded.
The contents of the other four newspapers would be coded in the same way. Table 12.3
presents the results of the analysis. The far left-hand column lists the codes for the five
newspapers; the middle set of columns show the number of times editorials were coded as
positive, mixed, negative, or not relevant; while the right-hand column gives the total number
of editorials in each newspaper. The totals across the bottom show how many editorials for all
newspapers were positive, mixed, negative, or that dealt with other topics.
PREV NEXT

Chapter 12 Page 10
Table 12.3. Coding of editorials about the United Nations

Introduction
Code
Sources of 4
available Newspaper 1 2 3 (Not Number of
data number (Positive) (Mixed) (Negative) relevant) editorials
01 5 6 0 15 26
Analyzing
02 0 3 3 12 18
available
records 03 8 2 0 9 19
04 0 3 1 10 14
Secondary
analyses 05 1 2 5 13 21
Total 14 16 9 59 98
Content
analysis Analyzing and interpreting results
The results of content analyses are frequently expressed as percentages. Let's say there were
Strengths & 98 editorials in the three-year period. As shown in Table 12.3, we find that 39 or about 39% of
limitations the 98 editorials dealt with the United Nations. These were the editorials coded as 1, 2, or 3.
of available Of these 39 editorials, 16 or 41% were mixed, while 14 or 36% were positive, and 9 or 23%
data were negative. These are results of the study and would serve as the basis for the conclusions
that would be offered.
Social The following comments apply mainly to uses of data from official records or surveys.
indicators
Strengths and limitations of available data
Aids Strengths
Analyzing available data is a quick, low cost way of doing research. The often costly and
certainly time-consuming process of data collection is eliminated. Secondary analysis also
offers opportunity to analyze data based on enumeration or very large samples. Data from the
Ministry of Education, analyzed by Badri and Burchinal (1988), were based on tens of
thousands of children. Farah (1981) re-analyzed data from the sample of 3,000 women living in
northern Sudan. Neither investigation could have been done without access to available data.
Another advantage is that you can study issues that may be too politically sensitive to study in
person. In addition, most analyses of available data can be carried out by a single investigator.
Limitations
The strengths just mentioned are offset by three limitations. One of these concerns the
relevance or validity of the data for the problem you want to address. Sometimes, despite the
great advantages of this method, the data available simply cannot be used as indicators of the

Chapter 12 Page 10
variables you want to investigate. As the investigator, you have to defend the use of any
available data in terms of its validity for the variables you are using the data to measure.
Second, the data may not be free of errors or other bias introduced at the time of data collection
or during earlier analysis of the data. When using available data, it is a good idea to read the
"Methods" or "Data Collection" section of a report to see what safeguards were taken to
minimize errors or bias in data collection and analysis.
These sections should also provide information on the validity and reliability of the data. If you
have doubts about the quality of the data, you should carefully consider whether to re-analyze
the data.
In addition, while content analysis is a quick and inexpensive way of collecting and analyzing
mass media information, it only provides a description of what is being communicated. It does
not give any evidence of the effect of the content on audiences.
PREV NEXT

Chapter 12 Page 11
Our discussion of content analysis represents one approach to this useful research technique.
Introduction Several Web sites provide alternative discussions on how to plan and conduct content analysis.
Any of three following sites will provide you additional information and guidance on
Sources of conducting content analysis.
available An Overview of Content Analysis provides an introduction to content analysis, with examples
data and definitions of key terms
Analyzing Overview: Content Analysis describes the history and uses of content analysis and the method
available itself; discusses issues of validity and reliability and the advantages and disadvantages of the
records method; provides examples of studies that used content analysis; defines key terms associated
with content analysis
Secondary Unobtrusive Measurement refers to techniques of data collection that do not influence the
analyses kinds of observations or responses obtained), this site describes various kinds of unobtrusive
measurements, including direct observation, content analysis, and secondary analysis of data
Content
analysis Social indicators
Social indicators are measures of wellbeing for a population in terms of some characteristic.
Strengths & These measures are frequently reported for a geographical area, such as a metropolitan area, a
limitations province, or a country. Social indicators are generally based on official government data. They
of available are frequently used by policy makers in international organizations and government ministries
data in deciding how to allocate funds and staff in meeting social and economic needs.
Table12.4 lists some commonly used social indicators. For example, the first indicator under
Social health is the indicator used for reporting infant mortality. The indicator is expressed precisely
indicators by the following formula:
Aids Number of deaths of infants

_______________________ (1000)
Number of infants
Infants are defined as babies less than one year of age. The indicator gives a number of deaths
per 1,000 infants in a population. Using a standard base, in this case 1,000, allows comparison
between groups with different sized populations. For example, there may have been 1,850
deaths for a certain province of a country out of 153,250 infants, giving an infant mortality rate
of 1,850/153,250 (1,000) or 12.1. In another province there might have been 520 deaths
among 28,446 infants, which would produce an infant mortality rate of 18.3. Thus, we can say
the second province has a higher infant mortality rate. Other bases are used as well. A base of
10,000 is used for some of the medical indexes because there are so few doctors and nurses in
terms of the population they serve.
Social indicators are useful in many kinds of research. They are used to describe conditions as
they currently exist or to conduct trend or longitudinal studies. Because they are standardized,
social indicators are particularly valuable for making comparisons among countries and regions

Chapter 12 Page 11
of the world. Some examples of social indicators that are available for all countries are listed in
Table 12.4.
PREV NEXT

Chapter 12 Page 12
Health
Introduction
Infant mortality (deaths) per 1,000 infants
Sources of Child mortality (deaths) up to 10 years of age per 1,000 children
Number of physicians per 10,000 persons
available
Number of hospital beds per 10,000 persons
data Average length of life for males, females, all persons
Analyzing Education
available Percentage of adults, males, females who are literate
records Percentage of children of primary school who are attending school
Percentage of secondary school children who are attending school
Secondary Percentage of children (all children, boys, girls) who complete each level of
analyses schooling
Ratio of the number of primary teachers to the number of primary children,
Content same for ratios for the intermediate and secondary levels
analysis Enrollment at institutions of higher education per 1,000 population
Strengths & Table 12.5. Sources of social and economic data for countries and regions, in print, on
limitations CD-ROM or available on Web sites
of available
The web sites and publications listed in Table 12.5 contain a large number of social and
data
economic indicators. Checking any of the sources listed will give you a much better idea of the
variety of social indicators available, including many that describe economic and social
Social conditions in your country.
indicators
Development of an indicator is a long and complex process. Researchers and policy-makers,
Aids therefore, generally use existing indicators, many of which are developed by governmental and
international organizations. Still, if you wish to consider development of a new or alternative
indicator, you can find a useful description of the steps involved at:
Steps in Designing an Indicator System, this article describes the steps involved and provides
guidance for carrying out each step.
Aids
Key terms
● Available data ● Intra-analyst reliability
● Content analysis ● Secondary analysis
● Ecological fallacy ● Social indicators
● Inter-analyst reliability ● Time series analyses
Main points

Chapter 12 Page 12
1. Available data are used in some way in virtually all research.

2. Available data include census and other statistical reports issued by government
ministries, data from previous surveys, and the content of mass media materials -
newspapers, magazines, films, speeches, etc.
3. Available data also exists in publications and on the Web sites of international
organizations.
4. When analyzing available data, researchers are careful to limit their generalizations to the
level at which the data are aggregated. Ignoring this rule may result in committing the
error of ecological fallacy. This occurs when generalizations are broader than the data
upon which they are based.
5. Secondary analysis is based on the re-analysis of previously collected and analyzed data.
Most secondary analyses are based on analysis of previously collected survey data or the
use of census data.
6. Content analysis is a systematic way of analyzing the content of materials and verbal
communications. It is based on developing a set of categories for coding and analyzing
the content of a body of material.
7. A set of categories used in content analysis should be tested for reliability between two
competent analysts, called inter-analyst reliability, and for the reliability of coding by the
main analyst, referred to as intra-analyst reliability.
8. Analysis of available data is a quick, inexpensive, and non-reactive way to do research.
Its main limitation is that the data are often only an approximation of the measures a
researcher would like to have.
9. Social indicators are measures of the economic and social wellbeing of a population.
International organizations develop and publish a large variety of these indicators, many
of which are available free on the Web sites of the organizations. Social indicators are
used by researchers in conducting trend studies and for comparing social and economic
development among countries.
PREV NEXT

Chapter 12 Page 13
Table 12.5 Sources of social and economic data for countries and regions, in print, on
Introduction CD-ROM or available on Web sites
Title and source Containing data related to:
Sources of
available General (providing indicators across many areas of human activity)
data Little Data Book 2004 A pocket-sized ready reference on key development data
for 206 countries
Analyzing
available
Social Indicators, Provides indicators across many areas such as
records UN Statistics Division child-bearing, education, health, housing, literacy,
economic activity, population, water supply and
Secondary sanitation, and youth and elderly populations
analyses
Statistical Databases, Provides brief descriptions of and links to a wide variety
UN by Statistics Division of databases produced by UN agencies and that are
Content
available with unrestricted access
analysis
U.S. Department of Provides links for online access to statistical and other
Strengths & Labor, Bureau of Labor information from more than 70 agencies of the U.S.
limitations Statistics government and statistical offices of most countries
Statistical Sites on the throughout the world
of available
data World Wide Web
Statistical Yearbook, An annual compilation of statistics for over 200
Social 48th issue (2004) UN countries and areas of the world, organized in four parts:
indicators Statistics Division world and region summary; population and social
statistics, economic activity; and international economic
Aids relations; includes 80 tables of data
World Statistics Provides basic economic, social, and environmental
Pocketbook, indicators for 209 countries and areas worldwide; covers
UN Statistics Division 57 key indicators in the areas of population, economic
activity, agriculture, industry, energy, international trade,
transport, communications, gender, education and
environment, drawn from over 20 international statistical
sources
World Development Presents the most current and accurate information on
Indicators 2004, global development on World Bank both the national
World Bank level and aggregated globally, with over 80 tables and
800 Indicators for 152 economies and 14 country groups
Africa

Chapter 12 Page 13
African Development Provides the most detailed collection of data on Africa

Indicators, The World available in one source; presents data from 53 African
Bank countries, arranged in separate tables for more than 500
development indicators
Live Database Africa, An extensive database containing a wide range of
The World Bank indicators for monitoring and evaluating programrs and
outcomes in client countries of the World Bank; the
World Bank arranges for installation of the LDA with
institutions in client countries
PREV NEXT

Chapter 12 Page 14

Introduction
Children
Sources of The State of the World's Includes a wide array of statistics on the lives of children
available Children, in developingcountries;describes programs for improving
data UNICEF lives of children; provides statistics data and down
loadable files for indicators related to nutrition, health,
HIV/AIDs, education, economic development, women
Analyzing
and child protection, on infant and rates of progress
available regarding protection for women and children
records
Monitoring the Situation A key Web site for information on the condition of
Secondary of Children and Women, children throughoutthe world, provides data specific to
UNICEF each country, including indicators for child survival,
analyses
children's health and nutritional status, maternal health,
water and sanitation, educational achievement,
Content immunizations, and many other conditions of
analysis
Education
Strengths & Statistical Yearbook, Covers all aspects of national educational systems,
limitations United Nations facilities, staffing educational attainment; enrollment by
of available Educational, Scientific, level of education, public expenditures for education;also
data and Cultural Organization provides data on science and technology and culture and
(UNESCO) Organization communication; includes data on book
Social production, libraries, newspaper and magazine
circulation, production of films
indicators
UNESCO Institute for Provides access to international educational statistical
Aids Statistics databases, statistical, country profiles, and fast facts
related to educational enrollment and attainment,
educational facilities, and other educational information
for countries throughout the world
World Education Provides statistics by country on access to schooling,
Indicators, participation in education, operation of primary schools,
United Nations Education, indicators of resources for education, rates of primary
Scientific, and Cultural literacy
Organization (UNESCO)
Food
FAO ORG, A set of databases with time series records related to
Food and Agricultural food production and distribution, land and irrigation
Organization of the United practices, fishing and fish production, and food aid and
Nations other fool related topics

Chapter 12 Page 14
Gender
GenderStats, A database of gender statistics and indicators, offers
The World Bank country data sheets showing summary gender indicators,
basic demographic data, population dynamics, labour
force structure, and education and health statistics
PREV NEXT

Chapter 12 Page 15

Introduction
Health
Sources of HNPStats (short for Offers country data sheets showing key indicators for
available Health, Nutrition and health status, health determinants, and health finance;
data Population Statistics) data from 1960 to the most recent year; The World Bank
provides links to may Web sites to many Web sites of
international and country statistical offices
Analyzing
available World Health Report, The text of the annual report of WHO, examines the
records 2005, World Health global health situation and some of the major threats to
Organization (WHO) health in today's world; contains five chapters, a
Secondary conclusion, and statistical annex
analyses World Health Report, Provides links to statistical annexes for World Health
Statistical Annex, World Reports from 1995 to the most recently issued report;
Content Health Organization permits searching by country and within countries a by a
analysis (WHO) large number of indicators health in today's world
Human development
Strengths &
limitations Human Development Provides the text of the 2005 Human Development
of available Reports 2005, UN Report, with 5 chapters; alsoincludes a wide array of
Development Program human development
data
(UNDP)
Social Labour
indicators
Key Indicators of the Based on 20 key indicators of the labor market with an
Labour Market, Third explanation of what the indicators represent and how
Aids they can be used in assessing labor market issues;
Edition, The International
Labour Office (ILO) includes data on employment, wages, labor markets by
country and gender and many other labour related
indicators
Population
Demographic Yearbook A comprehensive collection of population and
System, United Nations demographic statistics of for the world and countries;
Division (UNSD) covers population, fertility, foetal mortality, infant and
child mortality, general mortality, and marriage and
divorce data

Chapter 12 Page 15
The State of the World Discussion of issues related to population growth and
Population, 2005, impacts on the environment; also covers women and the
Unite Nations Population environment, health and the environment, and presents
Fund (UNFPA) actions for sustainable and equitable development;
includes global agreements on human rights,
environment and development, and reproductive health
and gender equality
United Nations A comprehensive guide to population information on the
Population Information Web sites of UN agencies; provides descriptions of
Network, (POPIN) publications; demographic and social statistics, national
statistics, and links to statistical databases maintained by
UN agencies
Refugees
The State of the World's Provides data and analysis of refugees and displaced
Refugees (2000), UN persons throughout the world; numbers, movement and
High Commissioner for locations of refugees, by region and country; repatriation
Refugees (UNHCR) of refugees
UNHCR Statistics, UN Provides data, charts, and graphs showing refugee
High Commisioner for populations and movements, demographic
Refugees (UNHCR) characteristics, major refugee locations, new refugee
outflows, and beneficiaries of UNHCR's programs


Chapter 13 Page 1
Chapter 13: Observing Behavior

Introduction
Introduction
Casual The previous four chapters described quantitative methods how to conduct experiments, how to
observation construct a questionnaire and use it for interviewing respondents, and the use of available data.
This chapter focuses on qualitative research methods. Most of the chapter deals with conducting
Participant field research, which relies on use of informal observation and unstructured interviewing of
observation persons in their natural settings. In addition, we discuss the case study approach to data collection
and the use of focus groups.
Conducting a In everyday life, we constantly rely on observation to keep track of events around us. Social
participant scientists also use observation to learn about social behavior, but scientific observation differs
observation from every day observation in two important ways.
study
Scientific observation
Establishing First, scientific observation differs from every day observation by being a planned, deliberate
validity process. Observation does not just happen as in everyday life. Researchers using observation
plan how observation will be carried out, how the observations will be recorded, and how the
Generalizing observational data will be analyzed. The purpose of this planning is to ensure that the record of
from a the observations, which become the data for an observational study, are as accurate and complete
as possible. Second, the recorded observations - the data - are scientifically analyzed, based on
qualitative
procedures developed specifically for observational studies. These include giving evidence for
study the validity and reliability of the observations, showing the basis for drawing conclusions, and
cross checking results with those of other comparable studies.
Personal
qualifications Forms of scientific observation
Observation is a flexible way of collecting data. It can be conducted in different kinds of social
Case studies settings; involve different forms of relationships with the persons being observed; and can be
conducted using either qualitative or quantitative approaches to data collection. The main settings
Focus groups for observing are either natural settings, as in homes, public areas, schools, offices, factories and
many other places where everyday activities occur, or controlled settings that are specifically
Strengths and created for the purposes of research. Many experiments, for example, are based on artificially
limitations of created situations that are controlled by the experimenter.
qualitative Observation studies also differ in terms of the role the observer takes in relation to the persons or
research groups being observed. At one extreme, the observer can simply observe without having any
interaction with the subjects being observed. At the other extreme, the observer can join the
Aids group being studied and participate in all its activities while, at the same time, observe the
activities of the members of the group. Participant observation, as this technique is called, allows
the observer to experience and learn about a group from the inside, by participating and observing
the daily life of members of the group. Other roles, with varying degrees of involvement can be
used as well.
Observation studies can also differ in the method used to record observations. In unstructured

Chapter 13 Page 1
observation the observer attempts to describe all the relevant behavior that occurs.This is done by
keeping a mental record of what happens or by taking notes that are later developed into a full
description of the behavior. Unstructured observation produces qualitative data. In contrast, an
observer can create a set of clearly defined categories for recording behavior, similar to that
described for content analysis. The observer uses the categories as a check list for noting each
time the behavior occurs. Structured observation, as this technique is called, produces
quantitative data.
PREV NEXT

Chapter 13 Page 2
These three conditions - the setting, the role of the observer, and the method of recording data
Introduction can be combined in a variety of ways. Two frequently used ways of conducting qualitative
observation are described in Box 13.1. Both are used for recording observations of subjects in
Casual natural settings.
observation
Box 13.1. Two frequently used forms of scientific observation
Participant 1. Casual or simple observation - the observer has no interaction
observation with subjects. Observation typically occurs in a natural setting
and is recorded in the form of field notes.
Conducting a 2. Participant observation - the observer immerses him or herself
participant in the group by becoming a group member and tries to
observation understand as fully as possible how the group sees itself, how it
study functions, and how it related to its environment.
Establishing Casual observation

validity In casual or simple observation, the observer simply watches and listens to what is occurring in a
situation. Observations are recorded in words as a narrative description of what was observed.
Generalizing The observer tries to describe the most important features of the behavior as fully as possible.
from a
qualitative Casual observation can be useful at any stage of research. If you were planning a survey of work
roles in a village, you could visit the village and informally observe who does what tasks. This
study
could provide initial ideas for the concepts you would want to measure and give you ideas for
operationalizing them. Casual observation is always a part of studies based on participant
Personal observation. An observer who is participating in a group's activities cannot help but observe a
qualifications large number of things during the course of daily interaction.
Case studies Casual observation may also be the first step in developing a structured observation study. In a
study of what students actually do in the library, we might use casual observation to develop
Focus groups categories for a later structured observation study. We could start with several periods of casual
observation conducted at different times for several days and record what students were doing.
Were they studying class notes, using reference books, talking, daydreaming, sleeping, or what?
Strengths and
Casual observation could reveal various forms of behavior and perhaps suggest additional
limitations of categories for a later structured observation of their library behavior.
qualitative
research One of the main strengths of casual observation is that the observer can remain unknown and
unobserved while observing. This eliminates the threat of reactivity - persons behaving
Aids differently than normal when they know they are being observed. Also, casual observation can be
used in virtually any social setting. Casual observation, however, has two significant limitations.
First, as with all observation methods, there is the danger of bias or error committed by the
observer. When the observer loses attention or becomes sloppy in observing and recording, errors
will occur. Second, casual observation does not provide information for why any behavior
occurs. Any set of observations can be interpreted in various ways by different observers. Still,
if you want to describe behavior as it occurs in a natural setting without getting involved

Chapter 13 Page 2
personally with the group, casual observation is a good way to do it. But if your purpose is to try
to understand why behavior in a group occurs as it does, you may want to consider undertaking a
participant observation study.
This note may not be necessary, but for some readers the words "casual" and "causal" may be
confusing. Casual refers to informal situations; causal refers to cause and effect relationships.
PREV NEXT
1.This note may not be necessary, but for some readers the words "casual" and "causal" may be
confusing. Casual refers to informal situations; causal refers to cause and effect relationships.

Chapter 13 Page 3
Participant observation
Introduction
Description
Casual As the name implies, participant observation is based on observation of people while
observation participating with them in their normal, daily activities. These kinds of studies are almost always
done in natural settings. The observer arranges to become a member or a guest of the group and
then over a period of time, observes and records what members do and what they talk about.
Participant
observation Involvement with the group can range from becoming a full member and actively participating in
all things the members do to less active roles. How active the observer will be has to be decided
Conducting a in advance of observation. Most observers assume less than a full membership role. For one
participant thing, no observer can really enter into all the activities of most groups, particularly life in a
observation village or community. In these settings, culturally defined roles for persons by gender, age,
marital status, social status, and other community-defined conditions frequently limit the extent
study
to which an investigator can participate and observe activities of certain members of the group. A
male observer, for example, could not participate in many of the activities of women, while a
Establishing female observer might have difficulty observing decision-making sessions of the male elders of a
validity village.
Generalizing Regardless of the level of involvement, the same data collection and analysis techniques are used,
from a mainly unstructured observation, based on repeated interaction with participants, and
unstructured interviewing. In addition, investigators make full use of any available data, such as
qualitative
historical records or census reports, and may even conduct surveys. Using all possible data, the
study investigator assembles information for answering his or her research question.
Personal Periods of observation may be a brief as several weeks or extend to a year or longer. Studies of
qualifications the organization and functioning of formal groups, such as offices, are generally on the shorter
side, whereas studies of villages or other communities frequently last longer. Regardless of the
Case studies length of time involved, most participant observation studies share certain common features.
Distinctive features
Focus groups
Most participant observation studies can be described as:
Strengths and ● Being focused on one social setting;
limitations of ● Using an open, flexible research approach;

qualitative ● Combining data collection with simultaneous analysis of data;
research ● For the purpose of understanding or explaining why certain behaviors occur among
members of the group.
Aids
Research is generally conducted within the confines of one social setting. This may be a village,
community, office, club, or other group. Methods of data collection are flexible. The investigator
can use extensive unstructured interviewing to follow up on things he or she has observed. New
questions can be explored at any time or the investigator can go back to previous points and get
further information to enrich what has already been learned. As the investigator learns about the
group, the topic or focus of the study can be changed and different forms of data collection can be

Chapter 13 Page 3
employed at any time.

Data collection is ongoing and continues up to the last day the observer spends with a group. In
participant observation and other qualitative approaches to research, analysis proceeds along with
data collection. Analysis is based on thinking about and trying to make sense of what is observed.
At any time, old ideas can be revisited and new concepts explored. Ultimately, the investigator
seeks to present a full description of the actions of the group along with an explanation for them.
Example
Haberer (1985) investigated the status and power of women in an Arab Muslim village in
northern Israel. Her hypothesis was that women, who lacked access to the official power structure
of the village, develop informal sources of power and use these to influence decisions affecting
themselves and the village as a whole. To test this hypothesis, Haberer decided to use participant
observation as the way to collect her data.
With her husband and two school-aged children, she rented an apartment and lived in the village
over a 24-month period from 1977 to1985. At the outset, she explained her purpose to her
landlord, who also became a valuable source of information and provided assistance, and to
everyone she met. Haberer lived like any other woman in the village, doing household chores,
taking care of her family, participating in the school activities of her children, and otherwise
living according to the patterns of life in the village. In addition, she visited extensively with
other women in their homes, both during the day and in the evenings when the men were home.
Soon, she came to know many women well and became close friends with some. One older lady
adopted her as a "daughter" and patiently answered her questions and provided insights into the
life of women in the village.
PREV NEXT

Chapter 13 Page 4
As guests in the village, Haberer and her family enjoyed the traditional Arab generosity and
Introduction hospitality of many families, particularly the parents of the schoolmates of her children. She was
free to come and go throughout the village and used this freedom to seek out conversations with
Casual women living in all quarters of the village, from the wealthier to the poorest sections. As a
observation member of the village, she joined in wedding festivities and shared the grief of friends at
funerals. She also exchanged visits with female schoolteachers, who were the only other women
Participant in the village with post-secondary education.
observation After ten months of participant observation, Harberer sought additional data by tape recording 25
life histories of women and conducting extensive informal interviews with women across all
Conducting a socio-economic levels, from large to small families, and older as well as younger women. She
participant also conducted structured interviews with 125 women. In addition, she arranged for secondary
observation students to write essays on "The Arab Village Woman." These essays were analyzed using
study content analysis methods. She visited schools and attended after school activities and lectures for
women in the village. She also sought and used every opportunity to talk with men, who were
quite interested in what she was writing down in her notebooks.
Establishing
validity Available data were not neglected. To put her observations in a historical context, she read all she
could find on the history of the area and the village, from Turkish occupation to current Israeli
Generalizing domination. This included examining 20 years of court records and analysis of unpublished data
from a about the village and local area from records from the Bureau of Statistics in Jerusalem.
qualitative Haberer based her analysis and the interpretation of her data on social science concepts. She
study analyzed the lives of the women in terms of the social dynamics of village life, its power and
authority structure, social control mechanisms, and individual and group strategies for
Personal influencing decisions. These concepts became real from her observations of day-to-day life in the
qualifications village. She concluded that the women were unhappy over the traditional prerogatives of men in
marriage making, securing education, employment, physical mobility, and in clothing styles.
Case studies Still, adult women felt secure, productive, and satisfied as mothers, wives, homemakers, and as
family members and villagers. The women participated in joint decision making with their
Focus groups husbands and had considerable authority within the household.
Conducting a participant observation study
Strengths and
limitations of Most participant observation investigations go through three phases:
qualitative ● Preparing for participant observation;
research ● Conducting the observation, called the field work phase of the investigation; and
● Analysis and interpreting the findings and preparing a report.

Aids
Figure 13.1 shows the sequence of these three phases
The double-headed arrows in Figure 13.1 show the interactions among decisions made as part of
the planning activities. All decisions begin with the formulation of the initial topic or problem to
be investigated

Chapter 13 Page 4
Participant observation methods are ideally suited when the investigator has a broad idea for a
study and wants to obtain detailed information about a topic before committing to a specific focus
for the investigation. In the process of data collection, the initial idea for the investigation may be
modified or even discarded in favor of a new set of questions
PREV NEXT

Chapter 13 Page 5
Introduction
Casual
observation
Participant
observation
Conducting a
participant
observation
study
Establishing
validity
Generalizing
from a
qualitative
study
Personal
qualifications
Case studies
Focus groups
Strengths and
limitations of
Fig. 13.1. Steps in conducting a participant observation study
qualitative
research PREV NEXT
Aids

Chapter 13 Page 6
Preparing for field work

Introduction
This includes all the activities an investigator has to do before beginning observation. These
normally include:
Casual
● Specifying the topic or problem to be investigated;
observation
● Conducting a review of the literature;
Participant ● Selecting a setting for the investigation;

observation ● Deciding on the degree of participation in the group to be studied; and
● Preparing for living in the field during the period of observation.

Conducting a
participant Specifying the research question. In Chapter 4, we stressed the importance of stating the
observation research question as clearly and precisely as you can. This requirement is less important when
study designing a qualitative investigation because qualitative methods allow greater flexibility. This is
one the great strengths of this technique of data collection.
Establishing Reviewing the literature. All the points made in Chapter 4 about how to conduct a literature
validity review apply to preparing for an observation study. Several additional points, however, need to
be mentioned. First, as the investigator, you will want to learn about participant observation
Generalizing studies in settings similar to the one you are planning. This kind of review will provide a wealth
from a of information on how participant observation studies are done; about what kinds of information
qualitative are collected and how data are analyzed, interpreted, and reported. What you learn will be useful
study in planning and conducting any participant observation you plan to do.
In addition, it is important to become thoroughly familiar with the group you intend to study. You
Personal will need to seek all available information you can find about the group. This certainly will
qualifications include any written sources, such as histories, research reports, census data, or results from
previous surveys. Haberer (1985), for example, obtained historical and current information about
Case studies the village and its culture, including its physical environment, dominant norms and social
values; how its social systems functioned; how power was exercised in the village; and how these
Focus groups conditions affected the lives of women. You can also seek information from persons who know
the group you want to observe or visit or live briefly with a group like the one you will be
Strengths and observing.
limitations of Selecting a setting. The initial idea for an investigation will probably include selection of the
qualitative setting for the study. In fact, the initial idea may be suggested by the investigator's knowledge or
research interest in a certain setting. A student in business management, for example, could decide to
conduct a participant observation study of how work is managed and carried out in a government
Aids ministry, business organization, or nongovernmental organization. Wanting to report on how
displaced persons are adapting to life in camps for displaced persons, another student could
decide to live with a family in a camp for a period of time. Other settings could prompt other
kinds of studies.
Deciding a relationship with the group. Deciding on a relationship really involves two
decisions. One decision concerns the degree of involvement; the other is whether the investigator

Chapter 13 Page 6
makes his or her presence known to the groups being observed. As shown in Table 13.1, at least
four arrangements are possible.
An investigator can assume one of several relationships with group members. He or she can
become a full member of the group and participate in all activities, but with his or her role
concealed from members of the group, as represented in the upper left cell of Table 13.1. As a
way of illustration, a student could work in an office or factory or join some informal group and
observe and record descriptions of activities and conversations without revealing his or her true
purpose. Hiding one's true intent, however, raises serious ethical questions because the purpose of
the observer is not known to the subjects and they have not provided their informed consent to be
observed. When the concealed approach is considered, the investigator has to ask him or herself
if the study is important enough to warrant invasion of the subjects' privacy. If you are doing such
a study as part of research required by your university, an ethical panel at your university should
also review your proposal.
The full participant observer role is most often and properly used with the full knowledge and
consent of all the members of a group. This approach is shown in the lower left cell of Table
13.1. While ethically responsible, this role also increases the reactivity effect. Knowing they are
under observation and what they do and say is being noted by a stranger in their group, persons
often act differently, showing them selves in the best possible way. If the observer is skilled and
good at establishing warm, trusting relationships, most members of the group will accept the
observer's presence, and, after awhile, generally behave in a normal fashion. Under these
circumstances, the threat of reactivity usually is greatly reduced.
PREV NEXT

Chapter 13 Page 7
Table 13.1. Possible relationships with the group being observed

Introduction
Awareness of
Casual observer's role
by the group Degree of observer's participation in the group
observation
Substantial Partial
Participant Not Known Substantial to full participant, Casual to limited
observation
role concealed participant, -role concealed
Conducting a Substantial to full participant, Casual to limited role, role
participant Fully known
role openly known openly known
observation
study
Participation may be less than full involvement, such as the role assumed by Haberer and
Establishing
illustrated by the right-hand column in Table 13.1. Haberer's role was based on partial
validity participation in village life but with her role fully known. Everyone knew her purpose and
accepted her in the village. In most extended participant observation studies, disguising one's role
Generalizing would be difficult as well as unethical. In addition, one could assume a substantial role in a group
from a without making one's purpose known, as illustrated on the upper left side of Table 13.1
qualitative
.Decisions about the initial focus of the study, the setting, and the role one assumes influence one
study
another. These decisions are generally worked through together, with the role one assumes
matched as best as possible with the setting the observer has chosen and the purpose of the
Personal investigation.
qualifications
Preparing for living in the field. The period of observation is referred to as the field work phase
Case studies of an observation study. Preparing for the field includes arranging for access to the group,
providing for daily living while observing, and obtaining the supplies needed for recording and
Focus groups analyzing observations. Access can be arranged in various ways. In organizational studies, access
is generally achieved through approval by the proper level of management. For village or
community studies, access almost always requires the permission and sponsorship of the local
Strengths and
leaders or political officials. Getting local approval may first require obtaining approval from
limitations of regional or national officials. If they approve, they can often act as the sponsor of the investigator
qualitative and guarantee approval from local officials. In some cases, obtaining sponsorship of a prominent
research local person is all that is needed. This person can then act as an intermediary and help convince
local leaders to approve the investigation.
Aids
Planning for fieldwork may also include making arrangements for "bed and board." Living
arrangements are particularly important for studies extending over many months. Who the
investigator lives with can be crucial to the success of an investigation. Selecting a "host" family
whose members are well known and highly respected by members of the group will help achieve
rapport and greatly aid in conducting field research. The head of the family and family members
can also open access to others in the group, supply an inside view of the group, allow you to get a

Chapter 13 Page 7
trusted view of your emerging perceptions, and provide a validity check on your conclusions.
Haberer's landlord fulfilled these functions in her study.
Before starting fieldwork, you will also want to have sufficient notebooks for recording
observations. If members of the group approve and are comfortable with audio or video
recording, you will need to get this equipment and plenty of spare batteries, tapes, or disks.
The objective of the planning phase is to prepare for observation in the selected setting. In
contrast to the design for a quantitative study, any plan for observing should be considered as
flexible, subject to change as the field work progresses. With an initial plan, one is ready to begin
observation.
PREV NEXT

Chapter 13 Page 8
Observing
Introduction
Figure 13.1 also shows the main activities carried out in conducting observation. Typically, these
activities include:
Casual
● Entering the group and gaining acceptance by its members;
observation
● Establishing one's role in the group;
Participant ● Observing and recording observations;

observation ● Analyzing observations; and
● Leaving the group.

Conducting a
participant Gaining acceptance. There is no formula for gaining entry and establishing one's role as an
observation observer in a group. The investigator's initial task is to establish rapport, a feeling of mutual trust
and confidence, with leaders and potential key informants and other members of the group. Key
study
informants are persons who are able to supply unusually valuable information relevant to the
investigation. They may be formal or informal leaders in the group; government officials; long
Establishing time friends of the investigator from the group; or persons who the investigator meets and
validity establishes a close bond with. Knowledge of the group gained during the planning phase can help
in identifying potential key informants. They also become the links to other members of the
Generalizing group. Often members of the host family are important key informants, as Haberer's landlord
from a was.
qualitative
To achieve rapport, a good rule is to assume the role of a respectful guest, showing respect for
study each member of the group and establishing rapport with them. A careful observer moves slowly
in establishing relationships and in gaining the respect and confidence of critical leaders and key
Personal informants, and then, through their sponsorship, extends his or her relationship to all members of
qualifications the group.
Case studies Building rapport is a process, similar to developing feelings of trust with friends. If you are
good at this process and can quickly establish warm, close bonds with others on the basis of
mutual respect and trust, you will probably be good at creating and maintaining rapport. But if
Focus groups
you have difficulties in this area, you should think carefully about using the participant
observation method.
Strengths and
limitations of Establishing one's role. In establishing a role, observers have to be flexible and be ready to
qualitative adjust quickly to changing perceptions of the observer's presence. Members may encourage fuller
research participation by the observer in various activities or they may remain suspicious of the observer
and limit his or her role. Encouragement for a greater role is a positive sign and, obviously should
be accepted with appreciation. Observers also need to react tactfully and accept any limitations
Aids
with good manners.
Observing and recording field notes. In participant observation, observing occurs mainly by
watching, listening, and asking questions. Generally this process goes on over a period of time,
with opportunities to observe and talk with members of the group a number of times and under a
variety of circumstances. Such prolonged interaction provides opportunities to get "deeper"

Chapter 13 Page 8
insights and to learn about the motivations of group members. Observing, however, takes
considerable skill and attention. The objective is to create a full, accurate description of what
occurred. Also, asking questions has to be done carefully, based on the readiness of respondents
to answer questions. In Chapter 10, as part of the discussion of questionnaire development, we
described how to phrase questions to produce valid data. If you are thinking about using
participant observation, look again at Chapter 10.
Observing activities and recording descriptions of them occur together. Each period of
observation is called a session. A record is prepared to describe what occurred during each
session. Records are prepared in written form and constitute the field notes of the study. The
field notes contain the data that is analyzed to produce the findings of the study. In creating field
notes, observers have to make three decisions: What to record? How to record? When to record
the information?
Deciding what to record. The "what" consists of detailed, concrete descriptions of what
occurred in a session. In any session, this might include:
● Who was present,
● When the session occurred,
● What each participant said, including their exact words of what is thought to be
particularly important comments, and
● How the interaction played out.
PREV NEXT

Chapter 13 Page 9
Be particularly alert to actions or comments that illustrate a significant action or reasons for the
Introduction observed behavior. How will you know when such an event occurs? To be honest, in the
beginning you may not be sure about what is or is not important. At first, try to capture and
Casual describe as much as you can. Try to record all the parts of the interaction you are observing,
observation including details of actions and conversation. It is important to include descriptions and
assessment of the emotions that accompanied actions or were expressed in conversations. Did
Participant participants express pleasure, unhappiness, anger, anxiety, or other emotions? Who were the
feelings directed to? Was the interaction relaxed and friendly or were there signs of tension or
observation
perhaps hostility? What kinds of body language were expressed? Events or comments that may
seem trivial at the time of observation may prove to be quite significant later. Will you miss some
Conducting a important comments and interactions? Yes, this will probably happen, particularly at the
participant beginning of your field work. With experience, however, you should develop a sense of what is
observation most relevant for your investigation and become more confident in focusing on and describing
study the really important actions and conversations.
Along with, but separate from your description of what you observed, you should also record
Establishing
your ideas about what happened and why. Your notes might include running reactions to what
validity you are observing and hearing. These may be in the form of questions: Why did the person say
what he or she did? Why was the emotion you observed expressed at that time? Reactions to
Generalizing observations may also be expressed as tentative hypotheses. Based on observations, you may
from a begin to see linkages among variables you are investigating. These insights can be expressed in
qualitative the form of a hypothesis and noted for testing against further observations. However, do not
study confuse your comments with your descriptions of what was observed. The latter are the data;
your comments are guides to yourself to help with the analysis and interpretation of what you
Personal observed. Both are subject to revision as you gain insight about the behavior of the group.
qualifications How to record. For recording observations, you have four choices:
● Use of an audio recorder or recording interviews
Case studies
● Use of a video camera for recording action and conversation
Focus groups ● Relying on recording notes by pen or pencil in a notebook
● Relying on your memory

Strengths and
Each recording method has strengths and limitations. The obvious appeal of audio recording is
limitations of that it produces a full record of any conversation. Both audio and video recording, however, have
qualitative several severe limitations. With audio recording, you get only what people say: There is no
research record of how they acted; no body language. Video recording captures action as well as verbal
content, but it is even more reactive than audio recording. Using a video camera is a sure way to
Aids alert persons that their actions and conversation are being recorded. Few persons can resist the
urge to "act" a bit, making their behavior different from what it normally would be in the
situation being observed.
Either method is impractical for recording activities throughout a day and evening. Also, the
apparent advantage of getting so much information on audio tape, however, is a distinct
disadvantage during data analysis. Every hour of data recording requires about six hours for

Chapter 13 Page 9
transcribing the content to a written record, and that assumes speedy typing. An investigator
using extensive audio or video recording can quickly find him or her self overwhelmed with
records and be faced with sifting through mountains of data to find significant information.
While being the least reactive, relying entirely on memory has its own limitations. With so much
going on during any period, most of us are unable to remember all the rich detail so important to
qualitative data collection. Worse, it is easy to confuse the names of persons who were present,
who said what to whom, and who took what kind of action.
This leaves the old fashioned way of taking notes by writing them down in notebooks. Note
taking, however, can also intrude upon observing and create some unease among persons being
observed. To minimize any problems, investigators frequently limit note taking during a session
to writing down just a few key words or phrases. These field jottings are used to help the observer
recall the main features of the observational session. Even jotting down brief phrases
occasionally can upset some persons. To the extent possible, therefore, try to find inconspicuous
times and places for creating field jottings. You can train yourself to make mental notes of key
interactions and conversions and then find a time as soon as possible thereafter to jot down notes.
PREV NEXT

Chapter 13 Page 10
Field jottings are only aids for creating the full record of what was observed. To protect against
Introduction memory loss or distortion, jottings or mental notes should be converted to full field notes
immediately a session, if possible. Natural breaks in actions, such as rest periods, can provide an
Casual opportunity to jot down mental notes. When opportunities do not occur naturally, you will have
observation to rely on your mental notes until you have an opportunity to complete a set of notes. If not done
earlier, field jottings should be expanded to full notes by the end of each day. Your field notes
Participant are your data for the investigation. These notes contain descriptions and your interpretations of all
that you observed. They should be as detailed as you can make them, with exact quotes of key
observation
conversations and from interviews. Writing field notes may take as much time as you spend in
observing. Field researchers learn to reserve time for preparing their field notes. In preparing field
Conducting a notes, investigators also include analytical comments. As the investigation progresses, analytical
participant notes tend to increase.
observation
study Guidelines for observing and recording field notes are summarized in Box 13.2.
Box 13.2. Guidelines for observing and recording field notes
Establishing
1. Train yourself to make mental notes of what you are
validity observing.
2. If possible, jot down brief notes describing the most important
Generalizing
actions or conversations that occurred during an observational
from a session.
qualitative
3. Convert mental notes to field jottings or full field notes as
study soon as possible after each observation session.
4. Include detailed descriptions of who was present, where the
Personal
action occurred, what happen, what was said by whom and to
qualifications whom, how the action played out, and other important features
of the interaction.
Case studies
5. Include your own reactions and analytical comments in your
field note - hunches, hypotheses.
Focus groups
6. Use any quantitative data as well.
Strengths and 7. Use triangulation to the extent possible.
limitations of
The last point, triangulation, is the key to successful qualitative research.
qualitative
research Triangulating
In everyday life we often use more than one way of getting information we need. We may ask
Aids
several persons about the same thing and then compare their responses. Or, we may consult
several books and, in addition, ask a person we trust to make sure we have correct information.
Social scientists use the term triangulation to describe ways of verifying or checking the accuracy
of information and the validity or conclusions. The term comes from the practice of navigators
who locate their position by taking sightings from different angles from known positions. Using
mathematics, they can pinpoint the position of their ship or airplane.

Chapter 13 Page 10
Triangulation is critical in qualitative research, where the collection, recording, and analysis of
data depend entirely on the perception and interpretation of a single observer. When using
participation observation, researchers try to verify each observation they intend to use as data.
Information from informants is checked against similar information from other informants. When
informants give the same information on some point, their responses can be assumed to be
accurate. When informants disagree, the observer has to seek additional sources of information to
establish what is true. When the same behavior occurs repeatedly, as in work performed by one
gender or the other or in family visiting patterns, an observer can accept it as valid description of
the behavior being studied.
Another way to verify observations is to use two or more methods for obtaining data. This is
what Haberer did. She used participant observation, unstructured and structured interviewing,
content analysis of written materials, and analysis of historical and current official records. Using
different methods has two benefits. One, we can benefit from the relative strengths of each
method and, at the same time, minimize their corresponding weaknesses. Two, each method can
provide unique information and insights. Haberer, for example, no doubt, got valuable insights
and new ideas from each of the methods she used. As is true in all qualitative studies, she
integrated the various ideas she derived from each method and reached her overall conclusions.
PREV NEXT

Chapter 13 Page 11
Analyzing and interpreting observations

Introduction
In quantitative studies, such as experiments or surveys, data analysis follows the collection of
data. Qualitative research is different: In qualitative investigations, data collection and analysis
Casual
proceed together, with each influencing the other. There are no precise guidelines for analyzing
observation qualitative data. The investigator interacts with his or her data, constantly looking for patterns in
the observed interactions and associations among forms of observed behavior.
Participant
observation Analysis is based on frequent review of accumulating field notes to identify patterns and
relationships. To help in this regard, most researchers develop a set of files, based on material
Conducting a included in the original field notes. A historical file containing information organized in a
chronological fashion usually proves helpful. This file can be started during the planning phase of
participant
a project and be updated throughout the project. Person or family files are frequently used as well
observation to organize information about the important persons in the group. Sometimes organizational
study files are needed to organize information about clubs, formal organizations, and government
bodies relevant to the study.
Establishing
validity A certain amount of trial and error generally occurs in creating files. You can start with a set of
files, such as we have mentioned, and, as you gain experience, you may decide to combine
certain files or create new ones. To facilitate creation of files, you may need to make several
Generalizing
copies of your field notes. Then, if you want to create additional files, you can cut up copies and
from a insert observational and analytical notes into the proper files.
qualitative
study Creation of files is the mechanical part of analysis. The heart of qualitative analysis is the
interaction of the observer with his or her data. Analysis of qualitative data is partly an art. Each
Personal individual brings a unique combination of skills, knowledge, and a way of thinking to this
process. Consequently, there are few specific, fixed rules to guide qualitative analysis.
qualifications
Nevertheless, we suggest several ways to analyze qualitative data. Not all suggestions will apply
to the analysis of a body of data, but some of these should help in most situations.
Case studies
First and foremost, triangulate to the extent possible. Constantly review your data: Look for
Focus groups consistencies and inconsistencies in what you have heard, seen, or read about. When
consistencies occur, look for variables that tend to occur together. This will give you an idea of
Strengths and possible relationships among variables. When differences occur, find out why informants
limitations of disagreed. Whenever you can, get additional data on matters in dispute. For example, if
informants disagree on the numbers of persons moving from an area, try to get the actual number
qualitative
from official records.Then, investigate why you got different reports from persons. The answer
research may give you insight into how different informants perceive local conditions, depending upon
their position in the local social structure or how much they feel threatened by social changes.
Aids
Watch for negative evidence. Be alert to conditions that differ from what you expect. Further
analysis of negative evidence often leads to important new insights, revised hypotheses, and the
search for additional critical data. When you discover something that does not fit your present
understanding of a situation try to find out why. For example, suppose you found that young
persons in a rural area preferred to remain farm workers rather than migrate and seek work in

Chapter 13 Page 11
cities. Now, you would be have to make a decision: Did this finding occur because of (1) errors
you made in observation; (2) is it a true, systematic variation from the general pattern; (3) did you
find some unusual cases that are exceptions to the general pattern of young persons leaving the
area; or (4) is the finding an indication of a new social trend.
Any result contrary to what you expected should be examined carefully. Any of the four
explanations might be correct. Use of triangulation may help establish which interpretation is
valid.
Develop categories for organizing and analyzing observations. Try to identify socially
meaningful categories for analysis purposes. Categories may be based on locally-used
descriptions (tribal or lineage names, references to "good" versus "no good" people, etc.) or may
reflect social science concepts (integrated versus alienated members of the community, social
status positions, etc.). The meaning of locally used terms is important for understanding the
behavior of group members toward one another. Initially you may want to use both local systems
of classification and categories based on social science concepts in developing your files. As you
gain experience, you can then merge the two systems and have one set of files for final analysis.
PREV NEXT

Chapter 13 Page 12
Look for deviations from the norms. Try to identify people or groups whose behavior does not
Introduction conform to the prevailing norms. In sociological terms these persons or groups are called
deviants. Identification of deviants and names for them may come from the deviants themselves,
Casual from other members of the group, or you may construct definitions for them based on social
observation science usage. When you identify deviants try to understand why they are different and how their
differences affect the behavior of the group.Are the deviants ahead of the main body of the group
Participant in setting new norms; are they a residual group left behind as the main group adapted to new
conditions; or do they represent a collection of unconventional individuals whose behavior has no
observation
major social significance?
Conducting a Watch for sequences of events and interactions. Are there patterns of association among
participant norms, deviants, events, or other concepts in your field notes and files? Which of these stand out
observation as most prominent or important? Why do they occur? Use connections you see to develop
study hypotheses based on your understanding of relationships between concepts. Test these
hypotheses with further observations and modify any emerging conclusions you might have
tentatively drawn.
Establishing
validity Use any available data in your analyses. Investigators using participant observation also seek
and use all available data they can find. These data may include census, population, crop or other
Generalizing economic data, and historical information.
from a Guard against premature or biased conclusions. Qualitative analysis is tricky: Frequently
qualitative there is no objective, outside criteria to use in analyzing data. Consequently, it is sometimes easy
study to think you have spotted a pattern in your data.Sometimes this pattern is not in the data, but in
your perception of the data. You will need to guard against hasty or poorly based conclusions.
Personal The best protection is to examine all the data and to look for data both for and against any
qualifications hypothesis or conclusion you wish to present Then base your conclusion on your best judgment.
Guidelines for analyzing qualitative data are summarized in Box 13.3.
Case studies
Box 13.3. Guidelines for analyzing qualitative data
Focus groups 1. Use triangulation to establish the validity of observations and to
verify findings.
Strengths and 2. Look for indicators of concepts that best describe or might help
limitations of explain what you have observed.
qualitative 3. Constantly examine field notes, testing the value of tentative
research concepts and relationships among observed actions.
4. Be alert to conditions that differ from what you expected.
Aids
5. Try to identify socially meaningful categories for analyzing the
content of your field notes.
6. Look for deviations from the norms.
7. Develop hypotheses for why recurring interactions occur and test
these by subjecting them to further observation.

Chapter 13 Page 12
8. Use any available data in interpreting your data and in formulating

conclusions.
9. Guard again premature or biased conclusions.
Ending fieldwork and leaving the group

In most quantitative investigations, such as surveys, the investigator knows when to stop. Data
collection ends when the last member of the sample is interviewed. In field research, there is no
specified time for ending data collection. Each researcher has to decide when he or she has
enough data in the form of a narrative description of the behavior of the group being investigated.
There is no precise basis for making this decision. A few criteria that might help, however, are
listed in Box 13.4.
PREV NEXT

Chapter 13 Page 13
Box 13.4. Criteria for stopping data collection

Introduction ● When observations no longer provide new data or insights
● When you have no new questions you need to answer

Casual
● When the interpretations you have developed from your
observation
data stand up against any alternative
● When informants agree that you have provided a full and
Participant
observation accurate description of what you set out to learn about
● When key informants agree that your explanation of the
Conducting a behavior you focused on is as valid as can be developed

participant
After you decide you have all the data you need, you are ready to leave the group. Advance
observation
thought should be given to the way you make your exit. One way is to make a quick exit, with
study little advance warning. This way may be the appropriate way to exit when the observer's role was
unknown to the group. When one's role is known, a better way is to make a gradual, slow
Establishing withdrawal over a period of time. When extremely close relationships are established,
validity leave-taking is similar to saying goodbye to close friends: We give them advance warning and we
visit frequently before the final departure. In field research, this would translate into a drawn out
Generalizing departure, with the departure data announced in advance and with possible periods of temporary
from a departure and returning, ending with the final departure of the investigator.
qualitative The protracted departure also fits well with the final stages of qualitative research that of
study developing a final analysis of field notes and preparing the final report on the project. Return
visits provide an opportunity to verify information, clarify discrepancies that become obvious in
Personal later analysis of field notes, and to get feedback on conclusions from key informants and others
qualifications who provided critical data.
Either way of exiting should be based on good manners, including showing appreciation to your
Case studies
hosts as any sensitive guest would. As a minimum you will want to express public as well as
private appreciation to any sponsors, leaders, officials, and key informants as well as to families
Focus groups and persons you observed.
Strengths and In making an exit you also have two ethical responsibilities. One is to maintain any protection
limitations of you promised to the subjects. The other is to your research colleagues and this is to leave in such
qualitative a way that opportunities for further research are enhanced.
research Final analysis and writing the report
Aids In most observational studies, analysis extends well beyond the end of the field work phase of the
study. Frequently, a period of further reflection and analysis follows during which the
investigator develops the conclusions of the investigation. The conclusions are the investigator's
interpretation of what the mass of observations, represented by his or her extensive field notes,
mean. Readers judge the validity of conclusions on the basis of the descriptive narrative and
quotes from participants. Because of this necessary descriptive material, reports based on
participation observation frequently are longer than those based on analysis of quantitative data.

Chapter 13 Page 13
For additional information and discussion of how to plan and conduct a participant observation
study, we recommend several sites:
The Five Dimensions of Participant Observation presents participant observation in terms of
(1) the role of the observer; (2) portrayal of the role to others; (3) portrayal of the study purpose
to others; (4) duration of observation; and (5) provides a discussion of what to observe and how
to take and record field notes
Qualitative Research: Essentials of Participant Observation describes and discusses the
relations between the observer and participants and other dimensions of participant observation;
includes guidelines for conducting participant observations
Participant Observation discusses key concepts and terms associated with participant
observation, arguments in favor of this research method and its limitations
PREV NEXT

Chapter 13 Page 14
Establishing validity
Introduction
In Chapter 6, we describe some frequently used techniques for estimating validity and reliability
in quantitative studies. Generally, these techniques do not apply to qualitative studies because
Casual
data are in the form of words and descriptions contained in the narrative prepared by the
observation investigator, not in the form of numbers that can be subjected to statistical analysis. Different
techniques, therefore, are used for asserting validity in qualitative investigations.
Participant
observation A field investigator has to convince readers that the observations reported in the narrative and the
conclusions included in his or her report are free of bias or error. This effort is part of the data
Conducting a collection process and proceeds through all stages or analysis. And this is why triangulation is so
important. Each major point has to be backed up with evidence of some kind - repeated
participant
observation of certain behavior or collaboration from several qualified persons.
observation
study Generalizing from a qualitative study
Generalizing conclusions from a field study is also different from that for a quantitative study. In
Establishing the latter approach, the sample is designed to allow for empirical generalization of results to the
validity population from which the sample was drawn. Most observational and other qualitative studies
are conducted within one or, at most, several sites. These sites cannot be considered as
Generalizing representative of any larger population of similar sites. Even if it could be argued that they are,
from a the sample of one or several sites would be too small to support any empirical generalizing. How
qualitative then can an investigator hope to extend conclusions to contexts other than the one studied?
study One answer to this question is that precise scientific generalization may not be the objective of
the investigation. Instead, a field investigation may be conducted for its practical, applied value.
Personal If the purpose of a field study in a community were to help the government authorities develop
qualifications plans to assist the community to stemming the spread of AIDS, for example, practical
recommendations based on evidence-based recommendations would substitute for scientific
Case studies conclusions and generalizations
However, scientific generalizations can also be derived from qualitative data. Doing so, however,
Focus groups
requires considerable skill in conceptual analysis. The investigator seeks to identify critical
concepts and to establish relationships among them. Using these empirically established
Strengths and relationships, the investigator builds a conceptual or theoretical explanation for behavior that was
limitations of observed. The resulting explanation is then tested against existing theory as a final way of
qualitative checking its validity. This approach goes by the name of grounded theory because any theoretical
research explanation that is developed is based on or grounded in the descriptive data collected by the
observer. For more information on this approach to generalizing the results of qualitative data, we
Aids suggest you consult one of the many books devoted to qualitative research.
Personal qualifications
Fieldwork can be exciting and highly rewarding: It also requires a range of skills and abilities,
and a lot of patience. Test yourself against the criteria listed in Box 13.5. Do you have the
combination of personal qualities and the writing and analytical skills needed? If you think you

Chapter 13 Page 14
do, you might want to undertake a brief period of participant observation with some group to test
yourself. The results would be a further and more realistic test of how well you will probably
perform as a field researcher.
Box 13.5 Qualitative research skills
Personal qualities
● An empathic person - able to feel the feelings of others
● Sensitive to nonverbal clues - posture, gestures, facial expressions
● Flexible - adaptable to new situations
● Likeable -able to get others to like and accept you; likes meeting and
talking with new persons
Observational skills
● Seeing both the total picture as well as separate parts
● Hearing what is said instead of what one thinks was said
● Not bothered by periods of silence
● Able to make and retain mental notes of what was observed
● Able to create field jottings that capture the most important elements of
an interaction or conversation
Analytical skills
● Ability to prepare and organize files and notes to see meaning in data
● Ability to extract meaning from a mass of material
● Ability to integrate findings into a cohesive, overall description of a

group and its behavior
● Being able to draw unbiased conclusions from findings
● Able to provide a convincing explanation for the observed behavior
Writing skills
● Able to reconstruct full notes accurately and fully from mental notes or
field jottings and that capture the most important actions or comments in
an interaction
● Ability to prepare interesting, informative narrative descriptions of
observed behavior and comments
● Ability to create an interesting story as part of the description of the
observed behavior
PREV NEXT

Chapter 13 Page 15
Case studies
Introduction
The term case study comes from medical research and refers to the intensive study of a single
person having some disease. By understanding the single case fully, doctors, for example, are
Casual
able to diagnose and treat other persons with similar symptoms more effectively. Psychologists
observation adopted the tradition of case studies for research on personality and psychological disorders.
From psychology, the method spread to other social sciences and to management studies.
Participant
observation In social research, case study methods are used to conduct in-depth investigations of a single
person, village or other community, a business organization, or some other entity. Case studies
Conducting a almost always are based on a multi-method of collecting data. Qualitative methods are generally
used, mainly in the form of unstructured interviews and observation. Available data are always
participant
analyzed to the fullest extent possible. Surveys are frequently conducted as part of case studies.
observation Results are generally expressed in narrative form, with rich, detailed descriptions of the essential
study features of the case being investigated. Using all available data, the investigator seeks to
understand the case thoroughly, as an integrated whole.
Establishing
validity A number of Web sites provide addition information about the case study method and guidance
for planning and conducting a case study. Case Studies, for example, examines case studies as a
Generalizing form of qualitative research; provides an introduction to case studies; discusses types of case
study designs; and provides several examples of well documented case studies as a way of
from a
illustrating how data are collected and analyzed in a case study.
qualitative
study Case studies can also be used in evaluation studies, the subject of the following chapter. For more
information, we refer you to The Application of Case Study Evaluations, which briefly
Personal discusses six kinds of case studies. Although written for use in evaluation studies, the discussion
qualifications has general value as well for understanding and appreciating the case study approach to research.
Example
Case studies
The investigation by Harberer (1985) of women in an Arab Muslim village in northern Israel was
Focus groups a case study. Harberer selected a village that she thought would serve as an example of the ways
women seek to influence decisions in an Arab village. She supplemented her qualitative data,
Strengths and derived from participant observation, informal interviews and the analysis of essays, with
quantitative data from official records, structured interviews, and a vast amount of historical data.
limitations of
Her objective, which she amply fulfilled, was to provide a rich, detailed description and
qualitative interpretation of the lives of women, how they regard themselves, and how they influence
research decisions in the household and at the village level.
Aids Analyzing data

The findings of a case study usually evolve from successive analyses of data. Generally the
investigator begins by describing each subsystem of the unit being studied as fully as possible.
In a community case study, for example, the researcher would analyze data to describe the main
community subsystems, such as functions of the marriage and kinship system, how the economic
life of the community is organized and carried out; how the religious system functions; and how

Chapter 13 Page 15
civil order is maintained. Descriptions usually evolve, becoming progressively more detailed and
comprehensive over the course of the fieldwork. When gaps or contradictions are found in the
data, the investigator seeks further data. Refinement continues until each major system or
functional part of the unit is fully understood and described in detail.
While this analysis is going on, the researcher looks for interrelationships among the parts of
subsystems. In a community study, for instance, one may see that certain changes are occurring
in both the marital and kinship systems and the economic system. At some point, the analysis
would shift to analysis of relationships among changes in the marital and kinship system with
changes in the economic system. The ultimate objective is to describe the case as a whole,
showing how the parts of subsystems fit together, influence one another, or how they are
changing.
PREV NEXT

Chapter 13 Page 16
Focus groups
Introduction
Conducting focus groups
Casual Focus groups are a special kind of group interview combined with techniques of qualitative
observation analysis. Most focus groups consist of five to ten persons who are selected to discuss a specific
topic or issue. Criteria for selecting participants depend on the purpose of the investigation. When
Participant a focus group is used as part of an exploratory study, as in preparation for constructing a
observation questionnaire, the investigator would select persons matching the characteristic of those who will
be interviewed later. When the purpose is to learn how opinions may be changing on some
Conducting a issue, participants might be selected purposively to represent a range of views of the issue.
participant Once selected, the investigator brings the participants together for one or more sessions and
observation initiates a discussion. The investigator decides in advance what he or she wants the group to talk
study about and prepares a set of questions to get the group discussion started. Although questions are
prepared in advance, they are presented informally to the group. The order in which questions are
Establishing presented to the group depends on how the discussion develops. As the conversation develops,
validity new questions may occur to the investigator and be presented to the group for discussion. The
investigator or an experienced moderator asks the questions and keeps the discussion focused on
the issue being investigated. As views are expressed, the moderator seeks clarification, asks for
Generalizing
agreement or disagreement on opinions that are expressed, and probes for additional comments.
from a
qualitative All the guidelines presented in Chapter 11 for effective interviewing also apply to moderating a
study focus group. The moderator establishes and sustains rapport with the group, maintains a neutral
role in asking questions, encourages all participants to speak without letting any one person
Personal dominate the discussion, and keeps the discussion focused on the issue. Managing a focus group
qualifications requires a great deal of skill and tact. The investigator may serve as the moderator or may select a
person who is skilled in group processes to moderate the focus group.
Case studies Focus group sessions generally last one hour or at the most two hours. Recording the discussion
may be done by an assistant, who maintains a set of notes on what is said, or by audio or video
Focus groups recording. Taking notes by hand is a difficult task. Discussions can become heated, with several
persons talking at once. Important points can easily be missed or incorrectly recorded. For these
Strengths and reasons, focus group discussions are usually recorded on audio tape. Audio is recommended over
limitations of video recording because it is easier, less expensive, and less noticeable to participants, and,
qualitative therefore, less reactive. Participants in focus groups generally ignore the microphone used for
audio recording, but are much more aware of the camera and lighting that may be necessary for
research
video recording. In addition, audio recordings are also easier to transcribe for later analysis. In
addition, an observer can add to the audio record by recording notes on the kinds and levels of
Aids emotion and facial and body expressions of agreement and disagreement that were expressed at
various times by participants. These additional observations can add meaning to the verbal
content of an audio recording.
Content analysis techniques are used to analyze the recorded discussion of focus groups. A set of
categories, based on themes or issues, is developed for coding the discussion. The steps involved

Chapter 13 Page 16
are the same as those described in Chapter 12 for conducting content analysis.
If you are considering using focus groups in your research, we suggest you visit:
Qualitative Research: Telephone Focus Groups, Face-to-Face Focus Groups presents lively
discussions on more than 15 full length articles on telephone focus groups; includes practical
advice on how best to manage telephone and face-to-face focus groups
Another site, Using Focus Groups for Evaluation, describes how use of focus groups can help
in conducting an evaluation study. While focused on evaluation, this site also has general value
for any use of focus groups. It describes and illustrates the organization and use of focus groups
and discusses what they can tell you and what they can't tell you.
PREV NEXT

Chapter 13 Page 17
Example
Introduction
Focus groups were the sole source of data in an investigation of the willingness of women in
southwestern Uganda to receive counseling and to be tested for HIV during pregnancy (Pool,
Casual
Nyahzi and Whitworth, 2001). Twenty four (24) groups of 6 to 10 women were organized. Each
observation was led by an experienced group facilitator who asked the groups to discuss four questions: (1)
Can a woman pass on HIV to her infant through breast feeding; (2) Can babies of HIV-positive
Participant women be born HIV-negative and, if so, can they be infected at birth; (3) Would you (a woman)
observation be willing to be counseled regarding HIV and be tested if you become pregnant; and (4) Would
you inform the medical staff if your test was positive?
Conducting a
The facilitator led the women through discussions of each question. An assistant took notes on
participant
body language and other expressions of feelings. Discussions were tape recorded and later
observation transcribed into English for analysis. Each session lasted about 90 minutes. The transcribed
study records of the discussions were analyzed using content analysis methods. Like most qualitative
reports, the data consisted of extensive quotes from the women that illustrated typical or main
Establishing points in the discussions.
validity
Analysis showed that virtually all women were willing to be counseled and tested in the event of
pregnancy. They were anxious, however, about confidentiality of the results of testing and
Generalizing
expressed fear that maternity staff might refuse to accept them, if they were known to be
from a HIV-positive, when their time came for delivery. The women were also concerned that if their
qualitative husbands found out that they were HIV-positive they would be blamed and that separation or
study violence might occur. The authors concluded that testing for HIV might be accepted in principle,
but much needs to be done to ensure confidentiality and to allay the women's fears during
Personal pregnancy and thereafter.
qualifications
Strengths and limitations of qualitative research
Case studies Strengths
Observation is superior to other methods of data collection for describing actual behavior in a
Focus groups
given situation. Available data describes only certain features of past situations while surveys
provide only what respondents say is true about something. These verbal responses may or may
Strengths and not match what the respondents actually do in real life. Observation provides descriptions of
limitations of what actually happens in real life situations.
qualitative
research Observational methods also yield richer and more complex data than other forms of data
collection. Observation is unequaled for describing the complexity of behavior, including the
kinds and intensity of the actions of people, the emotions that are involved, what is said, and how
Aids
different members of the group influence each other. In contrast, surveys generally provide far
less data.
Another advantage is that observation is a flexible way of collecting data. As new insights are
gained or new leads open up, the observer can shift perspective quickly and explore new areas of
inquiry. This is not possible with most surveys: Once questions are set, all are asked in the same

Chapter 13 Page 17
way for all respondents. With available data, the investigator has no choice; he or she has to use
whatever data is at hand.
Lastly, observation studies can be carried out with less community attention or disturbance than
surveys. This can be a powerful advantage when local officials or leaders have to grant
permission for a study. Gaining approval for a study based on one researcher living in a
community, for example, is easier than getting approval for a survey that will require interviews
with a large number of persons.
Case studies and focus groups share these strengths. In addition, case studies have several
appealing features. The emphasis on understanding the dynamics of a case meets the practical
needs of decision and policy-makers. A well written narrative with rich descriptions of actions
and quotes from persons provides graphic, easily understood information. This kind of
information can give additional meaning and value to statistical data from surveys or analysis of
existing records.
Case studies, unlike surveys, experiments, and other quantitative methods, also provide an
opportunity to engage members of the group being studied in developing awareness of the
group's situation. With increased awareness, the investigator can introduce training and other
activities to help strengthen the group's problem-solving skills.
Focus groups have unique strengths as well. This technique provides a fast, inexpensive way to
obtain a large, rich body of information on some topic. Focus groups can be organized quickly
and can be held almost anywhere - in an office, a classroom, or under a tree. During sessions the
researcher can probe to clarify issues or to get into the deeper meaning behind comments. The
open, flexible nature of focus groups also encourages participants to talk about topics they
normally would not discuss with strangers. Ghorayshi (1996), for example, makes this point in
relation to obtaining "suppressed discussion" among women. A focus group composed of only
women, she demonstrates, encourages women to talk about differences in gender norms and
practices more honestly than they would in a personal interview. Also, focus groups have a
special advantage in developing countries: They can be used with participants who are illiterate.
Focus groups also can be used with children.
Another strength of focus groups is that the technique can be used at any stage of research and for
a variety of purposes. Altareb (1997), for example, used five focus groups to get ideas for
developing items for measuring attitudes toward Muslims. A research team in Yemen (Inaoka
and others, 1999) used focus groups to supplement data obtained from structured interviews.
Kagimu and others (1998) also used focus groups to enhance information obtained in surveys
related to knowledge about AIDS.
PREV NEXT

Chapter 13 Page 18
Limitations
Introduction
A major limitation of the observational method is the considerable amount of training,
experience, and skill required to observe and record events accurately and completely. Not all
Casual
persons can acquire the needed skills. In addition, observation generally is limited to descriptions
observation of what happens in small groups of people, which also limits the ability to generalize the results.
Participant Observation can also be very demanding. Frequently, conversation and action occur at the same
observation time. The observer has to note what is said, to whom, in what way, with what effect, and what
kind of behavior was occurring. All of these things can occur very quickly as part of complex
Conducting a interaction among a number of persons. Inexperienced observers can miss part of the interaction
or fail to record the action accurately. The substantial demands placed on observers also raise the
participant
possibility of bias in how observations are made, recorded, analyzed, and interpreted. In
observation observational studies, the investigator is the research instrument. He or she observers, records the
study observations, and then analyzes and interprets them. This is why the accuracy and completeness
of the record of observation is so important. Other social scientists will examine and decide for
Establishing them selves whether the conclusions in a report are warranted, based on the record supplied by
validity the author.
These limitations also apply to research based on the case study or focus group techniques. For
Generalizing
each, bias introduced by the investigator in the collection and analysis of the data remains a
from a constant threat. Also, it is difficult to generalize findings from either a case study or results of a
qualitative focus group. Even if participants in a focus group are selected randomly from some population,
study the size of the sample is usually too small to warrant generalization.
Personal Aids
qualifications Internet resources
Case studies This chapter focused mainly on use of participant observation as a data collection technique.
Participant observation is only one of various methods of conducting qualitative research. A
number of Web sites describe and illustrate how to conduct other forms of qualitative research. If
Focus groups
you are thinking about using a qualitative approach to research, we suggest you also view some
of the following sites:
Strengths and
limitations of Methods-Qualitative, a guide to various qualitative research techniques, including natural
qualitative observation, interviewing, obtaining oral histories, and conducting focus groups
research Qualitative Research Resources on the Internet, this site provides on-line access to a
collection of qualitative research sites and materials available through the Internet, includes links
Aids to Web pages, journals, syllabi, and the current issue of The Qualitative Report, an online journal
Overview: Ethnology, Observation Research and Narrative Inquiry describes observational
and interpretative techniques for observing behavior of groups
Qualitative Measures presents important questions you should consider before undertaking

Chapter 13 Page 18
qualitative research; also provides links to pages on the same site that discuss and illustrate
various specific forms of qualitative research. These additional pages include topics such as:
The Qualitative-Quantitative Debate illustrates that all qualitative data can be coded
quantitatively and that all quantitative data are based on qualitative judgments; the major
categories of
Qualitative Data describes and discusses in-depth interviewing, direct observation, written
documents
Qualitative Approaches covers ethnography, phenomenolgy, field and grounded research theory
Qualitative Methods discusses participant observation, direct observation, unstructured
interviewing); and
Qualitative Validity discusses how to establish claims for the validity of generalizations based
on qualitative data
Key terms
● Case study ● Natural setting
● Casual observation ● Participant observation
● Controlled settings ● Record
● Field jottings ● Session
● Field jottings ● Simple observation
● Field notes ● Structured observation
● Field research ● Subjects
● Field work ● Triangulation
● Focus groups ● Unstructured observation
● Grounded theory
● Key informant
PREV NEXT

Chapter 13 Page 19
Main points
Introduction 1. Scientific observation is a planned, systematic process designed to produce an accurate
description of social interaction.
Casual 2. The distinctive characteristics of qualitative studies are: (1) intense focus on one social
observation setting; (2) an open, flexible approach to data collection; (3) continuous collection of data;
and (4) simultaneous analysis of data, concurrent with its collection.
Participant 3. Two widely used forms of scientific observation are casual or simple observation and
observation participant observation. Both are qualitative research techniques for observing everyday
social life.
Conducting a 4. In casual observation, the investigator observes activities without interacting intensely with
participant those being observed.
observation 5. In participant observation, the investigator observes a group from the inside by becoming a
study member of the group and experiencing life as a member of the group. Observation is
conducted by casual and more active forms of observation, by listening and asking
Establishing questions and by observing the everyday activities of members of the group. Observational
validity studies are also referred to as field studies or field research.
6. The purpose of a qualitative study is to provide a complete, accurate description of the
Generalizing dynamics of a group and to suggest an explanation of why the behavior being studied
from a occurs as it does.
qualitative 7. Participant observation generally involves three phases: (1) planning for field work; (2)
study conducting the field work; and (3) post field work tasks.
8. Planning for field work includes: (1) identifying the topic or focus of the investigation; (2)
Personal conducting a review of the literature on field research methods and becoming informed
qualifications about the group to be studied; (3) selecting the setting for the study (a village, office, or
other setting); (4) deciding on one's role in relation to the group; and (5) preparing for
Case studies living in the field during the period of observation.
9. Conducting a field investigation involves: (1) gaining entry into the group; (2) establishing
Focus groups rapport with leaders, key informants and others; (3) observing and recording observations:
(4) concurrently analyzing observations and developing tentative hypotheses and
Strengths and interpretations of emerging patterns; and (5) deciding when to stop and leave the group.
limitations of 10. In field research, investigators make mental notes of important events or jot down key
qualitative phrases and descriptions. As soon as possible thereafter, the mental notes or field jottings
research are expanded to full field notes containing detailed descriptions of what happened and what
was said.
Aids 11. Data analysis in qualitative studies consists of reviewing and analyzing field notes to
extract patterns of relationships among observations.
12. Analysis of qualitative data is partly an art and requires considerable skill.
13. Case studies are in-depth investigations of an individual, family, organization, village, or
some other single entity. Qualitative and quantitative methods are combined to produce a
comprehensive, integrated description of the social dynamics of the entity. The purpose of

Chapter 13 Page 19
many case studies is to explain why certain behaviors occur or why certain changes in
behavior have occurred.
14. Focus groups are a form of a group interview. An investigator brings five to ten persons
together and records their discussion of some issue or topic selected by the investigator.
Their comments are recorded and analyzed by methods of content analysis.
15. Qualitative research allows an investigator to produce deeper, richer descriptions of
behavior and interactions, describing these in all their complexity, and providing an
explanation grounded on data from the observations.
16. Limitations of qualitative methods of research include the considerable skill and
knowledge needed to carry out extensive field research and the question of the reliability
and validity of the observations, analysis, and conclusions drawn by the observer.
Investigator bias or systematic error is an ever present danger.


Chapter 14 Page 1
Introduction Chapter 14. Performing Evaluation Research

Introduction
Illustration of
evaluation In the previous chapters you learned how to use quantitative and qualitative research techniques.
research This chapter shows how these research techniques can be used to evaluate or assess programs
and services. To help you understand how evaluation studies are conducted, we begin this
Distinctive chapter with a description of an evaluation of a program carried out in Uganda (Kajimu, and
features of others, 1998). We next describe ways evaluation research differs from other social science
research. With this background, we present the steps involved in most evaluations. Even though
evaluation
ethical issues are discussed in Chapter 6, we return to this topic because evaluation research can
research raise special ethical and professional concerns. This chapter concludes with a review of problems
that can limit the value of an evaluation.
Conducting
an evaluation Illustration of evaluation research
Description of the program
Qualitative
evaluations In 1991, the Islamic Medical Association of Uganda (IMAU) conducted a survey to measure
knowledge about HIV infection and perceptions of the risks associated with getting the disease
Problems to (Kagimu and others, 1998). Based on the survey, the IMAU designed a two-year prevention
program. The program was based on the fact that the iman (mosque leader) is the teacher of
watch for
family values and behavior, including those related to risks associated with HIV. Therefore, the
program focused on training imans who would then conduct household visits with
Ethical families and provide information that could help prevent the spread of HIV. The objectives of
concerns the program, as operationalized in the household visits, were to: (1) increase knowledge about the
transmission of HIV; and (2) to change sexual behavior in ways to prevent its spread.
Aids
Twenty three (23) persons were trained to become the trainers of local teams of imans and their
assistants. The training sessions included 396 imans and 792 assistants. Each team then
conducted 1 or 2 visits per month with approximately 15 households each. Repeated visits
allowed the imans and their assistants to introduce new topics and to revisit topics discussed
previously. Over the course of the 2 years, 114,634 household visits were completed.
Evaluation of the household visits
At the end of the two years, a follow up survey was conducted using the same questions that were
asked in the original survey. This "before" and "after" quantitative measurement was
supplemented with qualitative data. Intensive interviews were conducted with 25 imans, their
assistants, and community leaders. In addition, 75 persons participated in 9 focus groups. Several
groups consisted only of adult males aged 25 to 40; others included only younger adult men, aged
20 to 29; and several were made up of females between the ages of 15 and 19. Separate groups
were organized by age and gender to encourage fuller and more honest discussion of the risks of
becoming HIV infected.
After reviewing both the survey and focus group results, the research team concluded that the
program had achieved its objectives. There was a substantial increase in correct knowledge about

Chapter 14 Page 1
HIV transmission and in methods of preventing HIV infection. In particular, participants

recognized that present practices in handling dead bodies and unsterile circumcision could cause
the spread of HIV. Evidence of changes in sexual behavior was found as well. There was a
significant decline in the number of sexual partners reported by respondents under 45 years of
age. Condom use increased among urban males, even though some religious leaders discouraged
their use. Qualitative data supported these findings and showed the importance of the repeated
household visits or imparting knowledge and encouraging changes in sexual behavior.
The Ugandan evaluation also illustrates the ways evaluation research differs from the usual
question-answering research.
PREV NEXT

Chapter 14 Page 2
Introduction Distinctive features of evaluation research

Evaluation research differs from question-answering research in five important ways:
Illustration of
● Having a practical purpose to provide practical information for measuring the effects of a
evaluation
program, policy, or some other activity;
research
● Conducted as sponsored research most evaluations are sponsored by an organization that
Distinctive wants the results for use in making a decision about a program or policy;
features of ● Having the potential for conflict evaluation researchers are frequently caught in conflict
evaluation among parties who have different views on the value or success of the program or service
being evaluated;
research
● Being a team effort evaluation research is often based on a team effort as opposed to
Conducting research carried out by a single investigator; and

an evaluation ● Producing less precise results results of evaluation research are often less precise than
expected from other social research.
Qualitative Done for a practical purpose
evaluations
The Ugandan study also illustrates how evaluation research differs from other research. Sponsors
Problems to of the program wanted evaluation data for assessing its impact on knowledge about HIV. In
response, the research team designed an evaluation to produce the information on the outcomes
watch for
of the program: Did it achieve the objectives set for it? Outcomes were measured by the extent
the program increased knowledge about the transmission of HIV and led to changes in sexual
Ethical behavior that would help slow the spread of the disease.
concerns
Measuring the outcome of a program is referred to as summative evaluation. This kind of
Aids evaluation is conducted when a program is considered mature or is completed. In the Ugandan
example, the program was planned for two years. At the end of this period, a summative
evaluation was conducted. Programs can also be evaluated while they are being developed. Using
the Ugandan example, again, the investigators could have conducted interviews at certain points
in the development of the program and then used the results to make improvements in the
program. This kind of evaluation is called formative evaluation.
Both kinds of evaluation can be applied to the same program. Program managers may arrange for
continuous monitoring of a program to provide feedback for modifying the program as it is being
developed. At the end of the program, as in the Ugandan case, the sponsoring organization may
conduct an evaluation to get information of the outcomes of the program. Formative evaluations
generally are designed to provide feedback to staff and program managers, while summative
evaluations are designed to give decision-makers information for use in deciding whether a
program will be continued, modified, or terminated.
Preformed as sponsored research
Individual researchers on their own may conduct an evaluation of some program Most
evaluations, however, are requested and paid for by an organization responsible for a given
program. In preparing for budget decisions, government ministries may request evaluations of

Chapter 14 Page 2
health, education, economic development, or other publicly-supported programs.

Nongovernmental organizations frequently hire evaluators to assess the effects of programs the
organizations support.
The Ugandan investigation described earlier illustrates the role of sponsoring organizations in
evaluation of programs. Three organizations were involved in the evaluation. The program was
developed and conducted by the Islamic Medical Association of Uganda, a non-governmental
organization, in cooperation with an international nongovernmental organization, World
Learning. The United States Agency for International Development, a governmental
organization, provided the funds for the program and its evaluation. As part of its conditions for
supporting the program, USAID, as the sponsor, required an evaluation.
PREV NEXT

Chapter 14 Page 3
Introduction Potential for conflict

Evaluation research is frequently carried out in the context of conflicting interests and opinions.
Illustration of
Often evaluation research is undertaken because there is disagreement about what should be done
evaluation regarding some program. Suppose some officials at the ministry responsible for a new rural
research extension effort think the program is providing effective services to rural communities while
other officials argue that it is a failure. Higher level management might call for an evaluation
Distinctive study to settle the dispute over the success or failure of the program. The sponsoring
features of organization, however, would not be the only group with a strong interest in the outcome of the
evaluation evaluation. The staff involved, including managers and field extension agents, naturally would
research have a strong interest in the future of "their" program. So would the farmers and others living in
the rural areas served by the program.
Conducting Individuals or groups with personal or vested interests in the outcomes of an evaluation study are
an evaluation known as stakeholders. Gaining the trust and cooperation of stakeholders is essential to the
success of most evaluations. Yet, conflicts among stakeholders, reflecting their different views
Qualitative about the how well a program is achieving its goals, can make gaining their trust difficult. A
evaluations powerful way of gaining their trust and cooperation is to consider their interests in designing and
carrying out an evaluation. This calls for skill and patience in working out conflicts in
Problems to developing evaluation designs and in collecting, analyzing and interpreting data. We describe
watch for some ways of doing this later.
Based on team efforts
Ethical
concerns Many evaluations are conducted by teams of researchers. The Ugandan investigation illustrates
this point well. The report of this study lists six authors (see the List of References, look for
Kajimu and the additional authors). Each of the persons listed as an author of the report assisted
Aids
in developing the design for the study, supervised collection of the data, or assisted with the
analysis of the data and the interpretation of the results. Some were university professors, some
were members of the Islamic Medical Association of Uganda, and some were from the
sponsoring organizations that provided the oversight and funding of the program.
Like the Ugandan evaluation, many evaluation studies involve more than one evaluator.
Sometimes several organizations have an interest in how the evaluation is designed and
conducted. Therefore, they want their "own" person involved. Another reason is that different
evaluation skills are frequently needed. Many times evaluations are based on more than one
method of data collection. In the Ugandan investigation, for example, methods included use of
surveys, interviews, and focus groups. Use of different methods frequently requires involvement
of different experts who work together as a team.
Less precise results
Finally, the results of evaluation studies are sometimes less precise than those from other social
research projects. Most social research is designed to answer questions posed by the investigator
who is in control of all aspects of the investigation. The investigator is free to decide what to
study and how to design and conduct the investigation. Evaluation studies generally are done

Chapter 14 Page 3
under different circumstances. Frequently, evaluation researchers face limits in what can be done.
For one thing, evaluators often have less freedom in selecting variables. Ways of measuring
variables may also be limited by how the program is being administered. Frequently the program
is already in operation, and may have been for years before an official decides that an evaluation
is needed. Too often, specific objectives or goals, which could be used to derive outcome
measures, are not specified when a program is started.
Faced with these conditions, an evaluator has to use whatever data are available for creating
measures for how well the program has performed. Many times these data are very imprecise
measures of outcomes. In addition, conflicts among stakeholders can complicate the process of
data collection. All of these conditions can combine to make measurement less precise than an
evaluator would like. As a consequence, results are often less precise as well.
The Ugandan evaluation illustrates the problem of imprecise results. The authors acknowledge
the limits of information collected in their interviews. For instance, many male respondents were
reluctant to admit using condoms because, if they did, they might be accused of being
promiscuous. (Kajimu and others, 1998, p. 227).Yet, use of condoms was taken as one of the
measures of changed sexual behavior, despite its weakness. This is often the case in evaluation
research.
PREV NEXT

Chapter 14 Page 4
Introduction Conducting an evaluation

As shown in Figure 14.1, conducting an evaluation follows the phases of other social science
Illustration of
research, but with several important variations. The main steps in conducting an evaluation are:
evaluation
● Establishing a clear agreement on the scope of the evaluation and the responsibilities of the
research
evaluator;
● Gaining the trust and cooperation of all stakeholders;
Distinctive
features of ● Reviewing the literature on similar kinds of evaluations;
evaluation ● Becoming informed about the program to be evaluated;
research ● Designing the evaluation, with input from stakeholders;
● Conducting the evaluation, with frequent feedback and periodical reports to stakeholders;
Conducting ● Preparing a draft report describing how the evaluation was done and what was found;
an evaluation
● Review of the draft report with stakeholders, followed by revision, as appropriate; and
● Submission of the final report of the evaluation.

Qualitative
evaluations
Problems to
watch for
Ethical
concerns
Aids
Figure 14.1. Model for conducting an evaluation

PREV NEXT

Chapter 14 Page 4

Chapter 14 Page 5
Introduction This model of the evaluation process is based on open, democratic relations among stakeholders,
participation of stakeholders in the design of the evaluation, open communication between the
Illustration of evaluator and stakeholders, and resolution of any conflict based on the empirical results of the
evaluation evaluation. These are the ideal conditions for conducting an evaluation. When they exist, the
research evaluator's work is much easier. But these ideal conditions do not always exist. A sponsor or
stakeholder can try to impose conditions or limit data collection in ways that will bias the results.
This can start with the terms and conditions of the agreement with the evaluator and continue
Distinctive
throughout the evaluation. To minimize potential problems later, the first step in thinking about
features of an evaluation sponsored by some organization is to reach a clear agreement with the sponsor and
evaluation stakeholders.
research
Establishing an agreement
Conducting All evaluations should be based on some form of agreement between the evaluator and the
an evaluation sponsor of the evaluation. When the evaluator is hired to do the evaluation, an agreement or
contract is usually prepared as the basis for paying the evaluator. But, even when an evaluator is
Qualitative not paid, the evaluator should know what is expected of him or her and what freedom he or she
evaluations has in conducting the evaluation. An agreement should set forth as precisely as possible the
scope of work, what is to be evaluated, what the sponsor will provide or how the sponsor will
Problems to assist in the evaluation, how long the evaluation will last, guarantees for access to data, freedom
watch for to use all appropriate methods of data collection and analysis, and other conditions that may
affect the outcomes of the evaluation. A clear agreement protects both the evaluator and the
sponsor. Both know what their responsibilities and roles are. An agreement will also help prevent
Ethical
disagreements that might arise later over who was responsible for what.
concerns
When the evaluation is sponsored by some organization, the organization generally prepares an
Aids initial agreement or contract setting forth what the organization wants done. Occasionally, and
sometimes innocently, a sponsor may propose an agreement that an evaluator may find
unacceptable. The sponsor may put limits on the kind or amount of data that can be collected or
suggest methods that the evaluator thinks are not the best ones to use. An evaluator, therefore,
needs to examine any agreement carefully before committing to the project. If conditions seem
too limiting or could bias the results, the evaluator will need to seek an agreement that will allow
objective evaluation. Once an agreement is reached, an evaluator's next task is to gain the trust
and cooperation of all stakeholders.
Gaining cooperation
From the very start of an evaluation project, experienced evaluators attempt to gain the
cooperation of key stakeholders. The objective is to give the stakeholders a sense of ownership
of the study. Ownership means they see the study as partly theirs. Instead of seeing the
evaluation as being done for someone outside their group, they see themselves as participating in
a joint effort with the evaluator and the sponsor of the evaluation. As a consequence, the
evaluator is more likely to get their cooperation and help which can make the difference between
a successful and unsuccessful study.

Chapter 14 Page 5
Gaining cooperation of stakeholders is not always easy, and there is no assured way of doing this.
An evaluator has to be prepared to spend a lot of time and effort in drawing stakeholders into the
evaluation. In many evaluations, evaluators have to overcome initial distrust or suspicious of
stakeholders. Many times, an evaluation threatens some stakeholders, who fear that a program
may be changed in ways they don't want, or, worse, be terminated. In this atmosphere, the
evaluator can be seen as representing the interests of the sponsor and as being against those of
stakeholders.
The evaluator, therefore, needs to show from the outset that the interests of all stakeholders will
be respected and that stakeholders will be able to participate in all phases of the evaluation.
Generally, this is done through a series of face-to-face meetings with groups of stakeholders and
later in meetings with individual stakeholders. In an evaluation of a community development
program, meetings might be held separately with government officials, local officials, village
leaders, managers and staff of the program, and with persons in the community who the program
is intended to benefit.
Meetings can range from large group meetings to meetings with small groups of stakeholders and
with individual, influential stakeholders. Large meetings are good for briefing stakeholders on the
purpose of the evaluation, for acquainting stakeholders with the evaluator, and, later, for
providing information about the findings of the evaluation. Meetings with small groups of
stakeholders allow the stakeholders to get to know the evaluator personally, help build the
essential bonds of trust between the stakeholders and the evaluator, and give stakeholders an
opportunity to express views and contribute to the development of the evaluation.
PREV NEXT

Chapter 14 Page 6
Introduction The evaluator's attitude and behavior toward stakeholders greatly affects how cooperative and
helpful they will be. Following are some guidelines that generally help in gaining the
Illustration of cooperation of stakeholders.
evaluation Treat stakeholders as partners. In conducting an evaluation, encourage the participation of
research stakeholders. Seek them out and ask for their ideas and suggestions. Keep them informed about
major decisions affecting the course of the study, such as formulation of the objectives of the
Distinctive study, selection of indicators for measuring the success of the program, and in the development of
features of measurements for the selected indicators.
evaluation
Spend time discussing developments with stakeholders. An evaluator must remember that
research evaluation studies, particularly summative ones, can arouse a lot of anxiety among stakeholders.
When people become anxious they also frequently become suspicious and disagreeable.
Conducting Evaluators have to be prepared to face and resolves these negative attitudes. Giving stakeholders
an evaluation an opportunity to say what is bothering them and engaging in honest give and take with them will
usually resolve negative feelings of stakeholders. Evaluators therefore allow plenty of time and
Qualitative demonstrate great patience in working through this process.
evaluations
Keep communication channels open. Smart evaluators seek suggestions from stakeholders and
provide periodic reports on how the evaluation is progressing. This may be accomplished
Problems to informally through conversations with stakeholders or may be done formally through the
watch for circulation of written progress reports. Either way the objective is to keep stakeholders informed
and to give them opportunities to participate in the evaluation.
Ethical
concerns Seek comments at critical stages of the study. Every evaluation project goes through certain
stages from the setting of the objectives for the evaluation to the preparation of the report for the
study. In conducting an evaluation study it is often useful to present a progress report to
Aids
stakeholders at the end of each critical phase in the project and to seek their comments.
Comments frequently can be used to improve the study. In preparing these reports, the evaluator
can show how previous suggestions from stakeholders were used or, if they were not used, can
give reasons why they were not. These interim reports also provide a history of the project and
can be easily modified to become sections of the final report of the study.
Reviewing the literature
In evaluation studies, two kinds of literature searches are generally carried out. One is based on a
search of the professional literature on evaluation methods, particularly as applied to programs or
services similar to the one to be evaluated. The evaluator looks for the kinds of research designs
and methods used, the kinds of variables and indicators used to assess the performance of
programs, and ways the data were analyzed. If one were evaluating the outcomes of a program
for teaching about the transmission of HIV, the Ugandan investigation would provide useful
information. For the evaluation of other kinds of programs, other evaluation reports would be
sought.
The second kind of review is done to understand and describe the program being evaluated as
thoroughly as possible.

Chapter 14 Page 6
Understanding and describing the program

To evaluate a program properly, the evaluator must learn as much as possible about the program.
When was it started? What objectives were established for the program? What has been its
history, in terms of management and staffing, funding, sponsorship, and other conditions that will
help the evaluator understand the operation of the program? What kinds of data are available in
the records of the program?
A good way to start is to read and analyze everything that has been written about the program
being evaluated. Look for copies of any legislation, decrees, or other documents that authorized
the program. Also examine program plans, budget documents, and administrative reports related
to the program. This review should help in identifying the objectives of the program, which will
aid in selecting the outcome variables to be measured. Ways of measuring these variables often
are suggested in the literature about the program.
PREV NEXT

Chapter 14 Page 7
Introduction Sources of information include program documents containing information about the
establishment and history of the program. These may be found in the offices of the program staff
Illustration of or in the files of the organization that is responsible for the program. Other documents may
evaluation provide information on the performance of the program. For an educational program, operational
research information may include data on the numbers of students enrolled in the program, the numbers
and qualifications of the teachers and other staff responsible for the program, attendance records
and grades of students, and previous evaluation or other reports by teachers and principals. In
Distinctive
addition, teachers and parents might provide important information. Having a thorough
features of understanding of the program is essential to designing a successful evaluation study.
evaluation
research Sometimes written descriptions are vague or sketchy. In these circumstances, the evaluator has
to use what can be learned from interviews with administrators and staff. When this is necessary
Conducting we recommend interviewing a variety of stakeholders to create the most accurate and complete
description possible.
an evaluation
In addition, it is always useful to observe what actually is happening on a daily basis in carrying
Qualitative out the program. As way of illustration, suppose you agreed to evaluate a change in teaching
evaluations methods, from lecture only, to the addition of discussion groups for teaching a certain course in
local secondary schools. Before starting the evaluation, you would need to understand what the
Problems to change was and how extensively and how well it had been implemented. From the written
watch for records and interviews you might think that all teachers had changed their method of teaching
and that all were effective leaders in the discussion groups. But suppose observation showed
otherwise: That, in fact, most teachers had continued to lecture and were not involving students in
Ethical
discussion. Under these circumstances the evaluation would be useless. It would be obvious that
concerns the so-called new method really was a continuation of the previous lecture method.
Aids This is probably an extreme example. For most new programs, at least some elements are
implemented, but sometimes the new practice falls below what is described in official documents.
Only first hand observation will tell you how the new approach is actually being implemented.
The evaluation study should be designed to evaluate the impact of the program as it is carried out
on a day-to-day basis rather than as it is described in official documents.
Also, the actual content of the program, not the intended content, should be fully and accurately
described in the evaluation report. This description tells readers what was evaluated and is needed
by them for assessing the results of the study.
The important step of understanding the program you are about to evaluate is explained in greater
detail in Evaluability Assessment. As explained on this site, the purpose of evaluability
assessment is to gain information from important documents and stakeholders about the
objectives and the content of the program to be evaluated. Viewing this site will help you
appreciate the importance of this step and learn how to plan and conduct an evaluability
assessment.
Designing an evaluation study
Well-developed designs are needed for all studies, but they are particularly important for

Chapter 14 Page 7
evaluation studies. Because of the impact of evaluation studies, their designs are often examined
in greater detail and more critically than those of other studies. This almost always happens when
the results do not agree with the expectations of the sponsor or powerful stakeholders.
Disappointed stakeholders may try to find weaknesses in how the study was carried out as a way
of discrediting the results. The design of an evaluation study, therefore, should be as "foolproof"
as possible. In so far as possible, it should meet all the criteria for the collection of valid, reliable,
and accurate data. Development of a design begins with establishing the objectives or goals of the
program being evaluated.
Establishing objectives of the program. To evaluate a program, the evaluator needs to know
what the program was expected to achieve? Most programs or services are based on a plan or
program statement of some kind. If done well, these documents will contain a set of objectives or
goals for the program. When these are stated clearly in program documents, the evaluator's task is
a lot easier. Sometimes, however, programs are started or continued without any specifically
stated objectives. Then, the evaluator has to infer these from language in the documents or derive
objectives from information supplied by the program staff or other stakeholders.
PREV NEXT

Chapter 14 Page 8
Introduction In most evaluation studies some consultation is necessary before the objective for the study can
be set. When the sponsor of the study sets the objectives of the evaluation, careful evaluators
Illustration of analyze this statement to make sure it is the one that should be pursued. Sponsors sometimes have
evaluation preconceived ideas about how successful or unsuccessful the program being evaluated has been.
research Their views can color how they present the objectives of the study to an evaluator. Careful
evaluators, therefore, examine the objectives set for the study by the sponsor and other
stakeholders to see if they should be used in the evaluation. In making this assessment, it is a
Distinctive
good idea to seek comments by other stakeholders and experts in the area being evaluated.
features of
evaluation The agreed-upon objectives of the program become the criteria for evaluating the program. Once
research the objectives are identified, indicators are developed for measuring achievement of the
objectives. Having clear, measurable objectives, therefore, is crucial to the design and ultimately
Conducting to the success of an evaluation. How do you know whether you have a clear set of objectives?
The answer is whether the objective can be measured. Measurable objectives are stated clearly in
an evaluation
specific, concrete terms. For example, the following objective for an adult literacy program meets
these criteria:
Qualitative
evaluations Seventy-five percent (75%) of the students who complete the course will be able to
accurately read and describe the contents of a local newspaper.
Problems to This objective is clear. There can be no argument over what it means. Success for the course is
watch for defined in terms of reading and describing the contents of a local newspaper. Further, successful
is measured in terms of a percentage, 75%, of the students not "some" or even "most reaching
Ethical this level of achievement.
concerns
Following is not an acceptable objective for the same course:
Aids "Seventy-five percent (75%) of the students will learn to read."
This objective properly specifies a percentage for passing as the objective for the course, but it
doesn't define what being able to read is. Is it reading at a child's level or at a functional adult
level, such as represented by reading and being able to describe what one has read in a local
newspaper? Or at some more advanced level? And the objective simply refers to students. Are
students who dropped out to be included in the evaluation or only those that complete the course?
We don't know. This objective would be useless for designing an evaluation of the adult reading
course. It could be converted to an acceptable objective by getting stakeholders to agree on what
learning to read means, how reading would be measured, and what percentage of students will be
expected to meet the criteria set for the course.
Selecting indicators for measuring objectives. After the objectives are agreed upon, the
evaluator can select indicators for measuring the extent to which the program or whatever is
being evaluated has met its objectives. Selection of indicators is crucial to the success of any
evaluation. The indicators, like all measurements, should be valid, reliable, and accurate
measures of the variables they represent. Indicators become the measures of the expected
outcomes of the program. In research terms, the program or service being evaluated is the
independent variable. As the independent variable, the program is expected to change or

Chapter 14 Page 8
influence some other variable or set of variables.

In the Ugandan program for reducing the spread of HIV, the program consisted of the repeated
household visits extending over two years. These visits and discussions of HIV with imans and
their assistants were intended to increase knowledge of the transmission of HIV and effect
changes in the sexual behavior of Ugandans. A number of indicators were used to measure the
outcomes of the visits. These included levels of knowledge of HIV transmission, which
increased; changes in number of sexual partners, which declined; and the frequency of condom
use, which increased. These results were interpreted as meaning that the program had achieved its
intended impact.
Many times outcome variables are easy to identify and measure. The outcome indicator for the
adult literacy program, for example, is fairly simple: Can an agreed-upon percentage of those
who finish the program read at some defined, minimal level? Sometimes indicators for outcomes
are more difficult to identify and measure. Suppose you were asked to evaluate the effectiveness
of training programs for women. Should the outcome indicator be whether the women gained an
acceptable level of whatever skill they were expected to learn, such as weaving, sewing, or
making handicrafts, or should the outcome indicator be based on what the women did with their
training, such as:
PREV NEXT

Chapter 14 Page 9
Introduction ● Did they get jobs because of their training?

● Did they establish their own businesses?
Illustration of ● How much money have they earned by using their new skills?
evaluation ● Did the training allow them to make things for use by their families and thereby save
research money they would have spent buying the same things?
● And if monetary criteria are to be used, how long after the training should the data be
Distinctive collected?
features of ● Should immediate earnings be used as the indicator of the value of the course or should
evaluation later earnings be used?
research ● If increased earnings were to be the outcome indicator, should data be collected a year or
so later to allow enough time for the women to apply their training?
Conducting
an evaluation These few questions show that selecting outcome variables is not always easy.
Selecting methods of data collection. A decision also has to be made regarding the method or
Qualitative methods of data collection. For some studies existing data are sufficient. Suppose, for example,
evaluations you were given the assignment of evaluating the effectiveness of a new policy for reducing
absenteeism among workers in a government office.The obvious outcome measure would be the
Problems to number of days workers were absent without permission for some period of time before and after
watch for the new policy was implemented. Data for this variable could be obtained from the official
attendance records maintained by the government office.
Ethical Observation and surveys are used in many evaluation studies.The Ugandan study, for example,
concerns used "before" and "after" surveys to collect data on changes in knowledge about the transmission
of HIV and for reports on changes in sexual behavior.
Aids
Experimental or quasi-experimental designs are often used in assessing the effectiveness of
success of programs. Grotberg and Badri (1986), for example, used a quasi-experimental design
in evaluating the effects of a home visit program for teaching mothers how to provide extra
stimulation for development of their children. The research literature contains many examples of
evaluations based on experiments. One study analyzed 16 true experiments designed to evaluate
family planning programs (Bauman, 1997).
We found two Web sites that provide useful discussions of additional research techniques for use
in evaluations:
Using Focus Groups in Evaluation describes what focus groups can tell you, examines the
advantages of focus groups and how to conduct them, and describes variations in using focus
groups
Using Existing Records in Evaluation describes what existing records are, what they can tell
you, the advantages and disadvantages of this research technique, ways of dealing with problems
connects to evaluating existing records, and how to find and evaluate existing records in
conducting an evaluation study

Chapter 14 Page 9
Measurement has to be taken very seriously in planning an evaluation study. Operational

definitions for outcome variables have to be able to withstand the most severe criticisms by
persons who will not like the conclusions of a study. Partly for this reason, but also to meet
scientific standards, it is wise to use tested, easy-to-understand measures whose validity and
reliability cannot be questioned. Also, when possible, multiple measures should be used to
overcome the limitations in any one measure. Going back to the Ugandan evaluation again, the
research team used both quantitative methods, based on surveys, and qualitative methods, based
on extensive interviews with key informants and analysis of the discussions of focus groups.
Using the illustration of the evaluation of training programs for women again, several outcome
criteria and measures might be used. These could include grades or ratings assigned by teachers;
observation and assessment of how well each women can do what she was trained to do; how the
women regarded the training they received; how many women found work as a result of their
training; how much those who were employed were earning; or whether the women saved family
income because of things they produced for household consumption, and, if so, how much had
they had saved.
PREV NEXT

Chapter 14 Page 10
Introduction Involving stakeholders. Previously we stressed the importance of involving stakeholders in the
design and conduct of an evaluation. This applies to the selection and definition of indicators for
Illustration of measuring achievement of objectives. Involving stakeholders in these decisions will increase
evaluation their willingness to help in the evaluation. Their suggestions can lead to identification of
research indicators that might not otherwise occur to an evaluator. Stakeholders can also suggest sources
of data for measuring indicators. On the other hand, when not involved in a meaningful way,
stakeholders can seriously disrupt an evaluation. If suspicious or distrustful of the evaluator or his
Distinctive
or her approach to an evaluation, staff of a program can withhold data or supply incorrect
features of information. Members of a community, who feel threatened by an evaluation, can enlist the help
evaluation of local leaders in stopping an evaluation or making it meaningless. Careful, patient interaction
research with stakeholders can avoid these problems.
Conducting Collecting data

an evaluation How the data are collected depends on the kind of evaluation being done and the design being
used. In formative evaluations, data may be collected just once or at periodical intervals. In
Qualitative summative evaluations, data are collected after the program is fully developed or at the end of a
evaluations planned period of operation, as was done in the Ugandan evaluation. Any method or
combination of methods can be used with either form of evaluation. Quantitative methods, such
Problems to as using available data or conducing surveys, provide precise data for measuring outcomes. These
watch for methods are useful when the objectives are expressed in quantitative terms, like 75% of the
graduates of a program will learn to read at a certain level. But quantitative methods are not
appropriate for all evaluations. Sometimes, qualitative methods are more appropriate. These are
Ethical
discussed later in this chapter.
concerns
Analyzing data and interpreting results
Aids
In Figure 14.1, we show analysis and interpretation as a cooperative activity with the sponsor
and other stakeholders. The evaluator is responsible for analyzing the data and for interpreting the
results of the evaluation. This is done in consultation with representatives of the sponsor and
stakeholder groups. A first draft is prepared and submitted to the sponsor and stakeholders for
review and comments. Feedback from them is critically important for developing a sound
evaluation. Stakeholders can point out and offer corrections for errors and suggest additional or
alternative interpretations of results. Their contributions can greatly improve the final report of
the evaluation. Feedback from the sponsor and stakeholders is used to prepare the final report,
which is then submitted to the sponsor. Unless the sponsor prohibits further distribution of the
report, which sometimes happens, copies are distributed to stakeholder groups.
When presenting results of an evaluation, we suggest you keep two considerations in mind. First,
evaluation studies are generally requested by administrators or other officials. Most of these
persons do not have training in research methods and are not interested in the details of analyses.
Second, the conclusions have to be based on data they can understand and trust. To meet the
needs of administrators, we offer two suggestions.
Use simple forms of analysis. Focus on the main outcome variables. Present results as simply

Chapter 14 Page 10
as possible. Generally percentages and averages are sufficient. Do not clutter up the report with
nonessential analyses that interest you personally. Save these for other reports you may write for
other readers.
State the conclusions clearly and simply and give your reasons for them. Stating the
conclusions is easy when the results clearly point to one conclusion. One clear conclusion might
be that a program is highly successful, fully meeting or exceeding the objectives set for it.
Another might be that the program is a dismal failure, falling fall short of its objectives.
Difficulties arise when the results are inconsistent or not clear. Sometimes the data suggest that a
program is achieving certain objectives but not others. In such cases many evaluators do not offer
an overall conclusion, but instead indicate the areas in which the program is achieving the goals
set for it and the areas in which it is not. Readers can then form their own conclusions about the
overall success of the program.
PREV NEXT

Chapter 14 Page 11
Introduction Writing an evaluation report

Following are some suggestions that may help you prepare a useful evaluation report. First,
Illustration of
organize the report into sections or chapters, each of which should present information about a
evaluation particular phase of the project. Many reports contain the following main sections:
research
1. An opening statement that describes the reasons for the evaluation;
Distinctive 2. Presentation of the objectives of the study;
features of 3. Description of the program being evaluated;
evaluation 4. Description of the design of the investigation;
research 5. Presentation of the results of the evaluation; and
6. Presentation of the conclusions and recommendations.
Conducting
an evaluation Dividing the main sections into subsections helps the reader move from one point to the next in
the report. When used, subsections should be identified by headings.
Qualitative Second, write simply and briefly. Keep sentences short. Avoid social science terminology. Use
evaluations everyday language to the extent possible. Anther way to simplify reporting is to use tables or
figures to show the key results. Well designed figures often help officials grasp the meaning of
Problems to the results. Use of figures can also reduce the length of a report, which, in itself, encourages
watch for persons to read the report. Chapter 17 shows how to develop tables and figures.
Third, leave methodological and design details out of the report. Most readers will not want to
Ethical know all these details; putting them in can discourage some readers from reading the report. If
concerns you believe it is necessary to describe the details related to measurement, sampling or data
collection, for example, put them in appendices at the end of the report. However, we want to
Aids warn you against making a report appear too long. Short is better. If the appendices make the
report look much longer than it really is, we suggest you drop them.
Fourth, after writing the report, it is a good idea to prepare a brief executive summary of the
report. Some summaries are as short as one page, rarely being more than five pages. The purpose
of the executive summary is to give readers the highlights of the study. A summary usually
includes a statement about why the study was undertaken, a statement of the objectives of the
study, a very brief description of the design of the study with a similarly brief description of how,
when and from whom data were obtained, the essential results of the study, the conclusions, and
perhaps some recommendations. Each statement or description is presented in a sentence or two
or at the most in a brief paragraph.
Executive summaries can be presented in one of two ways. The summary can be included as the
first section of the report or it can be prepared as a separate document with a title sheet followed
by the summary. The second option is recommended when the report is long. Persons are less
likely to begin reading longer reports and therefore might miss even the short summary at the
beginning of the report. They are more likely to read a separate short summary and learn what
was found. Then, if they want more details they can examine the main report.

Chapter 14 Page 11
One final suggestion, generally it is also useful to allow stakeholders to review the report in draft
form. This can be done in a meeting with stakeholders in which the evaluator verbally presents
the result and conclusions or by circulating copies of the draft report for comment. Comments by
stakeholders often suggest ways in which the analyses can be improved and lead to more firmly
established conclusions and recommendations.
Qualitative evaluations
Qualitative methods of evaluation include use of unstructured interviews, casual and participant
observation, and use of focus groups. These techniques are particularly valuable for:
● Evaluating programs that are still evolving;
● Understanding the meaning and value of programs from the views of stakeholders and
participants; and
● Getting at complex sets of variables.
PREV NEXT

Chapter 14 Page 12
Introduction Many educational and community programs are good candidates for evaluation with qualitative
techniques. In a child development evaluation, ratings by observers were used to measure the
Illustration of quality of and intensity of interactions of mothers with their young children (Grotberg and Badri,
evaluation 1986, p. 14). The variables included ratings of the mothers' skills in stimulating their children,
research their patience with their children, and ways mothers interacted with their children. These are hard
variables to measure. A survey approach, using a questionnaire, would not have produced the
depth and variety of data obtained from first-hand observation of the mothers with their children.
Distinctive
Also, data were accumulated over a number of visits. This allowed the observers to check the
features of accuracy of previous observations.
evaluation
research Well-documented and presented qualitative data have a compelling, built-in validity, and can be
very persuasive. A survey finding that most workers in a factory are dissatisfied with working
Conducting conditions, for example, could be dismissed by managers as complaints by lazy workers. But,
suppose that in addition to the survey results, the evaluator obtained extensive, unstructured
an evaluation
responses from the workers showing their dedication to their jobs and dissatisfaction with
specific working conditions; then the managers might be more willing to respond to the
Qualitative evaluation results.
evaluations
Frequently, qualitative evaluations of community activities are based on extensive interactions
Problems to between the evaluator and stakeholders. The evaluator acts more like a consultant to stakeholder
watch for groups, serving as a facilitator who helps the stakeholders work through the evaluation process.
Consultations are carried out in group meetings and in one-on-one meetings with individual
stakeholders. The questions to be answered by the evaluation are established in consultation with
Ethical
stakeholders. Stakeholders help select the indicators and ways of measuring them. The evaluation
concerns design is the result of extensive two-way communication between the evaluator and stakeholders.
Regardless of the extent of communication and planning, qualitative designs are frequently loose
Aids and flexible, sometimes simply based only on agreement for extensive interviews and periods of
observation of community events.
Data collection is done informally, similar to the processes described for participant observation.
The evaluator may observe and record how well he or she thinks some community program is
functioning, based on criteria developed cooperatively with stakeholders. Additional data may be
extracted from records of group meetings and focus groups. Notes on meetings with individuals
or small groups of stakeholders may provide additional data.
As in participant observation, the evaluator is an outsider to the group and its activities, but is
accorded a special status by the group. This status allows the evaluator to participant in the
activities of the community, observe and record information, ask questions, and to build up a
body of information on the activity being evaluated. Insights and assessments of the observer are
shared on a regular basis with stakeholders, who then provide feedback to the evaluator. Through
this process, the evaluator comes to understand the stakeholders' views and is able to test the
validity of his or her emerging conclusions.
Qualitative evaluation also makes great demands on an evaluator. The evaluator is constantly
aware that the conclusions being developed depend on his or her skill and judgment. There is the

Chapter 14 Page 12
ever present danger of faulty observation and analysis. Intensive consultations with stakeholders
protect against errors in observation and faulty analysis. Consultations, however, require a large
amount of time, and can create additional stress. Differences may arise between the evaluator and
stakeholders over means of data collection, the meaning of observations, and over the
interpretation of findings. Stakeholders may fail to realize the importance of proper methods of
collecting data and push for means the evaluator knows are open to bias. Should the evaluator
insist on the highest standards for data collection or should something closer to what stakeholders
think is sufficient be used? How far should an evaluator go in accepting alternative
interpretations by stakeholders? Differences may also arise among sets of stakeholders, putting
additional pressure on the evaluator.
There is no single best way to resolve differences with stakeholders. Evaluators have to balance
their commitment to adhering to the rules of data collection and analysis against the possibility
that observations or interpretations may be biased. The best course of action is to seek
confirmation of observations and other data by drawing on additional sources of information and
to be able to back up every conclusion with solid data. All the ways to triangulate should be
explored and used.
Qualitative community or organizational evaluations also provide an opportunity for building
skills and resources in the group participating in an evaluation. The process of consultation with
stakeholders can be used to build group cohesion, teach problem solving skills, introduce
elements of research methodology, and to demonstrate the analysis of observations and other
data. Communities and organizations can be left far richer and more resilient as a result of
participating in a consultative evaluation. This point is developed more fully as part of the
discussion of Rapid Rural Assessmen t in Chapter 15.
We found one Web site dealing specifically with qualitative evaluation. This site, The Use of
Qualitative Interviews in Evaluation, describes various ways qualitative interviewing is used,
the advantages and disadvantages of qualitative interviewing, how to conduct qualitative
interviews, and how to handle ethical issues that may arise.
PREV NEXT

Chapter 14 Page 13
Introduction Problems to watch for

Some problems that frequently arise in evaluation studies are summarized in Box 14.1 as a
Illustration of
reminder of things to avoid when designing and conducting an evaluation.
evaluation
research Box 14.1. Problems to watch for
1. Starting an evaluation without clear agreement with the sponsor on
Distinctive terms and conditions
features of 2. Lack of clear, measurable objectives to serve as criteria for the
evaluation outcomes of the program
research 3. Failure to fully understand the program, its history, operations, and
other factors affecting its outcomes
Conducting 4. Selection of weak or inappropriate indicators for measuring outcome
an evaluation variables
5. Use of inappropriate data for measuring outcome variables
Qualitative
6. Failure to consult with and gain the trust and cooperation of
evaluations stakeholders
7. Failure to obtain review of a draft report by the sponsor and
Problems to
stakeholders
watch for
8. Preparation of a poorly organized and written final report
Ethical
Ethical concerns
concerns
Evaluation research raises special concerns for the protection of persons, groups, and
Aids organizations that are involved in or affected by an evaluation. The American Evaluation
Association (AEA) has developed extensive ethical principles for protecting the rights of clients
and for ensuring that evaluators adhere to the highest levels of professional competence and
ethical standards. These principles are listed at Guiding Principles for Evaluators.
Box 14.2 lists some of the key provisions, taken from the AEA set of principles, that you should
follow in any evaluation you do.

Chapter 14 Page 13
Box 14.2. Guiding principles for evaluator researchers (from the

American Evaluation Association)
1. Evaluations should be based on systematic, data-based inquiries about
whatever is being evaluated.
2. Evaluators should provide the highest level of competent performance
to stakeholders.
3. Evaluators should ensure honesty and integrity in the entire evaluation
process.
4. Evaluators should respect the security, dignity and self worth of the
respondents, program participants, clients, and other stakeholders with
whom they interact.
5. Evaluators should recognize and take into account the diversity of
interests and values that may be related to the general and public
welfare.
PREV NEXT

Chapter 14 Page 14
Introduction Aids
Internet resources
Illustration of
evaluation In addition to the more specific sites we mentioned earlier, a number of Web sites provide
research valuable information on a broad range of issues related to evaluation research. For additional
information and guidance, we recommend the following sites:
Distinctive Free Evaluation Resources for Developing Countries
features of http://earth.prohosting.com/elecon/evaldevel/evaldevelopment.html provides resources for
evaluation conducting evaluation research, with particular reference to applications in developing countries;
research describes the best use of the site; goals, plans and programs; list of classes; project email list;
evaluation Wiki document work pages; and a set of links to evaluation sites, email groups,
Conducting organizations, etc.
an evaluation
The Application of Case Study Evaluation defines, describes, and illustrates six types of case
studies; presents evaluation questions that can be answered and some design features one might
Qualitative
use and problems to watch for when using each type of case study.
evaluations
Approaches to Evaluation provides a large number of links to sites describing various ways of
Problems to designing and conducting evaluations.
watch for
Guidelines for the Preparation of Independent Evaluations of ILO Programmes and
Projects provides definitions of key concepts of an independent evaluation; describes evaluation
Ethical
methods; and discusses planning for and conducting evaluations, including the responsibilities of
concerns
the evaluator and roles of stakeholders; also provides a suggested outline of an evaluation report.
Aids How to Deliver Negative Evaluation Results Constructively offers ten suggestions for
delivering negative evaluation results, based on the assumption that persons are more open to
learning and change when they do not feel threatened.
Practical Assessment, Research and Evaluation this site provides a peer-reviewed online
journal; provides the full texts of articles on research methodology, evaluation design and
methods, and articles of interest to research students.
Resources for Methods in Evaluation and Social Research presents practical, "how to"
guidance for conducting evaluation studies; includes links to various approaches to conducting an
evaluation study.
Steps for a Successful Policy Analysis describes how to conduct a policy analysis in six easy
steps and provides links to other policy analysis sites.
United States Agency for International Development, a rich source of evaluation reports
concerning the outcomes of assistance programs in many areas of development.
Key terms

Chapter 14 Page 14
● Assessment research ● Formative evaluation

● Evaluation research ● Stakeholders
● Executive summary ● Summative evaluation
PREV NEXT

Chapter 14 Page 15
Introduction Main points

1. Evaluation research is a special form of research designed to produce practical information
Illustration of about the effects or outcomes of a program or service.
evaluation 2. Evaluation differs from other social science research in five ways: (1) it is more likely to
research be sponsored (paid for); (2) its purpose is to provide information for making decisions
about a program rather than for advancing knowledge on some topic; (3) it has greater
Distinctive potential for conflict among persons and groups interested in the results of the evaluation;
features of (4) it is more likely to involve a team of researchers rather than a single researcher; and (5)
evaluation results are frequently less precise than expected for other social research.
research 3. The main steps in conducting an evaluation are: (1) establishing a clear agreement with the
sponsor of the evaluation regarding the terms and conditions of the evaluation; (2) gaining
Conducting and maintaining the cooperation of stakeholders; (3) reviewing the literature to learn about
an evaluation similar evaluations; (4) learning as much as possible about the program to be evaluated; (5)
preparing the design for the study; (6) collecting the data; (7) analyzing the data and
interpreting the results; (8) preparing a draft report, getting feedback from the sponsor and
Qualitative
stakeholders, and preparing and submitting the final report.
evaluations
4. Stakeholders are individuals and groups who have a strong personal interest in the program
being evaluated. They include staff of the organization responsible for the program,
Problems to
community leaders, and persons served by the program.
watch for
5. Gaining the trust and cooperation of stakeholders is essential for conducting a successful
evaluation. On the positive side, stakeholders can provide vital data and check the validity
Ethical
of the evaluator's observations, findings, interpretation, and conclusions. On the negative
concerns side, if suspicious of the evaluator and the evaluation process, they can greatly limit or
even destroy an evaluation.
Aids
6. Preparing a design for an evaluation includes: (1) identifying or developing the objectives
for the program being evaluated; which identify the expected outcomes for the program;
(2) developing a design appropriate for measuring the outcome variables; and (3) selecting
or developing indicators for measuring the outcome variables.
7. Evaluation designs may be based on quantitative or qualitative research methods.
Quantitative methods, such as use of available data or surveys, give more precise results.
Qualitative methods are better for understanding how stakeholders and participants view a
program, for understanding how a program operates in real life, and for measuring more
complex variables. When possible, both approaches should be used.
8. Evaluation reports should be brief and written in plain language without extensive use of
social science concepts. The sponsor and stakeholders should be given an opportunity to
review a draft version of the report. Their comments can help improve the accuracy of
observations and provide a validity check on findings and conclusions.
9. Evaluators should adhere to the highest standards of professional and ethical behavior.

Chapter 14 Page 15


Chapter 15 Page 1
Chapter 15. Using A Multi-method Design and Conducting

Multi-method
Rapid Rural Appraisals
approach to
research Multi-method approach to research
The past six chapters described alternative research methods and their applications. Each method
Rapid rural
was described and analyzed separately. This chapter focuses on the multi-method approach to
appraisal research. The second part of the chapter describes and illustrates a special way of conducting
multi-method research. Called Rapid Rural Appraisal (RRA), this approach has gained wide use
Strengths and for quickly providing information for addressing public policy issues. As the name implies, it
limitations began as a tool for research in rural areas, but the approach can be applied in any setting. The
chapter concludes with consideration of ethical issues that can arise in conducting a RRA
Ethical issues investigation.
The term multi-method means just what it says. A multi-method approach uses at least two ways
Aids
of measuring indicators for collecting data. Researchers use this approach whenever possible
because any single measurement based on one method, no matter how well applied, has
distinctive strengths and limitations. By using two or more methods to measure the same
indicators, we can minimize the weaknesses of each method and benefit from their corresponding
strengths.
Relative strengths and limitations of different methods
Table 15.1 summarizes the strengths and limitations of the most frequently used methods for
collecting data. Not all methods can be used in a single study, but some combination can be used
in most studies. Some ways researchers have used multi-methods of data collection are presented
next.
Examples of multi-method research
Davidson (1992) used a multi-method of data collection in his study of livelihood strategies
employed by households in the Nuba mountains of the Sudan. He began with informal
interviews, a qualitative method, with elders in a number of villages in the area. Based on
information gained in these interviews, he selected 12 villages that showed a variety of responses
to the rapidly changing socio-economic conditions in the area. Next, he conducted a detailed
survey based on 150 households in 3 of the villages. The survey was based on the usual features
of a quantitative investigation. His final set of data was based on indepth, open-ended interviews
with members of 20 households selected from the 150 households included in the survey. The
three research methods resulted in a rich blend of qualitative and quantitative data. This allowed
Davidson to identify and verify the strategies used by households in coping with socio-economic
changes. Strategies included various combinations of capitalistic and lineage-based modes of
production and different uses of labor.
Another multi-method study investigated how irritable men get during Ramadan, the month long
period when Muslims fast during the day. Kadri and colleagues (2000) set out to answer this
question, based on data from adult males living in Casablanca, Morocco. They knew irritability

Chapter 15 Page 1
was somewhat subjective and would be difficult to

PREV NEXT

Chapter 13 Page 2
Multi-method Methods Strengths Limitations

approach to
research
Experiments ● Control over dependent ● Possible difficulty in
Rapid rural variables and the creating experimental and
appraisal conditions for applying control groups
the experimental variable. ● Low external validity
Strengths and ● Precision in measuring the (difficulty in generalizing
limitations dependent and results)
independent variables
Ethical issues ● Provides rigorous tests of
causal hypothesis
Aids ● High internal validity
● Easy to conduct, lowcost
Quasi-experiments ● High on external validity ● Low on internal validity
● Can be conducted in ● Difficulty of obtaining
natural settings equivalent experimental
● Can approach but not and control groups
equal the strengths of
experiments
Surveys ● Simultaneous ● Time consuming, can be
measurement of numerous expensive
variables at one time ● Limited level of
● Can provide multiple measurement of variables
measurement of the same ● Provides limited
indicator, closed and understanding of why
open-ended items can be relationships occur
used in the same
● Potential for reactivity
instrument
● Possible bias in
● Collection of data from
constructing the
large samples
questionnaire
● Provides descriptive
● Possible bias in
information for a large
interviewing
number of variables
● Difficulty in detecting
● Data can be statistically
inaccurate or false
analyzed
responses
● Allows for generalization
in populations

Chapter 13 Page 2
Available and ● Non-reactive ● Bias during collection of

secondary data ● Can be done at the data unknown
(government convenience of the ● Data may be poor or
reports, census researcher invalid measures of
data, data from a concepts on wants to
● Specific data can be used -
previous surveys, measure
dates, frequencies,
mass media
qualitative data ● May be difficult to access
materials, etc.)
● Broad coverage, far more
than a single investigator
could collect
● Provides data for trend
studies
● Little or no cost involved
Causal ● Focus on events as they ● Only a limited number of
observation happen variables that can be
● Non-reactive observed at one time
● See events in their natural ● Potential bias on the part
social context of the observer in
observing, recording
● Easily repeated for
observations, analyzing
verification
and interpreting data
● Excellent for exploring a
● Limited to observation of
topic
small groups in specific
social setting
● Limited value for
generalizing (based on
small groups, relatively
few observations)
Participant ● Focus on real-life ● Time consuming, lengthy
observation processes and events process
● Descriptions of events in ● Requires high level of
their natural social context skill and experience
● Potentially reactive
PREV NEXT

Chapter 15 Page 3
Their solution to this problem was to measure irritability in different ways, using different
Multi-method methods of data collection - a clear example of the multi-method approach to research. One
approach to method was based on self assessments of irritability. Each participant rated himself on a scale
research from 0, for not feeling any increased irritability, to 10, for experiencing extreme irritability. In
addition, clinicians assessed levels of irritability based on responses given in semi-structured
Rapid rural interviews with participants. Indicators of irritability included amounts of coffee and tea
consumption, frequency of smoking, interruptions in normal sleeping patterns, the number of
appraisal
times the men hit children or wives, and incidences of breaking or throwing objects in anger.
Increases in these variables over the course of Ramadan were taken as indicators of increased
Strengths and irritability.
limitations
Assessment of irritability started one week before Ramadan, continued on a weekly basis during
Ethical issues Ramadan, and ended one week after the conclusion of Ramadan. Not surprising, irritability
increased during Ramadan, and far more for persons who smoked. Use of multiple indicators
Aids obtained by different research methods gives greater support for this conclusion than would have
been true if only one set of data had been used. If the data were based only on the subjective, self
assessment, social scientist might question the results. The same might be true if clinical
assessments, which also contain a subjective element, were the only source of data. But when the
two independent methods produced similar results, we can have greater confidence in the
conclusion offered by Kadri and colleagues.
Rapid rural appraisal
Rapid rural appraisal is a multi-method way of collecting data quickly and at least cost, while
seeking to obtain accurate, valid results. Rural development specialists developed RRA in
response to the needs of officials responsible for rural development programs. These officials
wanted trustworthy data quickly and inexpensively. Surveys, the traditional way of collecting
data, generally took too long to complete and were too expensive. Also, decisions about a policy
or program frequently had to be made before results from a survey would be available. Working
with decision and policy-makers, researchers developed methods for rapidly assessing the impact
of policies or programs. Because the techniques were developed and first applied in rural areas,
the word "rural" was attached to the approach. "Appraisal" is used instead of evaluation or
assessment because less precise methods of measurement are generally used.
Today, the terms "participatory rural appraisal" (PRA) or "participatory assessment" are used
along with RRA to reflect the fact that participation of local persons is a prominent feature of all
rural assessment efforts.
When we searched http://ww.google.com for Web sites dealing with RRA, we found over
15,000 sites. We selected 6 to get you started if case you want more information on this topic:
The Community Toolbox: The Idea, Methods and Tools for Participatory Assessment,
Monitoring and Evaluation in Community Forestry provides a comprehensive, detailed
discussion of the idea of participatory assessment, monitoring and evaluation (PAME), methods
used with this research technique, and descriptions and illustrations of tools used (meetings,

Chapter 15 Page 3
posters, murals, story telling, etc.)

Participatory Rural Appraisal contains a discussion of the key tenets of PRA, ways of
conducting a PRA, references, and an illustration of natural resource management in Burkina
Faso based on PRA
Rapid Rural Appraisal, Chapter 8, prepared by the Food and Agricultural Organization, this
sites provides a comprehensive discussion of RRA; including a definition and principals of rapid
rural appraisals, field operation principals, and other points
PREV NEXT

Chapter 15 Page 4
Rapid Rural Appraisal: Application at Charles Stuart University provides an introduction to

Multi-method RRA with a discussion of its features and methods used in practice
approach to
Rapid Rural Appraisal Method describes application of RRA in a study of farmers who were
research
active participants in semi-structured interviews; describes how this approach produces much
more detailed information on issues of importance to individuals in comparison with survey
Rapid rural
results
appraisal
Rapid Rural Appraisal and Rapid Assessment Procedures: A Comparison points out the
Strengths and common characteristics, potentials of RRA and PRA and suggests some distinctions that have
limitations been emerging between the two techniques
For simplicity, we use RRA to cover both the RRA and RPA approaches to assessment.
Ethical issues
Distinctive features of RRA research
Aids
The main features of an RRA investigation are:
● Multi-methods of data collection;
● Use of proxies for indicators;
● Reliance on triangulation for testing the validity of findings;
● Extensive and intensive participation of local persons in the investigation;
● Reversal of the researcher and informant roles;
● Acceptance of approximations and imprecision in results;
● Progressive and successive approximation to the final results; and
● An emphasis on training and empowerment of locals as part of the investigation.
Multi-methods of data collection. RRA investigations are almost always based on a

combination of quantitative and qualitative techniques, including any of those listed in Table
15.1. Data collection is limited only by the imagination and creativity of investigators.
Use of proxies. In previous chapters we stressed the importance of careful and precise
development of indicators. RRA investigators do not ignore these requirements. Frequently,
however, because of tight deadlines, they are not able to develop indicators in the usual fashion.
Without sacrificing validity and reliability, RRA researchers seek readily available substitutes
called proxies in place of the usual indicators. For example, instead of seeking exact
measurements of crop production, estimates provided by local farmers are used. The structure
and maintenance of homes might be used as a proxie for socio-economic status or wealth.
Through the use of proxies, investigators seek the best possible measurement of variables.
Triangulation, based on comparing data obtained by different methods, is used to compensate for
limitations in the indicators used.
Extensive use of triangulation. Triangulation, you will recall from Chapter 13, is based on
comparisons among measurements for some indicator. Most RRA investigations use triangulation
in some way. Howard (1987), for example, used a triangulated design in his investigation of

Chapter 15 Page 4
manufacturing and marketing strategies used by tailors, carpenters, and metal workers in
Khartoum and Omdurman, Sudan. He began by analyzing records dealing with the development
of operation of these trades in the Khartoum area. This was followed by long, informal
interviews, in the form of directed conversations with tradesmen. Additional data were obtained
from long observation periods in their shops. His conclusions were based on the analysis of data
from all sources - records, interviews, and observation. Findings based on one method were
compared to those from other methods. On certain points, the findings agreed: These became the
bases for his conclusions for describing the changes in the trades. Howard concluded that
production and marketing were shifting from made-to-order to ready-made production; to
increased use of wage labor instead of apprentice labor; and to increasing heterogeneity in place
of relative homogenous operations. If Howard had used only a survey, he would not have
obtained the rich, detailed results he found by using three methods of data collection.
PREV NEXT

Chapter 15 Page 5
Extensive participation by local persons. Another characteristic of RRA investigations is the

Multi-method extensive and intensive participation of local persons in the research. In addition to the usual
approach to steps of obtaining the permission of local leaders before starting research, RRA researchers
research frequently involve local persons to the maximum extent possible in all phases of an assessment.
Contributions of local persons can help at all stages in the process, from defining the question to
Rapid rural be investigated to reviewing results for accuracy and validly (another form of triangulation). In
some investigations, the best way to view a research problem is through the eyes of the local
appraisal
residents.
Strengths and Examples of use of local persons include research on health and sanitation conditions in a village
limitations in Alaska (Beradi, 1998); developing community based information and planning systems
(Ashton, 1996); planning for socio-economic development in Uganda (Namara, 1997);
Ethical issues developing projects for women in an Islamic context (Wakeman, 1992), and natural resource
management in Senegal (Freudenberger, 1993). In each of these studies, investigators consulted
Aids extensively with local leaders in deciding on the issues to be addressed and depended on
information supplied by them and other key informants.
Reversal of researcher and informant roles. In conventional research, the investigator decides
the question to be investigated and develops the design for the study. In most RRA studies, the
research and informant roles are reversed or, at least, established on a more collaborative basis.
The investigator enters the social setting of the research to become informed so he or she can see
the problem or issue from the perspective of the community. The investigator seeks to absorb and
understand local knowledge and perceptions regarding the issue under investigation. This
includes gaining an understanding of the solutions as seen by the community and learning how
the community would like to see the issue resolved. Investigators also supply information to local
persons as the research progresses and seek their feedback.
Acceptance of approximate results. RRA investigations can, but generally are not designed to
yield precise, quantified results. From the beginning, investigators know they will obtain only
approximate, but adequate, measures of variables and plan their research with this reality in mind.
We mentioned earlier that proxies or approximations are frequently used in place of carefully
developed indicators. Also, measurement may be in terms of approximations that make sense to
local persons who are supplying information. Instead of developing a scale to measure some
condition, in a RRA investigation the researcher may use local terms for describing things as
"good," "better," or "worse." Changes may be measured as "some," "a lot," or "little," and so on.
Locally descriptive terms are particularly useful in cultures that rely on oral traditions, as in many
rural areas in developing countries. Use of these terms also allows an investigator to see issues or
problems in local terms and to avoid misunderstandings.
Progressive approximation to final results. Final results in a RRA investigation are generally
reached through successive approximation. During the data collection phase of the research,
investigators develop tentative hypotheses and conclusions. Using triangulation methods,
investigators test the validity of emerging hypotheses and conclusions. Emerging findings and
conclusions, for example, are presented to key informants for review and correction. When
disagreements occur, additional key informants may be asked for their views and data is sought

Chapter 15 Page 5
from additional sources. As sources agree on the same point, the findings are considered valid
and used as a basis for establishing the conclusions of the investigation. Still, careful researchers
check their final conclusions and interpretations against the perceptions and interpretations
offered by key informants.
Combining training and empowerment of local participants. A RRA investigator also offers
an opportunity to train local leaders and others in methods of problem solving and finding ways
to improve local living conditions. Projects are frequently structured to achieve these goals.
Investigators involve local leaders and others in all phases of the project, from defining the
question to investigate to the final interpretation of the results.
Local participation in a RRA investigation can have both short and long term benefits. From their
participation in a project, local leaders and others come to see the project as "theirs" and not as
something imposed on them from the outside. The experience and training acquired can also
empower community members through their new capacity to acquire and use information in
solving problems. With their new capacities, leaders and others in the community are better
prepared to manage their own development and other community affairs.
PREV NEXT

Chapter 15 Page 6
Steps in conducting RRA research

Multi-method
There is no precise, step-by-step way to conduct a RRA. Like other qualitative research, RRA is
approach to
an open, flexible approach to research. Still, most RRA projects go through a set of phases,
research somewhat like those described for participant observation. This is not surprising since RRA
research almost always involves unstructured interviewing as well as casual and participant
Rapid rural observation. The following usual sequence of actions for conducting a RRA, therefore, should
appraisal look familiar, as they are similar to those for participant observation. Because these points are
covered in both Chapter 13 (Observation) and Chapter 14 (Evaluation), we will list them and
Strengths and comment only briefly to put them in the context of conducting a RRA. The usual steps are:
limitations ● Obtaining local approval and enlisting leaders and others as partners in conducting the
study;
Ethical issues ● Becoming thoroughly informed about the social setting of the focus of the research;
● Defining the problem or issue to be investigated;

Aids
● Agreeing on a plan for collecting data;
● Collecting and analyzing the data; and
● Developing the results and conclusions of the study and preparing recommendations for
action.
Obtaining local approval and participation. Officials of organizations who also act as sponsors
and introduce the investigator to local leaders frequently request RRA investigations. Under these
conditions, local approval is usually agreed upon in advance. But even when agreement exists
between local leaders and higher-level officials, a RRA investigator needs to obtain the
permission of local leaders before the project starts. During initial meetings, the investigator
describes the planned scope of the project and solicits comments and the participation of leaders
and others.
Berardi's (1998) investigation of sanitation issues in an Alaskan village illustrates this process.
Berardi visited the village, became acquainted with local families, and then sought and obtained
permission to do the investigation from the Elders at Community Services, the local social
services organization. A general plan for the investigation was approved, including the
participation of members of the community in the study. A similar process is essential to the
success of any RRA.
Becoming informed about the social setting and problem. Ideally, an investigator using a
RRA approach needs to be well informed in three areas:
1. How RRA investigations are organized and conducted;
2. About the social system and setting of the community, or organization where the
investigation will be done; and
3. About the issue or problem prompting the study.
This chapter provides only an introduction to RRA. Additional information is available on Web
sites cited earlier in this chapter. A library literature search will provide still further information

Chapter 15 Page 6
on how to conduct an RRA investigation. Information about local conditions may be found in
written records and from information supplied by key informants. Written records may include
census or survey reports, histories, tax records, school attendance records; marriage, birth, and
death records; and crop reports, livestock sales, or other economic records. In an organizational
study, available date could include annual reports, budgets, staffing patterns, revenue and expense
accounts, payrolls, production data, and other operational information.
In some situations, little written information may exist. Then, key informants become the major
source of information. Even when considerable available data exists, researchers verify these data
by checking data with local informants. In some cases, informants may be the only source of
current information on important points. Also, informants can expand on available information,
give you a perspective for organizing information, and serve as a check on information from
written sources.
Agreeing on the problem. Successful RRAs start from gaining the local perspective on the
problem under study. Investigators spend considerable time with local leaders and key informants
in reaching agreement on what is to be investigated. This is time well spent, as it can provide a
number of benefits. With key members of the community in support of the project, investigators
have much greater and easier access to a wide variety of data needed for the investigation; data
can be obtained much faster; and a basis is set for triangulating results by cross-checking findings
with key informants.
PREV NEXT

Chapter 15 Page 7
Agreeing on a approach to data collection. For this step, we have avoided use of "design,"
Multi-method which is the next step in most social science research. The idea of a formal design may be
approach to difficult for local leaders to understand. Besides, precise designs are not appropriate for most
research RRA studies. Agreement on "plans" or an "approach" to information gathering is a more
appropriate and accurate way of describing data collection procedures in a RRA study.
Rapid rural Some frequently used data collection methods include:
appraisal ● Casual and participant observation
● Structured and unstructured interviewing

Strengths and
● Group interviews and focus groups
limitations
● Surveys, frequently using local persons as interviewers
Ethical issues ● Participatory analysis of available and secondary data (reports, plans, aerial photographs,
life histories, etc.
Aids
Most RRA investigations are based on a combination of these methods.
Collecting and analyzing data. Collection and analysis of data often occur simultaneously in a
RRA. One reason for this is that RRAs generally involve at least some qualitative data. As
investigators observe and engage in daily conversations with members of the group, ideas,
hypotheses, and tentative conclusions begin to emerge. Constant review of field notes leads to
additional analyses, which can prompt further observations. This cycle of observation and
analysis continues through the process of a RRA.
Most RRA investigations are based on a team effort. This approach offers a number of benefits.
With additional researchers, data collection and analysis can be completed more quickly. Having
more than one researcher also creates opportunities for triangulation at all stages of the project.
Together, team members can shape the definition of the problem, agree on a plan for data
collection, help collect the data, participate in analysis, and can discuss, clarify, and verify
findings and conclusions. Errors or possible bias at nay stage of the project can be identified and
corrected by other team members.
Also, members of the team bring different skills, which can greatly increase the value of the
research. Some team members may be more skilled in quantitative methods while others can
bring qualitative research expertise. Together, they can plan and carry out a multi-method project
with greater skill and expertise than persons with primary skills in only one approach.
We have emphasized qualitative methods in RRA research, but quantitative methods are used as
well. Surveys are used to collect data from households, organizations, and other entities. Surveys
may be based on enumeration of all households in a village, for example, or may be based on
samples. When sampling is used, random samples are preferred. This allows for generalizing
results to the village or population from which the sample was selected. Sometimes time does not
permit use of random samples or the problem may call for data only from certain persons. In
some cases, a purposive sample of leaders or other persons may be the most appropriate way to
obtain data.

Chapter 15 Page 7
Establishing the results and conclusions and preparing recommendations. The RRA team,
specifically the lead investigator, is responsible for preparing a final report. Reports generally
include a statement of the problem, a description of the social setting and environment where the
investigation was conducted, how the data were collected, what was collected, the analysis
process, the main findings, conclusions, and a set of recommendations. Most RRAs are
conducted to address a policy issue or some problem. Therefore, unlike other social research
which typically end with conclusions, RRAs generally include a set of recommendations for
resolving some issue or problem. Similar to the process described in the previous chapter on
evaluation, the team prepares a preliminary draft report and obtains review and comments from
the sponsor of the investigator and key stakeholders. Based on their feedback, the final report is
prepared and disseminated to the offices and stakeholders interested in the results.
Reports for RRA investigations are written with the intended users in mind. Often, these are
officials of organizations, most of whom have little training in research methods. Therefore, data
should be presented clearly and simply, with the least use of statistics. Percentages and averages
generally are sufficient. Graphic presentation of results, in the form of bar or pie charts, help
officials and other readers quickly understand the results. Chapter 17 describes how to construct
graphs and charts). RRA reports tend to focus on the major findings, conclusions, and the
recommendations of the study. Authors keep reports as short as possible. An executive summary
is generally provided, either as the first part of a report or as a separate brief report.
PREV NEXT

Chapter 15 Page 8
Strengths and limitations

Multi-method
RRA has a number of appealing strengths. Foremost, investigations can be done quickly and
approach to
inexpensively - features that appeal to policy-makers and persons who control budgets. RRAs can
research be planned and conducted in a short time to meet critical decision deadlines. The flexibility of
RRA methods also allows investigators to make full use of local knowledge and expertise as
Rapid rural these become known. In addition, the RRA approach encourages full participation of local
appraisal leaders and others in all phases of an investigation. This opens opportunities for long term
capacity building among participants in a RRA.
Strengths and
RRAs also have potential serious limitations. The greatest limitation is the constant threat of bias,
limitations
misunderstanding of what is observed or heard, and errors committed by investigators. Bias can
be introduced by the way the problem or issue is defined, how data are collected and analyzed,
Ethical issues and how conclusions are drawn. In addition, persons with vested interests may try to bias the
results of an evaluation. Also, investigators can introduce systematic errors and
Aids bias.Triangulation, however, can reduce or eliminate bias and other errors.
RRAs serve a specific purpose, and that is provide relevant, sufficient, trustworthy data quickly
and inexpensively for use in deciding on a course of action. When precise descriptive data are
needed for hypothesis testing or when the purpose is to generate results for generalizing to a
population, rigorous designs, as described in the previous chapters, should be used.
Ethical issues
The previous chapter on evaluation ends with a review of ethical issues associated with
conducting evaluations. The same issues apply to RRA investigations. If you are planning to
conduct a RRA we suggest you review the ethical guidelines discussed in Chapter 6 and
Chapter 14.
Aids
Key terms
● Field notes ● Participatory rural appraisal
● Multi-method research ● Rapid rural appraisal
● Proxies ● Triangulation
Main points
1. Multi-method research is based on using more than one method or technique for the
collection of data.
2. Researchers use a multi-method approach to benefit from the strengths of each method and
to minimize its corresponding limitations.
3. Rapid rural appraisal (RRA) or participatory rural appraisal (PRA) uses a multi-method
approach to research with emphasis on quick, inexpensive ways of collecting valid, reliable
data.

Chapter 15 Page 8
4. Distinctive features of RRA or PRA research are: use of multi-methods of data collection;
use of proxies for indicators; reliance on triangulation for verifying observations, results,
and conclusions; extensive participation by local persons in the research; reversal or at
least more equal researcher and participant roles; acceptance of approximations and less
precise findings; successive approximations to results; and training of local persons in
problem solving.
5. Steps in conducting a RRA or PRA investigation include; obtaining local approval and
cooperation; becoming thoroughly familiar with RRA methods and the local social setting;
agreeing with local leaders on the research question and plans for data collection;
collecting and analyzing data; and establishing the conclusions, formulating
recommendations, and writing the final report.
6. Strengths of RRA or PRA research include: low cost, quick implementation and rapid
completion; flexibility in methods; and opportunities for involving local persons in the
research.
7. Limitations of RRA or PRA include the possibility of bias and error on the part of the
investigator or investigative team.
8. RRA or PRA researchers have to be sensitive to ethical issues. Frequently, these issues are
similar to those that arise in other evaluation studies.


Chapter 16 Page 1
Chapter 16: Preparing Data for Analysis

Introduction
Introduction
Hand versus The previous chapters described ways of collecting data. Now, we will examine how data are
computer analyzed to produce the findings of an investigation. We will look mainly at how data from items
analysis on a questionnaire are analyzed, although the techniques described can be used to analyze most
other kinds of data as well.
Coding Data analysis consists of two related sets of activities. First, the raw data - responses to items for
instance - have to be prepared for analysis. After data are organized, analyses can begin. The
Data entry following three chapters describe alternative ways of analyzing data. Second, the data are then
analyzed. How data are prepared for analysis, however, depends on the technique of analysis to
Aids be used. You can do analysis by hand or by computer. In hand analysis, responses to each item
are counted or tallied manually. Computer analysis requires the use of a statistical analysis
program, also called a software package. Today, there are a number of programs designed
specifically for use in analyzing social science data.
Hand versus computer analysis
Hand analysis
For many researchers undertaking their first project, hand tallying may be the fastest and easiest
way to analyze data. Conditions favoring hand analysis include:
● A small sample, less than 100 respondents;
● Relatively few variables, less than 10 or so;
● Mainly closed items, which are easy to tally;
● Simple forms of analysis and few statistical tests.
If your research design matches these conditions, hand tallying and analysis may be your best
choice. On the other hand, you may want to use a computer analysis method, even with a small
analysis task, in order to learn how to do this. If you do any large scale study, you certainly will
want to to use a computer analysis package.
Computer analysis
For studies with larger samples, a number of variables, and a lot of analyses, hand analysis can be
a tedious, slow task. This is especially true when you do a number of statistical analyses.
Computer analysis eliminates these calculations, saving you a lot of time.
Using a computer has several other benefits as well. Computer-based analyses eliminate errors
that almost always creep into hand tabulations, many of which are often hard to detect and
correct. Also, with a computer you can use more powerful statistical tests in analyzing your data.
This will strengthen your ability to draw clear, defensible conclusions from your data.
Computer analysis, however, also has its downside. A certain amount of careful work has to be

Chapter 16 Page 1
done before the computer can do its magic. First, you have to design a data file to tell the
computer what you are going to enter and where the data are to be stored. The data file contains
the short names you will assign to each variable; the codes for the attributes of each variable you
enter; and their locations in the data file for each variable and its attributes. Later in this chapter
we show how this is done. When the data are entered, the computer knows where the data for
each variable is stored and can follow instructions you provide.
PREV NEXT

Chapter 16 Page 2
If you plan to use a statistical analysis package, you will need to be thoroughly familiar with its
Introduction rules and procedures for setting up a data file. Each software package also has its own rules for
data entry, the process of entering the codes for the attributes of variables into the data file. Most
Hand versus packages have extensive tutorials that teach how to perform these operations. In addition, you
computer probably can get help from persons with experience in using software programs available at your
analysis university. Regardless of the help you might get, it is a good idea to enter data for a small number
of cases and do some test runs for analyses you plan to do later. This way you can discover and
correct errors before beginning you final analyses.
Coding
If your university does not have a software package already installed, there are a number of
Data entry packages available free from Web sites. Here are some of the ones we found after searching
google.com for free software packages:
Aids
BTS Guide to Good Statistical Practice in the Transportation Field, from the U.S. Bureau of
Transportation Statistics, a guide on how to present data analyses and discussion of data quality
and errors
Census and Survey Processing System, provided by the official statistical body of the U.S.
government, the Census Bureau, this package includes free software for entering, tabulating, and
mapping census and survey data
Data Analysis with Epi Info, a free source for Epi Info, a statistical analysis package; provides
for data entry and a wide range of statistical analyses
Free Software, an extensive guide with links to a variety of statistical analysis packages and
related resources; also provides links to guides for making statistical calculations
Web Pages that Perform Statistical Calculations provides over 600 links, including nearly 400
pages that perform calculations, and growing; a source of information on almost anything you
might need in conducting analyses and calculations
Even if you decide to use computer analysis, you will probably need to do hand tallying of data at
some point in your research. If you do an exploratory study, for instance, you might want to hand
tally the data for the few cases involved. Also, occasions arise when you may wish to look at
some variable without having to enter the data into a computer.
An important step in data preparation involves coding. Coding is essential for computer analysis
because computers can only "read" numbers. If you tally your data by hand, coding will help
reduce errors.
Coding
Coding is the process of assigning numbers to represent the attributes making up variables.
Coding can be done before or after data are collected. In Chapter 10, we gave examples of coding
responses to closed items at the time the items are written. Examples of precoding are provided in
Tables 10.1 and 10.2. The codes appear on the questionnaire and are used by the interviewer in

Chapter 16 Page 2
recording responses to the items.

Coding closed items
Coding closed items is relatively easy and straightforward. However, the level of measurement
has to be taken into account. For responses measured at the nominal level (male/female,
single/married/etc.), any set of numbers can be used. For such variables, numbers simply identify
categories and have no value in relation to one another. You could use "1" for male and "2" for
females or the reverse or any other set of numbers. For variables measured at the ordinal level,
codes assigned to attributes have real value. With ordinal measurement, the attributes can be
ordered along some dimension. Responses of "strongly agree," "agree," "disagree," and "strongly
disagree," for example, represent points along a dimension from agreeing to disagreeing about
some issue. Typically codes of 0, 1, 2, and 3 or 1, 2, 3, and 4 are used for responses to items like
these. The numbers used as codes preserve the idea of ordered variation among the responses.
Interval and ratio measurement produce numbers that do not require coding.
PREV NEXT

Chapter 16 Page 3
Coding open-ended items

Introduction
Open-ended items typically produce responses in words. These valuable, but non-numerical
responses have to be converted to numerical codes before they can be analyzed. To use an
Hand versus
example, let's say you included the following question in a survey at your university: "What do
computer you see as the most serious problem at our university today?" You would certainly get a wide
analysis range of responses to this question. In fact, as given by respondents, they probably would be too
varied to use by themselves. Before you could make any sense out the responses, you would need
Coding to reduce the number of responses to a few meaningful categories. The process used is similar to
that described in Chapter 12 for conducting content analysis. The same rules and techniques
Data entry apply: Only in this case, the content is the form of responses to items instead of some form of
available data.
Aids
Let's assume you noted the following responses for the first ten questionnaires you examined
from the survey of attitudes toward your university:
● Costs are too high
● Faculty members are not up to date
● Classes are too large
● Not enough financial aid
● Laboratory equipment is inadequate
● Faculty are not interested in students
● Too many special fees
● Library resources are limited
● Faculty miss classes frequently
● Poor Internet access
Later, as you examine the rest of the questionnaires, you would probably find that some of these
responses occurred more than once. Other responses might mean the same thing as those listed,
but are expressed in slightly different words. For example, instead of "costs too high," you might
find "tuition is too high" or "it costs too much to attend" or some other variation on the same
complaint. The first task, therefore, is to review all responses and note the different ways the
same problem is expressed. Examining all responses will also give you ideas for possible
categories to use for classifying the responses. Each category should contain responses with the
same meaning or intent, although the words in various responses may be slightly different.
Selection of categories is based on criteria suggested by the research question being investigated.
In the example we are using, the research question obviously deals with students' views of their
university. The categories we select, therefore, should capture the most serious problems the
students see. But there is no right or wrong set of categories. Construction of categories requires
creative thought and a certain amount of trail and error. To start, each response is examined to see
what broader category it could fit into. The objective is to construct the minimum number of
categories that include all responses. The following example illustrates how categories can be

Chapter 16 Page 3
created.
The first response, "costs are too high," suggests a category dealing with financial issues; so, let's
just call it that - "financial issues." With this as a start, we look for other responses that fit under
the category of financial issues. Table 16.1 shows the responses that reflect concern about
financial matters.
PREV NEXT

Chapter 16 Page 4
Table 16.1. Responses codes as financial issues

Introduction
Responses Related to financial issues
Hand versus
computer Costs are too high X
Faculty are not up to date
analysis
Not enough financial aid X
Lab equipment inadequate
Coding Faculty not interested in students
Too many special fees X
Data entry Limited library resources
Faculty frequently miss classes
Aids Poor Internet access
Three of the initial responses are clearly related to financial issues, but this leaves most of the
responses unclassified. The second response, "faculty are not up to date," suggests a new
category. A broader category that would include this and some other responses might be "quality
of faculty." Table 16.2 shows responses included under this category and those earlier classified
as related to financial issues.
Table 16.2. Responses coded as financial issues or as related to academic quality
Related to
Related to quality of
Responses financial issues faculty
Costs are too high X
Faculty are not up to date X
Classes are too large
Not enough financial aid X
Lab equipment inadequate
Faculty not interested in students X
Limited library resources
Faculty frequently miss classes X
Poor internet access
We could create a category called "inadequate facilities" to cover responses dealing with facilities
and services. But this still leaves one response, "classes too large." This response does not fit
under either financial issues or quality of faculty. It could be interpreted, however, as another
indicator of dissatisfaction with facilities. Questions about the proper placement of responses
often come up in constructing categories. Responses that do not initially fit into a classification

Chapter 16 Page 4
scheme can be dealt with in several ways. One solution is to expand the definition of a category
to include previously unclassified responses. In our example, we could decide that "large class
size" is related to not having enough classrooms and, therefore, represents complaints about the
adequacy of facilities. Based on this decision, we would have constructed three categories for
coding responses to the question about problems at the university. Having made these decisions,
our final coding is shown in Table 16.3. Further, we coded responses judged to be related to
financial issues as "1;" those related to academic quality as "2;" and responses dealing with
facilities and services as "3."
Responses that do not fit into coding categories can be treated in either of two other ways. You
can create an "other" category to account for the responses not categorized or you can simply
drop responses that don't fit easily in any category and complete the analysis with the responses
you could place in categories. If you decide not to use certain responses, this reduces the number
of cases the analysis is based on. The number of responses not used and the new total for the
analysis should be reported.
Different classification schemes can be used with the same set of data. As we proceed with our
analysis, for example, we might decide that our main interest was whether students saw problems
in terms of quality of faculty versus all other kinds of problems. With this as our interest, we
would recode all responses as fitting into either category. We could, for example, code responses
dealing with quality of faculty as a "1" and all other responses as a "2." Since this is a nominal
classification, 0 and 1 or other numbers could be used as well.
PREV NEXT

Chapter 16 Page 5
Table 16.3. Final coding for responses describing problems at the university
Introduction
Hand versus Responses Financial Faculty Facilities

computer Costs are too high X
analysis Faculty are not up to date
Classes are too large X
Coding Not enough financial aid X
Lab equipment inadequate X
Data entry Faculty not interested in students X
Aids Faculty frequently miss classes X
Poor Internet access X
Coding guidelines
Coding involves imagination and creativity. Categories emerge from repeated examination of
responses. Different persons, with differing ways of analyzing qualitative data, will come up with
different classification schemes. Construction of categories, nevertheless, is based on well
established guidelines. First and most important, categories have to be clearly defined. Responses
with the same meaning or the same thing in common, as defined for each category, are included
in the same category. Oranges and lemons, for example, properly belong in a category defined
as fruit, but rice would not belong in this category.
Second, as with attributes for all variables, categories must be mutually exclusive and exhaustive.
Each response should fit into one category and one category only; that is, be mutually exclusive.
No response should be left out (exhaustiveness), unless a decision is made to specifically exclude
certain responses. When these criteria cannot be met, the category scheme is clearly defective and
a new set of categories has to be developed.
Five to seven categories are adequate for most analyses. A scheme with ten or more categories
usually means that the categories are too narrowly defined. With too many categories, there is the
risk that there will not be enough cases in any category to support meaningful analysis. Fewer
categories generally are better. This gives a better chance of having enough cases in each
category for making comparisons among the responses. Finally, keep the meaning of categories
simple. A clear, brief, easily understood name for each category will help readers quickly grasp
the meaning of categories you develop.
Quality of coding
Validity and reliability issues arise with coding responses, as they did in connection with coding
the content of available materials. A coding system should meet at least the criteria of content or

Chapter 16 Page 5
face validity. To meet this criteria, researchers often provide a variety of responses to illustrate
the content and definition of categories. In our example, the validity of the category of "financial
issues" could be established by reporting several responses, such as "costs too much," tuition too
high" or other responses classified under this category. Two forms of reliability data also should
be reported. Inter-analyst reliability indicates the extent to which different qualified coders
placed the same responses in the identical categories. This is a test of the reliability of the coding
scheme. Intra-analyst reliability is the measure of consistency in coding by the person who does
the final coding. This is a measure of the coder's consistency in coding responses. Both forms of
reliability are described in Chapter 12 as part of content analysis.
Developing a codebook
After responses are coded, a codebook is prepared. A codebook is your guide to your data. It is
the complete listing of the variables you will analyze, including their names, definitions,
attributes, and the codes assigned to the attributes. When computer analysis is planned, the
codebook also identifies where each variable is located in the data file that will be created in the
memory of the computer. Box 16.1 shows the three essential features of a codebook.
First, each variable is identified by a short name or abbreviation, shown in Box 16.1 under the
heading of "Variable." ID for example, stands for the unique identification number assigned to
each respondent; GEN is used to designate gender; CLASS is short for the academic class level
of students; and PROB is the abbreviation for the question asking about problems at the
university. Each respondent is given a number that identifies all the data associated with that
individual. To illustrate, a set of numbers beginning with 01, 001, or 0001 might be used,
depending on whether the sample size is 99 or less (hence from 01 to 99); up to999 (hence
starting with 001 because three digits will be needed); or up to 9,999 (in which case the first ID
number would be 0001). It is critically important that every bit of data for each respondent be
clearly associated with the proper respondent. You can do this by entering the ID code for each
respondent with each set of data you create. If each respondent is not clearly identified and data
corrected associated with each respondent's ID code, you will end up worthless data associated
with that individual. To illustrate, a set of numbers beginning with 01, 001, or 0001 might be
used, depending on whether the sample size is 99 or less (hence from 01 to 99); up to999 (hence
starting with 001 because three digits will be needed); or up to 9,999 (in which case the first ID
number would be 0001). It is critically important that every bit of data for each respondent be
clearly associated with the proper respondent. You can do this by entering the ID code for each
respondent with each set of data you create. If each respondent is not clearly identified and data
corrected associated with each respondent's ID code, you will end up worthless data.
PREV NEXT

Chapter 16 Page 6
Box 16.1. Illustration of part of a codebook

Introduction
Variable Description Column
Hand versus ID The unique number assigned to
computer 01-03
each respondent
analysis
GEN 1. Male
04
Coding 2. Female
CLASS What is your class level?
Data entry
1. First year
Aids 2. Second year
05
3. Third year
4. Fourth year
5. Graduate level
PROB What do you see as the most
serious problem at our unversity
today?
1. Financial issues 06
2. Academic quality
3. Adequacy of facilities
Second, a description of each variable is included. For variables represented by items on a

questionnaire, the usual practice is to repeat the item as it appeared on the questionnaire. The
item and its response categories become the operational definition of the variable. Response
categories are listed along with their codes, as shown in the middle column of Box 16.1.
The third feature of a codebook is shown at the right side of the box. This is the location of the
data for each variable in the data file. Locations of the variables are frequently identified by
columns, a carry over from the pre-computer era. In our illustration, ID was assigned the location
of 01, 02, and 03 because, let's assume, we had a sample of 150 respondents; GEN was assigned
to a location identified as column 04, CLASS to location as column 05, and PROB to column 06
in the data file. When a variable requires two digits, as often happens with scores or indexes, two
columns are reserved for the data. For some variables, such as income, four, five, or even more
columns may have to be used.
Your codebook provides in one place all the information you will need for planning analyses.
The portion of a codebook shown in Box 16.1 tells us that CLASS, for example, is an ordinal
variable with five attributes and that PROB is a nominal variable with three attributes. With this
information, we can plan an analysis of these two variables. The codes listed will also guide you
in interpreting the results from an analysis.

Chapter 16 Page 6
For a study with only a few variables, a codebook may not be necessary. Also, a codebook may
not be needed for hand analysis of data, but it is a convenient way to maintain a complete record
of all your data and coding decisions. A codebook is essential for doing computer analyses.
Although the bulk of the work on a codebook is done when it is first constructed, changes occur
during analysis. As mentioned earlier, after coding a set of responses, you may decide to change
the coding scheme. The new scheme should be entered into the codebook for later use.
For computer analysis, data have to be entered into the computer. There are several ways of doing
this.
PREV NEXT

Chapter 16 Page 7
Data entry
Introduction
One way to prepare data entry into a computer is to prepare a code transfer sheet.
Hand versus Using a code transfer sheet
computer
A code transfer sheet is like a long, wide table, with a set of columns intersected by a set of rows
analysis
going across the sheet. Table 16.4 illustrates a part of a code sheet.
Coding Historically, code sheets were laid out with 80 columns. This is what the pre-computer data
processing machines required. Now, various formats are used for code transfer sheets, depending
Data entry on the software package being used. In our example, the numbers across the top of the first row
identify the columns of the code sheet. Notice the first three columns, 01, 02, and 03. The first
Aids columns of a code transfer sheet and a data file are usually reserved to store the identification
numbers assigned to cases. In this case, we had a sample of more than 99 respondents; therefore,
three columns are needed to identify them. The first row identifies case 001; the second row, case
002, and so on.
Table 16.4. Illustration of a partial code sheet
01 02 03 04 05 06 73 74 75 76 77
0 0 1 1 3 2 1 7 8 1 4
0 0 2 2 2 3 1 0 6 3 6
0 0 3 1 1 1 1 3 4 5 8
0 0 4 1 4 2 0 9 3 7 0
The remainder of each row contains codes for the attributes of various variables as given by each
respondent. For example, gender, coded as 1 for male and 2 for female, is shown in column 04.
Reading across row 1, we see that the first respondent is a male, as are respondents 003 and 004.
Respondent 002 is a female. Column 05 contains the codes for class level. From the code of 3 in
column 05 we know that respondent 001 is a third year student. The other students, in order of
their identification numbers are second, first, and fourth year students. Column 06 contains codes
for how responses to the question about problems at the university were coded.
In this way, each variable is given a location in the data file for our study. Codes for attributes of
each variable are located in the assigned locations. For simplicity, we have skipped locations
from 7 through 72. In actual coding, each column would contain codes for attributes of various
variables. Suppose we had filled these columns and then needed to add scores for a composite
measure of the student's overall satisfaction with conditions at the university. Let's say we
reserved columns 73 and 74 for these scores.
Scores for the first four students, as you can see, ranged from 09 to 17. The remaining columns
show codes for other variables. We ended with column 77, which, in this example, happened to
be the column for the last variable. Depending on the number of variables and how many
columns are needed for their attributes, the total number of columns or elements in a data file will

Chapter 16 Page 7
vary. For surveys with many variables, hundreds of "columns" might be needed.
Using edge coding
In edge coding, codes are written in the margin of the questionnaire immediately to the right of
the item to which they refer. This approach eliminates the need for a code sheet. For computer
analysis, some form of locator, such as column numbers, has to be added so that the codes will be
entered at specific locations in the data file. Box 16.2 gives an example of edge coding for use in
entering codes into a computer.
Each variable for which data is to be entered is given a data location. In the illustration in Box
16.2, for example, locations 01-03 are reserved for entering the i dentification or case numbers of
respondents. The code of 001 is shown for the first case. Locations and are also shown for the
gender (04) and the academic class levels (05) of respondents. The code for attributes for these
variables is shown below the location. The codes indicate that the first case (001) is a third-year
female. Each additional variable would be given a location in the data file and its code would be
written in the margin of the questionnaire under the designated location.
PREV NEXT

Chapter 16 Page 8
Direct computer entry

Introduction
Today, many software programs present "screens" or forms that allow for direct computer entry
of data. These programs, including the free ones mentioned earlier, are quite flexible and can save
Hand versus
a lot of time. They are help in reducing errors in entering data. You may wish to check with the
computer computer center of your university to see what kinds of direct computer entry programs are
analysis available for your use.
Coding Cleaning data

Errors creep into any operation in research. With so much detail involved, there are many
Data entry opportunities for making errors in coding responses and in transferring and entering data into a
computer. In transferring data for gender, for example, a 1 could easily be mistakenly entered as a
Aids 2 or the reverse. Editing or data cleaning, therefore, is an additional necessary step before
analysis can begin.
Fortunately, there are several ways to detect and correct errors in data preparation and entry
operations. If you use a code sheet, you can scan or proof read each sheet. Examine all codes in
each column to see if the codes are in the correct range. If any number other than a 1 or a 2
occurs in the field for gender (coded as 1 or 2), it is clear an error occurred. Similarly, codes for
all other variables should be in an expected range. This form of eye-balling works well when the
sample size is not too large. For larger samples, checking is done more easily by developing
frequency distributions for each variable. (We explain what a frequency distribution is and how
to construct these later in the next chapter). Some statistical analysis packages have built-in
safeguards against some forms of errors. The software is told what the codes for a set of attributes
should be and will "beep" to alert you when you try to enter an incorrect code. This is another
benefit from using a statistical analysis package.
Errors in coding and data entry can also be detected during analysis. One technique is to check
whether responses to one item are reasonable in light of responses to another item. For example,
when a person coded as five years old is also coded as married or as having children, you know
an error occurred somewhere in the process of coding, data transfer, or in data entry. When errors
are found, you need to locate and correct the error before continuing analyses. This requires
going back to the source of the data and correcting each subsequent step in processing the data.
Modifying data
During analysis researchers frequently get new ideas and hypotheses. Testing these ideas often
requires reorganization or modification of the original data. Earlier in this chapter we described
one form of data modification. After we had developed a set of categories for open-ended
responses, we decided to develop a new coding scheme by classifying responses as being
concerned about quality of faculty versus any other kind of problem. With this new interest, we
modified the data by creating a new coding scheme based on just two categories.
PREV NEXT

Chapter 16 Page 8

Chapter 16 Page 9
Development of composite scores from a set of responses is another form of data modification.
Introduction The responses to items are the initial, raw data. These data are analyzed to get composite scores,
which generate data for a new variable. Information about the new variable is then added to the
Hand versus codebook. After that, analysis proceeds either by hand tallying or the new date are entered into a
computer computer for analysis.
analysis Combining responses to items is a frequent form of data modification. Sometimes there are
relatively few responses in the extreme attributes for variables. To illustrate, suppose you got the
Coding following responses to an item stating that women have the same right to enter political life as
men:
Data entry Responses Frequency
Aids Strongly agree 5
Agree 32
Uncertain 10
Disagree 19
Strongly disagree 3
Compared to the middle attributes, there are relatively few frequencies for "strongly agree" and
"strongly disagree." In situations like this, the best choice is to collapse or combine the attribute
with few frequencies with the one immediately above or below it. In this case, we would combine
"strongly agree" and "agree" and make it one attribute and then do the same for "disagree" and
"strongly disagree." We would also update our codebook with these changes. Collapsing
attributes, however, results in throwing away a portion of the measurement of a variable and,
therefore, should be done with some caution.
Anytime you combine data, you should update your codebook so that you have an accurate map
to your data. This will include giving new codes to the responses and may require entering new
data in the database before beginning analysis.
With the data coded, you are ready to begin analysis. Whether you do this by hand or with the
help of a computer program, analysis begins with the examination of each variable, taken one at a
time. This is the focus of the next chapter.
Aids
Internet resources
In contrast to other topics, we found only a few web sites dealing with data preparation as would
be done by an individual researcher working by him or her self. We suggest you visit the one site
we found. This site should help you organize your data for analysis.
Data Preparation describes and illustrates steps involved with data preparation, includes
keeping track of the data, checking data for accuracy and completeness, developing a database
structure, entering data into a computer, and modifying data during the process of analysis

Chapter 16 Page 9
In addition, most of the sites for computer-based systems listed earlier in this chapter also include
instructions for preparing and entering data for these systems. Many of these instructions can be
applied to the process of data preparation and computer entry in general.
PREV NEXT

Chapter 16 Page 10
Key terms
Introduction
● Codebook ● Edge coding
Hand versus ● Code transfer sheet ● Hand analysis
computer ● Coding ● Inter-analyst reliability
analysis ● Computer analysis (computer based ● Intro-analyst reliability
analysis) ● Raw data
Coding ● Data cleaning ● Software package
● Data entry ● Statistical analysis package
Data entry
● Data file ● Tallying
Aids ● Data modification
Main points
1. When you have a small sample and only a few variables, tallying or counting responses by
hand may be the easiest way to analyze your data. For larger samples with many variables,
computer-based analysis, using a statistical analysis package, will save a lot of tedious
calculating and many errors.
2. To use a statistical analysis package, all data must be put in numerical form through a
process of coding. Coding can also help if you decide to do analyses by hand. Coding is the
process of assigning numbers in a proscribed manner to the attributes of variables.
3. Closed items may be coded at the time the responses are constructed, called precoding, or
may be done as part of the process of preparing data for analysis.
4. In coding, variables are given a short, abbreviated name. Giving names to variables is
essential for computer analysis. Also, numbers are assigned to the attributes making up
each variable. This information together with the definitions of variables is entered into a
codebook. A codebook is a directory to the variables and their codes.
5. When computer analysis is used, a data file has to be created. This identifies the locations
of variables in the memory of the computer and provides the codes used for their attributes.
This information is necessary for planning analyses and for interpreting what you find.
6. Coding open-ended items or other qualitative data requires developing a limited set of
categories that are mutually exclusive, exhaustive, and that have high face or content
validity. Generally, between 3 and 7 categories work well.
7. Reliability of coding is established by demonstrating agreement among independent
coders. Called inter-analyst reliability, this form of agreement establishes the reliability of
the coding scheme. In addition, the reliability of the main coder should be demonstrated.
This form of reliability is called intra-analyst reliability.
8. Codes can be prepared for data entry into the computer in several ways. Codes can be
copied on to transfer sheets that show the codes for each variable in a series of columns or
by use of edge coding. The latter is accomplished by copying codes in the blank margin of

Chapter 16 Page 10
the questionnaire opposite responses. Edge coding can also help when hand tallying is
used.
9. When computer analysis is used, it is necessary to check the accuracy of codes for all
entries in the data file. This process of data cleaning can be done by inspecting a printout
of all codes and by being alert to inconsistencies among codes for variables.


Chapter 17 Page 1
Chapter 17: Conducting Univariate Analyses
Frequency Frequency distributions

distributions The first step in analysis is to analyze data for each of your key variables Analysis of one variable
at a time is known as univariate analysis. This is done by first developing a frequency
Analyzing distribution of the attributes for each variable.
single
A frequency distribution, also referred to simply as a distribution, describes how many times
variables each attribute of a variable occurred. Distributions can be generated by hand tallying or by using
statistical analysis software on a computer. We will use hand tallying to show how distributions
Presenting are created. (The computer essentially does the same thing and presents the results on a monitor
univariate or as a print out). Hand tallying, however, is practical only when you have a small sample. With
data large samples, computer analysis is the only practical way to go.
Tallying involving categories
Measures of
central The way data are tallied depends upon the level of m easurement used. Different procedures are
tendency used for data in the form of categories in contrast to numerical data. Variables represented by
categories are measured at the nominal or ordinal level. Tallying responses for these items simply
Measures of requires counting how many times each response category was chosen for each item. Following
variability is an example of a nominal item used in a survey among villagers in central Sudan:
1. How do you get water for household use?
Standard (1) Wife and children bring it
deviation and (2) Use animals to bring it
the normal (3) It is brought by vehicle
distribution (4) Other (Specify)_______________________
A tally sheet, such as Table 17.1, can be used in tallying responses to an item. Our sheet shows
Computer the number of the item from the questionnaire (which, in our example, happened to be 11) and
analysis the four responses for this item. We listed a shortened version of the responses, although codes (1
reminder through 4) could have been used as well. Using codes is simpler, but can lead to mistakes in
coding. Using the verbal form generally results in fewer errors in recording responses for getting
Aids a tally. We also included a category labeled "other" to record any unexpected responses.
Using the tally sheet, we place a tally mark (/) following each response code for each time each
appears. Tallies are grouped in sets of five to aid in counting. Tally marks are then added to get
the frequency for the number of times each response category was selected. The letter "f" is
sometimes used for frequency in the headings of tables. Frequencies for each of the responses are
recorded and added to get the total for the analysis. In research, the total for an analysis is called
the "N" for the number of responses for an analysis. As a rule, the N for an analysis should equal
the N for the sample. This is why we added the "no data" category.
PREV NEXT

Chapter 17 Page 1

Chapter 17 Page 2
Table 17.1. Illustrative tally sheet

Code for Response Tally f
Frequency
distributions Wife/children //// //// //// //// /// 23
Animals //// / 6
Analyzing
single Vehicle //// //// /// 13
variables Other -
No data /// 3
Presenting
univariate N 45
data
By adding this category, the N for each analysis will be the same and should equal the N for the
sample. Without the "no data" category, the N could vary from one analysis to another and you
Measures of
won't know if you made a mistake in tallying or whether the difference was due to missing data.
central By adding the "no data" category, the N should always equal the N for the sample. If the N for
tendency some item does not equal the N for the study, you will know you have made a mistake in tallying
and will have to correct it before going on.
Measures of
variability Tallying numbers
Data in the form of numbers are tallied by placing a tally mark after each number as it occurs. To
Standard illustrate, imagine you had to tally responses for the following item: "How old is the head of this
deviation and household?" Let's say the youngest head of a household was 18 and the oldest was 84 years of
the normal age. You could list each age that was reported and then place a tally after each age as it occurred.
distribution The result would be a long list of numbers (ages), each followed by so many tally marks and a
number representing the frequency for each age. There is a better way, however, to present
Computer numbers that cover a wide range, such as the age distribution we are examining. Ages can be
combined or grouped into intervals. An interval is simply some span of numbers used to
analysis
represent age or any other continuous variable.
reminder
Generally, at least 3 and not more than about 10 or 15 intervals are used for most numerical
Aids variables. Also, the intervals have to be mutually exclusive and exhaustive, just as was required
for the creation of nominal or ordinal measures. Finally, in creating intervals it is useful to use a
span that corresponds to general usage. For intervals based on age, 5 or 10 year intervals are
commonly used. Intervals for income might be in terms of hundreds or thousands of some
currency.
To illustrate this process, Table 17.2 shows tallying in terms of 10-year intervals. Notice that the
two extreme categories are left open to achieve the criterion of exhaustiveness. For each head
aged 19 or younger, for example, a tally mark would be placed after that interval each time a man
was 19 or younger. In the same way, tally marks are made for each man 60 years or older. The
same procedure would be followed in recording the age of each head aged 20-29, 30-39, etc. As a
check, the tallies are converted to frequencies, as shown in the column to the right, and then

Chapter 17 Page 2
added. The total should equal the N for the analysis.

Table 17.2. Tally sheet for ages grouped by ten-year intervals
Intervals Tally Frequency
19 or younger /// 3
20-29 //// //// //// //// 19
//// //// //// ////
30-39 27
//// //
40-49 //// //// //// // 17
50-59 //// //// / 11
60 and older //// //// //// / 16
No data /// 3
N 96
PREV NEXT

Chapter 17 Page 3
Using intervals for reporting continuous data saves space and allows readers to more easily
understand patterns in the data. For this reason, g rouped data, based on intervals, is often used in
Frequency presenting data in the form of tables or graphs. We explain how to do this later in this chapter.
distributions
Calculation of summary measures, such as the mean (average), however, should always be based
on the original raw data. This preserves the original interval or ratio level of measurement.
Analyzing Otherwise, calculation of summary measures from grouped data can reduce the level of
single measurement from the ratio or ordinal level of measurement, depending on the nature of the
variables variable, to ordinal.
Presenting Any time original numerical data are reduced to intervals some data are, in effect, thrown away
because a less powerful level of measurement is substituted for the original ratio or interval level.
univariate
As a result, less powerful means of statistical analysis have to be used. Both conditions reduce
data the strength of the conclusions that can be drawn from the results of the analysis. Reduction is
more serious when fewer, broader intervals are used. Therefore, narrower intervals are generally
Measures of preferred over wider ones, and more intervals over fewer ones.
central
tendency Analyzing single variables
Frequencies are simply counts of raw data. By themselves, frequencies have little meaning. A
Measures of frequency acquires meaning through comparison with other frequencies. The fact that there are
variability 2,346 males at a university, for example, does not mean much by itself. But this frequency has
meaning when expressed as a proportion or percentage of male students relative to both male and
Standard female students at a university. In the context of the entire student body, the number of males
deviation and now has meaning. The same frequency can be expressed as ratio of males to females or the
the normal reverse. Frequencies also take on meaning when expressed as rates at which something occurs. A
rate expresses how often something occurs in relation all the chances that the event could occur.
distribution
Familiar rates are the birth rate or the unemployment rate. The birth rate says how many women
in a certain age range had live births in a year relative to the population of the area. The
Computer unemployment rate indicates how many persons looking for work can't find a job. These four
analysis forms of univariate analysis are used frequently in research reports. Even if you don't use any of
reminder these in your own research, you need to know what each is and how to interpret them.
Aids Proportion
A proportion is a fraction or part of something expressed as a decimal value between 0 and 1.0.
At a university with an equal number of male and female students, the proportion of each would
be 0.5. As the number of males increased relative to females, the proportion of males would
increase. If the number of females increased relative to the males, the proportion of females
would increase. A proportion is found by a simple formula:
Proportion = f/N
Where f is the frequency of the part of the distribution selected for analysis and N is the total
number of cases. In a university with 575 females and 2,456 males, the proportion of females
would be:

Chapter 17 Page 3
Proportion of females = f/N = 575/575+2,476 = 575/3,051 =.1885

Percentage
Proportions are converted to percentages by multiplying the proportion by 100. Percent literally
means per 100. Percentages reduce a distribution to a base of 100. The formula for percent is:
Percent = f/N(100)
For the illustration used previously for calculating a proportion, the percentage of females in the
university population is:
Percent females = f/N(100) = .1885(100) = 18.85 = 18.8%.
You will very likely calculate percentages in any research you do. The following guidelines will
help ensure calculation of accurate percentages:
PREV NEXT

Chapter 17 Page 4
1. Double check the addition for the frequency distribution. Calculation of a percentage
depends on the N used. If this number is wrong, all percentages will be wrong. As a first
Frequency step, therefore, add the frequencies to get the N and then add the column again to make
sure your addition is correct.
distributions
2. Decide on how many decimal places you are going to use in reporting the percentages.
Analyzing In social research, percentages are generally reported in whole numbers, meaning no
decimal place is used, or in terms of one decimal place. Percentages such as 83% or 21%
single
are given as whole numbers while 83.4% or 21.8% are reported slightly more precisely at
variables the level of one decimal.
3. Calculate the proportion of f/N for the frequency for which you want the percentage.
Presenting
Before you do the calculation, you will need to make a decision about how many
univariate decimals to report. If you use a hand calculator, you will get a string of decimals. If you do
data the calculation by hand, you have to decide how far to take the division involved. The rule
for calculating a proportion or percentage is to calculate to one number greater than you
Measures of plan to report. If you plan to report percentages in whole numbers, copy only the first three
central numbers to the right of the decimal place. Thus, if the result were listed as .8348665, you
tendency would note only .834 and ignore the rest of the numbers. If you plan to report the
percentage with one decimal place, note the first four numbers to the right of the decimal
Measures of place. In this case you would pay attention to .8348 and ignore the rest.
variability 4. Round the result to the desired number of places. Rounding is necessary to account for
the fraction of a number that is not used. Box 17.1 summarizes rules for rounding
Standard numbers. Earlier, when we calculated the percentage of females in a university population,
deviation and we got the proportion of .1885. Following the rules just given, we rounded this to .188
because the last number was a 5. Since the number to the left was an even number (8), we
the normal
left it as 8 and made the final percentage 18.8%.
distribution
Box 17.1. Rounding rules for numbers
Computer 1. Calculate the result to one more decimal place than you plan to use.
analysis 2. Use the end number at the right to adjust the number to its left, which will be
reminder the last number that is reported.
■ (a) If the end number is 0 through 4, leave the number to the left as it is.
Aids
■ (b) If the end number is 6 through 9, increase the number to the left by 1.
■ (c) If the end number is 5, round the number to the left to make it an even
number, like 2, 4, 6, or 8
5. After calculation all the percentages for a distribution, add the column of
percentages. The expected total is either 100%, when whole numbers are used, or
100.0%, when one decimal place is used. Because of rounding, however, totals sometimes
are slightly smaller or larger. For whole numbers, the total may vary between 99 and
101% and for percentages with one decimal place from about 99.8 to 100.2. If your total
is off by more than either amount, check your calculations: you probably have made a
mistake.

Chapter 17 Page 4
Percentage change over time

Some research questions call for comparisons of variables over time. Frequently, we want
to compare the percentages for the occurrence of the variable at two or more times. To
calculate a percentage change over time, compare a current or the most recent frequency,
called f t2, for a frequency at time two, for a variable with an earlier frequency, referred to
as f t1, for the frequency at a previous time for the same variable. The two frequencies are
put in the formula:
PREV NEXT

Chapter 17 Page 5
Percent change = (f t2 -f t1)/f t1 (100)
Frequency To make the calculation, find the difference between f t2 and f t1, divide the difference by f t1,
and multiply by 100. We present three examples. Suppose that the elementary school enrollment
distributions
in a rural area was 189 in 1995 and increased to 265 in 2005. The percentage change in
enrollment would be:
Analyzing
(265 - 189)/189(100) = 76/189(100) = .402(100) = 40%
single
variables Now, suppose enrollment had increased much more - to 463 students in 2002. In this case, the
percentage change would be:
Presenting (463-189)/189(100) = 274/189(100) = .449(100) = 144.9 = 145%
univariate
Any time the difference between the two frequencies is greater than the original frequency, the
data
percent change will be greater than 100.
Measures of Percentage change can also be negative. Let's say the enrollment in 2005 declined to 93 students.
central In this case, the percentage change would be:
tendency (93 189)/189(100) = - 96/189(100) = .507(100) = -51%
Any time the second frequency is less than the first the result will be negative. When this occurs,
Measures of
subtract the smaller frequency from the larger one to get the difference.
variability
Cumulative percentage
Standard
When attributes are measured at the ordinal level or higher, they can be combined or accumulated
deviation and at each level to include the frequencies or percentages at each lower or higher level. To illustrate,
the normal Table 17.3 presents the cumulative percent distribution for data originally presented in Table
distribution 17.2. Cumulative percents can be accumulated by adding percentages down a column, as shown
by the second column from the right or by adding percentages up a column, as shown by the far
Computer right column. In either case, the percentage for the lowest or highest interval is added to the
analysis percentage for the next interval to get a new sub-total. Thus, going down the column, the 3.2%
reminder for the interval of 19 or less is added to the 20.4% for the 20-29 interval to get a combined
percentage of 23.6% for ages of 29 or less. The same addition is used to create sub-totals for all
Aids the other combinations of intervals. At the final combined interval, the accumulated percentages
should equal 100 or something very close to 100. Because of rounding errors, the accumulated
percentages in Table 17.3 came to 99.9.
Cumulative frequency distributions are created in the same way. Cumulative frequency or
percentage distributions are useful for making statements such as "more than" or "less than" did
this or that. Looking at Table 17.3, we can quickly see that 47.3% of the household heads were
40 years of age or older and that 23.6% were less than 30 years of age.
Table 17.3 Illustrative cumulative percent distributions, percentage down and up

Chapter 17 Page 5
Cumulative
percent Cumulative
Ages f % down percent up
19 or less 3 3.2 3.2 99.9
20-29 19 20.4 23.6 96.7
30-39 27 29 52.6 76.3
40-49 17 18.3 70.9 47.3
50-59 11 11.8 82.7 29.0
60 or older 16 17.2 99.9 17.2
Total 93 99.9
PREV NEXT

Chapter 17 Page 6
Ratio
Frequency Ratios are based on comparisons of frequencies. Calculating a ratio is quite simple: Divide the
frequency you are interested in by the frequency of another variable or attribute and multiply the
distributions
result by a standardizing base. The standardizing base can be any number, but typically bases
such as 100, 1,000, or some larger multiple of 10 are used. Suppose we wanted to find the ratio of
Analyzing male to female students in the university with 575 female and 2,456 male students. Setting our
single standardizing base at 100, we do the simple arithmetic and find:
variables Ratio of female to male students = f for females/f for males (100) =
573/2,456(100) = .234(100) = 23.4
Presenting
univariate The result, 23.4, says that there are 23.4 females for every 100 males. A ratio, you might say,
data looks like a percent. Well, it does, except that the percent sign is not added and the result is
expressed as a frequency of females, in this case, per 100 males.
Measures of Rate
central
Rates are used to say how often something occurs in relation to a larger population. Calculating
tendency
rates is similar to ratios, except that the standardizing base is generally larger than 100. The birth
rate of a population, for example, is based on the number of live births per 1,000 persons in the
Measures of population. If 5,482 babies were born in a region with a population of 195,800, the birth rate for
variability that region would be:
Birth rate = frequency for live births/frequency for the population
Standard in the area (standardizing base) = 5,482/195,800 =.028(1,000) = 28
deviation and
the normal The number 28 says that 28 babies are born for every 1,000 persons in the population. By itself,
distribution the number 28 doesn't mean much. But when compared with rates elsewhere in the world, it takes
on meaning. A birth rate of 28 per 1,000, for example, is about two or three times what the rate is
in many industrialized countries.
Computer
analysis In calculating the birth rate, we used a standardizing base of 1,000. For events that occur less
reminder frequently, larger standardizing bases are used. A base of 10,000, for example, is generally used
in reporting rates for physicians in relation to the general population. For extremely rare diseases,
Aids a base of 100,000 or even 1,000,000 might be used. Steps in calculating a rate are summarized in
Box 17.2.
Box 17.2. Steps in calculating rates

1. Obtain the proportion. For events that occur very rarely,
the result will be a very small decimal.
2. Multiply the proportion by the standardizing base number,
which may be 100, 1,000, 10,000 or larger.
Uses of univariate measures

Chapter 17 Page 6
Except for cumulative distributions, the measures just described - proportions, percents, ratios,
and rates - can be used with any level of measurement. Cumulative distributions require ordinal
or higher measurement. Otherwise, each of these measures is useful for comparing attributes
among two or more groups with different Ns. For example, suppose you wish to determine which
of two universities has the largest proportion of female students. University A has 573 students
and 2,456 male students while university B has 1,250 female and 3,670 male students.
Calculation of female to male proportions, percents or ratios will give you the answer. The
calculation for university A is 573/2,456 = .233; the calculation for university B is 1,250/3,670 =
.340. University B has the larger proportion of female students. The result can also be interpreted
as meaning that university B has 34 female students for every 100 male students while university
A had only 23 female for every 100 male students.
Univariate analysis can tell you a lot about your data. Results can:
● Show where errors occurred in coding, data transfer or in data entry.
● Reveal whether modification of attributes for a variable may be needed.
● Indicate whether there is sufficient variation in variables to warrant further analysis.
The first two uses of frequency distributions were discussed in the previous chapter. In addition
to making these tests for each variable, you will need to check each variable for its amount of
variation. To establish an empirical relationship between two variables, each must show a
substantial amount of variation. If a variable has little variation, it can not co-vary or show a
relationship with any other variable. Under these conditions, you will not find an empirical
relationship. Examination of the frequency distribution for a variable will tell you whether it has
sufficient variability to warrant its use in testing for relationships with other variables. Going
back to the illustration at the beginning of this chapter on how families get water, if all but a few
of the families gave the same response, there would be little variation for the variable. It would
not be worthwhile to see if any other variable was related to how families got water.
PREV NEXT

Chapter 17 Page 7
Missing data and valid distributions
Frequency Earlier we indicated why it is important to count the number of times no response was obtained
for an item. In analyzing data, a decision has to be made about how to handle the problem of
distributions
missing data. We will use the data for the item, "How do you get water," to show how
percentages can be calculated when some respondents do not answer an item.
Analyzing
single For this item, N equaled 45 and there were 3 questionnaires for which no response had been
variables obtained. The data could be analyzed by finding percentages for all categories including the one
for no data. But knowing the percentage for missing data would not tell us about how water is
Presenting obtained. A more useful way to get percentages is to remove the frequencies for "no data," get a
new N, and then calculate the percentages. The second method is based on a valid frequency
univariate
distribution that produces a corresponding valid percentage distribution.
data
Table 17.4. Ways of acquiring water among households in a central Sudanese village: valid
Measures of percentages
central Means used
tendency for acquiring Valid Valid
water f Percent frequency percent
Measures of
Wife and
variability 23 51.1 23 54.8
children
Standard Vehicle 13 28.9 13 31
deviation and Animals 6 13.3 6 14.3
the normal
Missing data 3 6.6
distribution
N 45 99.9 42 100.1
Computer
analysis In most surveys we expect to find missing data for a few items, but a large percentage of "no
responses" for an item indicates a problem with the item. Either it is poorly phrased or some
reminder
systematic error occurred in the interviewing process. In such a case, the data for that item should
not be analyzed. A large number of "no responses" on a number of items generally means the
Aids whole questionnaire was poorly prepared, that interviewing was poorly done, or both.
Presenting univariate data
Frequency distributions provide the first results of data analysis. The next step is to convert
distributions to a form that is easy for readers to grasp and that best presents the meaning of your
data. Four common ways of presenting univariate data are:
Tables Pie charts Bar charts Line charts
In addition to our description of ways of presenting data graphically, we suggest you look at
another description of ways this can be done. A source we recommend is:

Chapter 17 Page 7
Presenting Data Graphically, and Presenting Results. This site provides brief descriptions of
and links to sources of information and guidance on ways of creating effective graphic
presentations of data.
PREV NEXT

Chapter 17 Page 8
Tables
Frequency Tables are the main way of presenting frequencies and percentages. You will probably use tables
to present your data. A well-constructed table is self-explanatory. It should have all the
distributions
information a reader needs to understand its contents. This information is conveyed through
seven essential elements.
Analyzing
single These are:
variables ● A table number
● A clear, brief title

Presenting ● Headings for each column in the table
univariate
● Names of the attributes of the variable, shown in the rows of the table
data
● Frequencies for the attributes of the variable
Measures of ● Percents corresponding to the frequencies for the attributes (in most tables)
central ● Totals of the frequencies (the N) and for the percentage columns
tendency We will use data from Table 17.1 to illustrate these features. In Table 17.5, The tally marks,
which represent the raw data, have been converted to frequencies and are listed under the column
Measures of headed with "frequency."Now, let's look at each element.
variability
The table number. Each table should be given a number. This helps in referring to them in the
Standard text of a report. In short reports, tables are numbered sequentially, as "Table 1," "Table 2," etc. In
longer reports or books with a number of chapters, tables are frequently numbered sequentially
deviation and
beginning with the number of the chapter, as we have done in this book. A period is placed after
the normal the table number.
distribution
Table title. Each table is given a brief title that clearly describes the content of the table. For
Computer univariate tables, this is done by giving the name of the variable being presented. The table
analysis number and table title are placed above the body of the table. When the data for many tables
come from the same source, such as the survey you conducted, the source of information can be
reminder
described in the text. When data come from other sources, the source should be described in a
brief note immediately below the total row. (See Table 18.3 for an example). This information
Aids allows readers to make judgments about the value of the data being presented. Also, when the
time of data collection is important, the date of data collection or publication of the data should
be given in the title as well. In Table 17.5, for instance, we show that the data were collected in
2005.
You also need to select a simple style for presenting titles. If your university has a required style
for creating table or figure titles, you will want to use it. For this book, we chose to use a simple
style based on normal type and lower case with only the first letter of the first word being
capitalized. However, when a proper name, like that of a person or city or country is included in a
title, it is capitalized.
Column headings. A table is composed of columns, which run down the table, and rows, which

Chapter 17 Page 8
extend across the table. As a rule, columns are used to present frequencies, percents, or other
numerical data. Rows are used to identify the attributes or categories for which the numerical data
are presented. During analysis, researchers use short hand references, such as "f" for frequency,
"N" for the total number of cases, and the symbol, %, for percent These symbols can be used for
tables, but could cause some confusion for readers without research training. A safer way to
identify columns is to write out each heading, as we did in Table 17.5.
Names of attributes. Each attribute for which data are presented should be clearly described.
Often, the response category is used, as we did in Table 17.5. If you use an "other" category, you
can describe its content by providing a few examples of responses in the text accompanying the
table. In constructing a table, you have a choice in how to list the attributes. One way is to list
them as they appeared as response categories on the questionnaire. But this is not always the best
way. Generally a more effective way to present frequencies or percents for discrete categories is
to list them in rank order. In a rank order, the highest frequency or percent is listed first,
followed by the next highest, and so, ending with the smallest value. This is what we did in Table
17.5. As response categories, the order of attributes was: "wife and children," "animals," and
"vehicles." In constructing Table 17.5, we reversed the position of "animals bring it" and "use a
vehicle" because "brought by vehicles" had a higher frequency and, hence, higher percentage
than "brought by animals." Putting numbers in rank order helps readers to quickly see the pattern
in the data.
PREV NEXT

Chapter 17 Page 9
Frequencies. The frequencies for each attribute are listed in the column labeled "Frequency."
The frequencies listed become the frequency distribution for the variable you are describing. The
Frequency sum of the frequencies should equal the N, as corrected for missing data.
distributions
Percentages. Percentages give readers a clear idea of the distribution among the attributes for a
variable. The percentages are listed in the column labeled "Percent." The percent column should
Analyzing show the direction in which the percentages are to be read. This is indicated by having a total,
single which should add to 100 or some number very close to it. (Because of rounding errors, some
variables percentage columns may add to a number slightly more or less then 100). It is not good practice
to include the "%" symbol after the percent numbers. Repeating the "%" after each number
Presenting clutters up a table, making it more difficult to understand. It is sufficient to identify the column as
univariate "Percent."
data Totals. The totals for the frequencies and percents are listed across the bottom of the table under
the columns for each. A table without totals is incomplete.
Measures of
central A few additional guidelines will help you present your data clearly to readers of your report.
tendency 1. Do not use lines to separate each of the rows or columns. Lines create a cluttered look and
distract from the data, which should be the focus of each table.
Measures of
variability 2. When you cite data from a source, such as a census or government report, cite the source
immediately below as table, as: Source: Sudan Fertility Survey, Department of Statistics,
Ministry of Planning and National Economy, Republic of Sudan. Giving readers the source of
Standard
data allows them to assess its value.
deviation and
Table number Table title
the normal
“ “
distribution
Table 17.5. Means used for acquiring water among households in a central Sudanese village,
Computer 2005
analysis Column headings Means used for
reminder ’ acquiring Frequency Percent
Aids Wife and children

23 51.1
bring it
Names for Brought by vehicle 13 28.9
attributes Use animals to bring
6 13.3
it
Missing data 3 6.6
Total ’ 45 99.9

Chapter 17 Page 9
Bar charts with discrete data

Data presented in tables can also be displayed in the form of bar charts, also called histograms.
Figure 17.1 shows a bar chart based on discrete data. These data were presented previously in
Table 17.1. In Figure 17.1, the attributes are shown along the horizontal axis, the base of the
figure, and the frequencies for the attributes are listed on the vertical axis. The heights of the bars
represent the frequencies for the attributes of the variable. In contrast to tables, the number and
title for a bar chart or any other graph is placed below the graph.
PREV NEXT

Chapter 17 Page 10
Frequency
distributions
Analyzing
single
variables
Presenting
univariate
Figure 17.1. Bar Chart: Means used to acquire water in a village in central Sudan
data
With discrete data, attributes are measured at the nominal level. Therefore, they can be arranged
Measures of in any order along the horizontal axis. As the creator of the graph, you can decide how you want
central to arrange the bars. They can be arranged from left to right in any of three ways: (1) according to
tendency the order of response categories on the questionnaire; (2) by decreasing size, starting with highest
on the left and placing the bars in descending order; or (3) the opposite, in ascending order,
Measures of with the shortest bar on the left and increasing to the highest bar on the right. The first choice
variability reflects judgments made during questionnaire construction. These judgments have no inherent
value for presenting data. Either of the other two alternatives is preferable.
Standard Generally, it is easier for readers to compare frequencies between bars next to one another than
deviation and those far apart. We decided to present results by decreasing frequencies. There are situations,
the normal however, when the orderly arrangement from low to high or the reverse should not be used.
distribution When you are presenting a series of bar graphs with the same categories, it is better to use the
same order for all graphs. This will help readers grasp variations from one graph to the next.
Computer An alternative way of presenting results in the form of a bar chart is shown in Figure 17.2. In this
analysis case, the results are shown in percentages, with the N for the sample shown below the horizontal
reminder axis. Since the data are in nominal form, the columns can be arranged in any way one would like.
They are simply different and have no ranking or value in relation to each other. With being the
Aids case, we chose to arrange the columns so that the relative percentages for each way of getting
water are obvious to the reader. We put them from highest to the lowest, going from let to right.
We could just have easily put them in order from right to left either way would allow readers to
quickly grasp the results. Data shown in Figure 17.3 could be put in percentages as well,
although in this case, because the age categories represent ratio date we would want to keep them
in order from the lowest to highest.
PREV NEXT

Chapter 17 Page 10

Chapter 17 Page 11
Frequency
distributions
Analyzing
single
variables
Presenting
univariate
data
Measures of
central Figure 17.2. Bar chart: Means used to acquire water in a village in central Sudan, in percentages
tendency
Bar charts with continuous data
Measures of Bar charts can also be used with continuous data. The data, however, have to be grouped into
variability intervals, as we illustrated earlier with ages of household heads. In this case, with age as the
variable, the age intervals are arranged in increasing order along the horizontal (X) axis of the
Standard chart. In Figure 17.3, we used ten-year intervals for ages. Each interval extends over a set of
deviation and numbers representing ages of respondents, such as 20-29. In statistical terms, the range of
the normal numbers representing an interval defines the class limits for that number. A single number, such
distribution as 20, also represents a class interval. In the case of 20 or any other number, the class interval
extends from the lower limit for the number to its upper limit. The number 20, for example, has a
Computer lower limit of 19.5 and an upper limit of 20.49. twenty (20) is the mid-point of the class interval.
In the same way, the class limit represented by the interval of 20-29 extends from the lower limit
analysis
for 20 or 19.5 to the upper limit of the interval or 29.49. The range from the lower to the upper
reminder values for a number or set of numbers is known as it class interval width.
Aids The important point from this digression into class limits is that the bars representing age
intervals are centered on the midpoint for each interval. As with discrete data, the height of each
bar equals the frequency for the age interval.
Use of open categories, such as "19 or less" and "50 or older" requires departure from the rule of
making the bars the same width. The width of bars at the lower limit for 19 or less and the line
for the upper limit for 50 or older are simply assumed and are made the same width as the bars
for the ten-year intervals.

Chapter 17 Page 11
Figure 17.3. Bar chart: Distribution of ages of household heads

PREV NEXT

Chapter 17 Page 12
Pie charts
Frequency Pie charts give readers a picture of your data. They show proportions for frequencies or
percentages in the form of slices of a pie or, if you prefer, a circle. Figure 17.4, for example,
distributions
shows the percentages for ways water was acquired in the Sudanese village mentioned earlier.
Analyzing
single
variables
Presenting
univariate
data
Measures of
central
tendency
Measures of
variability Figure 17.4 Pie Chart: Means used to acquire water in a village in central Sudan
Standard Line graphs

deviation and Athough continuous data can be presented in grouped form as bar charts, line graphs, also called
the normal frequency polygons, are the best way to present such data. Just as with bar graphs, values for the
distribution variable being presented are placed along the horizontal axis and frequencies are shown on the
vertical axis. Instead of bars, however, a dot or marker is placed on the graph above each number
Computer and in line with the frequency for the number. The dots are then connected by a line.
analysis As a way of illustration, we will construct of a line graph based on the following distribution for
reminder ages of a sample of students at a university. Figure 17.5 shows a line graph based on this
distribution.
Aids
Age Frequency Age Frequency
18 12 23 11
19 18 24 8
20 27 25 6
21 20
22 14

Chapter 17 Page 12
Figure 17.5. Line graph: Distribution of ages of students

PREV NEXT

Chapter 17 Page 13
Selecting a presentation form
Frequency Selecting a way to present data depends on the characteristics of the data.
distributions Tables are generally a better way to present data when:
1. There are less than 10 points of data (less than 10 categories or less than 10 intervals for
Analyzing continuous data)
single 2. You want to show exact values, such as numbers or percents.
variables
Bar or line graphs are better for:
Presenting 1. Presenting more than 10 points of data (categories or numerical values).
univariate 2. Showing the broad picture; to give readers a quick view of your results.
data
Guidelines for presenting figures
Measures of In line graphs, numerical values run from left, at the intersection of the two axes, to the right. On
central the vertical axis, numbers begin at the intersection and increase going up the axis. Values on each
tendency axis should be clearly identified. A uniform scale should be used for each. When individual
numbers are not used, it is preferable to use easily understood units, such as intervals of 10, 100,
Measures of or 1,000, depending on the values of the variable being presented. The ratio of the vertical to
variability horizontal axes affects how readers perceive and interpret the data being presented. A general
rule is to make the horizontal axis about one-third longer than the vertical one. This will prevent
distortion in presenting data. Guidelines for presenting figures are summarized in Box 17.3.
Standard
deviation and Box 17.3. Guidelines for creating figures
the normal 1. Create two axes at right angles, making the horizontal one
distribution about one-third longer than the vertical one.
2. Assign values in uniform increments along the horizontal line,
Computer increasing from left to right. Use easily understood values,
analysis such as 10, 100, or 1,000.
reminder 3. Place a label below the horizontal line describing the variable
being presented.
Aids 4. Assign easily understood values for frequencies along the
vertical axis, also in a uniform manner, beginning with zero
at the point of intersection of the axes and increasing going
up the axis.
5. Place a label on the vertical axis identifying it as "Frequency"
or whatever is measured on this axis.
6. Place a mark on each axis to identify the values spread out on
each.
7. Place a title (also referred to as the legend) below the figure.
It should briefly describe the content of the graph.

Chapter 17 Page 13
The choice between a bar or pie chart and a line graph is based on the level of measurement for
the variable being presented. Bar and pie charts are used to describe discrete variables. Line
graphs are better for presenting continuous variables.
A frequency distribution describes all the values for a variable. In addition, researchers use
several ways to summarize distributions in the form of the average or typical value. Statisticians
call these measures of central tendency.
PREV NEXT

Chapter 17 Page 14
Measures of central tendency
Frequency
In everyday life, we use descriptions like typical or average to describe events and behavior.
distributions
Your grade point average is an example of this. It is the typical grade you have received for all
the courses you have completed. In statistics, an average, like your grade point average, is
Analyzing referred to as a mean. It is the sum of all the numbers in a set divided by the number of cases that
single were added together. Two other measures of central tendency are also used. These are the mode
variables and the median. The mode is the value or attribute that occurs most frequently. The median is the
middle value in a set of scores that are arranged from low to high. Ordinal, interval, or ratio
Presenting measurements can be ordered in this way. Table 17.6 shows all three measures of central
univariate tendency for the same distribution. What is the typical size of household in this sample? There
data are actually three answers to this question. The easiest one to find is the mode.
Table 17.6. Illustration of mean, median and mode
Measures of
Size of Households Cumulative
central
(X) f fX frequency
tendency 2 3 6 3
3 - - 3
Measures of 4 5 20 8
variability 5 12 60 20
6 17 102 37
Standard 7 21 147 58
• Mean • Median
deviation and Mode = 8 34 272 92
the normal 9 29 261 121
distribution 10 9 90 130
11 4 44 134
Computer 12 3 36 137
analysis 13 1 13 138
reminder 14 or more 2 28 140
Aids N 1401 140 1,079= fX
Mode
Looking at the frequency column, the largest value is 34 and this occurred for the household size
of 8. Eight, therefore, is the mode for this distribution. In a percentage distribution, the mode is
the value with the largest percentage; in a bar graph, it is the highest bar; in a frequency polygon,
it is the value which has the highest point. A distribution, however, may have more than one
mode.
For example, look at the frequencies for codes for five responses to a questionnaire item:
1 Strongly agree 13

Chapter 17 Page 14
2 Agree 28
3 Uncertain 19
4 Disagree 28
5 Strongly disagree 9
This distribution has two modes for agree and for disagree. Each had an f of 28. When there
are two modes a distribution is said to be bimodal. The mode becomes less meaningful as a
measure of central tendency as the number of modes increases.
PREV NEXT

Chapter 17 Page 15
Mean
Frequency The common way to calculate a mean is to add all the values for a variable and then divide by the
number of cases that were added. To find the mean for the size of households listed in Table 17.6,
distributions
we could add 2+2+2 (because it occurred three times) and 4+4+4+4+4 (it had a frequency of 5),
and so on for each of 140 cases and then divide by 140. An easier way to get the sum of all the
Analyzing values is to multiply each value by its frequency and then add the sums and divide by 140. In
single mathematical operations like this, the values are referred to as scores, which are represented by
variables the symbol "X." The column headed "fX" in Table 17.6 is represents the f or the frequency
times X for the corresponding score. Thus, the first product of fX in the column is that of the
Presenting score of 2 times its frequency of 3, giving the product of 6, represented by fX.
univariate
The rest of the column shows the products for each score times its frequency. The sum of these
data products is shown at the bottom of the table, identified by the symbol of fX. The Greek letter ""
(pronounced as "sigma") means "the sum of of." Together, the symbol fX is read as "the sum of
Measures of scores" or, in this case, the sum of the products of fX for the household sizes from 2 through 14
central or more. You will see the symbol of a lot in statistical calculations. We'll use it right now to give
tendency the formula for calculating the mean for a distribution. In statistics, the mean is identified by the
Measures of letter M. The symbol, , pronounced as "bar X" or or "X bar" is also used to represent the
variability mean.
We prefer to use the letter M for the mean.
Standard
deviation and The formula for the mean is:
the normal
distribution
Computer
analysis
reminder
Median
Aids The median is the middle score in any distribution: Half of the scores are above the median value
and half are below it. For distributions having a small N, the median can be found by:
1. Determining whether N is odd or even;
2. Arranging the scores in order from the lowest to the highest;
3. And then counting off to find the middle score.
To illustrate, find the median for the following scores: 5, 3, 13, 4, 11, 6 and 10. To start, the
scores have to be put in numerical order, as 3, 4, 5, 6, 10, 11, and 13. Second, a count shows there
are 7 scores, an odd number. When N is odd there will be one middle score with an equal
number of scores above and below it. To find the position of the middle score, divide N by 2 and
round the result up to the next higher number. In the example N equals 7; dividing by 2 produces

Chapter 17 Page 15
3.5 that is rounded to 4. Therefore, the median is the fourth score from the top or bottom of the
distribution. Counting off from the lowest score, we find that the fourth score is 6, which is also
the median for the distribution. As a way of checking, you can see that there are 3 scores above
and 3 below the score of 6.
When N is even there will be two middle scores. Therefore, the median is calculated in a slightly
different way. To locate the two middle scores divide N by 2 which will give the lower of the two
scores. The median will be halfway between this score and the one above it. As way of
illustration, suppose the score of 16 were added to the distribution we are using. We would now
have 8 scores. Dividing N by 2, we get 4 which tells us that the median is halfway between the
fourth and fifth scores when they are put in order. When ordered, the list is: 3, 4, 5, 6, 10, 11, 13
and 16. The fourth score is 6 and the next higher score is 10: therefore, the median is the
arithmetic average of the two or 6 + 10 = 16 divided by 2 or 8.
We have used these simple illustrations to help you get the idea of the median. In practice,
generally a little more is involved in the calculation of the median. Typically, the N is much
larger which means a number of scores will probably occur many times. Under these
circumstances, the median is most easily calculated from a frequency distribution. We will use
the frequency distribution shown in Table 17.6 to illustrate calculation of a median.
The number of households is 140, an even number. Dividing N by 2, we get 70. This tells us that
the median is the 70th score beginning with lowest score. To find the 70th score we can add
down the frequency column until we come to the 70th score or we could develop a cumulative
frequency distribution, as shown in the last column to the right in the table. At the score of 7, we
see an accumulated frequency of 58, below the number we want, but, if we go to the score or 8,
we have an accumulated frequency of 92, above the 70 th frequency. We can now see that score
of 8 covers a range that includes all frequencies from the 59th through the 92nd in the distribution
of the 140 scores. The 70th score is in the range covered by the frequencies for the score of 8.
Therefore, in approximate terms 8 can be taken as the median. There are more precise ways of
calculating the median, but this way should be sufficient for most of your analyses. If you want a
more precise calculation for a median, consult a statistics textbook or visit a Web site with
information on univariate calculations.
PREV NEXT

Chapter 17 Page 16
Choosing a measure of central tendency
Frequency Selection of a measure of central tendency depends on the level of measurement used. For
nominal measurement, the mode is the only measure of central tendency that can be used. At a
distributions
university with 3,500 men and 1,250 women, it makes sense to say that men represent the mode
for enrollment by gender. It is impossible to calculate a median or mean with nominal data.
Analyzing Medians can be used for ordinal data, but when there are only a few categories, the mode is often
single a more appropriate measure of central tendency. For data measured at the interval or ratio level,
variables the mean is usually used as the measure of central tendency.
Presenting Sometimes, however, the mean is not the most appropriate measure to use with numerical data.
When scores occur at the extreme ends of a distribution, the mean will move in the direction of
univariate
the extreme scores, which can give a misleading report for central tendency. The median is less
data affected by extreme scores because it does not depend on the sizes of the scores as the mean does.
Some simple numbers will illustrate what we have just described. Consider the incomes
Measures of reported by two groups of workers:
central
tendency Sample A: 10, 90, 100, 120
Sample B: 80, 100, 120, 200
Measures of
variability The mean for sample A is 320/4 = 80 and the median is 95. For sample B the mean is 125 and the
median is 110. The low score of 10 in sample A pulled the mean down to 80, making the mean
less than the median. Even if the lowest score had been 0, the median would have remained 95,
Standard
but the mean would have declined further to 77.5. In the case of sample B, because of the high
deviation and income of 200, the mean moved upward to 125 while the median (110) remained close to the
the normal center of the distribution. Any higher score for income in sample B would further increase the
distribution mean, but would not affect the median. Suppose that instead of 200, a person in sample B
received 1,000 per month in local income. Then the mean for sample B would be 325, but the
Computer median would still remain at 110.
analysis
Distributions with a number of frequencies for scores at one end of the distribution are described
reminder as being skewed. In a negatively skewed distribution, the mean is pulled down in the direction of
the lower scores; in a positively skewed distribution, the mean is located nearer the higher scores.
Aids
When there are extreme scores the median is generally the more appropriate measure for
describing central tendency. If in doubt, calculate and report both the mean and median
Measures of variability
In addition to finding the central tendency of a distribution, univariate analysis also includes
describing the extent of variation among the scores for the distribution. The idea of variability is
quite simple, as we think you will see by examining the following two sets of grades for two
classes:
Class A: 40, 50, 55, 60, 70, 80, 85, 90, 100

Chapter 17 Page 16
Class B: 66, 67, 68, 69, 70, 71, 72, 73, 74.
Which class shows the greatest amount of variation in grades? With the scores arranged in order
from the lowest to the highest, one can easily see that there is more variation in the grades of
students in class A than in Class B. Grades in class A vary from 40 to 100 while whose in class B
vary from 66 to 74. Statisticians use three measures of variation. These are the range, variance
and the standard deviation of a distribution.
Range
The range (R) is simply the highest score minus the lowest score + 1. The "+1" is added because
we want to count the lowest score in the range. To clarify this point, calculate the range for the
scores of 10, 11, 12, 13, 14, and 15. The range is 15-10+1=5+1=6. If we simply subtracted 15-10,
we would get 5, but there are six scores: Count them. The range gives a quick idea of how far the
scores are spread out. For class A, as listed above, the range equals 100-40+1 or 61 while R for
class B is 74-66+1 or 9. Class A has almost seven times the variability of Class B.
While useful for giving a quick idea of variability, the range has limited value because it is based
on only the highest and lowest scores in a distribution. Also, the range will change any time
either of the end scores changes. Because of these limitations, researchers rely on the variance
and its derivative, the standard deviation, for more accurate descriptions of variability.
PREV NEXT

Chapter 17 Page 17
Variance
Frequency The variance of a distribution is based on an important property of its mean. The mean is the
balance point of a distribution. By this, we mean that the sum of the differences of each score
distributions
from the mean equals zero. This property is used to derive the variance of a distribution. Table
17.7 shows this property and how it is used in calculating the variance of the distribution of
Analyzing scores in Class A as compared with Class B.
single
variables Table 17.7. Illustrative derivation of variance based on deviations of scores from the mean
Class A Class B
Presenting X-
univariate X X-M (X-M)2 X M (X - M)2
data 40 -30 900 66 -4 16
50 -20 400 67 -3 9
Measures of 55 -15 225 68 -2 4
central 60 -10 100 69 -1 1
tendency 70 0 0 70 0 0
80 10 100 71 1 1
Measures of 85 15 225 72 2 4
variability 90 20 400 73 3 9
100 30 900 74 4 16
Standard
X 630 0 (X - M) 2 3,250 X 630 0 (X - M) 2 60
deviation and
the normal First, as shown in Table 17.7 we find the deviation of each score from the mean of the
distribution distribution. The scores for each distribution are listed under the columns headed X. The total or
the "sum of the scores" is shown at the bottom of each X column. For the purposes of this
Computer illustration, we have made the sum of Xs the same, X = 630. The N for each distribution is also
analysis identical (9). Consequently, the mean for each class is 70. Deviations of each score from the
mean of each distribution are shown in the columns headed with X- M. This represents the
reminder
difference of each score from its mean. For scores greater than the mean, the differences are
positive; for those less than the mean, the differences are negative. As expected, the sum of the
Aids deviations for each distribution equals zero. As a result, the differences with positive and
negative signs cannot be added. Mathematicians get around this problem by squaring numbers.
This eliminates negative signs and allows us to add all squared deviations to get a measure of
variability. The squared deviations appear in the column headed as (X - M) 2. The squared
deviations are added, represented by the symbol (X- M)2 and divided by N. The result is the
variance of the distribution, represented by the symbol s2. Now that we have explained what
variance is, here is the formula for calculating it:
(X - M)2
s2 = _________
N

Chapter 17 Page 17
For Class A the variance is: 3,250/9 = 361.111 = 361.11; for Class B the variance is 60/9 = 6.666
= 6.67. Class A has a larger variance than Class B, just as it had a greater range. Both measures
of variability are larger for Class A because the scores for Class A are spread out much more than
those in Class B. However, variance is seldom used in describing variation among scores in a
distribution. Instead, the standard deviation is used because of its valuable properties in
describing how many scores lie certain distances from the mean.
PREV NEXT

Chapter 17 Page 18
Standard deviation
Frequency The standard deviation, represented by the letter s, is simply the square root of the variance.
When the square root is taken from one side of an equation, the same operation has to be
distributions
performed on the other side in order to maintain the equality of the two sides. Thus, the formula
for standard deviation becomes:
Analyzing
single
variables
Presenting
univariate when the square root it taken from both sides of the equation, s2 becomes s and square root is
data taken of the value for calculation from the formula. Taking the square root of the variances for
the two classes yields the following:
Measures of
central
tendency
Measures of
variability
The standard deviations for the two distributions are expressed in smaller numbers than the
variances, but the difference in variability between the classes is maintained. Class A still has
Standard
over seven times more variation in scores than class B.
deviation and
the normal Mean and standard deviation
distribution When ever you report a mean, it is good practice to report the corresponding standard deviation
as well. Together, the two statistics tell readers a lot about the distribution of the variable you are
Computer reporting on. The mean indicates the central tendency of the distribution of the variable: the
analysis standard deviation shows the extent to which the values for the variable are clustered around the
reminder mean. A small standard deviation means that the values are closely clustered around the mean; a
larger standard deviation says that the values are spread out farther from the mean. Using data
Aids from Class B, we can describe the distribution as having a mean and standard deviation as:
M=7.01± 2.6.
Calculation formulas for variance and standard deviation
The methods described for calculating variance and standard deviation were used to help you
understand what the measures represent. There is a quicker and easier way is used to calculate
each measure. This method requires getting four numbers: the N, which is easy; the sum of the
scores (X), and two new values: (1) the sum of the squares of each score, represented by X²,
meaning each score is squared and then the squares are added to get the sum of squares of the
scores; and (2) the sum of the scores squared, which is represented by the expression of (X)²;
meaning the scores are added and this number is then squared. Table 17.8 shows the calculations
for Class A.

Chapter 17 Page 18
PREV NEXT

Chapter 17 Page 19
Table 17.8. Illustration of derivation of values for calculating variance and standard deviation
X X2
Frequency
40 1600
distributions
50 2500
Analyzing 55 3025
single 60 3600
variables 70 4900
80 6400
Presenting 85 7225
univariate 90 8100
data 100 10000
X=630 X 2=47,350
Measures of
For the calculation, we need:
central
● The N, which is 9;
tendency
● The sum of the scores squared, (X)2, which is 630 times 630 or 396,900; and the
Measures of ● X2, found by squaring each score and adding the products, which is 47,350.
variability
Substituting these numbers in the following formula, we get:
Standard
deviation and
the normal
distribution
Computer
analysis The new method of calculation gave the same variance as we got before. Standard deviation can be
reminder calculated the same way, but with the added step of taking the square root of the variance. The square
root of 361.11 is 19.00. The simplified formula can also be used with frequency distributions, as shown
Aids with the distribution shown in Table 17.9.
Table 17.9. Illustration of deriving variance and standard deviation from a frequency distribution
X f fX X2 fX2
2 3 6 4 16
3 - - 9 -
4 5 20 16 80
5 12 60 25 300
6 17 104 36 612
7 21 147 9 1029
8 34 272 64 2176
9 29 261 81 2349
10 9 90 100 900

Chapter 17 Page 19
11 4 44 121 484
12 3 36 144 432
13 1 13 169 169
14 or more 2 28 196 292
N= 140 fX=1,079 (fX 2)= 8,931
In Table 17.9, (fX2) is the same as X2 except that we get the sum of the squares of the scores by
squaring each and then multiplying by their frequencies. Also, the
fX is the same as adding every score each time it occurred, but in Table 17.9 we multiplied each score
by its frequency and then added the products. With these two values and the N, which is 131, we can
calculate the variance, as
The standard deviation is the square root of 4.39 or 2.09.

PREV NEXT

Chapter 17 Page 20
Standard deviation and the normal distribution
Frequency Properties
distributions Everything you will learn about probability and testing hypotheses rests on use of properties of
the normal distribution or what is generally known as the normal curve. A normal curve is shown
Analyzing in Figure 17.6. A normal curve has four unique properties. These are:
single ● A symmetrical, bell-shaped form;
variables ● The mean, median and mode are identical and coincide at the middle of the distribution;
● A fixed proportion of observations lie between the mean and specific units of the standard
Presenting deviation of the distribution; and
univariate
● A precise mathematical relationship exists between the scores and their frequencies, which
data specifies the shape of the curve.
Measures of The last point, although critical, can only be understood in mathematical terms. Understanding
central the math behind the normal curve, however, is not necessary for learning how the normal
tendency distribution is used in statistical analysis. Understanding the other properties is very important.
In a symmetrical distribution, the frequencies for scores are spread out at fixed and identical
Measures of distances to the left and right of the mean. This is why the shape of the curve is the same above
variability and below the mean. If you folded the one side over the over, they would overlap perfectly. This
match identifies a symmetrical distribution. In these distributions, scores tend to concentrate in
Standard the middle of the distribution and to fall off progressively toward either end of the distribution..
deviation and
the normal
distribution
Computer
analysis
reminder
Aids
Figure 17.6. Properties of the standard deviation in a normal distribution.

Also, in this kind of distributions the mean, median, and mode are identical Measurements for
many variables take the form of a normal curve. For large samples or populations, the distribution
for height, for example comes close to a normal distribution. There are relatively few really tall or
short persons; most persons tend to be in the middle of the range of heights. Intelligence scores
are another example.

Chapter 17 Page 20
PREV NEXT

Chapter 17 Page 21
Standard deviation in relation to the normal distribution
Frequency These features also give rise to several properties that are useful in research. For example, using
the properties of the normal curve we can state the exact percentage of scores that lie within
distributions
certain distances from the mean of the distribution. To illustrate, imagine that we gave a
mathematics test to 1,000 secondary students and found that the distribution of test grades looked
Analyzing like the one shown in Figure 17.6. The grades are shown as the scores along the bottom of the
single curve, beginning with a low of 35 and going up to 95. Let's say the distribution had a mean of
variables 65.0 and a standard deviation of 10. These two statistics would tell us a lot about the results of the
testing.
Presenting
Under conditions of a normal curve, we can use these two numbers to say how the scores of the
univariate
students varied from the mean of the scores. For the distribution in our example, each standard
data deviation (s) equals 10 points on the test or, put another way, each 10 points above and below the
mean equals one standard deviation. Thus:
Measures of
● One standard deviation (1s) above the mean would reach from the mean of 65 to the score
central of 75.
tendency
● Two standard deviations (2s) above the mean would be equal to 2s or 2 times 10 points on
the test and would reach to the score of 85 on the test.
Measures of
● Three standard deviations (3s) above the mean would be equal to 30 points on the test and
variability
would reach the score of 95.
Standard Similarly, we can calculate ranges for the standard deviation below the mean in terms of -1s, -2s,
deviation and and -3s. The score equivalent to -1s is 55,; for -2s, 45; and for -3s, extends for 35.
the normal These three ranges, 1, 2, and 3 standard deviations above and below the mean, are expressed as
distribution ±1s, ±2s and ±-3s. The standard deviation of ±1s represents the range of 55 to 75 in terms of test
scores; ±2s is from 45 to 85; and ±3s extends from 35 to 95. These three ranges also describe
Computer three important properties of the normal distribution
analysis
One property is that 68.26% of the scores lie within one standard deviation from the mean (±1s).
reminder
In our example, this means that out of the 1,000 students, 68.26% or approximately 683 students
had scores between ±1s or between 55 and 75. Another property of the normal curve is that
Aids 95.44% of the scores lie within two standard deviations from the mean (±2s). This means that
about 954 of the students had scores between 45 and 85. The third property is that 99.74% of the
scores lie within three standard deviations from the mean (±3s). In other words, 997 students
had scores between 35 and 95. Generally ±3s is the largest range used with standard deviations
because in a normal distribution less than one percent of the scores would lie outside this range.
These properties of the normal curve are frequently useful in describing variability among scores
for many of the empirical distributions. Even when the empirical distribution does not have the
exact shape of a normal curve, the properties just described generally apply. Frequently about
68% of the scores from a sample lie within one standard deviation above and below the mean;
close to 95% lie within two standard deviations from the mean; and almost all scores are
covered by the range of three standard deviations above and below the mean.

Chapter 17 Page 21
The broad term, descriptive statistics, is frequently used to refer to the calculation of central
tendency and variance for a set of data. As mentioned earlier, these two statistics describe the
essential features of any distribution. For alternative discussions of descriptive statistics and how
to calculate them, you might want to look at:
Descriptive Statistics, this site defines descriptive statistics as "the basic features of data in a
study" and describes and illustrates calculation of measures of central tendency and dispersion.
Also, most Web sites dealing with statistics include discussion of descriptive statistics. Some
Web sites will calculate descriptive statistics for you, based on the values you enter into a table
provided on the site. For example, go to:
Descriptive Sample Statistics provides directions for entry of up to 80 values; calculates the
mean, variance, standard deviation, estimates the sample mean, median, and provides other
descriptive information about the distribution formed by the values entered
For links to additional Web sites that perform statistical calculations, visit: Web Pages that
Perform Statistical Calculations , provides over 600 links, including nearly 400 pages that
perform calculations, and growing; a source of information on almost anything you might need in
conducting analyses and calculations
PREV NEXT

Chapter 17 Page 22
Computer analysis reminder
Frequency Before ending this chapter, we want to emphasize again the value of doing analyzes with the aid
of a statistical analysis package. Although we used only small samples to illustrate how to do
distributions
univariate analyses, you have seen the kinds of calculations that are required. Imagine the amount
of tedious work involved when a large sample is used or when a number of variables are
Analyzing analyzed. With larger samples, calculation of the X and X ² values requires many mathematical
single operations. Also, with more hand calculations, there is a greater risk of making errors. Other
variables techniques of statistical analysis, which are described in the following chapters, require even
more calculations. Computer-based analysis eliminates tedious calculations and the possibility of
Presenting mathematical errors. Therefore, we suggest you check to see if your university or another
univariate organization has a statistical analysis package and that you learn how to use it. Once you
data complete analysis of data with the aid of a computer, believe us, you will never want to be
without such assistance.
Measures of Aids
central
tendency Internet resources
In addition to the sites mentioned previously in this chapter, we call your attention to the
Measures of following sites that provide additional information and guidance about statistical operations.
variability
PEPI v3.01, a software package designed for use in planning and structuring analyses in the
Standard health field, although it can be used for other areas as well; interactive and described as "easy to
use"
deviation and
the normal Electronic Textbook StatSoft, this complete electronic statistics textbook can be downloaded
distribution for quick access on a local hard drive; part one provides an overview of elementary concepts in
statistics; subsequent sections cover a wide variety of analysis methods
Computer
Statistics and Design provides brief descriptions of and links to sites containing information and
analysis
guidance for analyzing data
reminder
We have introduced a number of statistical terms in this chapter. You will find short definitions
Aids of each in the Glossary. Other Web sites provide more extensive definitions of statistical terms,
provide a conceptual basis for understanding them, and show how to calculate them. We refer
you to the following sites:
Internet Glossary of Statistical Terms provides definitions of a wide range of statistical
concepts and terms
Statistics Glossary provides definitions of terms used in hypothesis testing, illustrates steps
involved, and shows how to interpret the results of hypothesis tests; provides links to Web sites
that perform a wide range of statistical calculations
PREV NEXT

Chapter 17 Page 22

Chapter 17 Page 23
Key terms
● Average ● Number (N)
Frequency
distributions ● Bar Chart ● Percent (percentage)
● Bimodal distribution ● Pie chart
Analyzing ● Central tendency ● Positively skewed distribution
single ● Class interval width ● Proportion
variables ● Class limits ● Range
● Continuous variable ● Rank order
Presenting
univariate ● Cumulative frequency distribution ● Rate Ratio
data ● Cumulative percent distribution ● Raw data
● Descriptive statistics ● Rounding
Measures of ● Discrete variable ● Score
central ● Distribution ● Skewed
tendency ● Frequency ● Standard deviation
● Frequency distribution ● Statistical software package
Measures of
variability ● Frequency polygon ● Table
● Grouped data ● Tally sheet
Standard ● Line graph ● Tallying (by hand)
deviation and ● Mean ● Univariate analysis
the normal ● Median ● Valid frequency distribution
distribution ● Mode ● Valid percent distribution
● Negatively skewed distribution ● Variance
Computer
analysis ● Normal distribution
reminder Main points
1. Univariate analysis is the analysis of data for a single variable. Univariate analysis
Aids
provides descriptions of variables. Univariate analysis begins with examination of the
frequency distribution of each variable. This tells how many times each attribute occurred.
2. A proportion is a fraction of something. Mathematically, it is expressed as the frequency
(f) for an attribute of interest over the total number of cases, referred to as the N (for the
number). Proportions vary from 0 to 1.00. The formula for a proportion is f/N. A
proportion of 0.5, for example, means that one attribute of a variable makes up half of all
the frequencies for that variable.
3. Percentages are proportions converted to a base of 100. To calculate a percentage, find the
proportion and then multiply by 100. The formula for percentage is f/N(100).
4. A ratio is a proportion of one frequency divided by another and multiplied by a

Chapter 17 Page 23
standardizing index such as 100.

5. A rate is a measure of occurrence of some event. It is similar to a ratio except that a larger
standardizing base such as 1,000, 10,000, or even larger is used to express the rate of
occurrence of an event, such as births in a population.
6. Proportions, percentages, ratios and rates are useful for comparing results from samples of
different sizes. By reducing results to a base of 100 or some other standardizing base,
differences in sizes of samples or population are eliminated.
7. A cumulative frequency distribution shows the sub-totals for each attribute and all
attributes above or below it. A cumulative percentage distribution expresses the same
property for a percentage distribution.
8. Measures of central tendency describe the average or typical occurrence of something. The
three measures of central tendency are the mean, median, and the mode. The mean is the
arithmetical average the sum of scores divided by the number of scores; the median is the
middle score in an ordered distribution (half the scores are above the median and half
below it); the mode is the score that occurs most frequently.
9. Selection of a measure of central tendency depends upon properties of a variable. For
variables measured at the nominal level, the mode is the only measure of central tendency.
Medians can be used for ordinal data, but the mode often is the best choice for measuring
central tendency.
10. The mean is used as measure of central tendency for most interval or ratio variables (age,
intelligence, etc.)
11. Scores at the extreme ends of a distribution affect the mean, but not the median, which is
affected only by the relative position of scores, not their values. When a distribution has a
number of extreme scores at the high or low end, it is said to be skewed. With skewed
distribution, the median is generally a more appropriate measure of central tendency. It is
not pushed in the direction of the extreme scores, whereas the mean is.
12. Another objective of univariate analysis is to learn about the degree of variation or
dispersion among scores making up a distribution. Three measures of variability are used:
the range (R), the variance (s2), and the standard deviation (s).
13. The range is measured by the difference between the highest minus the lowest score plus 1.
Variance is based on the deviation of each score from the mean of a distribution. Standard
deviation is the square root of the variance.
14. A normal distribution or normal curve has precise mathematical properties. It is
symmetrical in shape: If folded at the middle, the left and right halves of the curve would
match perfectly. In a normal distribution, therefore, the mean, median, and mode are
identical. But most important, a fixed proportion of observations lies between the mean and
specific units of the standard deviation.
15. In a normal distribution, approximately 68% of the scores lie within ±1s (standard
deviation) of the mean; approximately 95% lie within ±2s of the mean; and over 99% lie
within ±3s of the mean. Although most distributions for variables do not meet the exact
criteria for a normal distribution, standard deviation is frequently used to describe
variations among scores or other values.


Chapter 17 Page 23

Chapter 18 Page 1

Introduction
Chapter 18. Tabular Bivariate and Multivariate Analyses
Bivariate
Introduction
analysis:
nominal Bivariate analysis is the simultaneous analysis of two variables. It is usually undertaken to see if
variables one variable, such as gender, is related to another variable, perhaps attitudes toward male/female
equality. Multivariate analysis is the simultaneous analysis of three or more variables. It is
Bivariate frequently done to refine a bivariate analysis, taking into account the possible influence of a third
variable on the original bivariate relationship. Multivariate analysis is also used to test the joint
analysis:
effects of two or more variables upon a dependent variable.
ordinal
variables This chapter describes how to carry out bivariate and multivariate analyses by means of
constructing tables. Tables can be used to show bivarate and multivariate relations with any
Statistical variables, whether the variables are measured at the nominal, ordinal, interval or ratio levels .
tests of Also, variables measured at each of these levels can be analyzed by means of statistical tests of
significance. Chapter 19 includes descriptions of a few of the many tests of statistical
association
significance you can use. Any introductory statistics textbook can provide descriptions of
additional tests of statistical significance. So can most of the Web sites we mention in Chapter
Aids
19. We begin this chapter with the simplest form of bivariate analysis. This is the analysis of two
variables measured at the nominal level.
Bivariate analysis: nominal variables
Suppose we wanted to find out if gender was related to attitudes toward equality between men
and women and had measured each variable at the nominal level. The variables are:
Gender - measured as male or female; and
Attitude - measured simply as a "in favor of" or as "opposed to" gender equality.
To test the relationship between gender and attitude, let's develop a hypothesis based on the
following theoretical base:
● Formation of attitudes is based on socialization, which begins in infancy and extends over
many years;
● In all societies, males and females are socialized according to different norms;
● The female subculture generally favors imparting positive attitudes toward gender equality,
whereas the male subculture stresses maintenance of male prerogatives.
This theory of self interest suggests that females are socialized in ways to acquire more favorable
views toward gender equality. We therefore designate gender as the independent variable and
attitudes toward gender equality as the dependent variable for this analysis. On this basis, we can
establish the following hypothesis:
Adult females have more favorable views toward gender equality than adult males.
We will test this hypothesis by constructing a two-way or bivariate frequency distribution. Most

Chapter 18 Page 1
computer-based programs can do this quickly, once all the data are properly coded and entered
into the computer memory. However, we will use hand analysis to show the logic of bivariate
analysis.
PREV NEXT

Chapter 18 Page 2

Introduction
Obtaining two-way frequencies
Bivariate
For this research, imagine that we asked 150 adult men and 150 adult women whether they were
analysis: "in favor of" or "opposed to" women having the same rights as men. After getting these data, we
nominal would first conduct univariate analyses based the frequency distributions for each variable. Then,
variables we would be ready to proceed to testing the relationship between the two variables by looking at
their joint frequency distribution. To start, we need frequencies for of the following
Bivariate combinations:
analysis:
Females in favor of
ordinal Females opposed to
variables Males in favor of
Males opposed to
Statistical
tests of To find these four frequencies, we would first separate respondents by gender, putting all
questionnaires for females in one pile and those for males in another pile. Next, we would divide
association
each of the gender piles into two additional piles, putting all those who answered in favor of and
those who answered opposed to into separate piles. Figure 18.1 shows this process. The numbers
Aids at the bottom of the figure are the two-way frequencies for the cross classification based on
gender and expression of attitudes toward gender equality
Figure 18.1 Flowchart for obtaining frequencies for a bivariate analysis

With these frequencies, we can find the percentages of females and males who gave favorable
and unfavorable responses.
The percentage of females who gave favorable responses is 98/150 = 65%.
The percentage of females who gave unfavorable responses is 52/150 = 35%
The percentage of males who gave favorable views is 63/150 = 42%.
The percentage of males who gave unfavorable views is 87/150 =58%
The next step is to put these percentages into a properly constructed bivariate table.
Constructing a bivariate table
The percentages derived from the two-way sorting are shown in Table 18.1. Each percentage is
shown in terms of both variables. This kind of table is also known as a cross classification or
contingency table because the two variables are crossed with one another in the form of columns

Chapter 18 Page 2
and rows of a table. It is customary to show the independent variable in the columns and the
dependent variable in the rows of the table. In Table 18.1, gender, therefore, is shown in the
columns and the two attitude views make up the rows of the table. The frequency of 98 and the
corresponding 65% for females with favorable views are placed in the cell formed by the
intersection of the column labeled female and row labeled "in favor of." The frequencies and
percentages for the other three combinations are placed in their proper cells as well. In addition,
the column to the right shows the totals for each attitude response for both females and males.
The totals across the bottom show the totals for each gender.
PREV NEXT

Chapter 18 Page 3

Introduction
All bivariate tables have a built-in check. The sum of the frequencies for the "total" column must
Bivariate equal the sum of the "totals" listed across the bottom and each should equal the N for the
analysis: analysis. When these numbers do not equal each other, you have made a mistake somewhere in
nominal creating the table or in adding the column and row frequencies. In Table 18.1, the two
frequencies in the total column, 161 and 139, equal the expected 300. So does the total of 150 for
variables
each gender, as shown in the bottom totals. The sums for this table check out. This is a useful test
to apply to all bivariate tables.
Bivariate
analysis: Table 18.1. Attitudes of females and males toward gender equality
ordinal Attitude Females Males Total
variables Frequency % Frequency % Frequency %
In favor of 98 65 63 42 161 54
Statistical Opposed to 52 35 87 58 139 46
tests of Total 150 100 150 100 300 100
association
Reading a bivariate table
Aids All percentage tables, including bivariate ones, are based on "reading" or comparing differences
among the percentages. In reading a table, we compare percentages for each combination of the
independent and dependent variables. When differences among these combinations are about
15% to 20% or larger, we have reasons to conclude that the independent variable is associated
with the dependent variable. In research short hand, we say there is a relationship between the
two variables.
Looking at Table 18.1, we see that a larger percent of females (65%) than males (42%) expressed
favorable attitudes toward equality between the sexes. This comparison supports the hypothesis
we had formulated; namely, that females have more favorable attitudes toward gender equality
than males.
Reading a table may not always be as easy as illustrated with Table 18.1. As the number of
columns or rows increases, reading a table becomes more complicated. The three guidelines
summarized in Box 18.1, however, should help you read and interpret any table, no matter how
many cells it may have.
Making tables easier to read
In Table 18.1, we presented both frequencies and percentages in order to show how a bivariate
table is constructed. Showing both the frequencies and percentages for each cell, however, is not
necessary. An alternative way of presenting bivariate data is shown in Table 18.2. Notice that
only percentages are shown along with the Ns for each attribute of the independent variable
(gender). With these data, a reader can reconstruct the frequencies for any cell by multiplying the
percentage by the frequency for any attribute. (This is based on a simple conversion of the
percentage formula from % = f/N to f= %(N)).
Table 18.2 Percentages for attitudes toward gender equality by the gender of respondents

Chapter 18 Page 3
Attitude Female Male Total

In favor of 65% 42% 54%
Opposed to 35% 58% 46%
N 150 150 300
PREV NEXT

Chapter 18 Page 4

Introduction
Table 18.2, we think you will agree, is much easier to read. The differences in percentages stand
Bivariate out much clearer without the frequencies; yet all the data a reader needs to understand the table is
analysis: there.
nominal Box 18.1. Guidelines for reading a bivariate table
variables
1. Establish the direction in which the percentages are calculated. In a
properly constructed table, this direction will be shown by the row or
Bivariate column labeled "percent" - in the right-hand column or across the
analysis: bottom of the table. When the percentages are listed in columns, they
ordinal should add to 100 or close to it at the bottom. In this case, the table
variables has been percentaged down, as we did in Table 18.1. When the
totals for percentages are listed in a column at the right, the table has
Statistical been percentaged across rows.
tests of 2. To interpret a table, read in the opposite direction from the way the
association percentages were added. When a table is percentaged down, interpret
the results by comparing across the rows. This is how we interpreted
Aids the results in Table 18.1. We compared the percentages of females
and males who gave favorable responses. These percentages were
found in the same row. When percentages are totaled across rows,
read by table by comparing results down the columns.
3. In tables you construct, we recommend that you percentage down and
then interpret them by comparing across the rows, as we did for
Table 18.1. Readers seem to be able to make comparisons within
rows and across columns rather than the reverse.
Beyond the 2x2 table

Table 18.2 has only two columns and two rows, not counting the total column or the total across
the bottom of the table. A table with only two attributes for each variable is often called a 2x2
(two by two) table. Bivariate tables, however, can be extended, depending on the number of
attributes for either of the two variables. With more attributes, there are more columns or rows or
both. Thus, a table might have 3 or 4 or more columns and as many or more rows. Generally,
readers can quickly grasp tables with 3 or 4 columns and rows, but as the number of columns and
rows increase, tables become more difficult to read and interpret. With more cells, there are just
too many percentages to compare. When there are a large number of attributes, represented by
many columns and rows, it is better to use other measures of association. In the next chapter we
describe some measures you can use.
Bivariate analysis: ordinal variables
With ordinal measurement, the attributes for a variable can be arranged from low to high along
some dimension. Thus, in testing for a relationship between two ordinal variables, we can
determine whether the relationship is:
● Positive as one variable increases, so does the other (persons with more schooling, for

Chapter 18 Page 4
example, tend to earn more money); or

● Negative as one variable increases, the other tends to decrease (females with more
schooling tend to have fewer children)
Positive relationship. Halim (1995) wanted to know if there was a relationship between levels of
education among a sample of women and their attitudes toward holding a job after completing
their education. She created three categories for level of education, secondary, university, and
post-graduate. She also used three categories to measure responses to a question whether women
should work after completing their education. Responses were "strongly agree," "uncertain," or
"disagree." Using these ordinal data, she created the 3x3 table shown as Table 18.3. There was a
clear positive relationship between the level of education of women and their attitudes toward
holding jobs after they complete their education. Only 36% of the women with secondary
schooling expressed agreement with this view while 100% of the women who had a post graduate
level of education did, as did 99% of those with a university or special college education.
PREV NEXT

Chapter 18 Page 5

Introduction
Table 18.3. Levels of education and attitudes toward holding a job after completion of education,
Bivariate in percentages
analysis: Attitude Level of education in percentages
nominal Post-
variables Secondary University graduate Total
(414) (371) (25) (810)
Bivariate Strongly agree 13 13 68 15
analysis: Agree 23 86 32 52
ordinal Uncertain 6 1 0 4
variables Disagree 57 0 0 29
Total 99 100 100 100
Statistical
tests of Source: Halim (1995), The Ahfad Journal: Women and Change, 12:1, p. 9; adapted to illustrate a
positive bivatiate relationship for ordinal variables.
association
Negative relationship. In Table 18.4, we have created another 3x3 table. Cost, which originally
Aids would have been measured at the ratio level has been converted to three ordinal categories. The
same was done for amount of use, which could have been measured in minutes or hours, another
ratio measurement. The percentages in Table 18.4 show a negative relationship between use of
the Internet and cost of service. The negative relationship is shown in the percentages: As cost
increased, the extent of use declined. Where costs were low, 45% of the users were high users
and 20% were low users. For those experiencing high costs, 55% were low users and only 15%
were high users.
Table 18.4. Illustrative negative relationship: Use of Internet and cost of service, in percentages
Level of
Cost
Use
Low Middle High Total
(75) (50) (50) (165)
Agree 20 40 55 34
MIddle 35 35 30 34
High 45 25 15 32
Total 100 100 100 100
We have only touched on ways of expressing relationships in the form of tables. There are a
number of additional ways this can be done. One good source of information on constructing
tables and conducting tests of association is: Measures of Association in Crosstab Tables, this
site describes and shows how to calculate 19 measures of association. Some statistical knowledge
is needed to benefit from all parts of the site
For data measured at the ordinal level or higher, statistical tests of association, such as correlation
analysis, are frequently used to test for association between variables. Correlation analysis is

Chapter 18 Page 5
presented in Chapter 19. However, statistical tests require meeting certain data requirements
such as the normal distribution of a variable in the population from which a sample was drawn.
When this requirement or other requirements cannot be met, analysis in the form of tables will
give you an idea of the degree of association between two variables. As you have seen, tables can
be used as well to test for association between variables measured at any level.
PREV NEXT

Chapter 18 Page 6

Introduction
Mixed variables
Bivariate What do you do when you have a mix of variables measured at different levels? There are many
analysis: ways variables measured at different levels can be analyzed. Variables measured at the nominal,
nominal order, interval and ratio levels can be combined for bivariate or multivariate analysis. The
variables following analyses illustrate a few of the analyses that can be undertaken
We first illustrate some ways interval or ratio data can be analyzed with nominal or ordinal
Bivariate variables.
analysis:
ordinal Creating categories for bivariate comparisons
variables Halim (1995) wanted to determine the relationship between ages of women and their attitudes
toward various roles for women. One of the attitudes she investigated was whether the women
Statistical agreed with, were uncertain, or disagreed with the statement: "The new generation of girls should
tests of be allowed to choose their husbands." To test the relationship between the ages of the women, a
association ratio variable, and their attitudes about choosing their own husbands, an ordinal variable, Halim
placed each women in one of three age categories. With these three categories and three
Aids categories for expression of attitude, she constructed the 3x3 table shown in Table 18.5.
Table 18.5. Attitudes toward girls choosing their own husbands by the ages of women, in
percentages
Ages of Women Total
19-29 30-49 59 and
older
Attitude (541) (231) (38) (810)
Agree 83 88 68 84
Uncertain 12 7 0 11
Disagree 5 5 32 6
Total 100 100 100 101
Source: Halim (1995); The Ahfad Journal: Women and Change, 12:1, p. 19.
Most women, regardless of age, agreed with the statement. Still, much larger percentages of the
women under 50 expressed agreement (83% and 88%) than was true for women who were 50 or
older (68%). At the other extreme, a much larger percentage of the oldest women disagreed
(32%) than did either category of younger women (5% each).
Comparisons based on means
Means can also be used in tables. Hassan and associates (1988) wanted to examine the
relationship between gender and level of schooling on attitudes toward female/male equality.
Three levels of measurement were involved. Gender was a nominal variable. Level of schooling
was measured at the ordinal level by creating two levels: A low level of schooling was defined as
less than a secondary certificate: The high level was defined as having at least a secondary school

Chapter 18 Page 6
certificate. Attitudes were measured at the interval level.

With data in nominal, ordinal and interval forms, Hassan and associates chose to calculate means
for the attitude scores and classify the results in a 2x2 table based on gender and two levels of
schooling. Although Hassan and associates used seven indicators of attitudes, we will report on
only one dimension. This was attitude toward equality in pursuing social activities, which was
measured by five items. Two examples are: "A woman should be able to go places whenever she
wants to" and "A woman should only visit with other women outside her family." (Notice, that
one item represents a positive view and the other a negative one. As we pointed out in Chapter 7
on scale construction, both forms of items are used to obtain more reliable and valid responses).
Responses were scored as 2 for "agree," 1 for "uncertain," and 0 for "disagree." Thus, means
could vary from 0 for disagreement with all five items to 10 for agreement with all the items.
Means for the sample of 30 females and 30 males are shown in Table 18.6 according to their
levels of schooling.
PREV NEXT

Chapter 18 Page 7

Introduction
Table 18.6. Means for attitudes toward female/male equality by the gender of respondents and
Bivariate levels of schooling
analysis: Gender
nominal Level of Schooling Female Male Total
variables Low 6.7 4.1 5.4
High 9.3 6.1 7.7
Bivariate Total 8.0 5.1 6.6
analysis:
ordinal Adapted from Hassan and associates (1988), The Ahfad Journal: Women and Change, 5:1,p.8.
variables The results of the analysis are pretty clear. Even with the small Ns involved, the study provides a
basis for three tentative conclusions. First, at both the low and high levels of schooling, females
Statistical expressed more positive attitudes than males. This is clear from comparison of means within each
tests of level of schooling. Looking across the row for the low level of schooling, the mean for females
association was 6.7 compared to 4.1 for males; at the high level, the means were 9.3 and 6.1, respectively.
Second, for both males and females, means were higher for persons with a high level of
Aids schooling compared to those with a low level of schooling. You can see this by comparing means
down the columns for females and males. For females, the comparison is 6.7 for those with a low
level of education and 9.3 for those with a high level; similar results for males were found, 4.1
and 6.1 at the two levels of schooling. Third, there appeared to be an interaction or joint effect of
gender and level of schooling on attitudes. Females with a high level of schooling had the most
positive attitudes while males with a low level of school had the least positive views.
Because of the small Ns, just 30 for each gender, this study is best considered as an exploratory
study of the possible effects of gender and level of schooling on attitudes toward female/male
equality. Still, the study is useful for two reasons. It shows how means or other statistics can be
arranged in tables just like frequencies or percentages. In addition, as an exploratory study, it
suggests some results that would be worth following up with larger samples.
Multivariate variables
When variables are measured at the interval or ratio levels, statistical measures of association are
the most appropriate way to test for relationships. Since these methods are based on making
statistical inferences, they are explained in Chapter 19. The following discussion shows how
multivariate analyses of ordinal, interval or ratio data can be done by means of tables.
Most multivariate analyses are based on the simultaneous analysis of three variables. While more
than three variables can be analyzed jointly, the analysis becomes complex and usually requires
some form of advanced statistical analysis using a computer based program. We will show how
to do two kinds of three-way analyses. First, let's look at a controlled comparison. In this form of
analysis, we want to see if introducing a third variable changes a relationship between two other
variables.
Controlled comparisons

Chapter 18 Page 7
Controlled comparisons are used to test the association between two variables within specified
levels of a third variable. The third variable, called the control variable, is introduced to see
whether it has any effect on the relationship between the first two variables. An illustration will
show you how to do and interpret a controlled comparison.
Nasr (1986) wanted to know if depression among mothers was related to infant and child
mortality. She obtained data from 50 mothers diagnosed as depressed and from 50
"normal"mothers. Mothers in both groups provided information on the number of babies they had
had and the number of infants and children up to the age of 13 who had died. Based on bivariate
analysis, Nasr found one or more deaths had occurred in 50% of the families with a depressed
mother while this was true in 34% of the families with normal mothers. This result suggested that
depression among mothers was related to higher infant and child mortality. But, as we pointed
out in the discussion of cause and effect, we have to rule out the effects of other factors before we
can claim a specific variable might be the cause of change in another variable.
PREV NEXT

Chapter 18 Page

Introduction
From a review of the literature, Nasr learned that socio-economic status is also related to infant
Bivariate and child mortality. Death rates for these ages tend to be higher at the lower as compared with
analysis: upper socio-economic levels. Therefore, Nasr re-organized her data to see if controlling for the
nominal effect of socio-economic status changed the relationship between depression of mothers and
infant and child mortality rates.
variables
Nasr used the main occupation of the male head of the family as the indicator of socio-economic
Bivariate status. Occupations were classified as "low," "middle," and "high," based on ratings used in the
analysis: Khartoum area of the Sudan. Nasr began by separately sorting the sample of depressed and the
ordinal sample of normal mothers into three sub-samples based on the occupational classification. Each
variables of these six groupings was then divided into two subgroups based on whether or not an infant or
child had died. This process is illustrated in Figure 18.2. In the bottom row, "N" stands for no
deaths and "Y" means at least one death had occurred. The frequencies thus obtained were used
Statistical
to calculate the percentage of deaths that had occurred in the families with depressed mothers in
tests of comparison with families with normal mothers. These percentages are shown in Table 18.7.
association
The purpose of introducing the socio-economic levels of families was to compare infant and child
Aids mortality rates (in percentages) between families with normal and depressed mothers at each of
three socio-economic status levels. In this analysis, occupation was used as a control variable. If
differences in death rates occur between families with normal and depressed mothers within the
same socio-economic level, we could be more confident depression of mothers and deaths of
infant and children was related.
Figure 18.2. Flow chart for obtaining frequencies for a three-way classification
Table 18.7 is constructed to make these comparisons clear. Only the percentages for deaths are
listed. This greatly simplifies the table and makes it much easier to read. Also, the frequencies
upon which percentages are based are provided in parentheses under each percentage. Providing
the Ns for percentages allows readers to judge the quality of the data. With larger Ns, we can
have greater confidence in any percentage or measure of central tendency.

Chapter 18 Page
Table 18.7 clearly shows that depression of mothers is related to increased infant and child
deaths. At each of three occupational levels, a larger percentage of families with depressed
mothers experienced the death of at least one infant or child. At the low level, the comparison is
65% versus 33%; at the middle level, the difference is 44% compared to 33%; and at the high
level, 36% to 29%.
PREV NEXT

Chapter 18 Page 9

Introduction
If the differences in child death rates between the normal and depressed mothers had disappeared
Bivariate for one or more of the comparisons at the three occupational levels, the overall conclusion
analysis: would no longer be valid. Then, we would have to limit our conclusion to the occupational levels
nominal where differences were found.
variables Table 18.7 is a very economical way to present data. When a variable has only two attributes,
like "no deaths" versus "one or more deaths," you don't need to list both sets of data. Listing one
Bivariate set along with the Ns for the percentages will give readers the same information, but much more
analysis: clearly.
ordinal Table 18.7. Percentages of families with normal and depressed mothers with at least one infant or
variables child death by occupational levels of male heads
Occupation Mothers
Statistical
of male
tests of heads Normal Depressed
association 33 65
Low
(n=18) (n=20)
Aids 33 44
Middle
(n=15) (n=16)
29 36
High
(n=17) (n=14)
Total 34 50
However, recalling the discussion of cause and effect in Chapter 3, we cannot say that
depression of mothers is the cause of the greater death rates. At this point, we don't know if the
depression of the mothers is somehow linked to conditions that lead to greater deaths among their
infants and children or whether one or more deaths of their offspring caused or increased
depression among those mothers. Therefore, with the data at hand, we can only conclude that a
relationship exists apart from the effects of socio-economic status. More precise research would
be needed to provide an explanation for the relationship.
Interrelationships among variables
Multivariate analysis can also be used to see how two variables combine in their effect on a third
variable. To illustrate, we will extend the example involving gender and attitude toward gender
equality by adding schooling as a third variable. You already know about the first two variables.
Imagine that schooling was measured as being either "low," meaning 5 years or less or "high,"
meaning 6 years or more of schooling. In this analysis, we will designate gender and schooling as
independent variables and attitude toward gender equality as the dependent variable.
In most three-way analyses, the following relationships are typically examined. These are the:
1. Effect of the first independent variable on the dependent variable. In our example this is
the effect of gender on attitude toward sexual equality.
2. Effect of the second independent variable, in this case schooling, on the dependent variable

Chapter 18 Page 9
(attitude).
Both of these relationships could be obtained from separate bivariate analyses. We repeat them to
show that they can also be derived as part of a multivariate analysis.
3. Effect of the first independent variable (gender) on the dependent variable (attitude) when
the second independent variable (schooling) is controlled.
4. Effect of the second independent variable (schooling) on the dependent variable (attitude)
when the first independent variable (gender) is treated as a control variable.
Relationships 3 and 4 are similar to those described earlier, but in a three-way analysis each
independent variable takes a turn serving as a control variable as well.
5. The relative effect of each independent variable upon the dependent variable - which of the
two independent variables (gender versus schooling) has the greater effect on attitude
toward gender equality.
This is done by comparing the separate effects of the two independent variables upon the
dependent variable. In our example, we want to see whether gender or schooling had a stronger
relationship and, therefore, apparently a stronger effect on attitudes toward gender equality.
Now that you know what we are looking for, let's see how this three-way analysis could be
carried out.
PREV NEXT

Chapter 18 Page 10

Introduction
Illustrative three-way analysis
Bivariate
To get the data for the analysis we extend the sorting we did earlier for the bivariate analysis.
analysis: Starting with all respondents, we first separate them into two groups based on gender (male or
nominal female). Second, we separate each of these by level of schooling (into low and high as described
variables earlier. At this point, as shown in Figure 18.3, we now have four groupings. Each of these is
further subdivided into "favorable" or "opposed" categories, resulting in a total of eight
Bivariate groupings. The frequencies at the bottom of the flow chart are the raw data for the analysis. Using
analysis: these frequencies, we can calculate percentages for favorable and opposed attitudes for each of
ordinal the four groups, female or male by low and high levels of schooling.
variables
Statistical
tests of
association
Aids
Figure 18.3 Flowchart for obtaining frequencies for a three-way multivariate table
The first set of frequencies listed in Figure 18.3 beginning at left show that 34 of the 75 females
with a low level of schooling gave favorable responses while 41 were opposed to gender equity.
At the high level of schooling, 63 females were in favor of gender equity; 12 were not. You can
note similar frequencies for males with low and high levels of schooling. Each of the eight
frequencies was then converted to a percent so that comparisons could be made among responses.
The percentages are:
Females, low schooling, in favor of = 34/75 = 45%
Females, high schooling, in favor of = 63/75 = 84%
Females, low schooling, opposed to = 41/75 = 55%
Females, high schooling, opposed to = 12/75 = 16%
Males, low schooling, in favor of = 18/75 = 24%
Males, high schooling, in favor of = 45/75 = 60%
Males, low schooling, opposed to = 57/75 = 76%
Males, high schooling, opposed to = 30/75 = 40%
PREV NEXT

Chapter 18 Page 10

Chapter 18 Page 11

Introduction
Now that we have percentages for each subgroup, our next step is to transfer them to a properly
Bivariate constructed table. Table 18.8 shows the results of this step. In constructing this table we kept it as
analysis: simple as possible. First, only percentages for favorable responses are listed. We can do this
nominal because we have only favorable as contrasted to opposed views. Therefore, we need only to
report the percents for only one group or the other. The other percent in each case can be derived
variables
by subtracting the percentage provided from 100. Second, following the practice used previously,
the frequencies used in calculating the percentages are shown in parentheses below each
Bivariate percentage.
analysis:
ordinal Table 18.8. Percentages for favorable views toward gender equality by level of schooling and
variables gender
Level of schooling
Statistical Gender Low High Total
tests of 45 84 65
Female
association (75) (75) (150)
24 60 42
Male
Aids (75) (75) (150)
35 72 53
Total
(150) (150) (300)
Reading multivariate tables

The rules we gave earlier for reading a bivariate table have to be modified in reading Table 18.8
because only favorable views are given, there is no direction in the percentaging. Only half of the
percentages are listed and these are given in terms of the two independent variables. Still, the
table can be read by comparing either the percentages in each row going down the columns or in
each column or comparing by looking across the rows, depending on which effect one is looking
for.
In this way, Table 18.8 can be read to see the results for the six tests of relationships we listed
previously. The first relationship concerned the effect of gender on attitudes. This effect is seen
under the total column at the right of the table. These percentages are for all males and all
females, disregarding their levels of schooling. The percentages of favorable responses for
females (65%) exceeded that for men (42%); consequently, we can conclude, as before, that
females have more favorable attitudes toward gender equality than men.
The second relationship was the effect of schooling on attitude. This relationship is found by
comparing percentages for all respondents, males and females combined, having a low level of
schooling in comparison with those having a high level of schooling. The percentages for this
comparison are found in the row labeled "Total" across the bottom of the table. Thirty-five
percent (35%) of those having a low level of schooling gave favorable responses in comparison
with 72% of those with a high level of schooling. We can conclude, therefore, that schooling is

Chapter 18 Page 11
related to gender and specifically that favorable views are associated with a higher level of
schooling.
We now know that both independent variables are related to the dependent variable. But this is
only the beginning of the analysis. We are now prepared to conduct more powerful analyses that
go beyond what is possible with bivariate analysis.
Third, we can see whether gender remains associated with attitude responses when the effect of
schooling on attitude is taken into account. This is done by comparing percentages for favorable
responses between males and females who have the same level of schooling. Looking first at the
low level of schooling, we see that 45% of the females and 24% of the males gave favorable
responses. This says that among persons with low schooling a greater percentage of females had
favorable attitudes. The same result is seen at the high level of education: 84% of the females in
comparison with 60% of the males expressed favorable attitudes. Thus, with schooling
controlled, the hypothesis that women have more favorable attitudes toward gender equality than
men still holds.
PREV NEXT

Chapter 18 Page 12

Introduction
Fourth, we can also see if differences in attitudes associated with level of schooling remain when
Bivariate gender is controlled. For this comparison we need to examine percentages for favorable responses
analysis: between females having a low and high level of schooling. Among females the percentages for
nominal favorable responses were 45% for the low and 84% for the high level of schooling. We also need
to do the same comparison separately for males. Looking again at Table 18.8, we see that 24% of
variables
the males with a low level of schooling gave favorable responses in contrast to 60% of the males
with a high level of schooling. These results indicate that apart from the gender of the
Bivariate respondents, level of schooling is associated with attitudes toward gender equality. Thus, we can
analysis: conclude that for both males and females high levels of schooling are associated with more
ordinal favorable attitudes toward gender equality.
variables
At this point in our analysis we can further conclude that both gender and level of schooling are
independently related to attitudes toward sexual equality.
Statistical
tests of Fifth, we can determine which of the two independent variables, gender or schooling, had the
association stronger effect on attitudes. To do this, we compare two sets of percentages:
1. The differences in percentages for favorable responses apparently due to schooling,
Aids which are;
Schooling Females Males
High 84 60
Low 45 24
Difference 39 36
2. The differences in percentages for favorable responses apparently due to gender,
which are;
Gender Low schooling High schooling
Females 45 84
Males 24 60
Difference 21 24
These results show that differences in attitude due to differences in levels of schooling were
greater than differences due to gender. On this basis, we conclude that schooling apparently had a
stronger effect than gender on attitudes toward gender equality than did the gender of the
respondents.
Finally, we can examine how the two independent variables jointly affect the dependent variable.
For this analysis all four percentages based on both gender and schooling are compared. Looking
at these percentages, it seems that the two variables act jointly and consistently on the attitudes
expressed by the respondents. Males with a low level of schooling had the least favorable views
(24%), next came females with a low level of schooling (45%), followed by males with a high
level of schooling (60%); while females with a high level of schooling expressed the most
favorable attitudes (84%).
As you can see, multivariate analysis yields a large amount of useful information, far more that
can be obtained from univariate or bivariate analyses based on the same variables. But large

Chapter 18 Page 12
samples are required to do multivariate analyses. In our illustration, we assumed a stratified

sample based on 75 cases in each of four strata (two levels of schooling and by gender), resulting
in a total sample of 300 respondents. If used in actual research, this sample design and the
frequencies involved would probably have produced enough responses in each subgroup to
support any conclusions drawn from the findings. With smaller simple random samples of 50 or
even 100, multivariate analyses generally cannot be done because the frequencies in the various
cells are too small to support any conclusions that might be drawn
PREV NEXT

Chapter 18 Page

Introduction
Statistical tests of association
Bivariate
There are many ways to test for the association between variables. We have focused on use of
analysis: tables because this approach is relatively easy to understand and apply. Further, tables can be
nominal prepared entirely by hand, without the need for computer support. Sometimes tabular
variables presentation is not the only or even the best way to test for associations among variables. In the
next chapter we present several ways of conducting statistical tests of association.
Bivariate
Aids
analysis:
ordinal Key terms
variables
● Bivariate analysis ● Cross classification
Statistical ● Cell ● Multivariate analysis
tests of ● Contingency table ● Statistical inference
association ● Control variable ● Statistical test of significance
● Controlled comparison
Aids
Main Points
1. Bivariate analyis is the simultaneous analysis of two variables. It is used to test for the
association of an independent and dependent variable. Bivariate analyses can be completed
by constructing two-way tables with columns for one variable, usually the independent
variable, and rows for the dependent variable. Bivariate relations can also be tested using
statistical measures of association.
2. The way bivariate analyses are done depends on the level of measurement used for each
variable. Different methods are used for variables measured at the nominal, ordinal, and
interval or ratio levels.
3. For nominal or ordinal variables, cross classification or contingency tables generally are
used to present results.
4. Results in bivariate tables are usually expressed in percentages. To read percentages, first
determine the direction in which the percentages were calculated either down the columns
or across the rows - and then compare percentages in the opposite direction. If the
percentages add to 100% or close to this going down the columns, compare percentages
across the rows; do the reverse if percentages add across the rows.
5. In bivariate tables based on ordinal measurements, the direction of association can be
interpreted as positive or negative.
6. Multivariate analysis is the simultaneous analysis of three or more variables. It is
frequently used to see whether a relationship between two variables remains when a third
variable, called the control variable, is taken into account.
7. Multivariate analyses are also done to determine the separate and joint effects of two
variables upon a dependent variable.

Chapter 18 Page


Chapter 19 Page 1
Chapter 19: Performing Inferential Statistical Analyses

Samples
and Samples and populations
populations Most research is based on data collected from samples drawn from large populations. Frequently,
the sample represents only a small proportion of a population. Nevertheless, as we will show in
Probability this chapter, we can use statistics drawn from a sample to estimate the corresponding parameter
theory and in a population. We do this by making a statistical inference based on probability theory.
statistical
This chapter explains the logic behind making statistical inferences and describes a few of the
inferrence
statistical tests used in making inferences. We start with a review of the relations between
samples and populations.
Inferring a
population A sample, you will recall, is a randomly selected part of target population. Using random or
mean chance selection of the sampling elements, and nothing else, allows us as researchers to indicate
how well our sample represents the target population. Analysis of data from samples produces
Tests of statistics. Statistics are used in two ways. First, we use statistics to describe the variables we are
studying. Descriptions are usually based on the mean or some other measure of central tendency
statistical
and the standard deviation for a variable. Statistics are also used to describe relationships among
significance variables. The second use of statistics is to estimate values for variables in the population from
which a sample was drawn. The values of variables in a population are referred to as parameters.
Tests of Because we start with samples, we seldom know the values of parameters for the variables we
differences analyze. By using procedures based on inferential statistics, however, we can use statistics from a
between sample to estimate the corresponding values of parameters in a population.
means
Whether you make statistical inference or not depends on your research objective. If you are
conducting an exploratory study to get ideas for a larger study, you won't be interested in
Coefficient estimating any parameters. Calculation of summary measures limited to describing variables so
of you better understand them, such as means or percentages, will probably be all you need. Also,
correlation many descriptive studies are carried out to identify relationships between variables within the
samples without making estimates of corresponding parameters. Most of the research we cited in
Caution previous chapters did not involve making statistical inferences. But there may be times when you
with want to provide an accurate estimate of the parameter for some variable, perhaps the number of
association children born to some population of married women. In this chapter, we show you how you can
use statistics from a sample to do this. A mean and the standard deviation for a variable, for
Chi square example, can be used to estimate the mean for that variable in the population from which you
selected your sample. Estimating parameters requires understanding of probability theory.
Other Probability theory and statistical inference
tests of
Probability theory is based on the occurrence of random or chance events. It provides theoretical
significance
distributions that occur when events occur due to chance and chance only. When we select a
probability or random sample, we rely entirely on chance to produce a sample that is
Caution in representative of the population we want to learn about. Each random sample we select will
using contain a different combination of respondents or whatever we are sampling. If we drew a very
statistical large number of samples, the variation in means or other statistics would form a predictable

Chapter 19 Page 1
test results pattern entirely by chance. We can use this known, predictable pattern, resulting from random
variations that occur from one sample to the next, to estimate population parameters. The
Aids following illustration shows how we can use sampling variation as a basis for making statistical
inferences.
PREV NEXT

Chapter 19 Page 2
Our illustration is based on 25 workers in an office.Table 19.1 shows the number of days each
Samples worker was absent from work over the past two years.
and
populations Table 19.1. IIllustrative distribution of days absent for a population of 25 workers
Worker Days Worker Days Worker Days
Probability number absent number absent number absent
theory and 01 10 10 11 18 12
statistical 02 8 11 13 19 10
inferrence 03 10 12 9 20 9
04 12 13 12 21 14
Inferring a 05 9 14 13 22 8
population 06 11 15 10 23 13
mean 07 10 16 11 24 10
08 14 17 7 25 11
Tests of
09 6
statistical
significance The total number of days absent for all 25 workers was 263 and the mean was 10.5 days. The
mean of 10.5 is the parameter. It is the mean of the population. It is sometimes represented by a
Tests of Greek letter µ (mu). In most studies, we do not know the population mean or any other
differences parameter. We have only the mean for the sample we are analyzing. In this illustration, we start
between by showing the population mean so that you can see how sample variation occurs and its effect
means on any statistic based on sample data. We show this effect by calculating a series of means based
on independently selected random samples from the population of 25 workers.
Coefficient To illustrate sample variation, let's draw three random samples of five workers each and calculate
of means for each sample. Using a table of random numbers, we got the three samples listed in
correlation Table 19.2. (See Chapter 8 for guidance in using a table of random numbers.)
Table 19.2. Means based on three randomly selected samples

Caution
with Sample 1 Sample 2 Sample 3
association Worker Days Worker Days Worker Days
Number absent number absent number absent
Chi square 06 11 20 9 11 13
24 10 09 6 24 10
Other 19 10 12 9 16 11
tests of 10 11 19 10 08 14
significance 17 7 05 9 01 10
X 49 43 58
Caution in M 9.8 8.6 11.6
using
None of the sample means equals the population mean. Two are less than and one is greater than
statistical the population mean. Why? The variation in sample means was caused solely by the operation of

Chapter 19 Page 2
test results chance in the random selection of workers who comprised each sample. In each sample, each
worker had 1 chance in 25 or a probability of .04 of being selected. In the first sample we
Aids happened to select by chance 5 workers whose scores added to 49; for the second sample,
workers whose scores totaled 43; and for the third sample, workers whose scores totaled 58. If we
selected additional random samples we would find further variation in sums and means, solely
due to the operation of chance.
PREV NEXT

Chapter 19 Page 3
Inferring a population mean

Samples
and With all this variation in sample means, why do researchers use random selection of samples?
populations Doesn't this lead to confusion? No, on the contrary, the only way we can use a statistic to
estimate a parameter is by relying on known, predictable patterns of variation resulting from
Probability random selection of samples. An illustration should help you understand what how this is done.
Imagine that we selected a large number of successive samples of 100 each from a population of
theory and
5,000 households. In addition, let's say that we obtained the number of persons in each household
statistical and calculated a mean for each sample. With these data, we could construct a frequency
inferrence distribution of means, just as we did earlier with scores, but now the results are means. This
distribution is known as the sampling distribution of the mean or simply as the sampling
Inferring a distribution. It tells us how many times the values for various means occurred. If we actually did
population this, we would find that most means clustered near the center of the distribution with fewer and
mean fewer out toward the low or high end of the distribution. In fact, the sampling distribution would
take the form of a normal distribution, like the one described at the end of Chapter 17. All the
Tests of properties described earlier for a normal distribution apply to sampling distributions as well.
statistical Let's go over that point again because it is so important for understanding the process of making a
significance statistical inference. We are referring to a distribution of means. The distribution of means from
our many samples would take the form of a normal distribution. Further, the mean of the
Tests of distribution the mean of the means would be as close as we could get to knowing the value of
differences the mean of the population without conducting an enumeration of the population. Selecting all the
between necessary samples to prove this point, however, is obviously impossible. Fortunately, for us as
means researchers, it is also unnecessary. We can rely on the probability theory and statistical formula
estimating a population mean from just one sample. We can do this because the sampling
Coefficient distribution, like any distribution, has a standard deviation. Also, like any standard deviation, the
standard deviation of the sampling distribution indicates how the means from the many samples
of
are distributed around the mean of the distribution. The standard deviation of the sampling
correlation distribution is referred to as the standard error of the mean or simply as the standard error,
which is represented by the letters S.E.. Incidentally, standard error has nothing to do with errors
Caution made in measuring variables. It refers solely to variations among sample means arising from the
with chance variations that always occur in random selection of samples.
association
Now, here comes a very important point: The standard error has the properties of a normal
distribution. Approximately 68% of the means in a sampling distribution will lie within ±1
Chi square
standard error of the mean of the sampling distribution; 95% will lie within ±2 standard errors;
and over 99% will lie within ±3 standard errors. You might say this nice to know, but how does it
Other help us when we have only one mean and one standard deviation from our single random sample.
tests of Here is the beauty of the underlying mathematics. We can use statistics from just the one sample,
significance provided a random sample was used, to estimate the value of the mean in the population from
which the sample was drawn.
Caution in
An illustration should help make this point clear. Let's say that analysis of household sizes in our
using
random sample of 100 households produced a mean of 6.30 persons. Also, imagine that the in
statistical calculating the variance for the sample we found the following:

Chapter 19 Page 3
test results
£(X -M)2= 334.900
Aids To complete the calculation of the variance of the sample, we would divide the numerator, called
the "sum of squares" by N. To get the standard deviation for the sample, we extract the square
root of the variance. As a step toward the calculation of the standard error (S.E.), we need to
obtain an estimate of the standard deviation of the variable we are examining in the population
from which the sample was selected. This result is represented by the symbol of ^s, called "s hat."
The formula for s hat is similar to the one for standard deviation, except for the change to N-1 in
the denominator:
With these data, we can estimate the population mean. We know that each sample mean will
differ from the population by some amount. In estimating a population mean, however, we have
only one sample mean. Because of variations that occur from one random sample to the next, we
know that this single mean is only one estimate of the population mean. Absent any other
information, however, this single mean is our best estimate of the mean for the size of households
in the population. Fortunately, the extent of variation among sample means can be estimated,
using the standard deviation for the variable we are analyzing. In sampling, this random variation
among sample statistics is referred to as sampling error. When it is used to estimate a parameter,
it becomes standard error.
PREV NEXT
This discussion of the basis for inferring a population mean is based on the excellent presentation
by Dr. Trochim: at: Trochim, W. (2005). Selecting Statistics.Retrieved June 7, 2005.

Chapter 19 Page 4
We now have the two values needed for estimating the population mean: The sample mean, 6.30,
Samples and the estimate of the standard error of the mean, 0.184, persons. Before taking the next step, a
and decision has to be made, and this requires understanding a new concept. Earlier we indicated that
populations for a normal distribution, about 68% of the scores lie within ±1 standard deviation (s) of the
mean; that about 95% lie within ±2 s of the mean and that about 99% lie within ±3 s of the mean.
Probability These relationships hold as well for the standard error of the mean. We use these properties of the
theory and standard error when we use a sample mean to estimate a population mean. Generally, either ±2s
statistical or ±3s is used and the result is expressed in terms of the 95% or 99% level of confidence,
inferrence respectively. The level of confidence we select says how confident we want to be that the mean in
the population that we are estimating is within a certain distance from the sample mean.
Inferring a Once the confidence level is set, the population mean can be estimated. However, because of
population sampling error, we can only specify an interval in which we think the population mean will lie.
mean We start by assuming that the mean of the sample is the best estimate of the mean of the
population. Next, using one of three confidence levels, we calculate the confidence interval for
Tests of the parameter.
statistical ● At the 68% level of confidence, we would estimate that the population mean, represented
significance by the Greek letter mu, µ, lies within ± 1 S.E. or between 6.30 ± 0.184 which equals 6.13
to 6.48.
Tests of ● At the 95% level of confidence, we would estimate that µ lies within ± 2 S. E. or between
differences 6.30 ± 2(0.184) = 6.30 ± 0.376 = 5.92 to 6.68.

between ● At the 99% level of confidence, we would estimate that µ lies within ± 3 S.E. or between
means 6.30 ± 3(0.184) = 6.30 ± 0.552 = 5.75 to 6.85.

To attain a higher level of confidence, we have to accept a wider or less precise confidence
Coefficient interval. But the resulting interval can be expressed with greater confidence. At the 99 percent
of level of confidence, we would expect in 99 samples out of 100 µ would lie between 5.75 and 6.85
correlation persons. In only 1 sample out of 100 would µ be expected to be less than 5.75 or greater than
6.85.
Caution
with Increasing the precision of estimates
association
The precision of an estimate refers to the width of the interval for estimating a mean. A narrower
interval provides a more precise estimate. The width of the interval and hence the precision of the
Chi square
estimate depend on two factors: (1) the size of the standard deviation of the sample; and (2) the
size of the sample. The size of the standard deviation also depends in part on the size of N.
Other Remember that the formula for the standard deviation is the square root of the square of the
tests of deviations of the scores from the mean divided by the number of cases involved. Thus, increasing
significance sample size, N, reduces the standard deviation of a sample as well as the standard error of the
mean. A smaller standard error results in a narrower confidence interval. This, in turn, leads to a
Caution in more precise estimate of the population mean or any other parameter.
using
The reason for this is the influence of extremely high or low scores on the mean. Increasing
statistical sample size reduces this effect. With smaller samples, extremely high or low scores influence the

Chapter 19 Page 4
test results mean and the standard deviation more than in larger samples. With larger samples, each score,
regardless of its size has less influence on the mean. Generally, therefore, means based on smaller
Aids samples show greater variability from one sample to the next. But as sample size increases, the
distribution of means will show less variability because the means will tend to be more alike.
Also, with larger samples each sample has a greater chance of representing the population. With
larger samples, each mean will be closer to the population mean, and, therefore, will show less
sampling error and have a smaller standard error. As a result there also will be less variability
among means based on larger samples. Consequently, the standard deviations of samples
generally are smaller with larger samples, and this reduces the sizes of the estimates of the
standard error. As these estimates decline, confidence intervals becomes smaller, allowing for
more precise estimates of population means or any other parameters.
PREV NEXT

Chapter 19 Page 5
Tests of statistical significance

Samples
and Analyses are frequently done to see if there is a relationship between two variables. Almost
populations always some relationship is found. The question then is whether the observed result is large or
strong enough to say that there is a relationship between the variables in the populations from
Probability which the samples were drawn. In making this decision you have to choose between two
explanations. One explanation is that the relationship is the result of chance variations in the
theory and
selection of the sample and, therefore, the relationship found in the sample data does not describe
statistical a true relationship between the variables in the population. The other explanation is that the
inferrence relationship is too strong to be explained by sampling error and, therefore, most likely reflects a
genuine relationship between the variables in the population. A statistical test of significance is
Inferring a used in choosing between these two explanations. A test of significance is based on testing the
population null hypothesis.
mean
Tests of
Logic of the null hypothesis
statistical Well-established procedures are used in making this choice. The first step is to state a null
significance hypothesis which is the standard statistical way of saying that there is no relationship between
two variables or that there are no differences between two measures of central tendency. Forms of
Tests of the null hypothesis are:
differences
There is no relationship between variable X and variable Y. For example, years of schooling for
between females is not related to their fertility.
means
There is no difference between the means or percentages in one sample as compared to another
Coefficient sample. For example, there is no difference in means of males and females for attitudes toward
of gender equality.
correlation After a null hypothesis is formulated, it is subjected to a statistical test of significance. This
process is called hypothesis testing. Later in this chapter we describe a few of the many tests of
Caution significance available to social researchers. On the basis of the statistical test of significance, the
with null hypothesis is either accepted or rejected. When a researcher decides that the difference or
association relationship found could be due to chance variations, the null hypothesis is accepted. In this case,
the choice is that the independent variable, for example, had no effect on a dependent variable or
Chi square that two means are not statistically different. When the result of a statistical test suggests that the
difference or relationship is too large to have occurred due to chance, the null hypothesis is
rejected. In that case, we conclude that the difference or relationship we observed could not have
Other
occurred due to chance. The finding is assumed to reflect a real relationship or a real difference
tests of between two variables in the population.
significance
You may wonder whatever happened to the idea of a hypothesis established at the beginning of a
Caution in study. When a researcher decides to use a statistical test of significance, the fate of the original
using hypothesis depends on the outcome of the test of the null hypothesis. The original hypothesis
becomes an alternative hypothesis: that is, it is the alternative to be considered only if the null
statistical

Chapter 19 Page 5
test results hypothesis is rejected.
Aids The null and alternative hypotheses

The original research hypothesis, formed at the beginning of a study, states what the investigator
expects to find. Usually this hypothesis expresses a positive or negative relationships between
two variables or that a difference exists between two samples with respect to some variable.
When the null hypothesis is accepted, the original or now called the alternative hypothesis is
rejected. Acceptance of the null hypothesis indicates that there is not a sufficient basis for
accepting the alternative hypothesis. When there is a strong enough basis for rejecting the null
hypothesis, the alternative hypothesis established initially to guide the research, can be accepted.
All statistical tests of significance work on this reverse way of thinking. On the basis of
probability theory, we can only give the odds or chances that a finding is not due to chance. We
cannot say absolutely that a relationship between two variables exists; we can only give evidence
that it is unlikely that the relationship was due to chance. Statistical tests of significance are used
in deciding whether to accept or reject the null hypothesis. They allow us to express results in
terms of levels of confidence.
To repeat, in statistical analysis, we can never prove anything. Instead, we offer data that cast
doubt on the negative argument and, therefore, provide a sufficient basis for reaching the opposite
conclusion that there is a relationship between two variables. If you do not understand this point,
you will not understand any of the rest of this chapter. This kind of thinking takes getting used to,
but by the end of this chapter you should master it.
PREV NEXT

Chapter 19 Page 6
Samples Probability and confidence levels

and
Accepting or rejecting the null hypothesis is a calculated gamble. This decision rests on assessing
populations
the probability or odds that favor one choice over the other. Over the years, statisticians and
researchers have accepted two probability levels, referred to as "p" levels, as guides in making
Probability this decision. One, a p level of .05 indicates that the observed finding could have occurred due to
theory and chance variations in sample selection in less than 5 times out of 100. Researchers accept the .05
statistical or 5% level of confidence as the minimum level for rejecting a null hypothesis. We assume,
inferrence gamble if you like, that the result we found in our one sample is not one of the 5 instances that
could have occurred due to chance. We therefore reject the null hypothesis. When a p level is
Inferring a greater than 5%, the null hypothesis is accepted. In this case, we are unwilling to gamble that
population what was found was not due to random sample variation. We say, in effect, that the odds or
mean probability favor the result as having occurred due to chance and that there is no relationship or
difference among variables in the population.
Tests of When results fail to reach the 5% level of confidence, they are reported as nonsignificant The
statistical short hand version of nonsignificant, which you will see in research reports, is "ns."
significance
Some researchers prefer to use the .01 level of significance for accepting or rejecting null
hypotheses. This criterion indicates that a finding could have occurred due to chance less then 1
Tests of time in 100. The .01 level is a stronger basis for rejecting a null hypothesis than the .05 level.
differences Results at the .05 and .01 levels are referred to as being "significant." As you review literature,
between you may see reference to even higher levels of significance, such as the .005 or .001 levels of
means significance. The .001 level indicates that a finding could have occurred by chance only in 1
sample out of 1,000. Probability levels this low are described as being "highly significant."
Coefficient
of Steps in conducting a statistical test of significance
correlation
Three steps are involved in conducting any statistical test, including the t test. These are:
Caution ● Formulate the null hypothesis and, if you did not do it before, do the same for the
with alternative hypothesis;

association ● Conduct the statistical test; and
● Interpret the results based on the critical value found.

Chi square
The purpose of most research is to test for relationships among variables. Research can also focus
on identifying differences between groups. Obviously, different tests are used for establishing
Other relationships as compared with discovering differences between groups. Also, regardless of the
tests of hypothesis, different tests are used for continuous as opposed to discrete data.
significance
Selecting the right statistical test can be confusing. Fortunately, a lot of easy-to-use help is
Caution in available through the Internet. Selecting Statistics, is an excellent starting point. This site begins
using by asking you whether you are analyzing one variable, two variables, or more than two variables.
It then asks you to answer additional questions which should lead you to select the right way to
statistical

Chapter 19 Page 6
test results analyze your set of data. If this site does not provide the help you want, do a search on Google or
some other search engine using search terms like "selecting statistical tests." You will find
Aids additional sites that provide help with selecting statistics.
We will start by explaining how to conduct a test for the statistical significance of a difference
between two means following the three steps listed above.
Tests of differences between means

Means are used to summarize distributions based on continuous data (interval or ratio
measurement). A statistical measure called the t test is used to test for the significance of the
difference between two means. The t test assesses the degree of overlap in the distribution of
scores in each of two samples being compared. When the two distributions are highly similar,
there will be little difference between the means. The t test, therefore, will produce a high p
value, meaning that any difference found can be safely attributed to chance variations in the
selection of the samples. When scores in one distribution are distributed differently from the
other, there is a greater probability that the difference between the means will be greater. As the
difference increases, the p value will be lower, making it less unlikely that any difference found
was due to chance.
A t test can be used with large or small samples. However, as the sample size becomes smaller,
mean differences have to be larger to become significant. In addition to the requirement of
continuous measurement, the t test assumes that the variable being measured is normally
distributed in the population from which the sample was selected. Even when distributions for
samples are mildly skewed, it may be reasonable to assume a normal distribution for the variable
in the population. However, when the distribution for a sample is badly skewed or you doubt that
the variable is normally distributed in the population, you should not use a t test. As an alternative
you can compare medians or convert continuous data to a set of intervals and conduct a chi
square test. We describe how to do perform a chi square calculation later in this chapter.
PREV NEXT

Chapter 19 Page 7
Samples Formulating the null and alternative hypotheses

and
For the t test, the null hypothesis takes the form that two population means, represented by µ 1for
populations
the mean of one population and µ 1for the mean of the other population, are the same. The
Probability hypothesis can be expressed as:
theory and µ 1 = µ 2 or that µ 1- µ 2= 0
statistical
inferrence We will work through a simple illustration of a t test. Suppose we asked a random sample of 10
employees in Office A and another 10 in Office B to rate their morale on a 10-point scale. Now
Inferring a suppose, in addition, that the manager of Office B had conducted a special program to improve
morale. To find out whether his efforts improved morale, the manager persuaded the manager of
population
Office A to collect similar ratings for morale so there would be a basis for testing the effect of the
mean morale improvement program in Office B. In Table 19.3, the number of days absent are shown
under the column headed X1 for workers in Office A and under X2 for workers in Office B. (In
Tests of
practice, we would not use a sample as small as 10. We are using a small number to keep the
statistical calculations simple for this illustration). For this test, the null hypothesis is that there is no
significance difference in means for ratings of morale in the two offices. The alternative hypothesis is that the
mean for Office B will be larger than the mean for Office A. This hypothesis is based on the
Tests of assumption that the morale improving program would have a positive result. In addition, we need
differences to set the level of significant to use in choosing between the two hypotheses. Let's say we select
between the .05 level.
means Table 19.3. Illustrative calculations needed for a t test
Coefficient Office A Office B

of X1 X12 X2 X22
correlation 4 16 7 49
8 64 5 25
Caution 5 25 6 36
with 2 4 7 49
association 4 16 6 36
6 36 7 49
Chi square 5 25 5 25
7 49 4 16
Other 8 64 7 49
tests of 5 25 8 64
significance 54 324 62 398
N 10 10
Caution in
M 5.4 6.2
using
statistical At first glance, the manager of Office B would be pleased to see that the mean morale score for

Chapter 19 Page 7
test results the sample of his workers was higher than that for the control group of workers. But we know
that variations in scores and means occur among samples. The question therefore still remains: Is
Aids the difference of .8 on a 10-point scale enough to support rejection of the null hypothesis? The
answer lies in the result of the t test, and for this we need to do some
PREV NEXT

Chapter 19 Page 8
Samples Calculating the t test

and
The formula for calculating a t test is:
populations
Probability
theory and
statistical
inferrence
Inferring a
population
mean
Tests of
statistical To perform the calculation, we need to obtain the following numbers for each sample. For Office
A, the preparatory calculations are:
significance
● The mean = 5.4
Tests of ● The sum of the scores = X 1 = 54
differences ● The total of the scores squared = (X 1) 2 = 54(54) = 2,916
between ● The sum of the squares of the scores = X 1 2 = 324

means ● The N = 10
Coefficient After completing the same calculations for Office B, we can insert the results in the formula and
of solve for t as follows:
correlation
Caution
with
association
Chi square
Other
tests of
significance
Caution in
using
statistical

Chapter 19 Page 8
test results
Aids
The result, 1.119, is the t ratio for the test of difference between the two means. The next and
final step is to interpret this value and to decide whether to accept or reject the null hypothesis
PREV NEXT

Chapter 19 Page 9
Samples Calculating the t test

and
The formula for calculating a t test is:
populations
Probability
theory and
statistical
inferrence
Inferring a
population
mean
To perform the calculation, we need to obtain the following numbers for each sample. For Office
Tests of A, the preparatory calculations are:
statistical ● The mean = 5.4
significance ● The sum of the scores = X 1 = 54
● The total of the scores squared = (X 1) 2 = 54(54) = 2,916

Tests of
● The sum of the squares of the scores = X 1 2 = 324
differences
● The N = 10
between
means After completing the same calculations for Office B, we can insert the results in the formula and
solve for t as follows:
Coefficient
of
correlation
Caution
with
association
Chi square
Other
tests of
significance
Caution in
using
statistical

Chapter 19 Page 9
test results
Aids
The result, 1.119, is the t ratio for the test of difference between the two means. The next and
final step is to interpret this value and to decide whether to accept or reject the null hypothesis.
PREV NEXT

Chapter 19 Page 10
In t tables, the degrees of freedom are listed down the left hand column. In the table we used, t
Samples
critical values, we went down the column to df = 18 and then read across the row to the right to
and
find the .05 level, which we had set as our criterion for accepting or rejecting the null hypothesis.
populations With 18 degrees of freedom, a t ratio of 2.101 (under a nondirectional test, called an alpha
two-tailed in this table) is needed to support rejection of the null hypothesis of the .05 level. The t
Probability ratio for our illustration was 1.119, less than the critical value of 2.101. This means that the
theory and difference we found could have occurred due to chance more than 5 times out of 100. We
statistical therefore accept the null hypothesis and conclude that morale in the two offices is not really
inferrence different. By this test, a difference as large as we found between the means of 6.2 and 5.4 could
have occurred due to chance differences in sample selection more than 5 times out of 100 sets of
Inferring a samples. The alternative hypothesis that morale is higher in Office B than in Office A is
population therefore rejected.
mean Results of a t test are reported in the text of a report or in a line immediately below a table
containing the means being analyzed. Our illustration produced a non-significant result. This
Tests of result would be reported as:
statistical
t = 1.119 (df=18) p >.05
significance
which says that the t ratio of 1.119 with 18 degrees of freedom was less than the value needed to
Tests of achieve the .05 level of significance. The inequality sign, >, means "greater than." As shown
differences above, it says the p value found is greater than the .05 critical value for df = 18. If the df you need
between isn't listed, use the df closest to yours. For example, if your df = 50, you could use the value listed
for df = 40. Also, in any t table you will see that the sizes of critical values of t at any level of
means
significance decrease as the degrees of freedom increase. At the .05 level, for example, with 10
df, the critical value of t is 2.228, but at df =60, the value is 2.000.
Coefficient
of Before leaving the t test, we want to point out why three decimal places were used in the
correlation calculations. In calculating means, one or at most two decimal places generally are sufficient.
Three places were used in these calculations because the critical values of t are expressed at this
Caution level.
with Web-based calculation of t
association
But why do the calculations for t when you don't have to? There are sites that calculate t for you.
For example, t test calculator provides instructions for entering data for the scores in each of the
Chi square
two groups you are comparing, for selecting the right t test for the data you are analyzing, and
then does the calculation for you. It provides the means and standard deviations for each group
Other
and other information you will need for interpreting your results. For small samples, the t test
tests of calculator is easy to use and produces fast, error-free, accurate results. The same is true for other
significance Web-based calculators we mention later.
Caution in Other tests of mean differences

using The t test we just described is used when the two samples are independent of one another. This is
statistical the case when a sample is drawn from one population, like Office A in our illustration, and

Chapter 19 Page 10
test results another sample is drawn from a separate office (Office B). Samples can also be non-independent,
or more simply put, dependent in some way. In an experiment, for example, data are collected at
Aids least twice, once before and once after the application of the experimental variable. In these
circumstances, the t test is referred to as a "paired comparison" test. For any t test where the
groups are not independent, a different formula is used for conducting a t test. The t test
calculator we just described will also do a paired comparison t test. Also, the significance of
differences among three or means can be tested using a measure called the Analysis of Variance
(ANOVA). Ways of conducting dependent t tests and ANOVA are described in most statistical
textbooks and in many Web sites devoted to statistical analysis.
Our presentation of hypothesis testing is just one way of explaining this critical process. For
different explanations and illustrations of hypothesis testing and tests of statistical significance,
we suggest you view at least several of the following Web sites:
PREV NEXT

Chapter 19 Page 11
The Concept of Statistical Significance Testing, this online article should help you to better
Samples
understand the concept of statistical significance, the meaning of probability, the concept of
and significance testing, and the ways results can be misinterpreted
populations
Hypotheses provides a brief, clear and well illustrated discussion of hypothesis testing
Probability
Hypothesis Testing describes factors related to testing the null and alternative hypotheses,
theory and
including selection of the proper statistical test, consideration of significance levels, and
statistical one-sided to two-sided tests, and one sample or two sample tests
inferrence
Significance Tests: Hypothesis Testing discusses confidence intervals in estimating a
Inferring a population mean; provides a useful discussion of terminology involved in conducting a test of
population significance
mean Statistical Sampling Terms, a nicely presented and illustrated discussion of the process of
estimating a mean for a population; also includes definitions of terms and concepts used in the
Tests of process
statistical
significance
Coefficient of correlation
Tests of
differences For continuous variables, those measured at the interval or ratio levels, the appropriate measure
of association is Pearson's coefficient of correlation that is represented by the letter r. Correlation
between
analysis can also be used with data measured at the ordinal level. When one or either variable is
means measured at the ordinal level, the raw data can be coded to reflect increasing values for each
category of the ordinal variable. For example, the category of "strongly disagree' for an item can
Coefficient be coded with increasing values assigned from "strong disagree," codes as "1," "disagree" as "2,"
of "uncertain" as "3," "disagree" as "4," and "strongly agree" as "5." Other items used in a set to
correlation measure a variable would be coded in the same way. For detailed discussion of this method of
developing scores, go to Chapter 7, "Constructing Composite Measures, Scales and Indexes."
Caution Similar codes can be applied to any set of ordered responses. The resulting scores can be used in
with correlational or other statistical analyses with another variable in ordinal, interval or ratio form.
association Conducting statistical analyses with coded ordinal variables, however, assumes that the intervals
represented by each category of the ordinal variable are equal. In coding, we assume, for
Chi square example, that the difference between for the value assigned to "strongly agree" and "agree' is the
same as that for the between "uncertain" and "disagree" or between differences between two
Other adjacent categories. This assumption can not be verified. In fact, differences in what these words
tests of mean to various respondents may be quite different. Further, the distribution of coded responses
significance may not meet the requirements for certain statistical tests. If you chose to conduct statistical
analyses with coded ordinal data, you need to be aware of these limitations. Nevertheless,
Caution in statistical analysis with coded ordinal variables can give you an idea of association among yourr
using ordinal, interval and ratio variables.
statistical Represented as scatter plots

Chapter 19 Page 11
test results
An easy way to understand the coefficient of correlation is to visualize the relationship between
two variables. One way to do this is to show the relationship in the form of a scatter plot. We
Aids described this technique previously in Chapter 3, but repeat it so you can see its application to
understanding correlation. To illustrate, let's say you obtained the following data for the years of
schooling and number of babies born to a sample of 10 women. (In actual research, a larger
sample would be used. We are using only 10 cases to keep the illustration simple). With two
scores, one for schooling and one for fertility, the data could be listed as shown in Table 19.4.
PREV NEXT

Chapter 19 Page 12
Table 19.4 Raw data for years of schooling and fertility for 10 women
Samples
and Respondent
Number Schooling Fertility
populations
1 4 7
Probability 2 10 5
theory and 3 0 6
statistical 4 8 5
inferrence 5 11 4
6 3 8
Inferring a 7 7 4
population 8 12 3
mean 9 9 4
10 5 6
Tests of
Schooling, in this case, is referred to as the X variable and fertility as the Y variable. By tradition,
statistical X is used to represent the independent variable and Y is used for the dependent variable. Also,
significance notice that the data are listed in pairs. There are two scores for each woman, one for schooling
and one for fertility. Schooling is designated as the independent variable because it is more likely
Tests of that schooling could affect fertility than the other way around.
differences
Use of graph paper helps in drawing a scatter plot. To construct a scatter plot, draw a horizontal
between
line for the X axis and a vertical line for the Y axis beginning at the left end of the X axis. The
means point where the two lines meet is the 0 point for both axes. Mark off values for the independent
variable along the X axis and for the dependent variable along the Y axis. For our example, the
Coefficient scores for X varied from 0 to 12. These are shown along the X axis in Figure 19.1. Scores for
of fertility, from 0 to 8, are marked off on the Y axis. Each woman is then represented on the graph
correlation by a dot at the intersection of lines extending from her score for schooling and that for fertility.
To illustrate, woman number 1 had 4 years of schooling and 7 babies. A dot is placed at the
Caution intersection of line going up from 4 on the X axis and a line extending to the right from 7 on the
with Y axis. This point represents the pair of scores for woman number 1. The same process is
association repeated for the remaining 9 pairs, resulting in 10 points. The plot for these 10 points is shown in
Figure 19.1.
Chi square Looking at the line drawn through the set of dots, you can see that there is a relationship between
schooling and fertility: As the scores for schooling increase, fertility declines. Drawing a straight
Other line through the middle of the points helps in visualizing the relationship. To do this, draw a
tests of straight line that comes the closest to the largest number of points. Called the line of best fit, this
significance line shows the direction of the relationship. It can be used to estimate how strongly the variables
are related.
Caution in
using
statistical

Chapter 19 Page 12
test results
Aids
Figure 19.1 Years of school and fertility

PREV NEXT

Chapter 19 Page 13
The line of best fit in Figure 19.1 slopes from left to right. This indicates a negative association:
Samples fertility declines as schooling increases. A positive relationship is indicated when the line slopes
and upward from left to right, as shown in Figure 19.2.
populations
Probability
theory and
statistical
inferrence
Inferring a
population
mean
Tests of
statistical
significance
Tests of
Figure 19.2. Years of schooling and income (in thousands)
differences
between Figure 19.3 illustrates a third pattern. This graph shows an illustrative relationship between
means income and scores for measuring love and affection shown to children. In this case, the dots are
spread all over the plot with no particular pattern. There is no line of best fit. Almost any line we
Coefficient would draw would be flat. When no pattern can be established, the variables are considered to be
independent of one another. In other words, they are not associated.
of
correlation
Caution
with
association
Chi square
Other
tests of
significance
Caution in
using
statistical

Chapter 19 Page 13
test results
Figure 19.3. Annual income of
Aids
households and affection toward
children
You can also get a rough idea of how strongly the variables are related by how close the points
are to the line of best fit. When all the points are covered by the line, the two variables are
perfectly related: then a specific increase in the X variable is always accompanied by a specific
change in variable Y. Such relationships are seldom found in social research. But when the
points, as in Figure 19.1, are fairly close to the line of best fit there is evidence of a strong
association, but not a perfect relationship. Large distances between most of the points and the line
indicates a weaker relationship, as in Figure 19.3.
While the scatter-plot helps in getting an idea of the direction and strength of relationship, the
coefficient of correlation provides the direction and a quantitative measure of the relationship.
PREV NEXT

Chapter 19 Page 14
Calculating r
Samples
and To calculate the coefficient of correlation (r), you need to know the N, which is the number for which you have data
populations for both of the variables being tested for association, and you have to calculate seven quantities. You already know
how to get three of them. These are X, X 2, and (X)2, basic elements in many statistical calculations. The same
Probability quantities have to be calculated for the second (Y) variable as well, giving Y, Y2, (Y)2. In addition, you have to find
theory and the sum of the products of each X value times its associated Y value. These products are referred to as the cross
products (XY) and when summed are represented by (XY), and are referred to as the sum of the cross-products.
statistical
Table 19.5 shows the computation of the seven quantities using the pairs of scores listed earlier for schooling and
inferrence fertility.
Inferring a Table 19.5. Illustrative computation of values for calculating a coefficient of correlation
population Respondent X Y X2 Y2 XY
mean 1 4 7 16 49 28
2 10 5 100 25 50
Tests of 3 0 6 0 36 0
statistical 4 8 5 64 25 40
significance 5 11 4 121 16 44
6 3 8 9 64 24
Tests of
7 7 4 49 16 28
differences
8 12 3 144 9 36
between
9 9 4 81 16 36
means
10 5 6 25 36 30
Coefficient 69 52 609 292 316
of
correlation Five of the needed quantities are shown as the totals for the columns in the computational table. In addition, we need
the N, which is 10, and the value for (X)2 which is (69)(69) or 4,761; and for (Y)2 which is (52)(52) or 2,704.
Caution These seven values are then substituted in the following formula for correlation:
with
association
Chi square
Other
tests of
significance
Caution in
using
statistical
test results
Aids

Chapter 19 Page 14
PREV NEXT

Chapter 19 Page 15
Samples Interpreting the coefficient of correlation

and
A correlation coefficient can vary between -1.0 and +1.0. As r approaches either -1.0 or +1.0, it
populations
indicates a strong relationship, either negative or positive, respectively. As r becomes smaller it
indicates a weaker relationship, with 0 indicating no relationship at all between the variables
Probability under examination. The r of -.80 indicates a strong negative correlation.
theory and
statistical As with the t test, interpretation of an r depends on the degrees of freedom involved. In the case
inferrence of r, however, df = N-2. For our example, df is therefore 10-2 = 8. Just as with t ratios,
mathematicians have worked critical values for r at various degrees of freedom. Most statistical
textbooks contain a table of critical values for r. These tables have various names such as
Inferring a
"Critical Values for Pearson r" or "Critical Values for the Correlation Coefficient" just look for a
population table with "critical values" and "correlation coefficient" in its title. To read a critical values of r
mean table, you need to know the degrees of freedom upon the test is based, whether you are
conducting a one-tailed or two-tailed test, also called a directional or nondirectional test, and you
Tests of have to set the level of significance you will accept.
statistical
As mentioned with the t test, the two-tailed or nondirectional test is more conservation and is the
significance
most commonly used test. We found a value of -.80 for r between years of schooling and fertility.
Looking in a table of critical values for r, we find that at 8 degrees of freedom an r of .80 exceeds
Tests of the critical value of .765 listed for the .01 level of significance for a nondirectional test. This
differences result says that an r as large as found would be due to chance less than 1 time in 100. Therefore,
between we can safely reject the null hypothesis (that there is no relationship between schooling and
means fertility among women) and accept the alternative hypothesis (that a relationship exists between
the two variables).
Coefficient
In interpreting r, the positive or negative sign is ignored. The sign indicates the direction of the
of association. The size of the r and the N are used to derive the degrees of freedom determine
correlation whether the null hypothesis is accepted or rejected. Instead of finding a statistics books with a
critical table for r, you can use a Web-based way of finding significance levels for various values
Caution of r and df. We used Pearson Product-Moment Correlation Coefficient Table of Critical
with Values. With df = 8, we found that the r value of -.80 was greater than the critical value of .765 at
association the .01 level of significance. Therefore, the null hypothesis is rejected and the association is
assumed to be significant and due to chance association between the two variables.
Chi square
Web-based calculation of r
Other
tests of Calculation of r involves a lot of laborious calculating. You can avoid this by using a Web-based
calculator. If you need to calculate r, we suggest you trying using:
significance
Linear Correlation and Regression. This site provides instruction of entering the values for X
Caution in and Y and the site will calculate the r, provide descriptive statistics for each set of values, and
using give the probability level for the r. Nice and easy.
statistical

Chapter 19 Page 15
test results Coefficient of determination

The correlation coefficient can also be used to say how much of the variation in scores for the
Aids
dependent variable can be explained by its association with the independent variable. This is done
by squaring r. This produces a result known as the coefficient of determination. Thus, in our
example, we could say that about (-.80) 2 or .64 or 64% of the variation in fertility is explained
by differences in schooling among the women in the sample.
PREV NEXT

Chapter 19 Page 16
Rank order association

Samples
and Calculation of r is based on the assumption that each of the two variables is normally distributed
populations in the population from which the sample was drawn. In practice, researchers calculate r even
when sample distributions are not exactly normal. However, when the sample distributions for
Probability either or both of the variables are severely skewed, with a lot of extremely high or low scores
relative to the rest of the scores, it is better to use the Spearman rank-order coefficient, called rho.
theory and
In addition, rho is the appropriate measure to use for testing the association between two ordinal
statistical variables.
inferrence
Assigning ranks to scores
Inferring a In a rank-order analysis, each score is assigned a rank based on its relationship to the other scores
population in the distribution. The ranks are then used in place of the scores. Table 19.6 shows original and
mean ranked scores for grades received by 8 students in two subjects. The null hypothesis is that there
is no relationship between scores (grades) in one subject (X) and those in the other subject (Y). In
Tests of ranking scores, the highest score is given a rank of 1.The next highest score gets a rank of 2. Each
statistical lower score get the next highest rank. This process is repeated for all scores. The lowest score
significance gets a rank equal to the N for the analysis. When ties in scores occur, each score is given the
mean of the ranks for the set of tied scores. The ranks that would have been given to separate
Tests of scores are added and divided by the number of tied scores. Each score is given the mean of the
tied ranks
differences
between Table 19.6. Illustration of ranked scores and calculations needed for determining the Spearman
means rank-order correlation
Student Rx Y Ry d d2
Coefficient 1 80 5 73 7 4
of 2 82 3 90 1.5 2.25
correlation 3 80 5 85 1 1
4 85 2 90 .5 .25
Caution
5 75 8 77 2 4
with
6 90 1 86 2 4
association
7 80 5 70 3 9
8 78 7 82 2 4
Chi square
d2 = 28.50
Other
tests of
significance Applying these rules to the scores in the X distribution in Table 19.6, we look for the highest
score This is 90, which is given a rank of 1, as shown in R x column. The next highest score, 85,
Caution in is given a rank of 2, while 82, the third highest score gets a rank of 3. Then, we find 3 score of
80. These scores represent the ranks of 4, 5 and 6. Because they are ties, they get the mean of
using
three ranks; so, each gets a score of 5. The remaining scores, 78 and 75, are ranked as 7 and 8,
statistical respectively. The Y scores would be ranked the same way Rules for ranking scores are

Chapter 19 Page 16
test results summarized in Box 19.1.
Aids Box 19.1. Rules for ranking scores

1. The highest score in the distribution is given a rank of 1.
2. The next highest score gets a rank of 2.
3. This process is repeated for all scores with each lower score
getting the next higher rank. The lowest score gets a rank
equal to the N for the analysis.
4. When ties in scores occur, each score is given the mean of the
ranks for the set of tied scores. The ranks that would have
been given to separate scores are added and divided by the
number of tied scores. Each score is given the mean of the
tied ranks.
With the ranking completed, the next step is to find the differences between the ranking of the
scores for each student. These are listed under the column labeled d. Plus or minus signs are
ignored because the differences are squared, as shown in the last column to the right. Squaring
removes the minus signs and allows us to add the differences, now squared, which produces the
important element for calculating rho.
PREV NEXT

Chapter 19 Page 17
Calculating rho
Samples
and The formula for rho is:
populations 6d 2 6(28.50) 171.0
rho = 1 - _________ = 1 - _______ = 1 - ______ = 1 - .339 = .661
Probability N(N 2 - 1) 8(64-1) 504
theory and
statistical Like r, rho can vary from +1.0 to -1.0. The size of rho depends on the differences between each
set of scores and the N for the analysis. As differences between ranks increase, so do their
inferrence
squares. As the N increases, this reduces the value for 6 times the sum of the squares of the
differences. As the result of this division decreases, rho increases because a smaller number is
Inferring a subtracted from the 1 at the beginning of the formula.
population
mean If ranks for each set of scores match exactly, the differences in ranks will equal 0. Therefore, rho
will equal 1, representing a perfect correlation. As differences in ranks increase, rho will
Tests of decrease.
statistical The critical values for rho are provided in most statistical textbooks. Look for values of the
significance Spearman Rank Order Correlation Coefficient. With rho, critical values are associated with the
number of pairs used. In our example, there were 8 pairs of scores. For 8 pairs, the critical
Tests of value of rho at the .05 level is .643. The rho we found was .661 which exceeded the .05 level.
differences Therefore, we can reject the null hypothesis and accept the alternative hypothesis that grades in
between subject X are associated with those in subject Y.
means Web-based calculation of rho
Coefficient Web calculators are also available for calculating rho. We used Spearman Rank Order
of Correlation Coefficient. You can do the calculation by entering raw scores for X and Y or by
correlation entering ranks for each score. We confirmed our hand calculation for rho, as just described, using
the calculator for rho and got a slightly different result. The calculator gave a rho of 0.6504
compared to the 0.661 we got. This difference occurred because the Web-based calculator carried
Caution
out calculations with a larger decimal value than we used, but the important result was the same.
with The result was significant at the .05 level.
association
Caution with association
Chi square Finding a strong correlation between variables only says that the variables are related and nothing
more. Such a finding is no basis for claiming that the independent variable caused the changes in
Other the dependent variable. Claims for cause and effect can only be made when the effects of all
tests of independent influences on the dependent variable are eliminated. This is extremely hard to do in
significance social research. Therefore, as we pointed out in the section on "Cause and Effect" in Chapter 3,
social scientists are very careful when they interpret correlation results. Finding associations is
Caution in important. These results add to our knowledge of social relationships. But as every student in
using statistics learns: Correlation is not the basis for causation.
statistical

Chapter 19 Page 17
test results
Chi square
Aids
Chi square, written as , is frequently used for testing for association between two nominal or two
ordinal variables. It will tell you what the probability is that an observed variation of one variable
with another is the result of chance, due to the particular sample you selected, or whether the
covariation between the two variables is probably real.It will not tell you the strength of
association between the two variables. Other tests, such as the coefficient of contingency,
described later, can be used for this purpose. This and other statistical tests are described in most
statistical textbooks and on many Web sites, including those cited at the end of this chapter.
Chi square can also be calculated for any sized bivariate table containing two nominal or a
nominal and ordinal variable. Table 19.7 shows a 3x2 table for an ordinal variable,
socio-economic status of fathers, expressed in three levels, and a nominal variable, their attitude
toward completion of schooling by daughters, which is expressed as yes (to complete secondary
school) or no (not to complete secondary schooling). The upper half of the table gives the
observed frequencies for the relationship between these two variables. A quick calculation shows
that the percentages of fathers who said "yes" increased directly with the socio-economic status
of the families (low, 44%; middle, 67%; high, 75%). But, assuming random samples were used
to select the fathers, are these differences large enough not to be attributed to sampling error? We
can find out by conducting a chi square test.
PREV NEXT

Chapter 19 Page 18
Table 19.7. Observed and expected frequencies for chi square for a 3x2 calculation (Socio-economic
Samples levels of fathers and attitudes toward completion of secondary school by daughters)
and
Observed frequencies
populations
Socio-economic level
Complete school? Low Middle High Total
Probability Yes 35 40 30 105
theory and No 45 20 10 75
statistical Total 80 60 40 180
inferrence
Expected Frequencies
Inferring a Yes 46.67 35.00 23.33 105
population No 33.33 25.00 16.67 65
mean Total 80.00 60.00 40.00 180
Calculating chi square
Tests of
statistical Chi square is based on comparing differences between an observed set of frequencies and ones that
could be expected due to chance variations from sampling. The formula for making this comparison
significance
is:
Tests of (O - E)2
differences 2 = _______
between E
means
This formula tells us to find the difference between each observed frequency (O) and its
corresponding expected frequency (E); then square these differences; divide the result by expected
Coefficient
frequency for that difference; and then add the results of these operations. The result gives the value
of of the chi square.
correlation
We know the observed frequencies. These are listed in the upper set of cells of Table 19.7. For
Caution example, 35 of the fathers classified as low status said yes and 45 said no. For the middle status
with fathers, the frequencies were 40 and 20, and for the high status fathers, 30 and 10, respectively. The
total for all yes responses was 105 and 65 for no. Totals for each status group are shown across the
association
bottom of the upper half of the table. There were a total of 80 low status fathers, 60 middle status
fathers, and 40 high status fathers.
Chi square
The row and totals are used for calculating the expected frequencies for the chi square test. The row
Other total is represented by R t and the column total by C t. The formula for calculating the expected
tests of frequencies for each cell is:
significance (Rt)(Ct)
E (for any cell) = ______
Caution in N
using
Using the formula for finding the Es, we can start by finding the E for the observed frequency of 35 in
statistical
the upper left hand cell of the table. The total (Rt) for the row containing this cell is 105; Ct for the
test results column containing this cell is 80; and the N is 180. The calculation for the E for this cell then is

Chapter 19 Page 18
Aids (105)(80)/180 which equals 46.67.Es for the remaining cells were calculated in this way and are
shown in the lower half of Table 19.7.
Putting the values for the Os and Es into the formula for chi square, we find:
In this calculation, we didn't show the arithmetic for each step in the calculation. To gain experience
in doing a chi square, we suggest you work through the results and see if you get the answer we got.
PREV NEXT

Chapter 19 Page 19
Interpreting a chi square result

Samples
and In the case of chi square, degrees of freedom equals the product of the number of rows minus 1
populations times the number of columns minus 1. Expressed as a formula: df = (r - 1)(c - 1). The df for our
3x2 table, therefore, is (3 -1) (2 - 1) or 2. As with all tests of significance, we need to consult a
Probability table of critical values to decide on the level of significance of the result. Again, almost any
statistical textbook will have a table of critical values of chi square. Using such a table, we find
theory and
that with df = 2 the calculated value of 12.99 exceeds the critical value for the .01 level (9.210).
statistical This means that the chance of the differences among the three samples of fathers occurring due to
inferrence chance is less than 1 time in 100. Therefore, we can safely reject the null hypothesis that there is
no difference in their attitudes about schooling for their daughters. Having a strong basis for
Inferring a rejecting the null hypothesis, we can accept the alternative hypothesis that a direct relationship
population exists between the social-economic status of fathers and a preference for daughters completing
mean school. We would report this result as:
Tests of 2 = 12.99 (df=2) p<.01

statistical
significance This says that with 2 degrees of freedom, the 2 of 12.99 has a probability of occurring due to
chance less than 1 time in 100.
Tests of Instead of looking up the critical values in a statistics book, you can reply on sites such as the Chi
differences Square to P Calculaton to provide the probability value for any chi square you calculate. Simply
between enter the value of chi square that you have calculated and the df where indicated and click on
means "Calculate." When we entered 12.99 for the chi square in the previous example and df=2, we
were informed that the probability of getting a chi square of 12.99 with 2 degrees of freedom =
Coefficient 0.0015, which is considerably less than the .01 level.
of
Web-based calculation of chi square
correlation
You need not do all the calculations needed for determining a chi square value. You can use a
Caution Web-based calculator to the heavy work. One site you might want to try is Measuring
with Dependency of Two Variables from Categorized Data. This calculator is very easy to use. You
association just enter the values for the observations you in the cells of a contingency table and click on
"Calculate." You will get the chi square value, a correlation value, and P value for the level of
Chi square significance of the result.
Limitations of chi square
Other
tests of Chi square can be used with any nominal variable, but be careful when Ns are small. When any
significance expected cell frequency is zero (0) or when more than 25% of the expected cell frequencies are
less than 5, chi square should not be used. Also, some special conditions apply to its use with
Caution in 2X2 tables. When the N is less than 20, chi square is not the best statistical test to use with 2X2
tables. A test called the Fisher exact test is more appropriate. For 2X2 tables with an N over 40, a
using
correction, called the correction for continuity, will produce a more accurate chi square. The
statistical Fisher test and the correction for continuity are described in most statistical textbooks and on

Chapter 19 Page 19
test results many statistical web sites. Otherwise, chi square can be safely applied to tables with more than 2
columns and 2 rows, provided they meet the cell frequency requirements listed previously.
Aids
Other tests of significance
We have described only a few of the many tests of statistical significance available to social
researchers. There are dozens of additional tests that can be used for analyses involving nominal,
ordinal, or interval or ratio data. Some of these may be more appropriate for a specific analysis
you may need to do than any of the tests we have described. You can find information about
additional tests in any statistical textbook or in many Web sites. The most comprehensive site we
know of is: Web Pages that Perform Statistical Calculations!.
This site provides links to online statistical books, tutorials, downloadable software, and forms of
help for planning and conducting statistical tests of significance. This site is a real gold mine of
resources for researchers; it includes free software packages, random number generators,
probability distributions for commonly used statistical tests. With some looking around, you
should be able to find help with any statistical test you may be considering and find access to
distribution tables you need for interpreting any statistical test you perform
PREV NEXT

Chapter 19 Page 20
Samples Caution in using statistical test results

and
populations Tests of statistical significance have great appeal because they provide researchers with an
independent, objective basis for assessing their findings. But they only tell us the probability that
Probability our results could have been due to chance variations that occur in the selection of a random or
theory and probability sample. If the odds are low enough that the result could not have occurred by chance,
statistical we can reject the null hypothesis and accept the alternative hypothesis formulated at the
inferrence beginning of the research project.
While useful, we caution against uncritical dependence on levels of significance in the
Inferring a interpretation of results. First, statistical tests say nothing about the substantive value or the
population importance of the results. With a large enough N, almost any result will be statistically
mean significant. But a very weak level of association or a meager difference between means, while
significant at the .01 level, has very little theoretical or practical value. For example, with df = 50,
Tests of a correlation coefficient of .231 is significant at the .05 level. Although significant, this
statistical correlation indicates only a slight association between two variables. Its coefficient of
determination would be only 0.53, meaning that only about 5% of the variation in the dependent
significance
variable could be attributed to the effect of the independent variable. The result, while significant,
would have to be interpreted as indicating only a weak association and would not suggest much
Tests of theoretical or practical importance.
differences
between Second, there are times when it may be appropriate to bend the traditional .05 level of
means significance in assessing results. Remember, the .05 level is an arbitrary criterion established by
the research community. Why not a .07 level or .03 level? Any other level with a low probability
for results occurring due to chance could have been used. By tradition, the research community
Coefficient
settled on the .05 level. What do you do when results are close to but still slightly below the .05
of level for a test? It is entirely appropriate to point out the results are "approaching significance"
correlation and to give and interpret a result accordingly. For instance, a chi square of 11.35 with 6 degrees
of freedom lies between the .10 and .05 levels of significance. You could simply report the result
Caution as nonsignificant, and show it as p<.05; retain the null hypothesis; and conclude that the two
with variables are not related. Or, you could report your results more precisely by using the notation of
association .10<p<.05, which says the probability of the null hypothesis being true is between .10 and .05. In
similar fashion, significance levels can be reported between .30 and .20 or any other levels.
Chi square Third, there are situations where using a statistical test is not appropriate. When data are collected
from all members of population, there is no sampling error. Therefore, there is no basis for
Other making statements about the probability of relationships or differences occurring due to chance.
tests of For measurements based on enumeration of a population, the results are parameters. This is not to
significance say that whatever is found is absolutely accurate. There still is the possibility of error in
measurement. For more on this point, review the discussion of measurement error in Chapter 7.
Caution in But whatever is found does not involve sampling error.
using
Strictly speaking, statistical tests are also inappropriate when nonprobability samples are used.
statistical Probability theory, the basis of all tests of statistical significance, rests on the use of probability

Chapter 19 Page 20
test results or random samples. When convenience or other nonprobability techniques of sampling are used,
there is no basis for drawing on probability theory. For these samples, there is no sampling error
Aids and, therefore, statistics from non-probability samples cannot be used to estimate population
parameters. Results from nonprobability samples, however, can be and are frequently reported in
the literature. Careful researchers note this fact and interpret their results in theoretical terms or in
terms of their practical value.
PREV NEXT

Chapter 19 Page 21
Samples Aids
and
populations Key terms
● Alternative hypothesis ● Line of best fit
Probability
theory and ● Analysis of variance ● Nonsignificant
statistical ● Chi square ● Null hypothesis
inferrence ● Coefficient of correlation ● Parameter
● Coefficient of determination ● Perfect relationship
Inferring a ● Confidence interval ● Probability level
population
● Cross products ● Sampling distribution of the mean
mean
● Degrees of freedom ● Sampling error
Tests of ● Hypothesis testing ● Scatter plot
statistical ● Independent ● Spearman rank order correlation
significance ● inequality signs ● Standard error of the mean (standard
● Inferential statistics error)
Tests of ● Level of confidence ● Statistical interference
differences ● Statistic
● Level of significance
between ● Statistical test of significance 1 test
means
Main points
Coefficient 1. Analysis of data from samples produces results called statistics. Examples of statistics
of include means, percentages, and standard deviations. Corresponding values in a population
correlation referred to as parameters.
2. Values for statistics vary from one sample to the next because of variations resulting from
Caution random selection of samples.
with 3. Inferential statistics are used to estimate parameters from statistic.
association 4. Inferential analysis can be used to estimate a single parameter, such as population mean
from a sample mean, or to establish a relationship between two variables.
Chi square
5. The exact value of a parameter, such as a population mean, cannot be estimated precisely.
Instead, we can only estimate an interval in which the mean is probably located. This
Other interval is known as the confidence interval and is defined in terms of standard errors
tests of above and below the sample mean. Further, the confidence interval is defined in terms of
significance levels of confidence. The levels of confidence express how confident we want to be that
the result was not due to chance variation. The usual confidence levels are estimating that a
Caution in parameter is within the specified confidence interval is 95 times out of 100, referred to as
using the .05 level of confidence or that the parameter will be in the specified interval 99 times
statistical out of 100, referred to as the .01 level of confidence.

Chapter 19 Page 21
test results 6. Tests of statistical significance are used in deciding whether a relationship between two
variables observed in a sample also exist between the variables in the population from
Aids which the sample was selected.
7. Statistical tests of significance are based on testing the null hypothesis. This hypothesis
states that two variables are not associated or that two statistics, such as means, are not
different. Tests of significance establish whether the observed result is one that could be
expected due to chance variations in sampling. When the probability is high that an
association or difference could be due to sampling variation, the null hypothesis is
accepted. When the probably is low that an association or difference could have occurred
due to chance, the null hypothesis is rejected.
8. Traditionally, the .05 level of significance is used in testing the null hypothesis. This level
says that a result could have occurred due to chance in less than 5 out of every 100 samples
that could be selected from a population. With a result significant at the .05 level, we could
be wrong in rejecting the null hypothesis 5 times in 100. At the .01 level of significance,
we would incorrectly reject the null hypothesis 1 time in 100.
9. When the null hypothesis is accepted, the conclusion is that no relationship exists between
the variables being analyzed. When the null hypothesis is rejected, the alternative
hypothesis established at the beginning of the investigation, also called the research
hypothesis, can be accepted.
10. The t test is used to test the significance of the difference between two means.
11. Pearson's coefficient of correlation is used for measuring the association between two
continuous variables (measured at the interval or ratio levels). It varies between -1.0 and
+1.0. A scatter plot, based on plotting pairs of observations on the X and Y coordinates of
a graph, can show the direction and degree of association between variables. The
independent variable is traditionally labeled as the X variable and the dependent variable as
Y.
12. Spearman's rank order coefficient of correlation is used to test the association between
pairs of ordinal, interval, or ratio scores that have been converted to ranks. The ranks are
then used in place of the original scores in testing for association.
13. Chi square is used to test for the dependence between nominal or ordinal variables.
14. Statistical tests of significance are interpreted in terms of degrees of freedom, which
depend on the N used in the analysis. Critical values for tests, such as the t test or chi
square, indicate the levels of significance at various levels of confidence (.05, .01, or .001).
When the result for a test exceeds the specified critical value at the specified degrees of
freedom, the null hypothesis is rejected. When the result is less than the critical value at the
specified degrees of freedom, the null hypothesis is accepted.
15. Statistical tests do not mean that results have theoretical value or practical importance.
With a large enough N, almost any result can be statistically significant. In addition to
assessing the statistical significance of results, researchers also have to make judgments
about the theoretical or practical value of findings.
16. Strictly speaking, tests of significance should only be used to analyze data from properly
drawn probability samples. Nevertheless, tests of significance are used with nonprobability
samples. These tests are useful for establishing the extent of relationships among variables,
even though the conclusions cannot be safely generalized to any population.

Chapter 19 Page 21


Chapter 20 Page 1
Introduction Chapter 20: Reporting Your Research

Introduction
Improving
your In the previous chapters you learned how to design a research project and how to collect and
writing analyze your data. In this final chapter, we describe how to do the last but extremely important
step in the research process - that of reporting your research to others. A well organized, clearly
Guidelines written report will invite the interest of readers and increase the use of your research results. A
for clear poorly organized and badly written report will discourage persons from reading it. As a result, no
matter how well you conducted your investigation, all your efforts could be wasted because
writing
readers will not try to figure out what you are telling them.
Writing The purpose of scientific reporting is to contribute to understanding of some problem. To achieve
your report this objective, your report must describe clearly and fully what you have investigated (the
statement of the problem), how the study was conducted (the design that was used), how the data
Final were collected and analyzed, and what was discovered. In addition, reports often include ideas
checking for new approaches to the study of some problem or new problems for investigation. These
objectives can only be achieved if your report is well organized and written in clear,
understandable language.
Oral
presentations The importance of clear communication applies to all the ways you might choose to report your
results.
Using the
As a student, you probably will have to prepare a written report for the research project you are
mass media required to do. In addition, you may choose to prepare a manuscript for publication in a journal,
deliver talks based on your research, or to communicate your research findings in a local
Aids newspaper or magazine. Each of these ways of reporting requires writing about your research.
Improving your writing
There is no secret to good writing. Each person has his or her own style. There are ways,
however, for improving our writing skills. We offer some well tested guidelines for more
effective writing. We focus on preparing research reports since this may be a requirement you
have to meet, but the guidelines apply to other forms of reporting mentioned above.
In addition to the guidance we provide in this chapter, you may want to visit any of the following
Web sites. Each of these sites contain suggestions and instruction on how to write a research
report:
Final Year Projects provides practical advice and assistance on all aspects of writing a final year
project, thesis or dissertation; written primarily for undergraduate students, but has wide value for
preparing any research paper; includes guidance related to getting started, reviewing the
literature, references, time management, and other topics
Formatting provides detailed guidelines on how to prepare a research report; gives the
approximate number of pages for each section of a report (Introduction, Methods, etc.); describes
how to cite references in a report; and shows how to construct tables and figures

Chapter 20 Page 1
PREV NEXT

Chapter 20 Page 2
Introduction (A) Guide for Writing Research Papers based on Styles Recommended by the American
Pyschological Association provides detailed guidance for preparing a research paper, including
Improving formats for references based on the style required for submission to journals published by the
your American Psychological Association
writing
How to Write a Scholarly Research Report provides a brief introduction to and description of
sections of a scholarly report
Guidelines
for clear How to Write a Technical Report provides a good overview of what a research report should
writing contain, describes the order of the sections of the report, and offers guidelines on preparing a
report
Writing
Research Helper describes four steps for completing a research paper - prepare, find, think,
your report
share and reflect
Final The Research Process outlines a simple and effective strategy for finding information for a
checking research paper and documenting the sources you find
Steps in Research and Writing Process describes the steps involved as: Select a topic, frame a
Oral
central research question, survey the topic, locate materials, re-evaluate your research strategy,
presentations
re-evaluate your topic and research strategy, develop your supporting argument, use sources
fairly, write, revise, edit and proofread
Using the
mass media Write-Up, www.socialresearchmethods.net/writeup.htm, notes three considerations writers
should keep in mind: the audience for the report, the story you want to tell, and how the report
Aids should be formatted; also includes a sample paper to serve as a guide for persons writing their
first research report
Know what is expected
Before you start writing, make sure you know the reporting requirements of the organization you
will submit your report to. Your university may have a specific format and style for the
preparation of research reports. If so, find out what these are and follow them. Later, you may
be responsible for preparing a report for a government ministry or a nongovernmental
organization that sponsored research you directed. Their reporting requirements may be different
from those of your university. Before preparing a report for an organization, you will want to find
out how the organization wants reports prepared. If specific instructions are not provided, you
may want to use an internationally recognized style for preparing a research report. Writing for a
journal publication often involves still different requirements.
Start well before deadlines
Instead of waiting until just before your report is due, start writing early. This will allow you time
to organize your thoughts, prepare a rough draft of the material, and then have time to do what all
writers do: revise and revise until a carefully organized and well written product is produced.
Writing sections of a report as you complete each phase of a project can make writing a lot easier

Chapter 20 Page 2
and less stressful. After you finish your review of literature, for example, you can write the first
section of your report. This could cover the statement of the problem, formulation of a theoretical
framework and development of hypotheses as well as the review of the literature. The section of
methods can be completed after you prepare your design. Upon completion of the analyses, you
can describe how you analyzed your data and what you found. At this stage, you can also prepare
rough and then final copies of tables or graphs you will later include in your report. If you follow
this approach, at the end of the project you only have to add the conclusions, recommendations,
and other final material. Try to write each section of your report as you proceed through your
project. This will give you time to review your entire report when you finish to make sure it
meets all the expected requirements.
PREV NEXT

Chapter 20 Page 3
Introduction Organize before writing

Many students have difficulty in writing because they do not organize their material and thoughts
Improving
before starting to write. You can avoid this problem by maintaining a clear, well-documented
your record of the decisions you made and work you did at each stage of your project. You can either
writing write out sections of your report as you complete each phase or create detailed, complete, clearly
written notes for reconstruction of these decisions and actions. You might want to keep separate
Guidelines folders for material related to each section of your report. Your initial folder, for example, could
for clear contain the final approved statement of the problem or question you are investigating and your
writing reasons for undertaking the study. Another folder could contain summaries of material you have
reviewed and any hypotheses you are testing. Other folders can be created for the remaining
Writing major sections of your report. With a complete set of notes or draft material for various sections
your report of your report, writing the final report will be a lot easier.
Outline before writing
Final
checking Perhaps the single best way to both improve writing and to make it easier is to prepare an outline
before you start writing. Preparing an outline makes us think about what we want to say and to
organize our thoughts in a logical manner.
Oral
presentations There is general agreement about the main topics that should be included in a scientific report.
Also, it is customary to present these topics in a certain order. The topics and their order are:
Using the 1. The Problem
mass media 2. Review of the Literature
3. Method or Design
Aids
4. Findings or Results
5. Conclusions
6. Summar
7. List of References
8. Appendices
These topics, listed in the order given, can be used as the overall outline for your report. The
first six are generally written as separate sections or chapters of the report, while the last two are
added at the end of the report. For short reports, topics might be combined. The first section or
chapter, for example, might include an introduction to the study, the statement of the problem,
and the review of the literature. In some reports, conclusions and the summary are combined.
You have to select the arrangement that best meets your requirements.
Deciding on the main sections of your report, however, is only a start. Before starting to write a
section or chapter, we strongly advise that you prepare a detailed outline. First, list the topics or
ideas you think you will need to discuss to complete the chapter. Then put the topics in a logical
order. Several revisions may be necessary before you create an outline you like. After some
thought, a student might have created the following outline for Chapter 1:

Chapter 20 Page 3
Chapter 1
The problem
1. Statement of the problem
2. Significance of the problem
3. Limitations of the study
4. Outline of the remainder of the report
These four topics would become the main sections of the chapter.
PREV NEXT

Chapter 20 Page 4
Introduction For most sections, it is useful to prepare further, more detailed outlines. For example, further
thought about Chapter 1 could lead to the following detailed outline:
Improving
Chapter 1
your The problem
writing
● Statement of the problem
❍ Historical background
Guidelines
for clear ■ Long term influences
writing ■ Recent social developments
■ Combined influences
Writing ❍ Current setting for the problem
your report ■ Social conditions
■ Economic conditions
Final
■ Impact on the group being studied
checking
● Significance of the problem
Oral ❍ Scientific value of the study
presentations ❍ Practical value of the study
■ For government offices

Using the
■ For nongovernmental organizations
mass media
■ For educational purposes
Aids ■ For use by citizens
● Limitations of the study

❍ In relation to the samples used
❍ In terms of measurement
● Outline of the remainder of the report

❍ Brief introduction for each of the following chapters
Having produced a detailed outline, you would be ready to start writing the first chapter. You
would know what you want to cover and the order in which you want to present each main
section and any subsections. Now you would be ready to shift your attention to putting together
the sentences and paragraphs for writing the first subsection of the first section of Chapter 1.
Outlines may be more or less detailed than the one we have presented. If you are not sure how
detailed to make an outline, we recommend you prepare the most detailed one you can. Thinking
about details will help you sort out the ideas you want to present and will simplify the later
process of writing. But even with the best outline you prepare, once you get into writing new
ideas will often occur to you. When this happens, stop and think about the outline you are using.
You may need to revise it before starting to write again. Further ideas about outlining take a look
at:

Chapter 20 Page 4
Getting Started: Outlining. This site emphases the importance of outlining and presents
illustrative outlines for a research paper
Follow examples of good reporting

You can improve your research reporting by basing your writing on examples of effective
scientific reporting. While not all journal articles are examples of good writing, many are. As
you conduct your review of the literature, note how articles are organized; how sections of the
articles are introduced; how data, findings and conclusions are presented and discussed; and how
summaries are written. Later, as you write the various chapters of your report you may find it
helpful to reread some articles to get ideas for how to write the corresponding parts of your
report. A number of Web sites also provide guidance for effective research writing. We cite a few
of these at the end of this chapter.
Write, revise, and revise again
Writing and revision go together. Few of us express exactly what we want to say in one inspired
session of writing. Most writers, even the most experienced and accomplished, write and then
revise and, often, revise again. The great American writer, Ernest Hemingway, is said to have
revised the last paragraph of one of his famous books twenty eight times before he settled on the
final ending. You probably will not revise any part of your report that many times, but be ready
to make at least several revisions to parts of it. This is another reason for starting writing as early
as you can. This way, you will have time for revisions so important for producing a report you
can be proud of. After you have written a chapter we recommend that you set it side for about a
week or so and then critically reread it. During the interval you may get some new ideas or think
of more effective ways of saying what you want to say.
PREV NEXT

Chapter 20 Page 5
Introduction Get outside review

Getting comments from others on what you have written will also help improve your report.
Improving
Before you show your writing to your research advisor, ask classmates, friends, or others to
your review your material. What you think is clear may not be clear to others. f your friends or
writing classmates do not understand something, your advisor and readers of your final report will
probably not understand it either. Exposing one's self to criticism is not easy and can be
Guidelines unpleasant, but it can be a great help in writing a well organized, clearly written report.
for clear
Use a word processor
writing
If at all possible, type your report, beginning with your first drafts, on a computer with a word
Writing processing program. Earlier we urged use of a computer for analyzing your data. The same
your report computer will probably have some kind of word processing program. With a word processing
program, you can type your report (or have someone else do it), add charts and graphs, and
Final quickly revise and retype various drafts. Using a word processor makes revisions much easier to
checking do. Also, most word processing programs will identify misspelled words and give you the correct
spelling for them. These features will help you prepare a much more readable and professional
looking report.
Oral
presentations The preceding points are offered to help you get ready to start writing. Now, let's turn to the
process of writing.
Using the
Guidelines for clear writing
mass media
Following are a few additional suggestions for ways to make your writing clear and easily
Aids understood:
● Use simple language
● Use short sentences
● Be consistent in use of tenses
● Check your spelling of words
● Use headings
Use simple language

While use of some technical words is necessary, avoid excessive use of them. Whenever
possible, use simple, everyday, easily understood words and phrases. If you cannot express
yourself in simple language, you probably are not sure what you want to say.
Use short sentences
Another way to ensure clarity is to use short sentences and paragraphs. As a rule, keep most
sentences to 10 to 15 words or less. Short sentences are easier to read and understand. Some
sentences, of course, will exceed the 15 word limit, but these should be the exception and should
be mixed in with shorter ones. Using short paragraphs will also improve the readability of your
report. Most paragraphs should be less than one page long (typed, double-spaced).

Chapter 20 Page 5
Watch use of tenses

Scientific writing uses both past and present tenses. Past tense is generally used to state the
problem (the problem investigated was, etc.); to present the review of literature (Fattah found
that, etc.); and to describe the design used (a longitudinal design was used). Past tense is also
used to present the findings. After all, the writer is reporting on data that was collected at some
previous time. Most writers, however, shift to the present tense for presentation of the discussion
or interpretation of the findings and conclusions. This is proper because both forms of discussion
apply to the present. Also, hypotheses are usually expressed in the present tense (years of
schooling is positively related to later income). We suggest you use these practices past tense
for most of the report and present tense for discussing results and presenting conclusions.
PREV NEXT

Chapter 20 Page 6
Introduction Check your spelling

There is no excuse for misspelled words in scientific reporting. Using a word processing
Improving
program that identifies spelling errors is your best protection against making spelling errors. If
your you are not using a word processor and are not sure how to spell a word, look it up in a dictionary
writing and get the correct spelling.
Guidelines Use headings

for clear The short title above is a heading. Headings are used to announce and identify a section or
writing sub-section of a report. You no doubt have noticed headings throughout this and preceding
chapters. We have used headings to give you a sense of the organization of material and to
Writing introduce the content that follows. You can use headings in your report for the same purposes.
your report
Most reports have at lest two levels of headings. One level identifies the main sections of the
chapters of a report. These are generally placed in the center of the page or at the left hand
Final margin. To show that they are main headings they are frequently presented in larger and bold
checking type, as has been done in this book. The next level of headings identifies sub-sections with the
main sections. These are usually shown at the left margin. These secondary headings sometimes
Oral are in plain type and sometimes in bold type or italics. If third-level headings are needed, they
presentations can be shown at the beginning of a paragraph but set off from the text by being in bold, italics or
underlined.
Using the
Another point in style is whether you capitalize first letters of each word or just the first letter of
mass media the first word in any heading or title. We chose to capitalize first letters of each main word in the
titles for chapters, but only the first word in headings within chapters.
Aids
Whatever style you use, use it consistently. Check each chapter to see that you have used the
same style throughout your report.
The preceding discussion only touched on rules for grammar and composition. The Web sites
mentioned earlier also provide guidance on these points. In addition, we suggest you look at some
of the pages available on the following sites:
The Arthur C. Banks Library at Capital Community College. This is the home page for the
library of Webster Community College; it has links to a number of pages that will help you with
almost any writing problem you may have; to start, by clicking on MLA Guide, near the center
of the page, you will find a list of topics, each of which will link you to additional forms of
specific help with format, grammar, composition, paragraph construction and many more topics
Another helpful site is:
Paragraph Development and Topic Sentences. This site describes how to develop a paragraph,
using a topic sentence to provide a unified and focused discussion of an idea or topic that is then
presented in the paragraph
Do a final check

Chapter 20 Page 6
In addition, after you have completed all the chapters in your report, it is a good idea to reread the
entire report. This further review is necessary to make sure that all the points you need to present
are covered; that there are no inconsistencies or redundancies among the various parts of the
report; and that the report presents a clear, full and accurate description of your investigation and
what you discovered. This reading may lead to further revisions.
PREV NEXT

Chapter 20 Page 7
Introduction Writing your report

The following sections describe how to prepare each chapter of a research report. Typically,
Improving
research reports begin with a chapter titled "Introduction" or "The Problem."
your
writing The problem
Regardless of the title used, the purpose of the first chapter is to introduce the problem or topic
Guidelines being investigated and to provide the author's specific approach to the problem. For studies with a
for clear specific focus, this can be done by simply beginning with a heading such as "Statement of the
writing Problem" and then beginning the first sentence with words like: "This study sought to investigate
the relationship between variable A and variable B." or some other appropriate sentence. A
Writing different approach may be more appropriate for a study involving a large set of variables.For
your report instance, an investigation may seek information about knowledge concerning contraceptives and
their use, persons' attitudes toward using contraceptives, their actual use of them, and their view
Final about possible further use. For such studies it might be better to list a set of objectives, one for
checking each set of variables, rather than to try to state a single problem.
The first chapter may also give reasons for selecting the problem under investigation, describe
Oral how hypotheses were derived, discuss the significance and limitations of the study, and provide
presentations an outline of the remainder of the report. When a hypothesis is used, the introductory chapter
often contains sufficient information from other studies to serve as the basis for the hypothesis. In
Using the this case, authors frequently note that further details are provided in the section of the review of
mass media the literature.
Definition of the problem or question being investigated leads to the second chapter, the Review
Aids of the Literature.
Review of the literature
By the time you are ready to write your review of the literature, you will have spent a lot of time
reading and summarizing relevant research reports. Now your task is to tell your readers what
you have learned and to do this in a way that is both interesting and informative. One way to do
this is to provide a historical record of research on the problem or question you have selected. A
review generally begins with the oldest research on the problem, identifies trends or themes in the
research field, and provides the reader with an integrated summary of what has been discovered.
An alternative approach is to organize the review around opposing sets of findings or views.
Using this approach, you would summarize the findings or arguments on one side of the
controversy, do the same for the other side, and then indicate how your study relates to the
disagreement. You might propose a hypothesis that would resolve the controversy one way or
the other and offer your study as a way of testing the hypothesis.
The chapter on the review of literature can also be organized to reveal limitations or gaps in our
knowledge about some topics. You might find that current research has led to certain
conclusions, but the conclusions do not extend to the population you are studying. You could
summarize the literature, indicate where its application stops, and then show how your study will

Chapter 20 Page 7
extend current knowledge.

The review of literature is often used as the basis for generating hypotheses. In this approach, the
conclusions of previous studies are summarized to extract generalizations to form the theoretical
basis for a hypothesis.
Any of these approaches also can be combined. Within the context of the historical approach, for
example, you could show the contradictions or gaps in research findings and then describe how
your study will help resolve these. Regardless of the approach used, when a reader finishes
reading this chapter, he or she should have a good understanding of the current knowledge related
to your problem and a clear idea of how your study fits into the stream of scientific thought on
that particular problem.
PREV NEXT

Chapter 20 Page 8
Introduction In reviewing literature you will need to refer to reports you cite. In Chapter 4, we described how
to prepare references and the List of References that will become part of your research report. In
Improving addition, you will need to tell readers whose research you are citing when you report specific
your findings in the text of your report. Following are two ways researchers do this:
writing "Rahama (1997) found that women's roles during famine were crucial to the well being and
survival of their household."
Guidelines
for clear The same result can be expressed in a slightly different way:
writing "During periods of famine, women's roles were critical to the well being and survival of their
households (Rahama, 1997)."
Writing
your report To keep the writing interesting, authors generally use both forms, varying them from one citation
to the next.
Final Either form refers to an entire publication, which is as specific as needed for citing conclusions of
checking a study. However, when you cite a specific finding, such as a frequency, mean or percentage, or
quote a sentence or two from a report, you also need to provide the page number from which the
Oral information was taken. This is done by adding a comma after the date of publication and adding
presentations the page number or numbers. For example, Rahama also found that 63% of the women became
household heads as a result of adaptations to prolonged famine. This result could be reported as:
Using the "As a result of prolonged famine in western Sudan, women became heads of households in 63
mass media percent of the households studied (Rahama, 1997, p. 11)."
Aids Design or methods

Description of the design is an important part of every report. Readers depend on this
description to learn how the study was conducted and, based on this understanding, to decide to
what extent they will trust and accept your findings and conclusions. It is in your interest,
therefore, to describe your design clearly, fully, and accurately. Only then will your report be
seriously considered by other social scientists.
Although the contents of the design chapter vary, the following topics generally are covered: the
time period of the study; definitions and conceptual analyses of the variables being studied;
operations used to measure the variables; methods of data collection; the definition and size of
the target population, the method of sampling used, the size of the planned and the actual sample
and the response rate; and techniques of data analysis used.
Sufficient detail should be provided so that a reader could accurately repeat your study using the
description you provided of your design how you measured the variables you studied.
Analysis and findings
The chapter containing the analyses of the data and the findings is generally the longest one in a
report. Because of its greater length, extra care has to be taken to preserve the logical order of
the sections and to write each section clearly. The following suggestions should help.

Chapter 20 Page 8
Organize the chapter into sections. We suggest you create a separate section for each major
variable or set of related variables. Identify each section with a heading based on the name of the
variable or variables. This will help readers move through the chapter easily.
Organize analyses and present findings around the purpose of the investigation. Your
purpose was expressed in the introductory chapter. It may have been expressed as the problem or
question you are investigating or in the form of a hypothesis you are testing. Readers will be
looking for your answer to the problem or the results of hypothesis testing. Present these results
first. Giving the results central to your investigation will make your report much easier to read
and understand. After presenting your main findings, you can present any secondary findings.
For most studies, particularly ones based on questionnaires, a large number of analyses can be
generated. Do not try to report everything that occurs to you. Including too much, especially
minor or trivial findings, will only detract from the focus on your study and the important results
you may have. Our advice is to present only the data and findings for the main focus of your
study and save other things that interest you for other reports.
PREV NEXT

Chapter 20 Page 9
Introduction Write each section as an integrated whole. A useful rule is to follow the advice on how to
give a speech. The rule is: "Tell them what you are going to tell them; tell them; and then tell
Improving them what you have told them." In other words, first prepare the reader for the analysis and
your findings you are about to present. The headings you add will partly achieve this goal. In
writing addition, start each paragraph with a short introductory sentence that announces what the
paragraph contains. For the section dealing with the characteristics of the respondents, for
example, you might begin with: "The following sections provide information about the
Guidelines
characteristics of the respondents, beginning with their ages."
for clear
writing Second, tell them. This is accomplished by presenting the information referred to in the
introductory sentence. You might want to use a subheading, such as "Ages of Respondents,"
Writing which would be followed by the data and analyses for this variable. The data can be presented in
your report words only, which we will refer to as the text of the report; in the form of a table or figure; or in
both ways, in the form of a table or figure with accompanying text. The next section might focus
on "Educational Levels" and would have its introductory sentence, and so on for the remainder of
Final
the chapter.
checking
Third, it is a good idea to end the chapter with a brief summary of the findings.This helps readers
Oral integrate what you "told" them earlier.
presentations Present sufficient quantitative data. Particular care has to be taken in presenting percentages.
Always give the numbers upon which percentages are based. This will allow readers to check
Using the your results and combine data to get percentages for other combinations that interest them. Also,
mass media if you used indexes or scales, describe their construction in sufficient detail so that a reader could
use and score them the way you did. There are practical limits, however, to how much data you
Aids can present. Generally you cannot and should not try to present all your raw data for each
variable. Most readers will not be interested in this level of detail and it would make very tedious
reading. Statistical summary measures, such as the mean, standard deviation, and the N upon
which they are based, are generally sufficient.
Combine tables and figure with text. Previously, we emphasized the value of an integrated
presentation of data, analysis and findings. In keeping with this guideline, when the tables or
figures are used, they should be placed in the section of the report to which they apply. Be sure
tables and figures are properly presented. Chapter 17 and Chapter 18 provide detailed
instructions on how to construct tables and figures. When you get ready to prepare a table or
figure, review these instructions. Also, looking at how tables and figures are presented in journal
articles can help you construct your own.
Special care has to be taken in discussing data that are presented in tables or figures. Some
writers simply repeat in words what is displayed in tables or figures. This defeats the purpose of
using tables or figures, which is to provide large amount of data in the quickest and easiest way.
Trying to say in words what a table or figure presents is boring and repetitious. Skilled writers
avoid this problem by using a table or figure to present detailed information and then provide
only a brief summary of the results or an interpretation of what the data mean in the text. The
latter may include a statement indicating whether a hypothesis was or was not supported by the

Chapter 20 Page 9
data in the table or figure or comparison of the data with other results form the study or from
other studies. Comments that will help the reader grasp the meaning or significance of the
findings generally are useful to readers.
Present descriptive statistics first. Descriptive statistics are results for single variables, such as
means or percentages. Present these analyses first so that readers can undertand the characteristics
of your key variables. After you have done this, proceed to any bivariate or multivariate analyses,
including any involving statistical inference.
Each sentence should add new information. Repeating previous information wastes a reader's
time, produces boring material and can cause readers to abandon your report. As you reread what
you have written, eliminate any repetition you find.
PREV NEXT

Chapter 20 Page 10
Introduction Discussion
In short reports, the discussion section or chapter often includes the conclusions and summary as
Improving
well. In longer reports, discussion of the findings is presented in a separate chapter or section.
your Results are generally discussed in the order in which they are presented in the analysis section,
writing beginning with the most important results. Discussion consists of comments on the meaning or
interpretation of the findings. This can include whether or not the null hypothesis is accepted or
Guidelines rejected and, consequently, whether the research or alternative hypothesis is rejected or accepted.
for clear Additional points to consider in writing the discussion chapter include:
writing ● The significance of the findings what are their theoretical or practical importance?
● Whether the findings are consistent with or disagree with findings summarized in the
Writing review of literature.
your report ● Inconsistencies in your data, where findings conflict with one another.
● Discussion of the strengths and limitations of your investigation and how either might have
Final
affected your results.
checking
● Ideas for further research, what new directions are suggested by your findings.
Oral Conclusions
presentations
Your conclusions are the final and generally the most valuable part of your report. They are
what other researchers are most likely to quote in their reviews of literature and what most
Using the
officials and other non-researchers will look for in your report. Obviously, you want your
mass media conclusions to meet the criteria for scientific validity. The following guidelines should help in
achieving both goals.
Aids
Base conclusions on your findings. Conclusions are derived from and must be clearly and fully
supported by your findings. In deriving conclusions you need to guard against letting your
personal feelings or beliefs influence how you interpret your results and the conclusions you
provide. The best protection against making errors in drawing conclusions is to carefully review
all your data and findings. In addition to looking for results that support your hypothesis, look
equally hard for evidence against it and any conclusion you are tempted to draw. Base each
conclusion on careful, objective assessment of all the relevant data.
Try to generalize. Your findings describe what you have discovered, based on data collected
from a specific sample at a given time. In addition to reporting their specific findings, researchers
to offer broader statements that go beyond the limits of the specific results. They seek to
generalize their results. Most generalizing occurs in one of two ways. One form is empirical; the
other form is conceptual. Empirical generalizations are usually based on statistical inferences, as
described in Chapter 19.
Conceptual generalization is a bit trickier and depends on the relationships between the indicators
used for purposes of measurement and the concepts the indicators represent. Let's return to the
investigation of factors related to fertility in the Sudan (see Chapter 2). Empirical findings from
that study showed that fertility was inversely related to the years of schooling among women.

Chapter 20 Page 10
Separately, we know that years of schooling among females are increasing in the Sudan.
Therefore, we can expect that in the long run fertility will decline in the Sudan. This prediction is
based on the conceptual linkage of changes in one variable (schooling) and its expected effect on
another variable (fertility). By establishing this conceptual link, we are able to offer a
generalization describing changes we expect to occur. We have made a conceptual generalization.
Do not over-generalize. Over-generalizing means proposing a conclusion that goes beyond the
supporting data: the conclusion, in effect, is too broad or too general. To illustrate, suppose that a
researcher found that most of a sample of university educated men were politically active. But
suppose, based on these results, that the researcher offered the conclusion that most men from the
area, regardless of their levels of education, were politically active. Would this be a valid
conclusion? No, it would not because the university-educated men were not a representative
sample of all men in the area. In fact, they probably would be quite different in a number of ways,
including their interest in political activities. Most men from the area probably were not
university educated. Therefore, the conclusion went far beyond the facts and represented a case of
over-generalization.
Your best protection against over-generalizing is to think conservatively: limit conclusions to the
narrowest interpretation of your data. Returning to the example of the university-educated men,
the most conservative conclusion would be that university-educated men in the sample were
politically active. Assuming the data indicated this result, such a conclusion would be a limited
empirical statement. If a large enough random sample had been attained, it would be possible to
offer a broader empirical conclusion by extending the conclusion to the population of
university-educated men in the area. This would probably be the safe outer limits of
generalization.
PREV NEXT

Chapter 20 Page 11
Introduction Share your insights regarding your conclusions. Finally, as the investigator you have an
obligation to report any condition that you think might affect the degree to which your
Improving conclusions can be safely generalized. You will know the strengths and weakness of your study.
your You should assess these conditions and report how you think they could have affected your
writing findings and consequent conclusions.
Summary
Guidelines
for clear The purpose of the summary is to give readers the essentials of your report. A summary generally
includes a brief statement of the problem (one sentence is usually enough); statements of
writing
hypotheses used; a brief description of the design (a short paragraph is generally sufficient); a
description of the most important results; and a brief statement of conclusions. Most reports can
Writing be adequately summarized in one or two, double-spaced typewritten pages.
your report
List of references
Final The list of references provides the full reference for each publication, Web site, or other source
checking that you cite in the text of your report. Chapter 4 describes how to prepare a list of references. In
some reports, you may see "Bibliography" or simply "References" instead of "List of references."
Oral If your university does not have a required style, you can use any of these terms to identify your
presentations references.
Using the Appendices

mass media Appendices are used to present information that is too detailed to be included in the text of the
report. Appendices often include a copy of the questionnaire, interview guide, observation
Aids checklist, or other instruments used to collect data; the content of an illustrative interview or
observation session; or tables containing detailed data and analyses beyond those needed for
presenting the essential results of the study.
When two or more appendices are used, the first is usually labeled Appendix A and is given a
brief title; the second is labeled Appendix B and is given a brief title, and so on. Pages are
identified as A-1, A-2, etc.
Abstract
Many universities require an abstract of research reports. An abstract is a short description of the
important elements of a report. Abstracts generally vary between 100 and 250 words. Elements to
include in an abstract are:
● The statement of the problem or question that was investigated
● The basic features of the design used (survey, experiment, field study, etc.)
● Definition of the key variables
● Description of how the variables were measured
● Description of the analyses used to establish the findings
● Description of the key or most important findings of the investigation

Chapter 20 Page 11
● Statement of the conclusions and their significance or implications

Each of these points is described in a sentence or two. An example of an abstract is provided in
Chapter 4.
PREV NEXT

Chapter 20 Page 12
Introduction The title

We have saved discussion of the title until last because a title is best written after you have
Improving
absorbed the full implications of your investigation. Your title is the first thing a reader sees of
your your report. A brief, concise descriptive title invites readers to read the report. A long, poorly
writing written title that leaves the reader wondering what the report is about will turn them off.
Guidelines Writing a short, clear title can be challenging. Here are a few suggestions. First, consider the
for clear number of variables you studied. If you focused on the relationship between a few variables, base
your title on the names of the variables and their relationship. Here are a few examples from the
writing
literature:
Writing Status, Power, and Influence of Women in an Arab Village in Israel (Haberer, 1985)
your report
The Role of Rural Women in Family Decision Making In Iraq (Fattah, 1981)
Final Each of these lists specific variables, such as "power" or "family decision making," and each
checking gives a clear idea of what each study was about.
When a large number of variables are studied, titles generally indicate only the kinds of variables
Oral studied. For example:
presentations
Measuring Women's Work in Developing Countries (Donahoe, 1999)
Using the Women's Roles and Social Change in Sudan (Darkoh, 1994)
mass media
Each of these describes a broader set of variables, such as "women's work" or "social change"
Aids while still giving a clear idea of what was studied.
Titles should describe what was studied and not the results or conclusions of the study. Look at
the titles above: None indicates what was found: Instead, each describes the key variables that
were studied. Titles generally are not written in the form of questions, even though the
investigation was undertaken to answer a specific question. Words like "influence" or "effect"
should not be used in titles. These words imply cause and effect. It is better to use terms like
"association" or "relationship" between variables X and Y. These terms do not carry the image of
causal relationships.
Research papers that can serve as a model for preparing your paper are available on the Internet.
The following Web site provides the full text of a research paper as a guide for students facing
the task of writing a research paper.
Sample Paper presents the full text of a research paper prepared according to the rules of the
American Psychological Association
Journal submissions
In addition to writing a required research report, you may choose to prepare a manuscript for
submission to a professional journal. (Technically, a document prepared for submission to a

Chapter 20 Page 12
journal is called a manuscript. It becomes an "article" after the manuscript is published in a

journal). Most journals have very strict requirements for preparation of manuscripts. If you plan
to seek publication in a specific journal, be sure to follow the formatting and style requirements
for that journal. Manuscript requirements are provided on the inside of the front or back cover of
a journal or on the Web site of the publisher of the journal.
PREV NEXT

Chapter 20 Page 13
Introduction Many journals are published in developing countries. Your chances of having an article accepted
by a journal in your country or a neighboring country are much greater than by an international
Improving journal published in Europe or North America. For journals published in the latter regions, a vast
your majority of manuscripts are rejected outright or are accepted only after extensive revision and
writing resubmission. Publication in western journals also involves considerable time delay up to
several years after initial submission.
Guidelines In general, the organization of a journal manuscript is the same as for research reports. Journal
for clear manuscripts, however, are shorter and formats are often more flexible. Also, sections treated
writing separately in reports are frequently combined in journal articles. Articles may begin with an
"Introduction" or they may start without a heading. This first section, whether with a heading or
Writing not, generally includes a review of the literature and the development of the theoretical or other
your report basis for addressing the selected problem or question. "Methods" and "Results" generally are kept
as separate sections, but "Discussion" often includes not only the discussion or interpretation of
results, but also the conclusions drawn from the findings. Some authors keep the conclusions as a
Final
separate section. A summary section is not always used. The article may end with the discussion
checking section or, in place of a summary, there may be a section devoted to directions for future
research.
Oral
presentations For additional guidance and help in preparing a manuscript for submission to a scholarly journal,
take a look at the following site:
Using the Writing the Empirical Journal Article provides detailed guidance and illustrates the process of
mass media writing a journal article
Aids If you plan to submit to a specific journal, study articles published in that journal to see how they
are organized.
Final checking
Following is a checklist for reviewing the final draft of a research report or a manuscript for
journal submission.
Title
Short, concise, clear
Accurately describes the variables studied, relationships that were investigated or the problem,
question, or hypothesis central to the investigation
Abstract
Brief, clear, describes the problem or question central to the investigation
Describes the design used, describes and defines key variables and the ways they were measured,
identifies the target population, sampling method and results of sampling
Describes the main findings

Chapter 20 Page 13
Presents the main conclusions of the study

Introduction
Presents the problem or question being studied
Provides the rationale for selection of the specific problem or question selected for study
May indicate the significance of the problem or research question and the unique features of the
investigation and its particular strengths and limitations
May include the formulation of the theoretical framework used for the study and the derivation of
hypotheses
PREV NEXT

Chapter 20 Page 14
Introduction Review of the literature

● In journal manuscripts, this review is sometimes included in the introduction; in research
Improving reports, it is usually a separate section
your ● Provides an integrated summary of knowledge related to the problem selected for study
writing ● May be organized to show the historical trends in research related to the selected problem,
describe dominant research trends, show areas of agreement and disagreement among
Guidelines findings, or indicate gaps in research
for clear ● References to research cited include names and dates of publication set in parentheses
writing
Methods
Writing ● Description of design used (survey, experiment, multi-method approach, etc.)
your report ● Identification of the target population and sampling design and details (size, method, return
rate, etc.)
Final ● Description and definitions of variables (conceptual analysis)
checking ● Descriptions of operations used to measure the variables (items, scales, indexes, etc.)
● Description of coding and scoring procedures

Oral
● If an experiment was used, provide a full, clear description of procedures used for creating
presentations
the experimental and control groups, for administering the experimental variable, and for
collecting pre- and posttest data
Using the
mass media Findings
● Findings central to the research question or hypothesis are presented first
Aids ● Results of univariate analysis (means, standard deviations) are presented first, followed by
any bivariate, multivariate, or inferential analyses
● Tables and figures are clear, all parts are labeled, direction of percentaging is clear, each
table or figure has a brief, clear title
● Results presented in tables or figures are summarized or interpreted, not simply repeated in
the accompanying text
● Results of tests of null hypotheses are clear, showing the critical values obtained, degrees
of freedom involved, and the probability values for each test
Discussion
● Interpretation of results follows the order in which the analyses were presented, with the
key results related to the research question or hypotheses being discussed first
● If used, results of tests of hypotheses, including any null hypotheses, are discussed in terms
of their theoretical or practical importance
● Results may also be interpreted or discussed in relation to what was found in the review of
the literature, pointing out where your findings agree and where they disagree with current
knowledge on the problem being studied
● May include discussion of the implications of the results, provide recommendations for

Chapter 20 Page 14
future research, or recommendations for action on the problem that was studied
References
● List of references contains all references cited in the text of the report
● All references are complete and formatted according to the standards you chose to use
(those of your university, the American Psychological Association, etc.)
PREV NEXT

Chapter 20 Page 15
Introduction Review of the literature

● In journal manuscripts, this review is sometimes included in the introduction; in research
Improving reports, it is usually a separate section
your ● Provides an integrated summary of knowledge related to the problem selected for study
writing ● May be organized to show the historical trends in research related to the selected problem,
describe dominant research trends, show areas of agreement and disagreement among
Guidelines findings, or indicate gaps in research
for clear ● References to research cited include names and dates of publication set in parentheses
writing
Methods
Writing ● Description of design used (survey, experiment, multi-method approach, etc.)
your report ● Identification of the target population and sampling design and details (size, method, return
rate, etc.)
Final ● Description and definitions of variables (conceptual analysis)
checking ● Descriptions of operations used to measure the variables (items, scales, indexes, etc.)
● Description of coding and scoring procedures

Oral
● If an experiment was used, provide a full, clear description of procedures used for creating
presentations
the experimental and control groups, for administering the experimental variable, and for
collecting pre- and posttest data
Using the
mass media Findings
● Findings central to the research question or hypothesis are presented first
Aids ● Results of univariate analysis (means, standard deviations) are presented first, followed by
any bivariate, multivariate, or inferential analyses
● Tables and figures are clear, all parts are labeled, direction of percentaging is clear, each
table or figure has a brief, clear title
● Results presented in tables or figures are summarized or interpreted, not simply repeated in
the accompanying text
● Results of tests of null hypotheses are clear, showing the critical values obtained, degrees
of freedom involved, and the probability values for each test
Discussion
● Interpretation of results follows the order in which the analyses were presented, with the
key results related to the research question or hypotheses being discussed first If used,
results of tests of hypotheses, including any null hypotheses, are discussed in terms of their
theoretical or practical importance
● Results may also be interpreted or discussed in relation to what was found in the review of
the literature, pointing out where your findings agree and where they disagree with current
knowledge on the problem being studied
● May include discussion of the implications of the results, provide recommendations for

Chapter 20 Page 15
future research, or recommendations for action on the problem that was studied
PREV NEXT

Chapter 20 Page 16
Introduction Results may also be interpreted or discussed in relation to what was found in the review of the
literature, pointing out where your findings agree and where they disagree with current
Improving knowledge on the problem being studied
your May include discussion of the implications of the results, provide recommendations for future
writing research, or recommendations for action on the problem that was studied
Guidelines References
for clear List of references contains all references cited in the text of the report
writing
All references are complete and formatted according to the standards you chose to use (those of
Writing your university, the American Psychological Association, etc.)
your report Oral presentations
Final Another way to communicate your research results is to present them orally in a seminar with
classmates, to representatives of governmental or non-governmental organization, or to other
checking
groups. Preparing for an oral presentation is different from preparing a written report. Oral
presentations depend primarily on persons hearing and understanding what you say. This is
Oral different from learning about results by reading a report. When reading, persons can go back and
presentations clarify a point that was not clear on first reading and pause to think about what they are reading.
None of these can be done during an oral presentation. The speaker makes a point and moves on
Using the to the next. Some members of the audience may be ready to move on with the speaker; others
mass media may not. Unless persons show signs of confusion - frowns or disinterest - the speaker often is not
aware that he or she has lost the audience. Preparation for any kind of talk, therefore, should be
Aids based on how persons listen and what they can absorb from hearing what you say.
A few suggestions may help you make an effective oral presentation.
First, plan your remarks to fit the time period you are given. Timing how long your remarks last
is best way of knowing exactly how long your presentation is.
Second, select a few key points you want to make and build your presentation around them.
These should be the most important things you found and the ones you believe are most relevant
to the audience you will be addressing. In a seminar with classmates, for example, the key points
might be selected for their scientific value or significance. For a presentation to a governmental
or nongovernmental organization, you might want to emphasize the practical implications of
what you found. For most audiences, it is better to focus on results. Generally, this is what will
interest persons the most, rather than details of the methods used. If someone wants details on
methods, you can provide them in the question and answer period following your formal remarks.
Third, present your remarks enthusiastically. If you are excited about what you found, your
audience will sense this and pay more attention. Try to persuade them to accept what you think is
important from your investigation.
Fourth, practice your presentation before you make it. Rehearsals can have several important

Chapter 20 Page 16
benefits. By actually giving every word of your talk, you will have an accurate idea of how long
it is. You can then either shorten it or add material to fit the time span you will have. It is far
better to make this adjustment before you speak. Audiences get very restless when the speaker
says, "Oh, I will have to skip over this" or "Well, I guess I will have to shorten this, but you will
get the idea." They also resent anyone who goes well beyond the expected time.
Rehearsals can also build your confidence in your ability to give oral presentations. As a start,
you may want to do a rehearsal just by yourself, with no audience, to develop some confidence
and to get a "feel" for your material. As a follow up, it is a good idea to ask some persons to serve
as an audience and to give you feedback on your presentation and its content. As we said with
preparing draft material for a research report, getting criticism is not always pleasant, but it is the
only way to be sure that what we want to say will be clear and understood by others.
PREV NEXT

Chapter 20 Page 17
Introduction Fifth, be careful in how you use visual aids. Use of effective visual aids can help get your key
points across. But, if not properly prepared and used, visual aids can detract from your talk. Aids
Improving may be as simple as a "handout" listing your key points or you may choose to use transparencies
your or computer-based projections containing key points. If you use any of these, keep them simple,
writing with only a few key points listed in single lines on each transparency or slide. Audiences are
turned off by visual aids containing lines and lines of type. These "dense" presentations are
usually hard to read, contain more than persons can absorb, and distract from the verbal
Guidelines
presentation. With just a few main points on a slide, your audience can read your slides and hear
for clear your key comments.
writing
If you use any visual aid equipment, check in advance to make sure that power and connecting
Writing cables will be available, that the equipment will be available and working, and that you know
your report how to operate the equipment. Also, if you plan to use visual aids, pretest them: use them in your
rehearsals exactly as you will in your final presentation and see how effective they are in
communicating to your audiences.
Final
checking For more information on making oral presentation, visit the following Web site:
KU Medical Center: On-Line Tutorial Series describes and illustrates how to prepare for
Oral
effective oral presentations, how to design effective visual aids to support, not detract from, oral
presentations
presentations, and how to create an effective poster presentation
Using the Using the mass media
mass media
Your research results can also be applied to help create change in your country. This can be done
by publishing your results in local newspapers or magazines or by presenting results on radio or
Aids television programs. These forms of presentations also require adapting the content of your
research report to meet the requirements of the mass media. Editors or producers of mass media
material will focus on the implications or meaning of results for the public and be less interested
in the details of design or techniques of measurement. You may be asked to "simplify" your
results by putting them into everyday concepts and language. This can usually be done without
sacrificing or diluting the scientific integrity of your findings. To get greater public interest,
sometimes editors or producers ask researchers to claim more from their data than is warranted. If
asked to do so, standing by the limits of your research will win the respect of most editors and
producers and protect your integrity as a social researcher.
Aids
Key terms
You get a break. By now, we assume you know all the key terms used in this chapter, so none is
listed.
Main points
1. Research results can be communicated in the form of research reports, journal articles,
orally, and through the mass media.

Chapter 20 Page 17
2. Each form of reporting has its own preparation requirements. Learning through reading is
different that learning by hearing.
3. With practice and patience, each of us can improve our writing and speaking skills.
4. Effective writing depends in part on how one prepares for writing. Ways to prepare
include: know what you are expected to produce; start writing well before deadlines;
organize your material before writing; outline before you write; following examples of
good writing; spending time on revising what you have written; getting others to review
and comment on your draft material; and using a word processor to type your report.
5. Guidelines for improving the process of writing include: use simple language; use short
sentences; be consistent in use of tenses; checking your spelling; and using headings to
provide a structure for chapters in your report.
6. The typical order of chapters or major sections of a report is: introduction, review of the
literature, design or methods, findings or results, discussion, conclusions, and summary.
Reports may also include an abstract and appendices.
7. The introduction presents the problem or question being investigated and may include the
reasons for undertaking the study.
8. The review of the literature presents an integrated summary of what is known on the topic
selected for investigation.
9. The chapter on design or methods sets forth a full, detailed description of the methods
used, including how variables were measured.
10. The results or findings chapter presents how the data were analyzed and what was
discovered.
11. The discussion chapter provides the author's interpretation of the results, including a
discussion of their theoretical or practical value.
12. The conclusions chapter provides the most general statements the author is able to offer
based on the results of an investigation.
13. The summary provides a brief description of the entire study, including what was
investigated, methods used, what was found and what the author thinks it means.
14. Reports and journal articles also include a list of references, also called the bibliography or
simply "References." These contain the full reference for each publication or source of
information mentioned in the report or article.
15. Titles should be brief, clear, and describe what the research was about.
16. Presenting research results orally is different from presenting them in writing. We absorb
information differently by hearing than by reading. These differences have to be taken into
account when preparing to make an oral presentation.
17. Guidelines for making an effective oral presentation include: know how much time you
have and prepare to stay within the time limits; select and focus on a few key points; be
enthusiastic or persuasive in your presentation; practice your presentation before making it;
use visual aids carefully.
18. Newspapers, magazines, and radio and television offer additional ways to communicate
your research findings, particular to the public. Editors and producers of mass media
generally are more interested in your findings and not in the details of the methods used.

Chapter 20 Page 17

Glossary
Glossary
This Glossary provides definitions for key terms used in the previous chapters. Most
definitions include other key terms. Key terms used in the definition of other key terms
are in bold type. This lets you go to any term you encounter and find its meaning.
In addition, other glossaries of social research and statistical terms are available on Web
sites. The names and Web addresses of some of these Glossaries are listed under "Aids -
Internet Resources" at the end of Chapter 17
ABCDEFGHIJKLMNOPQRSTUVWXYZ
A
Abstract (abstraction) - a mental image of something that people experience and agree
to describe in a certain way; concepts for example, are abstractions derived from
observations and defined in scientific terms; abstract is the opposite of concrete, which
refers to the specific things we experience and can observe
(An) abstract: a short summary of a publication, usually about 250 words
Alternative hypothesis: the original hypothesis formulated at the beginning of a
research project
Analysis: the process of summarizing and organizing data to establish the results of an
investigation
Analysis of variance: a statistical test used to determine if differences among three or
more means are statistically significant
Anonymity: the assurance given to respondents that no one, not even the investigator
will be able to identify the respondent or any data supplied by or about the respondent
Area sample: see cluster sample
Assessment research: research undertaken to see if a program is achieving the
objectives set for it; also referred to as evaluation research
Association (or associated): refers to the extent to which one variable is related to
another variable; a measure of how changes in one variable influence changes in another
variable
Attribute: the elements that make up a variable; may be expressed either in words
(male or female) or in numbers
http://www.srmdc.net/glossary.htm (1 of 30) [2/11/2008 9:40:24 AM]

Glossary
Available data: data that already exists in the form of responses to previous surveys, as
mass media material, or as other written, audio, video, or cultural artifacts
Average: a loose term used in everyday language to describe one form of the central
tendency of a distribution; statisticians use mean in place of average; two other
averages or "typical" scores for a distribution are the median and the mode
B
Back translation: the translation of a document that was translated into a new language
and then back to the original language
Bar chart: a graphic way of presenting data in which bars representing the attributes
of a variable are arranged along the X axis of a graph and the height of the bars, as
measured on the Y axis, show the frequency for each attribute; also known as a
histogram
Bias: any tendency to see events in a certain way that causes distortions in the collection
or analysis of data or in drawing conclusions from findings
Bimodal distribution: a distribution with two modes
Bivariate analysis: the simultaneous analysis of two variables; bivatiate analysis is
generally done to find the extent of association between two variables
Bogardus social distance scale: a measurement technique for finding how closely
respondents say they are willing to associate with members of some designated group;
social distance scales are used to measure attitudes toward some group of persons
Browser: an Internet -based service that allows a computer to connect with the Internet
Browsing: casual examination of books or other materials in search of relevant
materials; one can also browse among Web sites, using links on sites to move from one
site to another; this form of browsing is called surfing
C
Call back - the act of making a second or third visit to a respondent to obtain an
interview
Case study: a detailed investigation of a person, organization, village or other entity for
the purpose of understanding the entity in all its complexity as fully as possible
Casual observation: observation of behavior in which actions are recorded in narrative
form; stands in contrast to structured observation where observations are noted in
terms of pre-defined categories
Categorical variable: a variable whose attributes form some kind of a classification;
the categories used form the elements of the classification; male and female, for
example, would be categories of the classification of persons based on gender;
categorical variables are also referred to as qualitative variables

Glossary
Causal hypothesis testing: testing a hypothesis under carefully controlled conditions,

as in a true experiment, to exclude the influence of any variable other than the
independent or experimenta l variable upon the dependent variable; under these
conditions, changes in the dependent variable are assumed to be caused by the
independent or experimental variable
Cause and effect (or causal relationship): refers to a relationship where one variable
is thought to be solely or substantially responsible for changes in another variable; see
the definition of causal hypothesis testing
CD-ROM: stands for "compact disk read only memory," a form of electronic storage for
music, data files and other information; is "read" or played with the help of a computer
Cell: a part of a table identified by the intersection of a column and a row of the table
Census: collection of data from all the members of some population; also called
enumeration
Central tendency: measures of the degree to which scores are clustered around the
mean of a distribution
Chart format: used when the same question is repeated with the same response
categories; example, when asking for the ages of all members of a household
Chain sample: see network sampling
Chance selection: see random selection
Chi square: a statistical test for determining whether two variables are independent of
one another; chi square is based on comparing differences between observed and
expected frequencies for various cells in a table
Chronbach's alpha: used in item analysis to select items that are highly associated
with the other items in a composite measure; items whose scores correlate moderately
with other items are assumed to be measuring the same thing and, therefore, the scores
can be safely combined to provide a composite measure
Class interval width: closely related to class limits; any whole number (22, 51, 175,
etc) is really the midpoint of range that extends 0.5 below and 0.5 above the number,
thus the interval of 20-29 has a class width of 19.49 to 29.49 with 25.0 as its midpoint
Class limits: the range of numbers that are created when continuous data are combined
to form broader categories or intervals; for example, exact ages can be combined into
intervals, such as 20-29, 30-39, etc.; the ten year categories are the class limits for the
age intervals
Classical experiment: a technique for testing hypotheses under carefully controlled
conditions, where the experimental or independent variable is administered to the
experimental group but not to an equivalent control group and measurements of the
dependent variable are compared between the two groups following the experiment;

Glossary
also called the true experiment
Closed item: a question or item with a fixed set of responses; respondents are asked
to select the response that most closely matches their views
Cluster sampling: a probability sampling design based on random selection of
successive clusters or units with a simple random sample used in the final cluster to form
the final sample; also referred to as an area or multistage sample
Codebook: a record book used to provide information about variables, their attributes,
and their locations in a data file; a codebook is used to plan analyses of variables and to
interpret the results of analyses
Code transfer sheet: a sheet of paper with columns for recording the attributes of
variables and with rows for each respondent or case; the code for each response or
observation is placed at the intersection of the column for the attribute and the row for a
particular respondent
Coding: the process of assigning numbers to represent the attributes of indicators;
coding is a necessary step before data can be entered into a computer data file because
computers can only "read" numbers
Coefficient of correlation: a statistical measure of association between two
quantitative variables; a coefficient of correlation can vary between ±1.00
Coefficient of determination: the squared value of the coefficient of correlation; it
indicates the percentage of the variation in dependent variable accounted for by the
effect of the independent variable
Coefficient of reproducibility: the measure of the extent to which responses to a set of
items form a Guttman scale; a coefficient of.90 or higher is the generally accepted
coefficient
Comparison analysis: a research design based on two or more independent samples,
used to estimate how much difference there is among the samples in terms of variables
being measured
Composite measure (or score): scores or other measures based on two or more
indicators; examples are scales and indexes, each of which consist of at least two items
Computer analysis: analysis of data using a statistical software analysis package
stored in a computer
Concept - an abstract description we use to describe things that are real to us but that
we cannot experience directly; mental images we share and use to describe things we
talk about
Conceptualization: the process of defining concepts central to an investigation; also
includes specifying and defining dimensions of a concept for which measurements will
be developed

Glossary
Conclusion: the most general statement derived from the results of an investigation; the
investigator draws conclusions from the analysis of data collected for the investigation
Concrete: refers to specific things we can experience directly; the specific, identifiable
chair you are sitting on is considered concrete; in contrast, the idea of a chair is abstract
Concurrent validity: a method for estimating the validity of a measuring instrument,
such as a scale, based on showing that scores on the instrument differentiate between
persons known to differ in the variable being measured; example, in developing a scale
to measure attitudes toward some group, the scores of persons known to hold strong
positive and negative views toward that group would be compared; if the mean scores of
the two groups were substantially different, the scale would be assumed to have
demonstrated concurrent validity
Confidence interval: the range of values that contain the population parameter at a
specified level of confidence; if a mean is estimated to lie between 5.05 and 8.15, the
confidence interval is 5.05 to 8.15
Confidence level: see level of confidence
Confidentiality: the assurance given a respondent that even though the investigator can
identify the respondent his or her responses, the investigator will protect the respondent's
identity respondent
Construct: another term used for concept; a construct is a definition of a variable we
intend to investigate; the term is used because, as social scientists, we construct a
definition of a variable for the purposes of measurement
Contamination: occurs when members of a control group are accidentally or

otherwise exposed to the experimental variable
Content analysis: a method for analyzing the content of written or verbal material; most
often used in the analysis of mass media materials; based on development of a set of
categories for the coding the content of the material
Content validity: a form of validity based on how well the content of an indicato r
reflects the concept it is intended to measure
Contingency question: a question or item used to select respondents for further
questions, depending on how they answer a preceding question; for example, before
asking persons which political party they belong to, they could be asked if they now
belong to a party, only those who answered "yes" would then be asked for the name of
the party; also called a filter question
Contingency table: see cross classification table
Continuous variable: a variable whose attributes can assume increasingly smaller or
larger values; examples are age or income, each of which can be measured in smaller
and smaller amounts
Control group: the group in an experiment that is not exposed to the experimental or

Glossary
independent variable but is selected to match the experimental group, which is

exposed to the experimental or independent variable, in all other ways
Control variable: a variable that is held constant to remove its influence on other
variables
Controlled comparison: a multivariable analysis in which a control variable is
introduced to see if it causes changes in a relationship between other variables
Controlled setting: any situation created by an investigator for the purpose of
hypothesis testing in which selected variables are controlled to minimize their
influence on the outcome of the research
Convenience sample: a nonprobability form of sampling based on collecting data
from who ever is available or encountered; also called a haphazard sample
Copy: the process of making a copy of material from a data file or Web site by using
the copy function of a computer program
Correlation (coefficient of): a statistical measure of the empirical association

between two indicators; also referred to as the coefficient of correlation; values for
correlation coefficients vary between ±1.00
Criterion validity: the extent to which an indicator for a concep t is associated with an
eternal criterion; for example, the validity of a test given in secondary school for
predicting success in the university is shown by its ability to predict grade point averages
at the end of the freshman year at the university; also referred to as predictive validity
Cross classification: analysis based on showing the relationship between two variables
in categorical form; done in the form of bivariate or multivariate tables
Cross classification table (cross-tabulation table):
a table showing the relationship between two variables; the data for one variable is
displayed in columns and data for the other variable in rows of the table; also referred to
as a contingency table
Cross products: the products of the scores of two variables, required for the
calculation of the coefficient of correlation and other statistics
Cross-sectional design: a design used for surveys; based on use of a probability
sample so that the sample represents a cross-section of a population
Cumulative frequency distribution: a distribution in which the frequency for each
attribute is added to the next higher or lower attribute in the distribution, beginning
with the lowest attribute and adding down the distribution or with the highest attribute
and adding up the distribution; cumulative frequency distributions are useful for saying
how many respondents answered above or below a certain attribute
Cumulative percentage distribution: a distribution in which the percentage for each
attribute is added to the next higher or lower attribute in the distribution, beginning with

Glossary
the lowest attribute and adding down the distribution or with the highest attribute and
adding up the distribution; cumulative percentage distributions are useful for saying
what percentage of respondents answered above or below a certain attribute
Curvilinear relationship: a relationship between two variables in which the direction
of the relation moves in one direction and then reverses; for example, infant mortality
rates are high for the youngest mothers, then decline as mothers are older, only to rise
again for the oldest mothers; also called a nonlinear relationship
D
Data: the specific bits of information collected by a scientifically valid method of
collection; can be in the form of observation, by means of an experiment, or by asking
persons questions as part of a survey
Data collection: the planned, systematic process of obtaining data to answer a research
question
Data cleaning: the process of reviewing codes for attribute s entered into a computer
data file to find and correct errors
Database: a searchable computer-based compilation of information on a topic or
covering a discipline
Data entry: the process of entering codes into a data file stored in a computer; data
entry must be done according to the rules of the software program being used
Data file: coded data stored in a computer according to locations specified in a
codebook
Date modification: changing or adding data after the initial data were coded; examples
include developing composite scores or recoding open-ended responses to form new
categories
Deductive logic: a form of reasoning from a general principle or statement, often based
on a theoretical framework; for example, derivation of a hypothesis from a theoretical
framework
Degrees of freedom: a value used in interpreting tests of statistical significance;
degrees of freedom are calculated in different ways for different tests of significance
Dependent variable: the variable that depends on or is influenced by another

variable; dependent variables are what researchers seek to understand and explain
Descriptive research: investigations whose purpose is to provide precise descriptions of
variables and their relationships; surveys are frequently used as designs for descriptive
research
Descriptive statistical analysis: analysis of data to describe the characteristics of
sample or for measuring relationships between variables; examples include measures of
central tendency (mean, median and mode), measures of variability (variance and

Glossary
standard deviation) or measures of association (correlation, chi square)

Descriptive statistics: statistics used to describe features of distributions of scores,
such as means and standard deviations
Design: a plan for the collection and analysis of data; includes selection of a method of
collecting data, ways of measuring variables, a sampling plan, and plans for the
analysis of the data to be collected
Dimension: a specified and defined aspect or component of a concept selected for
measurement; dimensions of a concept are identified by the process of c
onceptualization
Direct relationship: see positive relationship
Discrete variable: a variable whose attributes cannot be separated into smaller units;
for example, gender exists in only two forms - male or female
Distribution: an ordered set of numbers showing how many times each occurred, from
the lowest to the highest number or the reverse
Download: the act of copying information from a computer-based file, such as those
found on Web sites, to the hard or floppy drive of a computer
Draft questionnaire: the form of a questionnaire ready for pretesting; a draft
questionnaire is usually revised based on information obtained during one or more
pretests
E
Ecological fallacy: an error in drawing a conclusion about the behavior or attitudes of
individuals when data are collected at the level of groups to which individuals may
belong
Edge coding: a way of showing codes for responses in which the codes assigned to
responses are written in the margin of the questionnaire opposite the item to which they
refer
Email: stands for "electronic mail," a form of communication using the Internet as a
way of connecting to persons you wish to communicate with
Email survey: a survey conducted by sending a questionnaire by email to a list or
sample of email addresses; respondents are asked to complete and return the
questionnaire by email
Empirical: refers to using one's senses (sight, hearing, touch, smelling, and tasting) to
learn about events; empirical research is based on measurement of observable events
Empirical generalization: a statement or conclusion based on empirical results;
basing a conclusion on a relationship between two indicators is an example of an
empirical generalization

Glossary
Empirical relationship: a measured or observed relationship based on data for two

variables
Empiricism: the use of one's senses to observe and record events external to ourselves;
scientific inquiry is based on knowledge derived from observation
Enumeration: the process of collecting data from all the members of a population;
also called taking a census
Equivalent forms measure of reliability: a technique for estimating reliability based
on the degree to which results from two equivalent scales or sets of observations are
associated; a high level of association indicates high reliability
Evaluation research: research undertaken to see whether a program or activity is
meeting or has met the objectives set for it
Executive summary: a summary of a report prepared to give a brief but complete
description of the purpose, methods used, results, and conclusions of an investigation;
executive summaries are often written to be understood by persons in administrative
positions and those without research training
Experiment (experimental design):a research method used to test hypotheses under
carefully controlled conditions designed to rule out the effects of any variables other
than the experimental treatment; elements of an experiment include random
assignment of subjects to either the experimental or the control group, measurement
of the dependent variable in both groups at the beginning of the experiment;
application of the experimental or independent variable to the experimental but not the
control group; measurement of the independent variable at the end of the experiment,
and comparison of measures on the dependent variable for the pretest and posttest
measurements for both groups; due to the effect of the experimental treatment, larger
differences between pretest and posttest measurements are expected in the experimental
as opposed to the control group
Experimental effect: in an experiment, the measure of the impact of the

experimental treatment upon members of the experimental group; the experimental
effect is measured as the difference in pretest and posttest scores in the experimental
as opposed to the control group
Experimental group: the group of subjects in an experiment who receive the
experimental treatment as contrasted to the control group whose members are not
subjected to the experimental treatment
Experimental mortality: refers to the loss of subjects during the course of an
experiment; high experimental mortality undermines the validity of an experiment
Experimental treatment: in an experiment, this is the variable that is changed by the
experimenter to see its effect on the dependent variable; also called the independent
variable or experimental variable
Experimental variable: this is another name for experimental treatment

Glossary
Experimenter bias: any potential source of error introduced in an experiment in the

way the experiment is designed, the way data are collected and analyzed, or how
conclusions are drawn
Explanatory research: research undertaken to explain why certain behavior occurs;
seeks to provide an explanation for why a relationship exists
Exploratory research: research carried out to learn more about a problem or topic;
usually undertaken to collect data for designing a descriptive or explanatory
investigation
External validity: refers to the degree that the results of an experiment can be extended
or generalized beyond the conditions of the experiment to conditions in the real world
F
Face validity: the characteristics of indicators that suggest they are a reasonable
measure of a variable; example, questions about whether girls have the same right to
education as boys would be reasonably valid indicators of attitudes toward gender
equity
Field jottings: brief notes taken during an observation session to provide a basis for
preparing more extensive field notes
Field notes: the full, detailed descriptions, sometimes based on field jottings, used to
describe what occurred during an observation period; may also contain hypotheses and
tentative explanations for what was observed
Field research: generally refers to qualitative research conducted in natural setting, as
in a village or other public area
Filter question: see contingency question
Findings: see results
Focus group: a group of persons organized by an investigator to obtain detailed
information about a topic or issue through unstructured but guided discussion
Formative evaluation: an evaluation carried out during the development of a program;
used to produce data for guiding the future development of the program
Frequency: the number or count for the occurrence of an attribute of an indicator or
variable
Frequency distribution: an ordered list of the frequencies or counts for all the
attributes of an indicator
Frequency matching: a technique for creating equivalent experimental and control
groups based on randomly assigning the same number of subjects with similar specific
characteristics (so many of one gender, age, ethnic group, etc.) to each group
Frequency polygon: see line graph

Glossary
G
Generalization: a statement based on the conclusions of a study that extends the
conclusions to a broader or more general level
Generalizing: is the process, based on logic, for extending conclusions to a broader or
more general level; generalizing may be done empirically, as when a statistic, based on
a sample, is generalized to the population from which the sample was drawn or may be
done theoretically by generalizing from results based on indicators to theoretical
relationships among concepts represented by the indicators
Grounded theory: development of a theoretical explanation for behavior based on the
analysis of data; this approach differs from the traditional deductive derivation of a
hypothesis; grounded theory is used most often to generate explanations for behavior
observed in qualitative investigations
Grouped data: continuous data that are combined into larger intervals or groups;
example, instead of analyzing data for the exact ages of respondents, ages could be
combined into five or ten-year intervals
Guttman scaling: a composite measure in which the scores for items indicate the
expected pattern of responses
H
Halo effect: in interviewing, the tendency to expect to receive a response in a certain
(biased) way based on how previous respondents had responded; represents a
systematic error in data collection
Hand analysis: analysis of data by hand counting; also referred to tallying responses
Haphazard sample: see convenience sample
Histogram: see bar chart
History effect: the influence of events on subjects during the course of an experiment;
example, an experiment to change attitudes toward some group could be invalidated by a
major public event concerning the group in question
Home page: the initial screen or page shown when you visit a Web site; the home page
generally has links to other pages on the site and to other related sites
Hypothesis: a tentative statement of an expected relationship between variables, usually
derived deductively from a theoretical framework; hypotheses may also be based on
an empirical findings or conclusions; hypotheses are confirmed (accepted) or
disconfirmed (rejected), based on empirical data
Hypothesis testing: the process of obtaining empirical data to judge whether a
hypothesis is confirmed (accepted) or disconfirmed (rejected); statistical tests are used
in making this judgment

Glossary
Hypothetical-inductive process: based on the combined use of deductive logic to

derive a hypothesis followed by use of inductive logic to test whether the hypothesis is
confirmed (accepted) or disconfirmed (rejected)
I
Independent (independence): the lack of a relationship between two variables; when
no relationship is observed, the variables are said to be independent
Independent variable: the variable that influences the value of another variable (the
dependent variable); in an experiment, the independent variable is the one that is
manipulated by the experimenter; in an experiment, the independent variable is also
called the experimental or treatment variable
Index: a composite measure consisting of two or more indicators assumed to be of the
same level of intensity; the indicators may be selected because they represent different
dimensions of the concept the index is intended to measure
Index score: the interim composite score assigned to mixed type responses as a step in
deriving a final Guttman score for a set of items: see Guttman scaling
Indicator: a variable used to measure a concept or one of its dimensions
Indirect relationship: see negative relationship
Inductive logic: a form of reasoning used in deriving conclusions from the results of an
investigation; reasoning from the bits or separate pieces of data to a conclusion
Inequality signs (< and >): are used in reporting the results of statistical tests of
significance to show whether the result produced a probability level of "greater than,"
shown as >, or "less than," shown as <, the .05 or .01 level of significance
Inferential statistical analysis: analysis used in conducting statistical tests of
significance and for estimating parameters in a population from results obtained from
a sampl e drawn form the population
Informed consent: the ethical practice of providing respondents or subjects
information about a study, particularly any risks involved, so they can make an informed
decision about participating in the study
Instrumentation effect: any effect the process of measuring has on the data obtained
in an investigation; in an experiment, administration of the pretest could affect scores
on the posttest, thus posing a threat to the validity of the experiment
Inter-analyst reliability: the degree to which the observations or ratings of the main
investigator and one or more independent observers or analysts agree with one another; a
high level of agreement indicates that the rating or coding categories have a high level of
reliability
Internal validity: the degree to which the results of an experiment can be attributed

Glossary
to the effects of the experimental (independent) variable and to no outside variables

Internet: the set of telecommunication connections and standards for transmitting
information for exchanging information and accessing Web sites from one computer to
another throughout the world
Internet survey: a form of survey in which questions are posted on a Web site or sent
by email to r espondents who reply by completing the questionnaire on the Web site or
sending responses by email
Interpretation (of results): the process of saying what the results mean; the purpose of
interpretation is to develop the conclusions of an investigation or to explain what was
found
Interrupted time series design: a form of a quasi experiment based on one group,
with no control group; the occurrence of some variable is compared over time before
and after some event that is thought to have an influence of the variable; example, does a
large increase in the tax on cigarettes cause a decline in sales; data for sales before and
after the imposition of the tax would be compared to answer this question
Interval: the range of numbers used for grouping continuous data
Interval measurement: based on an ordered set of categories where the intervals
between the categories are assumed to be equal; the numerical values assigned, however,
are not based on an absolute zero (examples, intelligence scores. scores on an attitude
scale)
Interval sample: see systematic random sample
Interview schedule: the set of questions used to interview respondents; today, the
term questionnaire is used in place of interview schedule guide or schedule
Interviewing: the process of collecting data from respondents by asking questions and
recording their responses: in structured interviewing, a questionnaire with a fixed set
of questions is used; in unstructured interviewing, questions are asked informally and
in any order, more in a conversational style with respondents
Intra-analyst reliability: refers to the consistency in recording observations or in
coding data by a single investigator
Inverse relationship: see negative relationship
Item: a question or statement used in a questionnaire to obtain data about a variable
Item analysis: the process of determining the extent to which items used in a composite
measure are related to one another and how well each item contributes to the composite
score; item analysis is used to assess the uni-dimensional or internal validity of a items
making up a tentative composite measure

Glossary
Key informant: a well informed person who provides crucial information in a

qualitative investigation; may also review an investigator's description and explanation
of events for accuracy and validity; information obtained from key informants is often
vital to the success of field research
Key terms: words or phrase used in conducting a search of a database or for identifying
relevant Web sites; key terms are selected to represent all the ways a concept may be
expressed
L
Level of confidence: estimate of the probability that a parameter lies within a
specified range of values; example, a research might report a 95 level of confidence of
that the mean for the size of households in a population lies between 8.25 and 10.13
persons
Level of measurement: refers to the characteristics of measurements used to collect
data; there are four levels of measurement - nominal, ordinal, interval, and ratio
Level of significance: the probability that the result of a statistical test could be due to
sampling error; for example, a result said to be significant at the .05 level indicates that
the result could have occurred due to chance variations among sample less than 5 times
out of 100 random samples of the same size from the target population; at the .01
level of significance, the result would be considered as occurring due to sampling error
less than 1 time out of every 100 samples
Likert scale: a composite measure based on a set of responses that range from one
extreme to another; example, a scale may have a number of items with response ranging
from strongly agree, agree, uncertain, disagree, or strongly disagree
Line graph: graphic way to present data in which the frequencies for attributes of a
variable are represented by dots at the intersection of the attribute, as arranged along the
X axis of the graph, and the values for frequencies, listed along the Y axis; the dots are
then connected by a line which creates a line graph; also known as a frequency polygon
Line of best fit: in a graph, shows the relationship between two variables; the line of
best fit is the line that comes the closest to the largest number of dots representing the
values for each pair of attributes for each respondent
Link: a connection provided on a Web site to other pages on the site or to a related Web
site
List of references: the list of the publications, Web sites, or other sources of
information cited in a report; references are prepared according to rules and listed
alphabetically by the last name of the author; the list of references is placed at the end of
the report
Longitudinal design: a research design used to measure changes in variables as they
occur; data are obtained through successive waves of data collection from the same
sample over a period of time

Glossary
M
Matrix format: a table format for presenting items that vary in content but all have the
same response categories; used frequently in presenting items asking about attitudes or
views about some topic or issue
Maturation effect: any naturally occurring processes over time that may produce
changes in subjects in an experiment; as people grow older, they change in many ways;
thus, maturation is a threat to the validity of experiments conducted over long periods of
time
Mean: one of the three measures of central tendency; the value of the sum of a set of
scores divided by the number of scores; in everyday communication, the term average
is used to indicate the mean
Measure: an indicator or set of indicators used to obtain data for a variable; also
referred to as a measuring instrument
Measurement: the process of assigning numerical values or qualitative descriptions to
attributes of an indicator or variable
Measurement error: the difference between the true value for an indicator and its
observed value; the observed value is almost always different from the true value
because of systematic and random errors that occur during data collection and analysis
Measuring instrument: see measure
Median: one of three measures of central tendency; the median is the middle score in a
distribution
Mixed types: in Guttman scaling, mixed types are response patterns that do not match
the expected pattern of responses; mixed types represent errors and reduce the
coefficient of reproducibility, which is the measure of success in creating a Guttman
scale
Mode: one of the three measures of central tendency; the mode is the most frequent
score in a distribution
Mortality effect: refers to the loss of subjects during the course of an experiment; high
mortality a threat to the validity of an experiment
Multimethod research: an investigation using more than one method of collecting data;
for example, an investigator may collect data on the same variables by means of
observation, use of a survey, and analysis of available data
Multiple measures, before and after design: a quasi-experimental design in which
data are obtained for a dependent variable from an experimental group and a
nonequivalent control group at several or more time before and after an event; pre- and
post-event data for the two groups are compared to see if the event had any effect on
the dependent variable

Glossary
Multistage sampling: see cluster sampling

Multivariate analysis: the simultaneous analysis of data for three or more variables;
may be done in the form of tabular analysis or using statistical tests
N
Natural setting: any setting where people carryout normal, everyday activities;
examples, life in the home, village, office, or other public places
Navigating: using navigation buttons and other aids to easily move among pages of a
Web site
Navigation buttons: buttons or aids on a Web site one can click on to move quickly
from one page of the site to another
Negative relationship: a relationship between two variables in which changes in one
variable are associated with changes in the opposite direction for the other variable;
example, years of schooling and fertility are negatively related; as schooling increases,
fertility tends to decline
Negatively skewed distribution: a distribution in which most scores are located near
the low end of the distribution
Network sample: a nonprobability sampling technique in which respondents who are
initially contacted are asked to identify other members of the target population for
inclusion in the investigation; example, in a study of female entrepreneurs, the first
entrepreneurs who were interviewed would be asked to name other females
entrepreneurs they know who would then be contacted, interviewed, and asked to
identify additional female entrepreneurs to be included in the sample, and so on; also
called chain or snowball sampling
Nominal measurement: the lowest level of measurement; consists of giving names to
categories or the attributes making up an indicator; nominal measurement simply
indicates that the categories differ; for example, male and female, are the categories or
attributes of the variable of gender
Nonequivalent control group design: a form of quasi-experimental design based on
use of a control group that is thought to be similar to the experimental group but
whose members were not selected by random assignment
Nonlinear relationship: see curvilinear relationship
Nonprobability sampling: - any form of sampling not based on random or chance
selection of the members of the sample
Nonreactive measure: see unobtrusive measurement
Nonsignificant: any result judged to be within the range of chance variation that occurs
from random sampling

Glossary
Normal distribution: a distribution with a distinctive bell shape and that has certain
specific properties; the most important for researchers is that approximately 68% of the
scores in a normal distribution lie within ±1 standard deviation of the mean of the
distribution, approximately 95% lie within ±2 standard deviations, and over 99% lie
within ±3 standard deviations
Null hypothesis: a hypothesis established as a basis for conducting a statistical test of
significance; the null hypothesis states that no relationship exists between two variables
in the population from which a random sample was drawn; the null hypothesis is
accepted or rejected, depending on the level of significance of result of the test
Number: refers to the size of a sample or the frequency for the number of cases in an
analysis
O
Objectivity: the ability to observe or reason without personal bias; while objectivity is
virtually impossible to attain in all aspects of research, it is an ideal scientists strive to
achieve
Observation: is the process of using one's senses to perceive and record information
about some aspect of the natural world; social scientists observe human interaction and
behavior
Observational design: a flexible plan for conducting observations; usually the basis of
field research
Observed value: the value for an indicator obtained as a result of measurement or
observation; this is the value we know and almost always differs from the true value of
the indicator because of random or systematic errors in data collection
One group, pre- and posttest experimental design: a quasi-experimental design
based on a single group, with a pretest measurement of a dependent variable, followed
by an experimental treatment and then a posttest of the dependent variable; this design
is subject to all the threats to internal validity
Online: refers to connecting to the Internet, databases or other computer-based sources
of information by means of a computer
Open-ended items: questions where the respondent answers in his or her own words; a
question is followed by blank space where the response is recorded or written; there are
no response categories as they are with closed items
Operational definition: the definition of a concept as expressed by the way it is
measured; the operational definition of social status, for example, is given by the item or
items used to measure it
Operationalization: the process of developing measurements for indicators
Ordinal measurement:a measurement based on ranking or ordering of the attributes
of a variable according to some criteria; level of schooling is an example of an ordinal

Glossary
measure - no schooling, primary level, secondary level, post-secondary level

Over-generalization: a statement or conclusion that goes beyond any supporting
findings or results
Over-generalizing: the act of drawing a conclusion that is not supported by data;
example, claiming that most men in a town prefer a certain political candidate when data
were collected only from men who had attended a college or university and who
represent a minority of men in the town
P
Page: a section of a Web site containing information on one of the topics or issues
covered by the site
Panel design: a research design based on successive data collection from the same
sample to measure changes in variables as they occur; panels are used in longitudinal
research
Parameter: the value of any indicator in the target population; an enumeration or
census produces parameters; generally we can only estimate parameters from statistics
that summarize data from a probability sample taken from the target population
Participant observation - a qualitative research technique in which the investigator
participates substantially in the activities of a group; used to develop an in-depth
understanding of the behavior of the group and to see things as members of the group do
Participatory rural appraisal (PRA): an approach to data collection in which
respondents are encouraged to participate fully in all phases of the research; is similar to
and employees many of the features of Rapid Rural Appraisal
Paste: the process of adding material taken from a database, Web site, or other source
to the document you are writing
Percent: a proportion multiplied by 100; literally means per 100; example, if 13
workers out a workforce of 170 were absent on a given day, the proportion absent is
13/170 or .076 and the percent is .076(100) or 7.6%
Perfect relationship: a perfect relationship occurs when a coefficient of correlation
equals - 1.00 or + 1.00; this means that a certain amount of change in one variable is
associated with a specific amount of change in the other variable; in physics, pressure
and volume are perfectly related, an increase in pressure is always associated with a
decrease in volume; in the social sciences, perfect relationships are seldom found
Personal interviewing: refers to the process of collecting data in face-to-face contact
with respondents as opposed to conducting telephone interviews
Pie chart: a graphic presentation of results in which the slices of a circle (the pie)
represent the proportions of each attribute of a variable
Population: the entire group of persons or other cases of interest to an investigator; the

Glossary
group to which an investigator may want to generalize from the sample used in an
investigation
Positive relationship: a relationship between two variables in which they change in the
same direction; as one increases or decreases in value so does the other; also called a
direct relationship
Positively skewed distribution: a distribution in which most scores are toward the
high end of the distribution
Posttest measurement: in an experiment, measurement of the dependen t variable
taken at the end of the experiment
Precision matching: a technique for establishing equivalent experimental and

control groups by randomly assigning subjects with exactly matching characteristics to
one group or the other; example, for two persons of the same gender and age one is
randomly selected for the experimental group, the other to the control group; this
process would be repeated for each set of persons with matching characteristics;
precision matching is the strongest basis for creating equivalent experimental and control
groups
Precoding responses: the process of assigning numbers to represent the attributes of an
indicator at the time the response categories are created; pre-coding is frequently used
with responses for closed items
Predictive validity: a way of estimating the validity of a measuring instrument, such
as a scale, based on the association of scores on the instrument with a scores for some
variable taken at a later time; example, the accumulated grade point average of
university students could be used to validate a test for university success given while the
students were in secondary school
Premature closure: occurs when one draws a conclusion based on insufficient data
Pretest: a test to see if a questionnaire is ready for use in a survey; generally based on
selecting a small sample similar to the one to be used in the actual survey; all the
elements of the questionnaire are tested, from the introduction to analysis of responses to
obtained
Pretest measurement: in an experiment, measurement of the dependent variable at
the beginning of the experiment, before the administration of the experimental variable
Pretesting: see pretest
Print out: the copy of materials printed from a data file, Web site, or a file stored in a
computer
Probability level: refers to the extent to which the results of a statistical test of
significance could be due to random variation that always occurs in sampling (called
sampling error); two probability or "p" levels are typically used in reporting results -
the .05 and the .01 levels; the .05 level indicates that the result could have occurred due

Glossary
to chance 5 less than 5 times out of every 100 samples; the .01 level indicates that the
results could be due to chance not more than once in every 100 samples
Probability sampling: any method of sampling based on random or chance selection,
where each sampling element or unit has an equal chance of being selected
Probe: a technique used in interviewing to encourage a respondent to provide a clearer
or more complete response
Proportion: a fraction or part of something, expressed as a decimal between 0 and 1.0;
example, the proportion of females at a university with 750 females and 4,500 males is
750/4,500 or .166 or .17
Proxies: easily developed substitutes for more precise forms of measurement; proxies
are frequently used in Rapid Rural Assessment to provide data quickly and
inexpensively in place of indicators that take longer to develop and test for validity and
reliability; example, the size or construction materials used in a house could be used as
a proxy for family wealth
Purposive sampling: a nonprobability method of selecting a sample based on
selecting respondents because they are uniquely able to provide needed information;
example, to learn about how decisions in villages are made, an investigator might select
samples of village leaders and elders
Q
Qualitative analysis: examination of data in the form of verbal descriptions rather than
numbers; the purpose of qualitative analysis is to describe behavior and provide an
explanation for what was observed
Qualitative interviewing: a loose, flexible approach to interviewing based on
exploration of topics that are discussed in depth with respondents; respondents are
encouraged to talk at length about issues presented by the interviewer
Qualitative research: a flexible approach to data collection, based mainly on written
descriptions of observed behavior; casual and participant observation and
unstructured interviewing are the main ways of conducting qualitative research
Qualitative survey: a survey based on use of an unstructured questionnaire; the
interviewer uses a conversational style of interaction with r espondents to get responses
in the respondents' own words and with emotional content
Qualitative variable: a variable described in words or by the names of the categories of
which it is composed as opposed to a quantitative variable, which is measured in
numbers; gender is an example of a qualitative variable
Quantitative analysis: analysis of data in the form of numbers; begins with the analysis
of each variable, one at a time (univariate analysis), and may proceed to bivariate and
multivariate analyses
Quantitative research: based on numerical measurement of indicators; used to

Glossary
establish quantitative relationships among variables

Quantitative variable: a variable that is measured in numbers as opposed to a
qualitative variable, which is not; the number of faculty of a university is a quantitative
variable
Quasi-experimental designs are based on some but not all the features of the classical
experiment; most quasi-experiments lack complete control over the independent
variable, but they have the advantage of estimating the effects of variables under real
social conditions; quasi-experiments may be low on internal validity, but are often high
on external validity
Questionnaire - a set of carefully phrased and tested questions or items prepared for the
collection of data; surveys are based on use of questionnaires
Quota sampling: a nonprobability method of sample selection based on setting quotas
for cases from defined components of the target population; once the criteria and quotas
are set, convenience or other nonprobability methods are used to select the sample;
quota sampling has the advantage of at least including sample elements from various
segments or components of the target population
R
Random error: any form of error that may occur in a particular instance during data
collection, coding, transfer, or analysis; examples, a poorly asked question, a
misunderstanding in recording a specific response, or an made in coding data
Random selection: selection based on chance and chance along, with no human
judgment or preference involved; can be accomplished using a table of random numbers
or by selecting sampling elements by chance from a box
Randomization: a process of assigning subjects to either the experimenta l or control
group by chance
Range - a measure of the dispersion or variation among scores; measured as the
difference between the lowest and highest score plus 1
Rapid rural appraisal: an approach to data collection using approximations, called
proxies, for measurement of indicators, that permits collection of data quickly and
inexpensively; often used to help make decisions about the development or future
directions of programs; includes an emphasis on the participation of local persons to the
maximum extent possible in the conduct of the investigation; also known as
participatory rural appraisal
Rank order: the result of arranging scores in descending order from the highest to
lowest; the highest score is given a rank or 1, the next lower score is given a rank of 2
and so on; rules are followed for assigning tied scores
Rapport: is the feeling of trust and confidence an interviewer seeks to establish and
maintain with respondents

Glossary
Rate: a measure of how frequently something occurs within the limits of larger
population; example, the birth rate is the number of babies born within the population
of an area; in social research, rates are expressed in terms of a standardizing base to
eliminate differences in the sizes of the populations being examined; a base of 1,000 is
used for calculating birth rates; thus, if 55 babies were born in a region with 2,400
persons, the birth rate would be 55/2,400(1000) or 22.9
Ratio: the relation between two frequencies; a ratio is found by dividing one frequency
by another; in social research ratios, like rates, are generally expressed in terms of a
standardizing base of 100, 1,000, or some other base; example, using a standardizing
base of 100, the ratio of females to males in a university with 750 females and 4,500
males is 750/4,500(100) or 17; this days that there are 17 females for every 100 males at
the university
Ratio measurement: the highest level of measurement, based on a real zero point; thus,
any number is a ratio of any other number; for example the age of 40 is twice as large as
the age of 20 by a ratio of 2
Raw data: the original data obtained by some form of data collection before data are
coded or modified in any way
Reactivity: occurs when the process of measurement influences the results obtained;
knowing they are being observed, persons, for example, may act differently than they
would in normal situations; in that situation, measurement would bereactive
Record: has two meanings: (1) the written description of observations made during a
session in the course of field research; and (2) the part of a data file, such as a set of
data for a single respondent or the full description of a document retrieved from a
database
Reference: a description of a source cited in a report, such as a book or journal article,
prepared in a specified fashion or format
Reliability: is the degree to which an indicator produces essentially the same result
with repeated measurements
Respondents: the individuals from whom data are obtained, usually by means of
interviewing or by completing a questionnaire
Response rate: the percentage of successfully completed interviews or
self-administered questionnaires over the number that was expected to be completed;
the latter usually is the size of the selected sample
Response set: the tendency of respondents to answer questions or items in way they
answered previous questions; to avoid response set, positive and negative items are
mixed up in any set of items, making the respondent think about each item before
answering
Results: what is discovered when the data are analyzed; findings represent the answer
to the question being investigated; also called findings

Glossary
Review of the literature: the process of reading research reports on a topic of interest;
learning about the results of research on a particular problem or topic
Rounding: the process of establishing the last digit in a number derived from a
calculation; rules for rounding are given in Chapter 17, Box 17.1
S
Sample: a part of target population; samples are selected by either probability or
nonprobability methods; with probability samples we can generalize results from a
sample to the target population; this cannot be done with nonprobability samples
Sample design: the plan prepared for the selection of a sample from a target
population; the simple random sample is one kind of sample design
Sample frame: a list of the sampling elements or units comprising a target population
Sampling element: a single member or unit of the target population; example, a single
member of the full time teaching faculty of a university in the spring of 2003; also called
a sampling unit
Sampling distribution (of the mean): a distribution of means that could be calculated
for all possible samples of a given size that could be drawn from a population
Sampling error: the error in measuring a variable that occurs because of variations
due to random selection of samples; when random samples are used, the amount of
sampling error can be calculated and used in estimating population parameters and in
conducting tests of statistical significance
Sampling interval: the ratio of the size of the sample to the size of the target
population; used as the basis for selecting a systemic or interval sample
Sampling unit: see sampling element
Scale: a composite measure based on multiple items of varying intensity; used for
measuring beliefs and attitudes
Scale types: in Guttman scaling, scale types are response patterns that match the
expected set of responses
Scatter plot: a form of graphic presentation of relationships between two variables;
each pair is represented by a dot at the intersection of the value for the attribute of one
variable, as displayed on the X axis, and the value for the attribute of the other variable,
displayed on the Y axis
Scientific inquiry: a way of examining the world around us based on logical analysis of
what we learn through use of our senses
Scientific method: the approach used in scientific inquiry to establish knowledge about
the natural world, based on principles for identifying concepts, developing hypotheses,
collecting and analyzing data to test hypotheses and generating findings which are

Glossary
incorporated into theories for explaining natural processes

Score: any numerical value used to represent an attribute of an indicator or some
dimension of a variable
Scoring: the process of assigning numbers to the attributes of a variable
Scroll: to move up or down the content of a page of a Web site
Search engine: a software program specially designed to allow persons to search the
Internet to find Web sites of interest
Search service: see search engine

Search strategy: the plan developed for selecting relevant records from a database or to
guide a search for Web sites
Secondary analysis: an investigation based on analysis of previously collected data;
example, reanalysis of survey data collected by another researcher or further analysis of
data available by a government ministry
Selective observation: the tendency to give extra emphasis to certain observations that
agree with a preconceived position and to ignore observations that do not agree with the
preconception
Self-administered questionnaire: a questionnaire designed for completion by
respondents without the assistance of an interviewer
Self weighted sample: a sample selected so that each segment represents its proportion
of the population; using self weighting simplifies analysis of data from samples
selected through successive stages, such as area, cluster, or multistage samples
Session: a period of observation as part of a field or observational study; also used to
describe a period of time for the operation of a focus group
Significance level: see level of significance
Simple observation: observation of behavior, generally in a natural setting, in which
actions are recorded in narrative form and later analyzed; also known as casual
observation
Simple random sample: a probability sample in each sample element has an equal
chance of being selected
Skewed: a distribution that differs greatly from a normal distribution; instead of most
scores occurring near the mean of the distribution, most scores occur at the high or low
end of the distribution
Snowball sample - see network sample
Social distance scale: see Bogardus social distance scale
Social indicators: broad, standardized measures of the quality of life or other

Glossary
socio-economic conditions of geographic areas such as nations, metropolitan areas, or

other areas; used to assess health conditions, educational levels, food availability,
violence, and other conditions
Software package: see statistical analysis package

Spearman rank order coefficient of correlation: measurement of association
between scores for two indicators based their rank order instead of the original values
of the scores
Split half reliability: a measure of the reliability of a scale or other measuring
instrument based on the degree of association between two equivalent forms or halves
of the scale; data for both forms are collected at the same time
Spurious relationship: a false relationship; a spurious relationship becomes apparent
when the initial relationship between two indicators disappears after the effect of third
variable is taken into account
Stakeholders: are individuals who have a strong interest in the outcome of an
evaluation; in the evaluation of an educational program, stakeholders could include
teachers, administrators, and parents, each of whom might have different expectations
for the results of the evaluation
Standard deviation: a measure of variability among a set of scores; it is based on the
sizes of the deviations of each score from the mean of the scores; in a normal
distribution, approximately 68% of the scores lie within ±1 standard deviation,
approximately 95% within ±2 standard deviations, and over 99% lie within ±3 standard
deviations
Standard error: see standard error of the mean
Standard error of the mean: the standard deviation of a sampling distribution; it
shows how much sample statistics, such as a mean, will vary from one random sample
to the next
Statistic: is any finding or result based on a sample; when probability samples are
used, statistics based on the analysis of data from the sample can be used to estimate the
corresponding parameters of the target population from which the sample was drawn
Statistical analysis package (or program): a software program designed to analyze
data stored in a computer
Statistical inference: using the results of a statistical test of significance from a
sample to make an estimate about relationships among variables in a population;
estimates are based on probability levels
Statistical tests of significance: calculations conducted to determine whether
differences between means or relationships between variables, for example, are within
the range that could be expected due to chance variations that occur from one random
sample to the next; statistical tests of significance are based on testing the null

Glossary
hypothesis
Stratified random sample: probability samples selected from two or more sub-groups
or strata of a target population; example, random samples of males and females drawn
separately from the population of university students
Structured interviewing: interviewing based on the use of a questionnaire, which, in
this kind of use is sometimes called an interview schedule; questions are asked in
exactly the same way in all interviews; responses are recorded as given
Structured observation: a quantitative observation technique in which the observed
behavior is recorded in terms of pre-established categories; tally marks are recorded each
time the defined behavior is observed
Subject reactivity: in an experiment, describes the tendency of subjects to act
differently than normal because they know they are being observed; a threat to the
internal validity of the experiment
Subjectivity: the tendency to form opinions or draw conclusions on personal

grounds without sufficient regard for empirica l evidence; the opposite of objectivity
Subjects: in an experiment, persons included in either the control or experimental
groups
Summative evaluation: an evaluation carried out after a program has been fully
developed; the purpose of a summative evaluation is to see whether the program has
achieved the objectives set for it
Surfing: the process of moving from one Web site to another using addresses supplied
by search engines or links on sites that are visited
Survey: a method of gathering data from persons, usually by means of getting them to
respond to items comprising a questionnaire; in developing countries most surveys are
carried out by interviewing persons
Systematic error: any kind of error that affects every case or a substantial number of
cases in an investigation; for example, a poorly worded question that gives unreliable
responses, a mistake in coding that affects all responses for that item
Systematic random sample: a probability sample based on selection of sample
elements at a specified interval beginningwith a randomly selected first interval; using a
sampling interval of 10, for example, one would select the first sampling element
randomly between the first and tenth element on a list and then every tenth element
thereafter; also called an interval sample
T
Table: a way of presenting a large amount of data in very little space; tables can be used
to display frequency distributions for one variable or to show bivarite or multivariate
relationships among variables

Glossary
Tally sheet: a sheet used is recording the counts or tallys for the frequencies of
attributes of variables; example, male and female, as the attributes of the variable
gender, could be listed as rows in a tally sheet and tally marks, such as ///, could be
recorded for each time either attribute occurred
Tallying: the process of counting responses or other data by hand to develop
frequency distributions
Target population: the specific, concrete population defined in terms of its sampling
elements; abstract or general populations are converted to target populations by defining
them precisely as possible such as the population of full time employees of a company
on the first work day of a given month
Telephone survey (interviewing): the process of conducting a survey by means of
telephone interviews
Test-retest reliability: a technique for estimating the reliability of a measuring
instrument based on the degree of association of between scores obtained at one time
and those obtained at a later time; a high degree of association would indicate high
reliability of the instrument
Testing effect: effects on the measurement of an indicator caused by the process of
measuring the indicator; in an experiment, obtaining the pretest measurement can
change how subjects respond to the posttest measurement of the same variable; a threat
to the internal validity of an experiment
Theoretical framework: a set of theoretical statements used for deriving a hypothesis
or for supporting an explanation for some behavior
Theory: the logical expression of relationships among abstract concepts; generally
developed to explain a set of related behaviors or events
Time series analysis: analysis using data available for a number of points in time for
the same indicator; can be used to establish trends or changes in social indicators or
other variables
Time series design: plan for data collection and analysis based on repeated
measurement for a variable at two or more times, such analyses are used to measure
changes or trends in variables over time
Trend studies (designs): investigations undertaken to measure changes that have
occurred in variables; data are collected for variables at two or more points and
compared to see what changes or trend is found; trend studies may involve two or more
points for data collection in the past or past data collection supplemented with data for
the variable at the present time
Triangulation: collection and comparison of data from two or more sources or

using two or more methods of data collection; triangulation is important in qualitative
investigations to ensure that observations are accurately recorded and interpreted; an
example would include collecting data for some indicators by means of observation,

Glossary
from interviewing several key informants, and by checking observations against

available data
True experiment: a technique for testing hypotheses under carefully controlled
conditions, where the experimental orindependent variable is administered to the
experimental group but not to an equivalent control group and measurements of the
dependent variable are compared between the two groups following the experiment;
also called a classical experiment
True value: the actual or real value of a score or other measurement; because of
random and systematic errors that can and do occur in research, we seldom know the
true value of anything we measure
(The) t test: a statistical test to determine if the differences between two means exceed
the difference that could be due to sampling error
Two group, posttest only design: a form of quasi- experiment based on an
experimental group and a control group (often a nonequivalent control group) in
which data are obtained only after the experimental variable has occurred; only
posttest data are obtained and compared for the two groups
Type: any group or category of persons sharing a common set of characteristics that
distinguish them from others; in social research, types are constructed by an investigator
from data describing the special characteristics of respondents; example, an official may
be classified as the "bureaucratic type" based on his or her obsessive attention to detailed
rules and regulations and desire to please his or her superiors
Typology: a classification of persons or groups based on distinctive types created by the
investigator for the purposes of analysis; typologies are useful as measures for dependent
variables but are hard to interpret as independent variables
U
Unidimensional: defining a concept so that it has only one dimension or measurable
set of characteristics
Unit of analysis: the entity used as the basis for combining data for analysis; may be
individuals, families, other groups, organizations, geographic areas, or other entities
Univariate analysis: analysis of a single indicator; univariate analysis is generally the
first step in the analysis of a body of data; it is undertaken to describe each variable in
terms of measures of central tendency (mean, median or mode) and variability (range,
variance or standard deviation)
Universal Resource Locator (URL): the unique address of each Web site
Unobtrusive measurement: any technique of data collection that does not influence the
results obtained; for example, observing how persons are dressed and using this as an
indicator of social status, analyzing data already collected; also called nonreactive
measures

Glossary
Unstructured interviewing: a flexible form of interviewing, more in the style of a

conversation; the interviewer adjusts the timing and content of questions to be asked and
seeks to obtain full answers in the respondent's own words
Unstructured observation: observation of behavior or events as they occur, generally
in a natural setting; the action being observed is described in narrative form;
participant observation is a form of unstructured observation
Unweighted index: an index in which the indicators making up the index are assigned
equal value
Unweighted score: responses to items are simply added to form a composite score; as
distinguished from a weighted score in which responses to some items are given more
importance by assigning a greater value to them
V
Valid frequency distribution: a frequency distribution based on the number of
useable responses obtained for an indicator; example, if 68 respondents out of a sample
75 provided useable responses to an item, a valid frequency distribution would be based
on an N of 68 rather than the N of 75 for the sample
Valid percentage distribution: a set of percentage based on the number of useable
responses obtained for a variable; example, if 68 respondents out of a sample 75
provided useable responses to an item, a valid percentage distribution would be based on
an N of 68 rather than the N of 75 for the sample
Validity: the extent to which an indicator measures a given concept or one of its
dimensions
Value judgment: a statement or opinion based on one's beliefs or values and not on
empirical evidence; Variable: any characteristic that varies; that can take on two or
more numerical values or has two or more qualities; the various values or qualities of a
variable are its attributes
Variance: a measure of dispersion or variability among scores in a distribution;
variance is the mean of the squared deviations of each score from the mean of the
distribution
W
Web address: see Universal Resource Locator
Web site: an electronic source of information accessible through the Internet, the
worldwide telecommunication network and software that links computers to Web sites
Web survey: a survey conducted by posting a questionnaire on a Web site and
inviting viewers to complete the questionnaire; also referred to as an Internet survey
Weighted index: an index in which the indicators making up the index are assigned
different values to reflect the greater importance of some of them to the composite score

Glossary
Weighted sample: in analyzing data from samples, assignment of different weights or

values to cases in proportion totheir probability of selection
Weighted score: in constructing a composite score, the process of giving greater value
to some indicators over others
Weighting indicators: the process of assigning greater importance to certain indicators
over others in the construction of a composite measure
World Wide Web: the original name for the connections among Web sites, now
preserved in the "www" in many Web addresses
TOP TOC


SRMDC.net Research Links
Selected Social Research Methods Web Sites

Research Methods
gsociology.icaap.org this site provides reports on major global social, demographic,
economic, and political changes; data sets related to these and related topics;
presentations based on these data sets useful for teaching and public education; links to
research resources, data, history, and theory from research organizations around the
world, including information about social research practices and methods, ways of
collecting, analyzing and presenting data and results; issues related to research ethics;
links to free software; online books, manuals, guides, statistical software packages and
many other topics.
The Research Methods Knowledge Base
www.socialresearchmethods.net/kb/contents.htm this site is the home page for a
number of additional Web pages, each of which provides brief, easily understood
descriptions and illustrations of virtually any social research method you might want to
use; covers the foundations of research, sampling, measurement, design, analysis, and
the process of writing up a research report..
Free Evaluation Resources for Developing Countries
http://earth.prohosting.com/elecon/evaldevel/evaldevelopment.html provides
resources for conducting evaluation research, with particular reference to applications in
developing countries; describes the best use of the site; goals, plans and programs; list of
classes; project email list; evaluation Wiki document work pages; and a set of links to
evaluation sites, email groups, organizations, etc.
Qualitative Research Resources on the Internet
www.nova.edu/ssss/QR/qualres.html this site provides links qualitative research sites
and materials available through the Internet, including links to Web pages on a variety of
topics, journals, syllabi, and the current issues of The Qualitative Report, an online
journal.
BUBL LINK (Catalogue of Selected Internet Resoures)

bubl.ac.uk provides links to hundreds of social science Web sites organized under
more than 300 specialized topics; also provides links to Internet resources covering all
subject areas.
Social Science Information Gateway (SOSIG)

sosig.ac.uk provides selected, high quality information for students and researchers in
the social sciences, business, and law; also provides links to over 50,000 social science
Web pages.
http://www.srmdc.net/research_links.htm (1 of 2) [2/11/2008 9:41:45 AM]

SRMDC.net Research Links
Web Pages that Perform Statistical Calculations

statpages.org provides over 600 links, including nearly 400 pages that perform
calculations, and growing; a source of information on almost anything you might need in
conducting analyses and calculations, including links to interactive statistics, free
software, books and manuals, and demonstrations and tutorials.
Home

http://www.srmdc.net/research_links.htm (2 of 2) [2/11/2008 9:41:45 AM]

References
References
Abdelrahman, A. I. (1989). Marriage patterns, trends and timing in northern and urban
Sudan. Unpublished doctoral dissertation, University of Pennsylvania, Philadelphia.
Abdelrahman, K. A. (1995). Fertility differentials in an agricultural area; A case study of
Geriza and Managil (Sudan). Unpublished doctoral dissertation, Mississippi State
University, Mississippi State University, MS.
Alrareb, B. Y. (1997). Attitudes towards Muslims: Initial scale development.
Unpublished doctoral dissertation, Ball State University, Muncie, IN.
Anker, R., & Anker, M. (1995). Improving Measurement of Women's Participation in
the Egyptian Labour Force: Results of a Methodical Study. Geneva, Switzerland:
International Labour Office, Population and Labour Policies Program, Working Paper
No. 163.
Arthur, J. A. (1988). Social change and crime in Africa. Unpublished doctoral
dissertation, Pennsylvania State University, State College.
Ashton, W. (1996). Cooperative Webs: A Way of Improving the Environmental Health of
Alaskan Villages. Coordinator's Manual. A Report to the U.S. Environment Agency.
Wasilla, WA: The Delta Institute.
Badri, A. E., & L. G. Burchinal. (2005). Methods for Social Research in Developing
Countries. Omdurman, Sudan: The Ahfad University for Women.
Badri, A. E., & Osama, S. (1995). Women and mass media: A critical and analytical
study of the portrayal of Sudanese women in printed media. The Ahfad Journal: Women
and Change, 12:1, 4-23.
Badri, H. S., & Burchinal, L. G. (1985). Educational attainment as an indicator of the
status of women in the Sudan. The Ahfad Journal: Women and Change, 2:1, 30-38.
Bauman, K. E. (1997). The effectiveness of family planning programs evaluated with
true experimental designs. American Journal of Public Health, 87: 666-669.
Beradi, G. (1998). Application of participatory rural appraisal in Alaska. Human
Organization, 57:4, 438-446.
Bogardus. E. S. (1933). A social distance scale. Sociology and Social Research, 17:
265-271.
http://www.srmdc.net/references.htm (1 of 8) [2/11/2008 9:42:08 AM]

References
Brems, S., & Berg, A. (1998). Eating down during pregnancy: nutrition, obstetrics and
culture considerations in the third world. Discussion paper prepared for the UN Advisory
Group on Nutrition, Subcommittee on Nutrition, Population and Human Resource
Division: Washington, D.C.
Bulluck, B. P. (1986). Basic needs fulfillment among less developed countries,
1960-1980: Social progress over two decades of economic growth. Unpublished doctoral
dissertation, Vanderbilt University, Nashville, TN.
Campell, D. T., & Stanley, J. S. (1966). Experimental and Quasi-Experimental Designs

for Research. Chicago: Rand McNally.
Campell, D. T. (1985) Factors related to the validity of experiments in social settings.
Psychological Bulletin, 54: 297 - 312.
Chavez, A., Martinez, C., & Soberanes, B. (2000). The effect of malnutrition on human
development. In A. H. Goodman, D. L. Dufour, & G. H. Pelto. Nutrition and
Anthropology: Biocultural Perspectives on Food and Nutrition. Mountain City, CA:
Mayfield Publishing Company.
Chronbach, L. (1951). Coefficient alpha and the internal structure of tests.
Psychometrica, 16: 297-334.
Cook, B. J. (2001). Islam and Egyptian higher education: Student Attitudes.
Comparative Education Review, 45:3, 379-409.
Cook. T., & Campbell, D. T. (1979). Quasi-Experimental Design and Analysis Issues for
Field Setting. Chicago: Rand McNally.
Darkoh, C. A. A. (1994). Women's roles and social change in Sudan. Unpublished

doctoral dissertation, Iowa State University, Ames, IA.
Davidson, A. P. (1992). Household, community, and individual: The formation of
livelihood strategies in the Nuba mountains, Sudan. Unpublished doctoral dissertation,
Michigan State University, East Lansing, MI.
Dei, G. J. S. (1992). Hardships and survival in rural West Africa: A case study of a
Ghanaian community. Dakar, Senegal: Codesria.
Department of Statistics, Ministry of Planning and National Economy, Republic of
Sudan. (1982). The Sudan Fertility Survey, 1979. Volumes 1 and 2. Voorburg: The
Netherlands: International Statistical Institute.
Donahoe, D. A. (1999). Measuring women's work in developing countries. Population
and Development review, 25:3, 543-576.

References
Farah, A. M. (1981). Child mortality and its correlates in Sudan. Unpublished doctoral
dissertation, University of Pennsylvania, Philadelphia, PA.
Fattah, Y. M. (1981). The role of rural women in family decision making in Iraq.
Unpublished doctoral dissertation, The University of Tennessee, Knoxville, TN.
Freudenberger, K. S. (1993). Fields, fallow, and flexibility: natural resource
management in Ndam Mor Fademba, Senegal: Results of a rapid rural appraisal carried
out 25-30 October 1991. London: International Institute for Environment and
Development.
Frey, S. R., & Field, C. (2000). The determinants of infant mortality in the less
developed countries: A cross-national test of five theories. Social Indicators Research,
52, 215-234.
Ghorayshi, P. (1996). Women in developing countries: Methodical and theoretical

considerations. Women and Politics, 16:3, 89-109.
Gimbel, R. W., Lehrman, S., Strosberg, M. A., Ziac, V., Freedman, Savicki, K., &
Tackley, L. (2002). Organizational and environmental predictors of job satisfaction in
community-based HIV/AIDS service organizations. Social Work Research, 26:1, 43-55.
Grotberg, E. H., & Badri, G. (1986). The effects of early stimulation by Sudanese
mothers: An experiment. The Ahfad Journal: Women and Change, 3:2, 3-16.
Guttman, L. (1980). The basis for scalogram analysis. In: S. A. Stouffer, et. al. (eds.)
Studies in Social Psychology in World War II, Vol. 4, Measurement and Prediction.
Princeton, NJ: Princeton University Press.
Haberer, R. B. (1985). Status, power, and influence of women in an Arab village in

Israel. Unpublished doctoral dissertation, Purdue University, West Lafayette, IN.
Had-Elzin, M. F & Ahmed, H. A. (2001). Perceptions and visions of Sudan youth
towards conflict resolution and culture of peace. The Ahfad Journal: Women and
Change, 18:1, 59-60.
Halim, A. M. A. (1995). Girl's education and their change of attitudes: A case from
Sudan. The Ahfad Journal: Women and Change, 12:1, 4-23.
Harrison, G. G. (1992). Nutrition and child feeding patterns in the Sudan. The Ahfad
Journal: Women and Change, 9:2, 4-22.
Hassan, A., Butrus, R., Rahman, A., & Burchinal, L. G. (1988). Factors associated with
attitudes towards male-female equality in Omdurman, Sudan. The Ahfad Journal:
Women and Change, 5:1, 3-13.

References
Hevi-Yoboe, L. A., Fanslow, A. M., & Cowan, D. L. (1986). Ahfad students: Attitudes
and behaviors regarding women's roles. The Ahfad Journal: Women and Change, 3:2,
28-36.
El Hiday, M., & Zumrawi, F. Y. (1992). The effect of a nutrition education programme
on pregnant women attending Khartoum model clinic. The Ahfad Journal: Women and
Change, 9:2, 23-36.
Howard, W. S. (1987). Social strategies in petty production: Three small scale industries
in urban Sudan. Unpublished doctoral dissertation, Michigan State University, East
Lansing, MI. 1`1``
International Labor Organization. (2002). Key Indicators of the Labour Market

2000-2002. Geneva, Switzerland, International Labor Organization.
Julia, M., & Ridha, H. (2001). Women and war: The role Kuwaiti women played during
the Iraqi occupation. Journal of International Development, 13: 583-598.
Kadri, N., Tilane, A., El Batal, M., Taltit, Y., Tahiri, S. M., and Moussaouri, D. (2000).
Irritability during the month of Ramadan. Psychosomatic Medicine, 62: 280-285.
Kagimu,M., Marum, E., Wabwire-Mangen, F., & others. (1998). Evaluation of the
effectiveness of AIDS health education intervention in the Muslim community in
Uganda. AIDS Education & Prevention, 10:3, 215-228.
Khalafalla, L. A. (2001). Towards gender sensitivity policies in Sudan: A study on
Sudan National Comprehensive Strategy (1992-2002). The Ahfad Journal: Women and
Change, 18:1, 60-62.
Khamis, A. H., & Alsuni, A. (1998). Prevalence of tuberculosis in Khartoum State in
Sudan: Correlation of tuberculin reactivity with age. The Ahfad Journal: Women and
Change, 15:1,4-14.
King, R. P., & Byerlee, D. (1977). Income distribution, consumption patterns and
consumption linkages in rural Sierra Leone. African Rural Economy Paper No. 16.
East Lansing, MI: Michigan State University.
Kridli, S. O. (1997). Establishing reliability and validity of an instrument measuring
attitudes, subjective norms, perceived control, and behavioral intentions of Jordanian
Muslim women toward the use of oral contraceptives. Unpublished doctoral dissertation,
University of Missouri, Columbia, MO.

References
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology,

No. 140.
Lloyd, C. B., Kauffman, C. E., & Hewett, P. (2000). The spread of primary schooling in
sub-Saharan Africa: Implications for fertility change. Population and Development,
26:3, 483-515.
Lobban, C. F. (1987). Islamic Law and Society in the Sudan. London: Frank Cass.
Madani, A. O. (1999). Depiction of Arabs and Muslims in the United States news media.
Unpublished doctoral dissertation, California School of Professional Psychology, Los
Angeles, CA.
Mageed, M. A., Sulima, B. A., & Kawther, D. M. (2000). Re-circumcision: the hidden
devil of female genital mutilation (FGM) in Sudan. (Case study on the perception and
attitudes of Sudanese men). The Ahfad Journal: Women and Change, 17:2, 3-14.
Marenin, O. (1997). Victimization surveys and the accuracy and reliability of official
crime data in developing countries. Journal of Criminal Justice, 25:6, 463-475.
Mathews, H. (1985) "The weeping woman: Variation and homogeneity in folk theories
of gender in a Mexican community. Presented at the annual meeting of the American
Anthropological Association, Washington, D.C.
Merton, R. K. (1968). Social Theory and Social Structure. New York: The Free Press.
Mohamed, M. A. K. (2002). Assessing determinants of departmental innovation: an
exploratory multi-level approach. Personnel Review, 31:5, 620-641.
Mulinge, M., & Mueller, C. W. (1998). Employee job satisfaction in developing
countries: the case of Kenya. World Development, 26:12,2189-2199.
Muneer, S. T. (1989). Agricultural cooperatives as a means for agricultural development:
The case of western Sudan small farmers' cooperatives. Unpublished doctoral
dissertation, Iowa State University, Ames, IA.
Mustafa, A. F. (2001). Factors that influence the legislation of personal status laws. A
case study of Sudanese personal status laws on divorce and polygamy. The Ahfad
Journal: Women and Change, 18:1, 59-60.
Mwango, E. (1986) The sources of variation in farmer adoption of government
recommended technologies in the Lilongwe rural development program area of central
Malawi. M. A. thesis, University of Florida, Gainesville, FL.
Nachmias, C. F., & Nachmias, D. (2000). Research Methods in the Social Sciences. New
York: Worth Publishers.

References
Nag, M. (1994). Beliefs and Practices About Food During Pregnancy, retrieved July
22, 2002, www.hsph.harvard.edu/grhf/SAsia/suchana/0317/nag.html.
Namara, A. (1997). Socioeconomic Survey of Fishing Villages in the Buffer Zone of Lake
Mburo National Park. Kampala, Uganda: Makerere Institute of Social Research,
Makerere University.
Nandy, S. (1995). A content analysis of health articles on developing countries published
in three health journals from1975 to 1994. Unpublished doctoral dissertation, Southern
Illinois University at Carbondale, Carbondale, IL.
Nasr, Z. (1986). Infant and Childhood Mortality Among Depressed and Non-depressed
Mothers in Sudan. Senior Paper, Ahfad University for Women, Omdurman: Sudan
Nouri, M. O. A. (1983). Fertility differentials in the Democratic Republic of the Sudan.
Unpublished doctoral dissertation, Mississippi State University, Mississippi State, MS.
Omer, A. B. (1991). Ethnicity, women's employment, and fertility: A case study of the
Gezira area in central Sudan. Unpublished doctoral dissertation, University of Michigan,
Ann Arbor, MI.
Osman, R. A. (1995). The fertility impact of rural development projects: The case of the
Rahad irrigation project, Sudan. Unpublished doctoral dissertation, Brown University,
Providence, RI.
Pool, R., Nyahzi, S. & Whitworth, J. A. G. (2001). Attitudes to voluntary counseling

and testing for HIV among pregnant women in rural south-west Uganda. AIDS CARE,
13:5, 605-615.
Rahama, A. (1997). Gender roles in crisis situations: the case of the famine of 1985/86.
The Ahfad Journal: Women and Change. 14, 2: 4-15.
Roa, A. P. (1981). The level of organizational involvement of women in develop
projects. Unpublished doctoral dissertation, Columbia University, New York, NY.
Ross, J. A., & Mauldin, W. P. (1996). Family planning programs: efforts and results.
Studies in Family Planning, 27:3, 137-147.
El Sayed, S. M., & Ahmed, S. E. (2001). Socio-cultural aspects of Kala-azar among

Masalit and Hawsa tribes. The Ahfad Journal: Women and Change, 18:1, 51-58.
Stone, A. (2002). "We've got work to do on image," says Bush. USA Today, 28
February, p. 4A.

References
Stycos, J. M. (1983). Sample surveys for social science in undeveloped areas. In Bulmer,
M., & Warick, D. P. (eds). Social Research in Developing Countries: surveys and
censuses in the Third World. New York: Wiley. pp. 53-64.
Swanjord, T. S. (1988). Images of Kuwaiti women: A content analysis of representative
selections from the literature with accompanying annotated bibliography. Unpublished
doctoral dissertation, South Dakota State University, Vermillion, SD.
Tessler, M. (1973). Measuring abstract concepts in Tunisia. In W. M. O'Barr, D. H.

Spain, & M. A. Tessler. (eds). Survey research in Africa: Its Implications and Limits.
Evanston: Northwestern University Press.
El Tigani, E. E. (1998). The socioeconomic aspects of child health in Geriza, Sudan.
Unpublished doctoral dissertation, The Johns Hopkins University, Baltimore, MD.
United Nations. (2001). Demographic Yearbook. New York: United Nations.

United Nations. (2001). World Statistics Pocketbook. New York: United Nations.
United Nations. (2002). Statistical Yearbook. New York: United Nations.
United Nations Children's Fund (UNICEF). (2002). The State of the World's Children.
New York: UNICEF.
United Nations Children's Fund (UNICEF). (2002). Statistical Data on Children. New
York: UNICEF.
United Nations Development Program. (2002). Human Development Report 2001. New
York: United Nations Development Program.
United Nations Educational, Scientific and Cultural Organization (UNESCO). (1999).
Statistical Yearbook. Lanham:, MD: Bernan Press.
United Nations High Commissioner for Refugees. (2000). The State of the World's
Refugees. New York: Oxford University Press.
United Nations Population Division. (2001). World Population Prospects. Vol. I and II.
New York: United Nations Population Division.
United Nations Statistical Division. (2002). Statistical Yearbook. 45 th Edition. New
York: United Nations Statistical Division.
United Nations Statistical Division. (2001). World Statistics Pocketbook. New York:
United Nations Statistical Division.

References
Vredenburgh, A. G. (2002). Organizational safety: which management practices are

most effective in reducing employee injury rates? Journal of Safety Research, 33:2,
259-276.
Wakeman, W. E. (1992). A case study of women, Islam and development: Strategies and
models for conducting projects with women in the Islamic context of Um Ruwaba,
Sudan. Unpublished doctoral dissertation, Tufts University, Medford, MA.
Wang, G. (1996). Women's reproductive rights in developing countries: A causal
analysis. Unpublished doctoral dissertation, University of North Texas, Dallas, TX.
Williams, M. (2000). Science and Social Science: An Introduction. London: Routledge.
World Bank. (2002) African Development Indicators 2000. Washington, D.C.: The
World Bank.
World Bank. (2002). World Development Indicators 2002. Washington, D.C.: The
World Bank.
World Bank. (2002). Little Data Book 2002. Washington, D.C. The World Bank.
World Bank. (2001). World Population Projections. Washington, D.C. The World Bank.
World Health Organization. (2001). World Health Report: Mental Health: New
Understandings: New Hope. Geneva: World Health Organization.
Zuckerman, H. (1988). The Sociology of Science. In Smelser, N. J. (ed.). Handbook of

Sociology. Pp. 511-574.
TOC


Help Page 1
Help In Using This Site

Introduction
Introduction
Navigating This Help section covers three topics:
this site ● Help in moving around this site - called " navigating "
● Descriptions of ways you can " read " this site; and
Reading
this site ● A brief introduction to Searching the Internet
Navigating This Site
Intro to
searching You can move about or navigate among the sections of this site in three ways:
the ● By clicking on topics that identify the main sections making up the site, like "Home,"
Internet "TOC," or "Glossary" as listed above the title of a section or chapter;

● By clicking on topics describing the sections within a main section, like those listed to the
Aids left; or
● By clicking on one of the four navigation buttons at the end of each chapter.
The topics that identify the main sections of the site are listed in a row immediately below title of
the site Methods for Social Researchers in Developing Countries. The sections title "Parts,"
as shown above, contain a list of chapters making up each Part of the site.
The topics describing the sub-sections of each chapter or other section of the site are listed in a
column on the left side of the first page of each chapter or section, as shown to the left on this
page.
The four navigation buttons at the end of each chapter or section allow you to move quickly
about the chapter or other section you are reading. The Previous (PREV) button takes you back to
the previous screen; the Next (NEXT) button moves you forward one screen full of information;
the Beginning (BEGINNING) button moves you to the beginning of the section you are viewing;
and the TOC (Table of Contents) button opens the Table of Contents where you can scroll
among topics, meaning you can move up or down the list of chapter titles and their contents.
We suggest you try navigating the site using each of these buttons. A little practice will help you
use any of these four ways to find pages you want to read
Reading this site
We have organized this site so that you can "read" it three ways:
● By reading it from the beginning to the end, just as you would a book;
● By selecting a part of the site and beginning there;
● By clicking on TOC.
PREV NEXT
http://www.srmdc.net/help/1.htm (1 of 2) [2/11/2008 9:42:15 AM]

Help Page 1

Help Page 2
Introduction Reading like a book

If you are a beginning researcher or a student in an introductory social research methods class,
Navigating
you may want to begin at the beginning of this site. In this case, click on Parts, shown as the third
this site term in the set of buttons at the top of this page. When you do this, a drop-down menu will
appear showing a short title for each of the five Parts. By clicking on a title for any Part, you will
Reading taken to the beginning of that Part. Once in a selected Part, you will see a list of chapters included
this site in that Part. Chapter titles are shown in a column on the left side of the first page of each Part. For
Part 1, for example, the chapters are: Chapter 1: Understanding Scientific Inquiry; Chapter 2:
Intro to Sudan Fertility Study; and Chapter 3: The Language of Research.
searching
the By clicking on the name of the chapter, you can go directly to that chapter. Once in a chapter, you
will see a list of topics in a column down the left side of the first page of the chapter. These topics
Internet
are listed from top to bottom in the order in which they are discussed. You can read from one
topic to the next and from one chapter to the next. In this way, you can read from the beginning
Aids of the site to the end. Or, by clicking on the buttons for various topics, you can jump from one
topic to the next within any chapter.
Selecting a part
This site is divided into five main parts. Part 1, "Thinking Like a Researcher," introduces you to
social research and the way researchers look at the world around them. If you want to learn more
about how scientists and researchers think and look at things, begin with Part 1 and proceed
through its three chapters.
Part 2, "Planning Research," describes how to plan a research project, beginning with the
selection of a problem to investigate. If your interest is in how to select a problem to investigate
and how to develop a plan for addressing that problem, this is the place to start. From Part 2, you
can go to any of the chapters making up this part and learn about the various steps involved in
planning a research project.
Research is based on the collection and analysis of information, called data by researchers. By
the way you plan your research, you will be creating the kinds of data you will collect and later
analyze. Therefore, Part 3 is called "Creating Data." The seven chapters of this part describe
ways of collecting data. Each of these chapters can be accessed from the first page of Part 3.
After you collect your data, you have to analyze it to see what you have found. Ways of
organizing and analyzing data are described in Part 4, "Analyzing Data." The four chapters in
this part show how to prepare data for analysis and how to conduct various statistical analyses of
your data. Clicking on a title name will take you to that chapter.
The final chapter, in Part 5, "Reporting Your Results," describes how to plan and write a
research report and how to share your research results with others in additional ways. You can go
to any of the specific section of this chapter by clicking on the topics listed in the column on the

Help Page 2
left side of the first page of this chapter.

Using the table of contents
The table of contents (TOC) of a book allows you to find a specific section you want to read. The
TOC on this site works the same way. Any time you want to go to a specific section, you can
click on TOC to see the table of contents for material on this site. The TOC provides a list of the
Parts of the site, each with their respective chapters, and then within each chapter, the main topics
discussed within each chapter. You can scroll up or down in the list to find the specific section
you want to go to.
PREV NEXT

Help Page 3
Go for it
Introduction
With this brief introduction, enter the site in your preferred way. We hope you will be able to
move easily about the site and find what you are looking for. In addition, you may wish to search
Navigating
other Web sites accessible through the Internet. For persons not familiar with the Internet, we
this site provide the following brief introduction.
Reading Introduction to searching the Internet
this site Understanding the Internet and Web sites
Intro to The terms Internet and Web sites are often used to describe the same thing, but, technically, they
searching refer to different things. The Internet is the set of connections and common means for linking
the users to Web sites throughout the world. The Web sites contain the information one can access
through the communications and linkages provided by partners who make up the Internet. Today,
Internet
Web sites are maintained by most major organizations, including universities, research centers,
government ministries, professional organization, international organizations, and businesses.
Aids The ever-increasing number of Web sites offers information on almost any topic you can think
of. Because of the growing importance of Web sites as a source of research information,
researchers routinely check Web sites for information. We recommend you include a Web search
for any research you plan to undertake.
Web site addresses
Each Web site has a unique address, referred to as its Universal Resource Locator or URL. In the
beginning, the addresses for all Web sites began with "www," which stands for the World Wide
Web. Now, many Web sites do not include the "www" prefix; instead, they begin with some short
version or the initials of the name of the organization sponsoring the site. For example, the Web
address for the World Bank is www.worldbank.org while that for the Ahfad University for
Women, in Omdurman, Sudan, is ahfad.org. You will see Web addresses in both forms with
various sets of letters and numbers. Also, Web addresses may end in a variety of ways. Some
may include the letters "html," but others may have only "htm" or some other ending.
Somewhere in their addresses, sites include three letters that identify the kind of organization that
maintains the site. For example, "org" stands for organization; "edu" for an academic institution;
"gov" for a governmental body; and "com" for a commercial establishment. There are other
three-letter identifications endings as well.
Be sure to note the full and exact address of any site you see and wish to return to; otherwise, you
will have difficulty returning to it.
Kinds of information on Web sites
Many kinds of information are available on Web sites. Chapter 4 describes five kinds of
information sources that are particularly useful to researchers. These are:
● Social science databases

Help Page 3
● Sites of international organizations

● Other social science Web sites
● Guides to specialized Web sites
● Full text sites
PREV NEXT

Help Page 4
Chapter 4 also provides examples of each of these kinds of sources of information and shows
Introduction how to locate and search these sources.
Navigating Finding Web sites

this site Since you are reading this, you are already connected to the Internet. If you are an experienced
Internet searcher, you dont need any more help. However, if you are a beginner in Internet
Reading searching, you may want a little help in getting started. We suggest you go to one of the popular
this site search engines, such as:
Google at www.google.com
Intro to
Yahoo at www.yahoo.com
searching
Ask at www.ask.com
the
Internet And type internet searching or some similar phrase in the search box on the home page of the
site. After hitting ENTER, you will see a list of sites with information about how to search the
Aids Internet. One or several of the sites should start you on your way to becoming an expert Internet
searcher.
Limitations of the Internet
Searching for useful Web sites can be fun and rewarding. It can also be frustrating and produce
misleading results. Perhaps the greatest limitation of the Internet is the variation in quality and
trustworthiness of the content on Web sites.
Unlike articles in journals, most of which are independently reviewed for quality or books edited
by experts, persons responsible for many Web sites make their own decisions about the content
of their sites. As a result, sites vary greatly in the accuracy or value of their contents. Viewers
have to decide whether they trust the information provided on any site. Generally, this is not a
problem on sites maintained by a majority of academic, professional, governmental, or
international organizations. Some of these, however, may have a particular point of view on
issues and seek to influence viewers to adopt their positions. So, be warned: You are on your own
when viewing and deciding what to use from a Web site.
Another problem is overload. For example, a search on Google on August 10, 2007, for
information on designing questionnaires produced over 28,300 responses or hits. Getting far
more responses than you can possibly read happens often. In Chapter 4 we describe some ways
you limit the number of responses you get.
Sometimes, however, you will not find the expected site. This can happen for a number of
reasons. You may have made an error in typing the address. The address has to be typed exactly.
An error in even one letter or other character will cause you to get a message that the site cannot
be found. After you type in address, read it carefully to make sure it is correct.
PREV NEXT

Help Page 4

Help Page 6
Another problem is that sites come and go. Some sites are started and maintained for a while and
then, when the work becomes too much, are discontinued. Links to discontinued sites, however,
Introduction remain on other sites. Don't be surprised, therefore, if you follow a link only to get a message that
the address for that site no longer exists. This is simply a fact of life on the Internet.
Navigating
this site The rapid pace of technological change can create problems for some Internet users, particularly
those in developing countries. Designers of Web sites seek to use the latest software and the
Reading increased speed and power of computers to present more attractive and useful pages. As these
technology-driven changes are made, users with older searching software and computers are left
this site
behind. Consequently, some users, and this may include you, may find they cannot read the
content on some sites.
Intro to
searching Finally, any search engine you use will not find all possibly relevant Web sites. Each engine has
the its own way of searching and selecting sites to match searches entered by users. Consequently,
Internet the same set of search terms entered on different search engines will produce different results.
Some of the same sites will be found in all searchers, but each engine will list sites the others do
not. This is another fact of life on the Internet. Experienced Web searchers get around this
Aids
problem by repeating the same search using different search engines. This way they are more
likely to find all sites corresponding to their search terms.
Still, even with these problems, the Internet offers researchers in developing countries access to
information and data that would otherwise be unavailable.
Aids
Key terms
● Browswer ● Page
● Data ● Scroll
● Databases ● Search engine
● Internet ● Search service
● Home Page ● Surf
● Link ● Universal Resource Locator
● Navigating ● Web Site
● Navigation Buttons ● World Wide Web


Help Page 6

Acknowledgements
Acknowledgements
Three persons have contributed significantly to the development of this site. Foremost is
Dr. Amna Badri, Vice President for Academic Affairs, The Ahfad University for
Women, who was the co-author of the book, Methods for Social Researchers in
Developing Countries, upon which this site is based. This book was originally published
by AUW in 1990 and republished by AUW in 2005. I accept responsibility for revising
and converting the content of the book in its present form as a Web site. In addition, I
am deeply indebted to Dan Jaeggi, of Abayumi Web Design, who designed and mounted
this web site.

http://www.srmdc.net/acknowledgements.htm [2/11/2008 9:42:53 AM]

Social Research in Developing Nations

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Social Research in Developing Nations

Diunggah oleh

Hak Cipta:

Format Tersedia

Methods For Social Researchers In Developing Countries

Home TOC Parts Glossary Links References Contact Us Help

Focus group: Dar ElSalam Camp, near Omdurman, Sudan

http://www.srmdc.net/ (1 of 2) [2/11/2008 8:39:05 AM]

http://www.srmdc.net/ (2 of 2) [2/11/2008 8:39:05 AM]

Home TOC Parts Glossary Links References Contact Us Help

http://www.srmdc.net/contactus.htm (1 of 2) [2/11/2008 8:39:14 AM]

Hosted by the Sudan-American Foundation for Education, Inc.

http://www.srmdc.net/contactus.htm (2 of 2) [2/11/2008 8:39:14 AM]

Home TOC Parts Glossary Links References Contact Us Help

Part One: Thinking as a Researcher

Chapter 3. Learning the Language of Research

http://www.srmdc.net/toc.htm (1 of 7) [2/11/2008 8:39:20 AM]

http://www.srmdc.net/toc.htm (2 of 7) [2/11/2008 8:39:20 AM]

Chapter 8. Selecting a Sample

http://www.srmdc.net/toc.htm (3 of 7) [2/11/2008 8:39:20 AM]

http://www.srmdc.net/toc.htm (4 of 7) [2/11/2008 8:39:20 AM]

Chapter 15. Using a MultiMethod Design and Conducting Rapid Rural

http://www.srmdc.net/toc.htm (5 of 7) [2/11/2008 8:39:20 AM]

❍ Hand versus compute analysis

Chapter 18. Tabular Bivariate and Multivariate Analyses

Part Five: Telling What You Did

http://www.srmdc.net/toc.htm (6 of 7) [2/11/2008 8:39:20 AM]

Chapter 20. Reporting Your Research

http://www.srmdc.net/toc.htm (7 of 7) [2/11/2008 8:39:20 AM]

Home TOC Parts Glossary Links References Contact Us Help

http://www.srmdc.net/overview.htm (1 of 3) [2/11/2008 8:39:22 AM]

http://www.srmdc.net/overview.htm (2 of 3) [2/11/2008 8:39:22 AM]

Hosted by the Sudan-American Foundation for Education, Inc.

http://www.srmdc.net/overview.htm (3 of 3) [2/11/2008 8:39:22 AM]

Home TOC Parts Glossary Links References Contact Us Help

Part One: Thinking Like a Researcher

Hosted by the SUDAN-AMERICANFOUNDATION FOR EDUCATION, INC.

http://www.srmdc.net/parts/1.htm (1 of 2) [2/11/2008 8:39:30 AM]

http://www.srmdc.net/parts/1.htm (2 of 2) [2/11/2008 8:39:30 AM]

Home TOC Parts Glossary Links References Contact Us Help

Chapter 4 Part Two: Planning Research

http://www.srmdc.net/parts/2.htm [2/11/2008 8:39:36 AM]

Home TOC Parts Glossary Links References Contact Us Help

Part Three: Creating Data

Hosted by the SUDAN-AMERICANFOUNDATION FOR EDUCATION, INC.

http://www.srmdc.net/parts/3.htm (1 of 2) [2/11/2008 8:39:39 AM]

http://www.srmdc.net/parts/3.htm (2 of 2) [2/11/2008 8:39:39 AM]

Home TOC Parts Glossary Links References Contact Us Help

Part 4. Analyzing Data

Hosted by the SUDAN-AMERICANFOUNDATION FOR EDUCATION, INC.

http://www.srmdc.net/parts/4.htm [2/11/2008 8:39:43 AM]

Home TOC Parts Glossary Links References Contact Us Help

Part 5. Writing Your Report

Hosted by the SUDAN-AMERICANFOUNDATION FOR EDUCATION, INC.

http://www.srmdc.net/parts/5.htm [2/11/2008 8:39:47 AM]

Home TOC Parts Glossary Links References Contact Us Help

Chapter 1: Understanding Scientific Inquiry

http://www.srmdc.net/chapter1/1.htm (1 of 2) [2/11/2008 8:39:50 AM]

http://www.srmdc.net/chapter1/1.htm (2 of 2) [2/11/2008 8:39:50 AM]

Home TOC Parts Glossary Links References Contact Us Help

http://www.srmdc.net/chapter1/2.htm (1 of 2) [2/11/2008 8:39:54 AM]

http://www.srmdc.net/chapter1/2.htm (2 of 2) [2/11/2008 8:39:54 AM]

Home TOC Parts Glossary Links References Contact Us Help

http://www.srmdc.net/chapter1/3.htm (1 of 2) [2/11/2008 8:40:00 AM]

http://www.srmdc.net/chapter1/3.htm (2 of 2) [2/11/2008 8:40:00 AM]

Home TOC Parts Glossary Links References Contact Us Help

The scientific approach to knowledge