Survey Research in Social Science

Myths and Reality?

Mr. Komil Kumar

Assistant Professor
Department of Psychology
Hari Institute of Technology,
Randevi [Nakur]
The common myths regarding survey
research?
 Surveys hardly helps us in measuring
correctly the social behaviour, attitudes and
opinions
 Surveys are if not always, but mostly
unreliable
Because--- it is not possible to measure the
attitude of big population referred as
“Universe” with speaking to only few people
referred as “Sample” in Survey research
Myths about survey research
contd.------
 Not only are samples small, but also
unreliable since they hardly represent
the characteristics of the Universe.
 One can get the answer you want by ay
of asking one kind of Question.--- Issue
of Questionnaire Design
 Let’s look at the evidence
Findings from survey Yes P.M should come under
conducted by--- Lokpal
India Against corruption 74%

Times of India 86%

CSDS 27%
How do you judge, which evidence is
more reliable compared to others?
 Various checks for judging if the empirical
evidence from the survey is reliable or not?
But 3 basic checks for judging the quality of
any empirical evidence:
 How representative is the sample?
 The issue of Questionnaire.
 Data collection method
What differentiated the three surveys?

Sample Questionnaire Data collection

technique
Indian Against Largely urban Not properly Filled by the
Corruption educated young designed, loaded respondent
men
Times of India Only internet Same was used Through website,
users by this survey internet
CSDS Cross section of Carefully Face to face
adults 18+ designed, interview
carefully worded
Let us look at another Research Question
Research Questions:
 Is there a shift among Muslim voters away
from Congress during last decade?
Hypothesis:
 Last few elections has witnessed major shift
in the Muslim vote for the Congress
What may be a possible explanation of
this research Question?

 Qualitative method of research would

provide answer to this question mainly in the
form of “Yes”, “No”, “to a great extent”
“some exent”, “not sure”. This would be
substantiated with may be some statements,
quotes etc.
 Quantitative Method would help us in
answering this question with “numbers”, will
help in quantifying the findings.
Let us look at findings of this Research
Question using empirical evidence
 At the national level, there is hardly any decline
in the support base of the Congress amongst
Muslim voters.
 In states which witness bi-polar contest. Muslim
are sharply polarised in favour of the Congress
 In states were Congress faces challenge from the
Regional parties, the Muslim support for the
Congress declines, regional parties take away
large chunk of Muslim vote
The empirical evidence from National Election
Study conducted by the CSDS

Muslim vote for 1996 1998 1999 2004 2009

Congress

All India 31% 33% 40% 37% 38

Bi polar Contest (BJP 67% 60% 84% 69% 70%

Contest against 26% 28% 37% 29% 33%

Regional parties
How representative is the Sample?
Why is survey an important
research tool?
 It helps us with evidence, which is
important in any research.
 We may be aware of some facts, but
there is a need to verify those fact, the
findings of the survey may confirm it,
refute it, etc.
To define Survey---

“Survey is a technique, a system

for collection of information from a
smaller number of people
(Sample), which helps to describe,
compare or explain the attitudes
and behaviour of a larger
population”
But are all surveys reliable?
 No, all surveys are not reliable,
 There are various things which make the
survey reliable
 Sample: Size of sample, how it had been
selected
 Instruments
 Field work and data collection etc.
Sample: Sampling technique
 “Sample” is referred to a small number of
people, selected from the larger number of
people which more or less resembles the
characteristics of the larger population from
which the sample if selected.”
 The technique of selection of the smaller
number of people from the larger group is
referred as “Sampling”
How big should be the sample?
 Sample is always a subset of the
universe, should this subset be 100,
1000, or 5000 or 10,000?
 Sample Size is guided by two things
1. Unit of analysis
2. Size of the Universe
Sample Size contd.---
 If unit of analysis are more, there is a need
for taking bigger sample, but if unit of
analysis is small, one can do even with
smaller sample.
 If the universe is very big, we can only think
of selecting the sample in some numbers
taking into account unit of analysis, but if the
universe is small one could think of taking the
sample in percent 10% or 20 % or so.
Sampling Techniques
There are broadly two kinds of sampling
techniques
 Probability Sampling

 Non Probability Sampling

Probability Technique
“Probability sampling technique is a technique
which ensures that the chances of being
selected as the sample of each and every
unit in the universe is more or less equal”
 All units will not be selected as the sample,
but they would have an equal chance of
being selected.
 Possible only if listing of elements of the
universe is available.
Non Probability sampling technique
“Non Probability Sampling technique is a
technique where chances of being
selected in the sample of each and every
unit of the universe are unequal”
Sometimes it systematically excludes one
or more sections, groups, locations etc.
from being selected as the sample
Probability Sampling
 Simple Random
 Systematic Random
 Multi Stage Systematic Random
Sampling technique
Simple Random Sampling Method
Starting point of any discussion on sampling,
most widely used method. This technique
ensures selection of required number of sample
units from the universe randomly without
anybody’s bias or preference or judgment.
Disadvantage: Though one has applied his or her
bias, but still there are chances of clustering
effect in the sample, if not selected properly.
There are chances of missing our some sections,
(groups, communities, regions etc.)
Systematic Random Sampling Method
 This is a refined version of the simple random
sampling technique. Samples are selected at
some regular interval. The first element of the
sample is selected randomly, and subsequent
samples are selected at regular intervals. This
ensures better spread of the sample across
region, different categories, communities etc.
 Prerequisite is a listing of all the elements in
the universe. The sample cannot be drawn with
this technique, incase the listing is not available.
Multi Stage Systematic Random
Sampling Method
 This is a refined version of systematic
random sampling technique.
 Since there is a prerequisite of a list and
it is difficult to have one comprehensive
list, the sample is selected using
different stages/steps.
Non-Probability Sampling Technique

 Convenience Sampling
 Snowball Sampling
 Quota Sampling
Convenience Sampling Technique
The sample for the survey is selected not
randomly, but are per the convenience of
both the interviewer as well as the
respondents. Those who are willing to be
interviewed (easily and readily available) are
selected as sample for the study.
Drawback: The chances of sample being
unrepresentative is very high
Snowball Sampling Technique
“This is the sampling technique where
the entire sample is not selected at one
go, but subsequent samples are
selected based on the reference of the
previously selected sample”.
Useful for research for which the universe is
small and there is hardly any listing available
for universe.
Quota Sampling Technique
 The sample is drawn first by fixing quota for different
sections which the research aims to study.
 Once the Quota for different sub groups are
allocated, the sample could be drawn randomly or
purposively.
 Useful if the sample to be studied is relatively small.
 So for smaller sample, the sample is normally drawn
purposively once the quota is decided in advance.
Can only a good sample guarantee a
reliable survey?
 No, there are issues about
questionnaire design, field work, data
collection, coding, checking entry and
analysis
 The most important tool for data
collection is a questionnaire
Issues about Questionnaire Design
 What is a Questionnaire?

“Questionnaire is a written document, which

contain more than one question, which
states, in a clearly worded format, the topic
and the issues on which the enquiry has been
designed”
Why there is a need for Questionnaire for
conducting Survey?
 One key word in Survey research is
standardisation
 It implies that the questions should be asked
to all the respondents in more or less same
manner conveying similar meaning.
 Questionnaire is necessary to maintain
standardisation.
Questionnaire design, some important
aspects
 Good Questionnaire vs. Bad Questions

 Open ended vs. Closed ended questions

 Opinion vs. factual questions

Questionnaire design: Some
elementary tips
 Use simple word and common language
 Questions should be essentially short, though
there could be exception
 Avoid framing “double barrel” questions
 Questions probing issues at certain time
frame must specify a specific time frame in
the question.
Questionnaire design contd.---
 Responses should be as far as possible in
shades/degree and not necessarily in “Yes”
and “No”
 Always leave an option “No Opinion” as a
valid answer category
 For multiple choice answer mix answer
categories to minimize “Primacy” and
“Recency” effect
Trained field investigators
 Trained field investigators are necessary
for reliable data collection.
 Training is necessary to maintain
standardization in technique of data
collection
Issues with coding, checking and
data entry
 Coding, checking and data entry are
necessary for a reliable survey.
 The data collected from the field needs
to be coded, checked and entered into
the computer for analysis.
 System of double entry necessary for
checking the reliability of data entry.
Analysis using SPSS
 The data entered into the computer is
analysed using SPSS package.
 Marginal or frequencies are the basic
tables which are generated.
 It is important to check if the sample is
representative or not.
 Apply weights if sample is not
representative.
Endless Analysis
 One can proceed with cross tables to
figure out the relationship between two
or more variables and similar such
analysis

Thank You