Advantages and Disadvantages of the Survey Method

Strengths:

 Surveys are relatively inexpensive (especially self-administered surveys).

 Surveys are useful in describing the characteristics of a large population. No other
method of observation can provide this general capability.
 They can be administered from remote locations using mail, email or telephone.
 Consequently, very large samples are feasible, making the results statistically
significant even when analyzing multiple variables.
 Many questions can be asked about a given topic giving considerable flexibility to the
analysis.
 There is flexibilty at the creation phase in deciding how the questions will be
administered: as face-to-face interviews, by telephone, as group administered written
or oral survey, or by electonic means.
 Standardized questions make measurement more precise by enforcing uniform
definitions upon the participants.
 Standardization ensures that similar data can be collected from groups then
interpreted comparatively (between-group study).
 Usually, high reliability is easy to obtain--by presenting all subjects with a
standardized stimulus, observer subjectivity is greatly eliminated.

Weaknesses:

 A methodology relying on standardization forces the researcher to develop questions

general enough to be minimally appropriate for all respondents, possibly missing what
is most appropriate to many respondents.
 Surveys are inflexible in that they require the initial study design (the tool and
administration of the tool) to remain unchanged throughout the data collection.
 The researcher must ensure that a large number of the selected sample will reply.
 It may be hard for participants to recall information or to tell the truth about a
controversial question.
 As opposed to direct observation, survey research (excluding some interview
approaches) can seldom deal with "context."
Types of Surveys

Surveys can be divided into two broad categories: the questionnaire and the interview.
Questionnaires are usually paper-and-pencil instruments that the respondent completes.
Interviews are completed by the interviewer based on the respondent says. Sometimes, it's
hard to tell the difference between a questionnaire and an interview. For instance, some
people think that questionnaires always ask short closed-ended questions while interviews
always ask broad open-ended ones. But you will see questionnaires with open-ended
questions (although they do tend to be shorter than in interviews) and there will often be a
series of closed-ended questions asked in an interview.

Survey research has changed dramatically in the last ten years. We have automated telephone
surveys that use random dialing methods. There are computerized kiosks in public places that
allows people to ask for input. A whole new variation of group interview has evolved as
focus group methodology. Increasingly, survey research is tightly integrated with the delivery
of service. Your hotel room has a survey on the desk. Your waiter presents a short customer
satisfaction survey with your check. You get a call for an interview several days after your
last call to a computer company for technical assistance. You're asked to complete a short
survey when you visit a web site. Here, I'll describe the major types of questionnaires and
interviews, keeping in mind that technology is leading to rapid evolution of methods. We'll
discuss the relative advantages and disadvantages of these different survey types in
Advantages and Disadvantages of Survey Methods.

Questionnaires
When most people think of questionnaires, they think of the mail
survey. All of us have, at one time or another, received a
questionnaire in the mail. There are many advantages to mail
surveys. They are relatively inexpensive to administer. You can
send the exact same instrument to a wide number of people. They
allow the respondent to fill it out at their own convenience. But there are some disadvantages
as well. Response rates from mail surveys are often very low. And, mail questionnaires are
not the best vehicles for asking for detailed written responses.

A second type is the group administered questionnaire. A sample

of respondents is brought together and asked to respond to a
structured sequence of questions. Traditionally, questionnaires were
administered in group settings for convenience. The researcher
could give the questionnaire to those who were present and be
fairly sure that there would be a high response rate. If the
respondents were unclear about the meaning of a question they
could ask for clarification. And, there were often organizational
settings where it was relatively easy to assemble the group (in a company or business, for
instance).
What's the difference between a group administered questionnaire and a group interview or
focus group? In the group administered questionnaire, each respondent is handed an
instrument and asked to complete it while in the room. Each respondent completes an
instrument. In the group interview or focus group, the interviewer facilitates the session.
People work as a group, listening to each other's comments and answering the questions.
Someone takes notes for the entire group -- people don't complete an interview individually.

A less familiar type of questionnaire is the household drop-off

survey. In this approach, a researcher goes to the respondent's home
or business and hands the respondent the instrument. In some cases,
the respondent is asked to mail it back or the interview returns to
pick it up. This approach attempts to blend the advantages of the
mail survey and the group administered questionnaire. Like the mail
survey, the respondent can work on the instrument in private, when
it's convenient. Like the group administered questionnaire, the interviewer makes personal
contact with the respondent -- they don't just send an impersonal survey instrument. And, the
respondent can ask questions about the study and get clarification on what is to be done.
Generally, this would be expected to increase the percent of people who are willing to
respond.

Interviews
Interviews are a far more personal form of research than questionnaires.
In the personal interview, the interviewer works directly with the
respondent. Unlike with mail surveys, the interviewer has the
opportunity to probe or ask follow-up questions. And, interviews are
generally easier for the respondent, especially if what is sought is
opinions or impressions. Interviews can be very time consuming and
they are resource intensive. The interviewer is considered a part of the
measurement instrument and interviewers have to be well trained in how
to respond to any contingency.

Almost everyone is familiar with the telephone interview.

Telephone interviews enable a researcher to gather information
rapidly. Most of the major public opinion polls that are reported
were based on telephone interviews. Like personal interviews, they
allow for some personal contact between the interviewer and the
respondent. And, they allow the interviewer to ask follow-up
questions. But they also have some major disadvantages. Many
people don't have publicly-listed telephone numbers. Some don't
have telephones. People often don't like the intrusion of a call to their homes. And, telephone
interviews have to be relatively short or people will feel imposed upon.
\

Info Pages

All lecturers come in contact with questionnaires whether it is the standard end of year course
questionnaire or one that is used in research. These questionnaires come in many different
forms from: factual to opinion based, from tick boxes to free text responses. Whatever their
form, questionnaires are often viewed as quick and easy to do. This is not always the case. To
get useful responses, in a cost-effective way, it is important to be clear about the aim of the
questionnaire and how the responses will help you improve the learning technology or its
implementation. Think also about the analysis of results. It can be sobering to consider the
amount of data you will generate and the time it will take to analyse.

Some advantages and disadvantages of questionnaires follow. Notes on how to deal with
some of the disadvantages are also provided, as are some references to more comprehensive
information on questionnaires.

Some disadvantages of questionnaires:

 Questionnaires, like many evaluation methods occur after the event, so participants may
forget important issues.
 Questionnaires are standardised so it is not possible to explain any points in the questions
that participants might misinterpret. This could be partially solved by piloting the questions
on a small group of students or at least friends and colleagues. It is advisable to do this
anyway.
 Open-ended questions can generate large amounts of data that can take a long time to
process and analyse. One way of limiting this would be to limit the space available to
students so their responses are concise or to sample the students and survey only a portion of
them.
 Respondents may answer superficially especially if the questionnaire takes a long time to
complete. The common mistake of asking too many questions should be avoided.
 Students may not be willing to answer the questions. They might not wish to reveal the
information or they might think that they will not benefit from responding perhaps even be
penalised by giving their real opinion. Students should be told why the information is being
collected and how the results will be beneficial. They should be asked to reply honestly and
told that if their response is negative this is just as useful as a more positive opinion. If
possible the questionnaire should be anonymous.

Some advantages of questionnaires:

 The responses are gathered in a standardised way, so questionnaires are more objective,
certainly more so than interviews.
 Generally it is relatively quick to collect information using a questionnaire. However in
some situations they can take a long time not only to design but also to apply and analyse (see
disadvantages for more information).
 Potentially information can be collected from a large portion of a group. This potential is
not often realised, as returns from questionnaires are usually low. However return rates can
be dramatically improved if the questionnaire is delivered and responded to in class time.