Strengths of Qualitative Research

 Issues can be examined in detail and in depth.

 Interviews are not restricted to specific questions and can be guided/redirected by the
researcher in real time.
 The research framework and direction can be quickly revised as new information
emerges.
 The data based on human experience that is obtained is powerful and sometimes more
compelling than quantitative data.
 Subtleties and complexities about the research subjects and/or topic are discovered that
are often missed by more positivistic enquiries.
 Data usually are collected from a few cases or individuals so findings cannot be
generalized to a larger population. Findings can however be transferable to another
setting.

Limitations of Qualitative Research

 Research quality is heavily dependent on the individual skills of the researcher and more
easily influenced by the researcher's personal biases and idiosyncrasies.
 Rigor is more difficult to maintain, assess, and demonstrate.
 The volume of data makes analysis and interpretation time consuming.
 It is sometimes not as well understood and accepted as quantitative research within the
scientific community
 The researcher's presence during data gathering, which is often unavoidable in qualitative
research, can affect the subjects' responses.
 Issues of anonymity and confidentiality can present problems when presenting findings
 Findings can be more difficult and time consuming to characterize in a visual way.

PRESENTATION OF QUALITATIVE RESEARCH FINDINGS

The following extracts are examples of how qualitative data might be presented:

Data From an Interview.

The following is an example of how to present and discuss a quote from an interview.
The researcher should select quotes that are poignant and/or most representative of the research
findings. Including large portions of an interview in a research paper is not necessary and often
tedious for the reader. The setting and speakers should be established in the text at the end of the
quote.
The student describes how he had used deep learning in a dispensing module. He was able to
draw on learning from a previous module, “I found that while using the e learning programme I
was able to apply the knowledge and skills that I had gained in last year's diseases and goals of
treatment module.” (interviewee 22, male)
This is an excerpt from an article on curriculum reform that used interviews 5:
The first question was, “Without the accreditation mandate, how much of this curriculum reform
would have been attempted?” According to respondents, accreditation played a significant role
in prompting the broad-based curricular change, and their comments revealed a nuanced view.
Most indicated that the change would likely have occurred even without the mandate from the
accreditation process: “It reflects where the profession wants to be … training a professional
who wants to take on more responsibility.” However, they also commented that “if it were not
mandated, it could have been a very difficult road.” Or it “would have happened, but much
later.” The change would more likely have been incremental, “evolutionary,” or far more
limited in its scope. “Accreditation tipped the balance” was the way one person phrased it.
“Nobody got serious until the accrediting body said it would no longer accredit programs that
did not change.”

Data From Observations

The following example is some data taken from observation of pharmacist patient consultations
using the Calgary Cambridge guide.6,7 The data are first presented and a discussion follows:
Pharmacist: We will soon be starting a stop smoking clinic.
Patient: Is the interview over now?
Pharmacist: No this is part of it. (Laughs) You can't tell me to bog off (sic) yet. (pause) We will
be starting a stop smoking service here,
Patient: Yes.
Pharmacist: with one-to-one and we will be able to help you or try to help you. If you want it.
In this example, the pharmacist has picked up from the patient's reaction to the stop smoking
clinic that she is not receptive to advice about giving up smoking at this time; in fact she would
rather end the consultation. The pharmacist draws on his prior relationship with the patient and
makes use of a joke to lighten the tone. He feels his message is important enough to persevere
but he presents the information in a succinct and non-pressurised way. His final comment of “If
you want it” is important as this makes it clear that he is not putting any pressure on the patient
to take up this offer. This extract shows that some patient cues were picked up, and appropriately
dealt with, but this was not the case in all examples.

Data From Focus Groups

This excerpt from a study involving 11 focus groups illustrates how findings are presented using
representative quotes from focus group participants.8
Those pharmacists who were initially familiar with CPD endorsed the model for their peers, and
suggested it had made a meaningful difference in the way they viewed their own practice. In
virtually all focus groups sessions, pharmacists familiar with and supportive of the CPD
paradigm had worked in collaborative practice environments such as hospital pharmacy
practice. For these pharmacists, the major advantage of CPD was the linking of workplace
learning with continuous education. One pharmacist stated, “It's amazing how much I have to
learn every day, when I work as a pharmacist. With [the learning portfolio] it helps to show how
much learning we all do, every day. It's kind of satisfying to look it over and see how much you
accomplish.”
Within many of the learning portfolio-sharing sessions, debates emerged regarding the true
value of traditional continuing education and its outcome in changing an individual's practice.
While participants appreciated the opportunity for social and professional networking inherent
in some forms of traditional CE, most eventually conceded that the academic value of most CE
programming was limited by the lack of a systematic process for following-up and implementing
new learning in the workplace. “Well it's nice to go to these [continuing education] events, but
really, I don't know how useful they are. You go, you sit, you listen, but then, well I at least
forget.”
The following is an extract from a focus group (conducted by the author) with first-year
pharmacy students about community placements. It illustrates how focus groups provide a
chance for participants to discuss issues on which they might disagree.
Interviewer: So you are saying that you would prefer health related placements?
Student 1: Not exactly so long as I could be developing my communication skill.
Student 2: Yes but I still think the more health related the placement is the more I'll gain from it.
Student 3: I disagree because other people related skills are useful and you may learn those from
taking part in a community project like building a garden.
Interviewer: So would you prefer a mixture of health and non health related community
placements?

Am J Pharm Educ. 2010 Oct 11; 74(8): 141.

PMCID: PMC2987281

PMID: 21179252

Presenting and Evaluating Qualitative Research

Claire Anderson, PhD, BPharm

Qualitative research. British Medical Journal Web
site. http://resources.bmj.com/bmj/authors/checklists-forms/qualitative-research. Accessed
August 31, 2010.
10. Qualitative research review guidelines – RATS. BioMed
Central.http://www.biomedcentral.com/info/ifora/rats Accessed August 31, 2010.

Micro-interlocutor Analysis: A New Method

of Analysis for Focus Group Research
According to Wilkinson (1998), most focus group analysts use the group as the unit of analysis.
However, using the group as the unit of analysis precludes the analysis of individual focus group
data. In particular, it prevents the researcher from documenting focus group members who did
not
contribute to the category or theme. As such, their voices, or lack thereof, are not acknowledged.
These focus group members might include those who are relatively silent (e.g., members who are
too shy to speak about this issue; members who do not want to reveal that they have a different
opinion, attitude, experience, level of knowledge, or the like; members who do not deem this
issue to be worth discussing), members who are relatively less articulate, members who have a
tendency to acquiesce to the majority viewpoint, and members who are not given the opportunity
to speak (e.g., due to one or more members dominating the discussion, due to insufficient time
for
them to speak before the moderator moves on to the next question).
As noted by Crabtree, Yanoshik, Miller, and O’Connor (1993), a sense of consensus in the data
actually might be an artifact of the group, being indicative of the group dynamics, and might
provide little information about the various views held by individual focus group members.
According to Sim (1998), “Conformity of opinion within focus group data is therefore an
International Journal of Qualitative Methods 2009, 8(3)
8
emergent property of the group context, rather than an aggregation of the views of individual
participants” (p. 348). As such, when discussing emergent themes, we recommend that in
addition to providing verbatim statements (i.e., quotations) made by focus group participants,
whenever possible, researchers delineate information about the number or proportion of
members
who appeared to be part of the consensus from which the category or theme emerged.
Furthermore, researchers should specify the number or proportion of members who appeared to
represent a dissenting view (if any) as well as how many participants did not appear to express
any view at all. In addition, because merely agreeing to a majority view either verbally (e.g., by
using statements such as “I agree” or “Yes”; by making an utterance such as “Uh-um”) or
nonverbally (e.g., nodding one’s head or smiling) might reflect some level of acquiescence, we
suggest that researchers document how many focus group members provide substantive
statements or examples that generate or support the consensus view. Similarly, we recommend
that they document how many members provide substantive statements or examples that suggest
a dissenting view. Thus, researchers/moderators are reliant on the assistant moderator who is in
the best position to record information about the level of consensus and dissension. To facilitate
this information-gathering process, we recommend that the assistant moderator use template
sheets. For example, he or she could use a matrix similar to the one in Figure 1.
The following notations can be entered in the cells:
A = Indicated agreement (i.e., verbal or nonverbal)
D = Indicated dissent (i.e., verbal or nonverbal)
SE = Provided significant statement or example suggesting agreement
SD = Provided significant statement or example suggesting dissent
NR = Did not indicate agreement or dissent (i.e., nonresponse)
International Journal of Qualitative Methods 2009, 8(3)
9
Using Figure 1 would allow the researcher to count the number of focus group members falling
into each category, which could appear in the final report (e.g., “Four of the six focus group
members believed…, with the remaining two members not providing any response to this
question”; “All six focus group members had this experience….with one member vividly
recalling…”).
Some focus group methodologists (e.g., Carey, 1995; Kidd & Marshall, 2000; Morgan, 1993,
1997; Silverman, 1985) have promoted the use of simple descriptive counts of categories. We
agree that counts can provide very useful information, not only about level of consensus/dissent
but also about response patterns among the focus group members. However, we believe that
counts should never be used to replace any qualitative data arising from focus groups because by
themselves they can present a misleading picture. In particular, the fact that the majority or even
all of the focus group members express a particular viewpoint does not necessarily imply that
this
viewpoint is important or compelling. However, when contextualized, the use of counts can
provide richer information than would be obtained by using qualitative data alone (cf.
Sandelowski, 2001). Indeed, supplementing qualitative data with counts yields a form of mixed
methods data analysis, or what is also known as mixed analysis (Onwuegbuzie & Teddlie, 2003;
see also Morse, 2003). When used in this manner, enumerating the frequency of a particular
viewpoint or experience actually expands the data set rather than reduces it. For example, we
believe that it is more informative to report that 7 out of 8 participants held a certain viewpoint
(i.e., data expansion) than to state that the majority of participants held a certain viewpoint (i.e.,
data reduction). Moreover, in addition to helping emergent themes be situated in a more
meaningful context (i.e., enhancing representation), enumerating the data (where possible) can
help to validate any inferences made about the level of consensus.
As stated by Sechrest and Sidani (1995, p. 79), “qualitative researchers regularly use terms such
as ‘many,’ ‘most,’ ‘frequently,’ ‘several,’ ‘never,’ and so on. These terms are fundamentally
quantitative.” Thus, focus group researchers can obtain more meaning by disclosing information
about level of consensus/dissent. Moreover, just as using counts by themselves can be
problematic, mainly reporting and describing the themes that emerge from an analysis of focus
groups also can be misleading because, to the extent that any themes that might stem from
dissenters are ignored, it can lead to unwarranted analytical generalizations (i.e., in which
findings are “applied to wider theory on the basis of how selected cases ‘fit’ with general
constructs”; Curtis, Gesler, Smith, & Washburn, 2000, p. 1002) and internal (statistical)
generalizations (i.e., which, in the context of focus groups, involve making generalizations,
inferences, or predictions on data obtained from one or more focus group members to all focus
group members; Onwuegbuzie, Slate, Leech, & Collins, 2009). Thus, the inclusion of frequency
data helps the researcher to disaggregate focus group data, which is consistent with the
qualitative
researcher’s notion of treating each focus group member as a unique and important study
participant. Further, as noted by Onwuegbuzie, Johnson, and Collins (2009), the ontological,
epistemological, and methodological assumptions and stances representing constructivist and
other qualitative-based paradigms (e.g., participatory paradigm) do not prevent descriptive
statistics from being combined with qualitative data.
Lazarsfeld, the same methodologist who co-developed focus group methodology, as we noted
previously, and his colleague Allen Barton (Barton & Lazarsfeld, 1955) advocated the use of
what they coined as quasi-statistics in qualitative research. According to these authors,
quasistatistics refer to the use of descriptive statistics that can be extracted from qualitative data.
Furthermore, Howard Becker (1970), a prolific symbolic interactionist, concluded that “one of
International Journal of Qualitative Methods 2009, 8(3)
10
the greatest faults in most observational case studies has been their failure to make explicit the
quasi-statistical basis of their conclusions” (pp. 81–82). Consistent with this assertion, Maxwell
(2005) observed,
Quasi-statistics not only allow you to test and support claims that are inherently
quantitative, but also enable you to assess the amount of evidence in your data that
bears on a particular conclusion or threat, such as how many discrepant instances
exist and from how many different sources they were obtained. (p. 113, emphasis in
original)
Of course, authors (e.g., Ashbury, 1995; Sim 1998) and others who are quick to declare that the
use of counts in the analysis of focus group data can be misleading are slow to acknowledge that
omitting numeric information also can be misleading, especially when the group is dominated by
one or a few focus group members.
Pre-Experimental Design
Pre-experimental designs are so named because they follow basic experimental steps but
fail to include a control group. In other words, a single group is often studied but no
comparison between an equivalent non-treatment group is made. Examples include the
following:

The One-Shot Case Study.

In this arrangement, subjects are presented with some type of treatment, such as a
semester of college work experience, and then the outcome measure is applied, such as
college grades. Like all experimental designs, the goal is to determine if the treatment had
any effect on the outcome. Without a comparison group, it is impossible to determine if the
outcome scores are any higher than they would have been without the treatment. And,
without any pre-test scores, it is impossible to determine if any change within the group
itself has taken place.

Advantages of focus groups

The main advantages of focus groups are:

 they are useful to obtain detailed information about personal and group feelings,
perceptions and opinions
 they can save time and money compared to individual interviews
 they can provide a broader range of information
 they offer the opportunity to seek clarification
 they provide useful material eg quotes for public relations publication and presentations

Disadvantages of focus groups

The main disadvantages of focus groups are:

 there can be disagreements and irrelevant discussion which distract from the main focus
 they can be hard to control and manage
 they can to tricky to analyse
 they can be difficult to encourage a range of people to participate
 some participants may find a focus group situation intimidating or off-putting;
participants may feel under pressure to agree with the dominant view
 as they are self-selecting, they may not be representative of non-users.

McNicol, Sarah (2004), "The evalued toolkit: a framework for the qualitative evaluation of electronic
information services", VINE, The Journal of Information & Knowledge Management Systems 34(4), pp.
172-175.
Questions for student focus groups

https://www.dmu.ac.uk/documents/.../questions-for-student-focus-groups.pdf

Example of an ANOVA Calculation

It will be easy to get lost in what follows. Here is the list of steps that we will follow in the
example below:

1. Calculate the sample means for each of our samples as well as the mean for all of
the sample data.
2. Calculate the sum of squares of error. Here within each sample, we square the
deviation of each data value from the sample mean. The sum of all of the squared
deviations is the sum of squares of error, abbreviated SSE.
3. Calculate the sum of squares of treatment. We square the deviation of each
sample mean from the overall mean. The sum of all of these squared deviations is
multiplied by one less than the number of samples we have. This number is the
sum of squares of treatment, abbreviated SST.
4. Calculate the degrees of freedom. The overall number of degrees of freedom is
one less than the total number of data points in our sample, or n - 1. The number
of degrees of freedom of treatment is one less than the number of samples used, or
m - 1. The number of degrees of freedom of error is the total number of data
points, minus the number of samples, or n - m.
5. Calculate the mean square of error. This is denoted MSE = SSE/(n - m).

6. Calculate the mean square of treatment. This is denoted MST = SST/m - `1.
7. Calculate the F statistic. This is the ratio of the two mean squares that we
calculated. So F = MST/MSE.
Software does all of this quite easily, but it is good to know what is happening behind the scenes.
In what follows we work out an example of ANOVA following the steps as listed above.

Data and Sample Means

Suppose we have four independent populations that satisfy the conditions for single factor
ANOVA. We wish to test the null hypothesis H0: μ1 = μ2 = μ3 = μ4. For purposes of this
example, we will use a sample of size three from each of the populations being studied. The data
from our samples is:

Sample from population #1: 12, 9, 12. This has a sample mean of 11.
Sample from population #2: 7, 10, 13. This has a sample mean of 10.
Sample from population #3: 5, 8, 11. This has a sample mean of 8.
Sample from population #4: 5, 8, 8. This has a sample mean of 7.
The mean of all of the data is 9.

Sum of Squares of Error

We now calculate the sum of the squared deviations from each sample mean. This is called the
sum of squares of error.

For the sample from population #1: (12 – 11)2 + (9– 11)2 +(12 – 11)2 = 6
For the sample from population #2: (7 – 10)2 + (10– 10)2 +(13 – 10)2 = 18
For the sample from population #3: (5 – 8)2 + (8 – 8)2 +(11 – 8)2 = 18
For the sample from population #4: (5 – 7)2 + (8 – 7)2 +(8 – 7)2 = 6.
We then add all of these sum of squared deviations and obtain 6 + 18 + 18 + 6 = 48.

Sum of Squares of Treatment

Now we calculate the sum of squares of treatment. Here we look at the squared deviations of
each sample mean from the overall mean, and multiply this number by one less than the number
of populations:

3[(11 – 9)2 + (10 – 9)2 +(8 – 9)2 + (7 – 9)2] = 3[4 + 1 + 1 + 4] = 30.

Degrees of Freedom
Before proceeding to the next step, we need the degrees of freedom. There are 12 data values and
four samples. Thus the number of degrees of freedom of treatment is 4 – 1 = 3. The number of
degrees of freedom of error is 12 – 4 = 8.

Mean Squares
We now divide our sum of squares by the appropriate number of degrees of freedom in order to
obtain the mean squares.

The mean square for treatment is 30 / 3 = 10.

The mean square for error is 48 / 8 = 6.
The F-statistic
The final step of this is to divide the mean square for treatment by the mean square for error.
This is the F-statistic from the data. Thus for our example F = 10/6 = 5/3 = 1.667.

Tables of values or software can be used to determine how likely it is to obtain a value of the F-
statistic as extreme as this value by chance alone.

Dependent t-test for paired samples

What does this test do?

The dependent t-test (also called the paired t-test or paired-samples t-test) compares
the means of two related groups to determine whether there is a statistically significant
difference between these means.

What variables do you need for a dependent t-test?

You need one dependent variable that is measured on an interval or ratio scale (see
our Types of Variable guide if you need clarification). You also need one categorical
variable that has only two related groups.
Join the 10,000s of students, academics and professionals who rely
on Laerd Statistics.TAKE THE TOUR PLANS & PRICING
What is meant by "related groups"?

A dependent t-test is an example of a "within-subjects" or "repeated-measures"

statistical test. This indicates that the same participants are tested more than once.
Thus, in the dependent t-test, "related groups" indicates that the same participants are
present in both groups. The reason that it is possible to have the same participants in
each group is because each participant has been measured on two occasions on the
same dependent variable. For example, you might have measured the performance of
10 participants in a spelling test (the dependent variable) before and after they
underwent a new form of computerised teaching method to improve spelling. You would
like to know if the computer training improved their spelling performance. Here, we can
use a dependent t-test because we have two related groups. The first related group
consists of the participants at the beginning (prior to) the computerised spell training
and the second related group consists of the same participants, but now at the end of
the computerised training.

Paired-samples t test (dependent t test)

This is used to compare the means of two variables for a single group. The procedure computes
the differences between values of the two variables for each case and tests whether the average
differs from zero. For example, you may be interested to evaluate the effectiveness of a
mnemonic method on memory recall. Subjects are given a passage from a book to read, a few
days later, they are asked to reproduce the passage and the number of words noted. Subjects are
then sent to a mnemonic training session. They are then asked to read and reproduce the passage
again and the number of words noted. Thus each subject has two measures, often called before
(or pre) and after (or post) measures.

An alternative design for which this test is used is a matched-pairs or case-control study. To
illustrate an example in this situation, consider treatment patients. In a blood pressure study,
patients and control might be matched by age, that is, a 64-year-old patient with a 64-year-old
control group member. Each record in the data file will contain responses from the patient and
also for his matched control subject.

Write down the null and alternative hypotheses for the Paired-samples t test (dependent t test):

Null Hypothesis (Ho): There is no difference in the average number of words recalled before and
after training.
T & P: The Tweedledee and Tweedledum of a T-test
T and P are inextricably linked. They go arm in arm, like Tweedledee and Tweedledum. Here's why.

When you perform a t-test, you're usually trying to find evidence of a significant difference between
population means (2-sample t) or between the population mean and a hypothesized value (1-sample
t). The t-value measures the size of the difference relative to the variation in your sample data. Put
another way, T is simply the calculated difference represented in units of standard error. The greater
the magnitude of T, the greater the evidence against the null hypothesis. This means there is greater
evidence that there is a significant difference. The closer T is to 0, the more likely there isn't a
significant difference.

Remember, the t-value in your output is calculated from only one sample from the entire population.
It you took repeated random samples of data from the same population, you'd get slightly different t-
values each time, due to random sampling error (which is really not a mistake of any kind–it's just
the random variation expected in the data).

How different could you expect the t-values from many random samples from the same population to
be? And how does the t-value from your sample data compare to those expected t-values?

You can use a t-distribution to find out.

Alternative Hypothesis (H1): There is a difference in the average number of words recalled
before and after training.

CALCULATING INDEPENDENT TWO SAMPLE T TEST : EXCEL

TEMPLATE
Deepanshu Bhalla Add Comment Statistics Using Excel

I developed an excel template that calculates independent two sample t test. It also writes summary
report which is based on p-value. This spreadsheet can handle up to 10,000 cases.

Download link : Independent t-test

Meaning :
The independent t test allows researchers to evaluate the mean difference between two
populations using the data from two separate samples.

Purpose :
The general purpose of the independent t test is to determine whether the sample mean difference
obtained is a real difference between the two populations or simply the result of sampling error.
Examples :

1. Do males and females differ in terms of their exam scores?

2. Is diet A better than diet B for asthma patients?

Assumptions :
1. The two groups must be independent.
2. The scores in each population must be normally distributed.
3. The two populations must have equal variances.

Interpretation :

Check the p-value. It is the lowest level of significance at which we can reject our null hypothesis. It
helps researchers determine if their hypotheses are correct.

Suppose the p-value is 0.39, i.e. greater than 0.05 (or 5 percent), it can be concluded that we don't
reject the null hypothesis at 5% significance level. That implies there is no mean difference between
the two groups.

Suppose the p-value is 0.029, i.e. less than 0.05 (or 5 percent), it can be concluded that we reject the
null hypothesis at 5% significance level. That implies there is a mean difference between the two
groups.