INSTITUTE OF PUBLIC ENTERPRISE

Sampling Theory,
Correlation, & Linear
Regression Techniques
Case Studies
Dr.Shaheen
8/9/2017

This document presents case studies on Sampling Theory & Techniques, Correlation, Linear Regression
Techniques (Simple Linear Regression, Multiple Linear Regression & Logistic Regression)
 Sampling Theory & Techniques

Case Study - Can this Survey be saved?

“What’s troubling me is that you can’t just pick a new random sample just because
somebody didn’t like the results of the first survey. Please tell me more about what’s been done.”
Your voice is clear and steady, trying to discover what actually happened and, you hope, to
identify some useful information without the additional expense of a new survey.
“It’s not that we didn’t like the results of the first survey,” responded R.L.Steegmans,
“it’s that only 54% of the membership responded. We hadn’t even looked at their planned
spending when the decision (to sample again) was made. Since we had (naively) planned on
receiving answers from nearly all of the 400 people initially selected, we choose 200 more at
random and surveyed them also. That’s the second sample.” At this point, sensing that there’s
more to the story, you simply respond “Uh huh...” Sure enough, more follows: “Then
E.S.Eldredge had this great idea of following up on those who didn’t respond. We sent them
another whole questionnaire, together with a crisp dollar and a letter telling them how important
their responses are to the planning of the industry. Worked pretty well. Then, of course, we had
to follow up the second sample as well.”
“Let me see if I understand,” your reply. “You have two samples: one of 400 people and
one of 200. For each, you have the initial responses and follow-up responses. Is that it?” “Well,
yes, but there was also the pilot study – 12 people in offices downstairs and across the street.
We’d kinda like to include them, average them, with the rest because we worked so hard on that
at the start, and it seems a shame to throw them away. But all we really want is to know average
spending to within about a hundred dollars.”
At this point, you feel that you have enough of the background information to evaluate
the situation and to either recommend an estimate or an additional survey. Here are additional
details for the survey of the 8,391 overall memberships in order to determine planned spending
over the next quarter.

Dr.Shaheen, Institute of Public Enterprise Page 2

Initial Mailing Pilot First Second Both All
Study Sample Sample Samples Combined
Mailed 12 400 200 600 612
Responses 12 216 120 336 348
Average $39,274.89 $3,949.40 $3,795.55 $3,894.45 $5,114.47
Standard Deviation $9,061.91 $849.26 $868.39 $858.02 $6,716.42

Follow-up Mailing Pilot First Second Both All

Study Sample Sample Samples Combined
Mailed 0 184 80 264 264
Responses 0 64 18 82 82
Average $1,238.34 $1,262.34 $1,243.60 $1,243.60
Standard Deviation $153.19 $156.59 $153.29 $153.29

Initial & Follow-up Pilot First Second Both All

Mailing Study Sample Sample Samples Combined
Mailed 12 400 200 600 612
Responses 12 280 138 418 430
Average $39,274.89 $3,329.73 $3,465.13 $3,374.43 $4,376.30
Standard Deviation $9,061.91 $1,364.45 $1,179.50 $1,306.42 $6,229.77

Discussion Questions

1. Do you agree that drawing a second sample was a good idea?

2. Do you agree that the follow-up mailings were a good idea?
3. How might you explain differences among averages in the results?
4. Are the results useful? Which ones are useful? Are they sufficient or is further study
needed?

Dr.Shaheen, Institute of Public Enterprise Page 3

This case is about a sampling plan with some complications: changes were made, with a second
sample having been drawn. In addition, there were subsequent follow-up efforts to obtain
responses from those who initially did not respond, as well as a small & apparently non-
representative pilot study.

1. Do you agree that drawing a second sample was a good idea?

o It’s not entirely clear that this was a good idea and it may well have been unnecessary.
o While it’s usually better to have more information, it is not always worth the extra cost.
o The second sample may not have been necessary because the standard error of the first
sample initial returns ($57.78) is already less that the $100 to within which they want to
know average spending.
o Those who chose to respond initially may not be entirely representative. Rather than
choose a new sample & receive another biased group of responses, the follow-up of
non-respondents is preferable.
o However, there is nothing fundamentally wrong with choosing a second sample for the
purpose of obtaining more information. It is permissible to draw two random samples
and then combine them, forming a single random sample with a larger size, provided the
decision to draw the second sample was not based on estimates obtained from the first
sample.

2. Do you agree that the follow-up mailings were a good idea?

Yes, this probably was a good idea. Since the 400 in the initial mailing of the first sample
were chosen to represent the entire membership, the opinions of the initial non-respondents are
important. It can happen that those who respond initially are different from the others; perhaps
they have more time, or are more actively involved in the issues. Since the purpose is to survey
the membership (& not just the most active members) these extra efforts should help ensure
representation.

Dr.Shaheen, Institute of Public Enterprise Page 4

3. How might you explain differences among averages in the results?
o The pilot study averages are very much higher than any of the others. Recall that these
were not randomly drawn and so we have no reason to consider them representative of
the membership in general.
o The follow-up averages are much lower than the others. Evidently there is a relationship
between planned spending and sending in the questionnaire. Those with high spending
were more likely to send it in. Differences from one sample to the other are not large.

4. Are there useful results here? Which ones are useful? Are they sufficient, or is further
study needed?
o There are some useful results here, but even after follow-up efforts, only about 70%
have responded. We still lack information from about 30% of these people. Considering
the differences between initial mailing and follow-up responses, these 30% could well
be below any of the reported averages. We don’t know. In the real world, there often is
non-response that cannot be eliminated as a problem.
o The pilot study should probably not be combined with the other responses. The pilot
study was not a random sample to begin with and has served its purpose in testing the
questionnaire.
o It might be argued that all other responses (first and second sample, initial mailings and
follow-up responses) are useful and should be combined. The resulting average ($3,374)
has a standard error of $64, which is less than the $100 required. This then represents
the planned spending of the approximately 70% of members who are likely to answer
these questionnaires.
o Ideas for further study could include different design methods or additional follow-up of
non-respondents.

Dr.Shaheen, Institute of Public Enterprise Page 5

 Linear Regression Models

Case Study – Just one more production step: Is it worthwhile?

The techies (scientists) in the laboratory have been lobbying you, and management in
general, to include just one more laboratory step. They think it’s a good idea, although you have
some doubt because one of them is known to be good friends with the founder of the start-up
biotechnology company that makes the reagent used in the reaction. But if adding this step works
as expected, it could help immensely in reducing production costs. The trouble is, the test results
just came back and they don’t look so good. Discussion at the upcoming meeting between the
technical staff and management will be spirited, so you’ve decided to take a look at the data.

Your firm is anticipating government approval from the Food and Drug Administration
(FDA) to market a new medical diagnostic test made possible by monoclonal antibody
technology, and you are part of the team in charge of production. Naturally, the team has been
investigating ways to increase production yields or lower costs.

The proposed improvements are to insert yet another reaction as an intermediate

purifying procedure. This is good because it focuses resources down the line on the particular
product you want to produce. But it shares the problem of any additional step in the laboratory:
one more manipulation, one more intervention, and one more way for something to go wrong. It
has been suggested that, small amounts of the reagent may be helpful, trying to purify too well
will actually decrease the yield and increase the cost. The design of the test was to have a series
of test production runs, each with a different amount of purifier, including one test run with the
purification step omitted entirely (i.e., 0 purifier). The order of the tests was randomized so that
at any time trends would not be mistakenly interpreted as being due to purification.

Discussion Questions

1. Does the amount of purifier have a significant effect on yield according to this regression
analysis? Based on this alone, would you be likely to recommend including a purifying
step in the production process?

2. What would you recommend? Are there any other considerations that might change your
mind?

Dr.Shaheen, Institute of Public Enterprise Page 6

 Amount of Purifier(s) Observed Yield
0 13.39
1 11.86
2 27.93
3 35.83
4 28.52
5 41.21
6 37.07
7 51.07
8 51.69
9 31.37
10 21.26

This case is about the need to look at data and not to jump to conclusions based only on an
analysis that requires specific assumptions (e.g., the linear model) that may not be satisfied. It is
also about the problems of the real world where answers may not be clear, even once some data
have been obtained.

Dr.Shaheen, Institute of Public Enterprise Page 7

1. Does the amount of purifier have a significant effect on yield according to this regression
analysis? Based on this alone, would you be likely to recommend including a purifying step
in the production process?

No, the amount of purifier does not have a significant effect on yield. The t statistic is
1.805. The p-value, 0.105, is not even significant if a one-sided test is used. (credit should
certainly be given to students who realize that a one-sided test is appropriate here: the p-
value would be 0.052=0.105/2 in this case). Based on this alone, there is no evidence that
purification affects yield other than randomly.

But this hypothesis testing conclusion is a weak one: we accept the null hypothesis
perhaps for lack of enough data against it. It is close to significant with the one-sided test
and we do have a small sample size (n=11). Perhaps the effect of the purifier has been
masked by randomness of the production process, and more data would be needed in order
to detect it.

2. What would you recommend? Are there any other considerations that might change your
mind?

From the scatter-plot it looks like the purifier helps (increases yield) somewhat through
about amount 8, but a higher purifier amounts the yield decreases. A regression omitting the
last point is highly significant (t=3.49 and p=0.008), while a regression omitting the last two
points is highly significant (t=7.12 and p=0.0002). On the other hand, it can be argued that
we are not really free to delete points we don’t like. After all, by doing that you could end up
with a significant relationship in nearly any situation.

If the true relationship is an increase in yield at low purifier levels and a decline at high
levels, then the linear model assumption is violated and statistical inference may not be
valid. Thus the finding of “not significant” is open to some question. If the cost is not too
high, you might consider viewing this as a pilot study. In the next stage, more data would be
collected to guide us towards the optimal level of purifier to obtain the highest yield. Some
students may want to consider the cost of the purifier, arguing that the optimal level will be
somewhat under the amount that would produce the highest yield.

Dr.Shaheen, Institute of Public Enterprise Page 8

Case Study – Controlling Quality of Production

Everybody seems to disagree about just why so many parts have to be fixed or thrown
away after they are produced. Some say that its the temperature of the production process, which
needs to be held constant (with a reasonable range). Others claim that its clearly the density of
the product, and that if we could only produce a heavier material, the problems would disappear.
Then there is Ole, who has been warning everyone forever to take care not to push the equipment
beyond its limits. This problem would be the easiest to fix, simply by slowing down the
production rate; however, this would increase costs. Interestingly, many of the workers on the
morning shift think that the problem is “those inexperienced workers in the afternoon,” who,
curiously, feel the same way about the morning workers.

Ever since the factory was automated, with computer network communication and bar
code readers at each station, data have been piling up. You have finally decided to have a look.
After your assistant aggregated the data by four-hour blocks and then typed in the AM/ PM
variable, you found the following note on your desk with a printout of the data already loaded
into the computer network. The variables include:

• Temperature actually measures temperature variability as a standard deviation during the

time of measurement.
• Density indicates the density of the final product.
• Rate indicates the rate of production.
• AM/ PM is an indicator variable that is 1 during morning production and is 0 during the
afternoon.
• Defect is the average number of defects per 1,000 produced.

Naturally you decide to run a multiple regression to predict the defect rate from all of the
explanatory variables, the idea being to see which (if any) are associated with the occurrence of
defects. There is also the hope that if a variable helps predict defects, then you might be able to
control (reduce) defects by changing its value.

Discussion Questions

1. What are the obvious conclusions from the hypothesis tests in the regression output?

2. Look through the data. Do you find anything that calls into question the regression
results? Perform further analysis as needed.

3. What action would you recommend? Why?

Dr.Shaheen, Institute of Public Enterprise Page 9

 Temperature Density Rate AM/ PM Defect
0.97 32.08 177.7 0 0.2
2.85 21.14 254.1 0 47.9
2.95 20.65 272.6 0 50.9
2.84 22.53 273.4 1 49.7
1.84 27.43 210.8 1 11.0
2.05 25.42 236.1 1 15.6
1.50 27.89 219.1 0 5.5
2.48 23.34 238.9 0 37.4
2.23 23.97 251.9 0 27.8
3.02 19.45 281.9 1 58.7
2.69 23.17 254.5 1 34.5
2.63 22.70 265.7 1 45.0
1.58 27.49 213.3 0 6.6
2.48 24.07 252.2 0 31.5
2.25 24.38 238.1 0 23.4
2.76 21.58 244.7 1 42.2
2.36 26.30 222.1 1 13.4
1.09 32.19 181.4 1 0.0
2.15 25.73 241.0 0 20.6
2.12 25.18 226.0 0 15.9
2.27 23.74 256.0 0 44.4
2.73 24.85 251.9 1 37.6
1.46 30.01 192.8 1 2.2
1.55 29.42 223.9 1 1.5
2.92 22.50 260.0 0 55.4
2.44 23.47 236.0 0 36.7
1.87 26.51 237.3 0 24.5
1.45 30.70 221.0 1 2.8
2.82 22.30 253.2 1 60.8
1.74 28.47 207.9 1 10.5

Dr.Shaheen, Institute of Public Enterprise Page 10

This case is about doing good investigative work. The results are not as useful or clear as they
first appear. It may be surprising for students to discover just how sensitive multiple regression
can be to the effects of a single outlier (look down the AM/ PM data column). The case also
encourages thinking about how to identify and deal with a problem that has multiple causes that
cannot all be disentangled from one another (multi-collinearity).

1. What are the obvious conclusions from the hypothesis tests in the regression output?

The F test is very highly significant (p=4.37E-12, so p<0.001). The t tests for Temperature
and for AM/ PM are significant. This says that temperature variability has a significant
effect on the defect rate, adjusting for the other factors. Similarly for AM/ PM. This suggests
that you should concentrate your attention on these two factors: controlling temperature
variability (because higher temperature variability is associated with significantly more
defects), and seeing why defects seem to be significantly lower in the morning, all else
equal.

Dr.Shaheen, Institute of Public Enterprise Page 11

2. Look through the data. Do you find anything that calls into question the regression results?
Perform further analysis as needed.

None of the t-tests for the explanatory variables is significant. However, the F test
remains highly significant. This suggests multi-collinearity, which is confirmed by the
correlation matrix for these variables, which show three variables (Temperature, Density and
Rate) correlated with one another at ±0.9 or stronger.

Since, in the multiple regression, no single X variable is significant (holding the others
constant) some students may want to consider these variables one at a time. Here are results
of ordinary bivariate regressions to predict defect from each of the X variables. All are
significant now, except AM/ PM, which makes no significant difference.

3. What action would you recommend? Why?

One suggestion that is consistent with the analysis of the corrected data is to concentrate on
the variable that “would be the easiest to fix”, namely slowing down the rate of production.
The high correlations between rate and each of the other X variables (except AM/ PM)
suggest that a lower rate may result in less temperature variability and a higher density, each
of which is associated with fewer defects. Of course this will raise costs. Some students may
have something to say about the cost-benefit trade-off between a lower defect rate (the
benefit) and fewer items produced (the cost).

Dr.Shaheen, Institute of Public Enterprise Page 12