Eng Mgt 6113

Advanced Personnel Management

(Strategic Human Resource Management & Measurement)

The Payoff from Enhanced Selection

David G. Spurlock, Ph.D., Instructor
 The Payoff from Enhanced Selection
Introduction & Overview
The Logic of Investment Value Calculated Using
Utility Analysis (Measures, Analytics, & Example)
Economic Factors (Variable Costs, Taxes, Discounting)
Employee Flows
Probationary Flows, Multiple Selection Devices, &
Departures from Top-Down Hiring
Risk & Uncertainty
Communicating to Decision Makers
Supply Chain Analogy & Conclusion
emgt6113ch10 2
 Introduction & Overview
Interesting comments about evolution of Googles
selection process
(Note: Remarks most relevant to highly selective
organizations with positions differentiated by impact)
Main point of chapter is to consider selection as
another important business process where certain
kinds of information improve ability to predict the
performance of a risky asset
Integrates ideas from Ch. 8 & Ch. 9 to demonstrate
calculations of the value of improved selection and
other aspects of the talent supply chain
emgt6113ch10 3
The Logic of Investment Value Calculated Using Utility
Analysis (Measures, Analytics, & Example)
Figure 10-1illustrates the enhanced BCG model
from Ch. 8 (quantity, quality, and cost factors)
Extended example of using a test to select
computer programmers (Programmers Aptitude
Test or PAT)
Note how tests are used primarily for selection at lower
levels of experience (thus emphasis on aptitude)
while positions at higher levels rarely use tests but
instead focus on demonstrated achievements and
experience
emgt6113ch10 4
The Logic of Investment Value Calculated Using Utility
Analysis (Measures, Analytics, & Example)
PAT study used actual data where relevant but
incorporated sensitivity analyses where data
wasnt available (e.g., SR values of .05, .10, .20,
, .80 and r values for previous selection
procedures of .00, .20, .30, .40, .50)
PAT study also incorporated expert judgment
(e.g., SDy) & other assumptions about cost)
Modified BCG model used in study shown on next
slide
emgt6113ch10 5
The MODIFIED Brogden-Cronbach-Gleser Model
( )
=

is the change in utility per applicant selected,
is the tenure in years of the average selectee
is the numbers of selectees annually
& are the predictive validities of the new & old selection procedures
is the standard deviation of the monetary criterion (e.g. job performance)
is the height of the normal curve at the predictor cutoff score (in std score
units)
is the selection ratio
is the cost of applying the new selection procedure to one applicant
c2 is the cost of applying the old selection procedure to one applicant

emgt6113ch10 6
The Logic of Investment Value Calculated Using Utility
Analysis (Measures, Analytics, & Example)

PAT study example with numbers in text showed

estimated gains in productivity for the entire
federal govt. from using the PAT (FOR ONE YEAR)
that ranged from $19.5 million to $334 million
depending upon the values used for some of the
assumed parameters

emgt6113ch10 7
The Logic of Investment Value Calculated Using Utility
Analysis (Measures, Analytics, & Example)

NOTE that while the PAT is only used for one

years cohort of selectees, individuals in that cohort
are, on average, with the govt. for 9.69 years, thus
the numbers above (e.g., $334 million) should be
divided by the total number of selectees (here
618) and then by 9.69

emgt6113ch10 8
Adjustments for Variable Costs, Taxes, Discounting
Great to be able to estimate financial impact of a
selection procedure BUT:
To make the numbers comparable to those used by
management for other investments, adjustments to the
numbers that take into account variable costs, taxes,
and discounting should be incorporated into the model
LONGER formula on the bottom of page 261
(Eq. 10-2) is second modification to the BCG
model & it includes those three factors

emgt6113ch10 9
 Adjustments for Variable Costs, Taxes, Discounting
Notes on the LONGER BCG formula (Eq. 10-2):
Discount factor expressed with summation notation
Subtle but important shift of SDy to SDsv so number
represents the std dev of sales value of productivity or
SV (essentially a revenue quantity distinct from profits)
Inclusion of V (proportion of SV due to variable costs)
Inclusion of TAX to represent the TAX RATE
Subtle shift in the meaning of r reflecting the use of SV
Use of Ct to represent the total selection cost for all
applicants (number selected divided by selection ratio)
emgt6113ch10 10
 Employee Flows
Notion of compound interest is applied to the
selectees as improved selection procedures are
applied year after year (additive cohort
effects)
By including a term for Nk = the number of
treated employees in the workforce k periods in
the future, these effects of employee flows can
be calculated

emgt6113ch10 11
 Employee Flows
Important to recognize that treated employees
in the example at the moment are those who are
selected through an improved selection procedure
but in principle any improved HR program
(training, performance appraisal, compensation,
etc.) could be considered a treatment in this sense
although the particular model (equation) of the
procedures utility would vary by context

emgt6113ch10 12
 Employee Flows
The resulting modification to the LONGER BCG
formula of Eq. 10-2 is formula on p. 264 (Eq. 10-5)
**IMPORTANT** The formula given in Eq. 10-5 is
ONLY for the change in utility for the kth period
NOTE that they assume several of the parameters
are constant over time (see top of p. 265) even
though this might not generally be true
Other assumptions about cost of treatment and the
discount factor are mentioned on the top of p. 265 as well
BUT WAIT! THERES MORE..
emgt6113ch10 13
 Employee Flows
The FULLY MODIFIED EXTREMELY LONG BCG
model is given in Eq. 10-6 on the bottom of p. 265
An example using it is worked with actual numbers so
you can see the effects over time of the improved
selection procedure
Fig. 10-2 on p. 267 tabulates the actual # of treated
employees remaining in workforce per year for 25 years
BOTTOM LINE: Improved selection procedures
can generate very large payoffs to an firm with
large numbers of employees so even multi-million$
selection systems can be justified
emgt6113ch10 14
 Probationary flows & other variations

Example of Whole Foods use of a (short)

probationary period as part of the selection
procedure
On its face, including a probationary procedure
seems like a good idea BUT its usefulness is
entirely dependent upon its relative validity, the
severity of selection errors, and the overall
variability of the applicant population

emgt6113ch10 15
 Probationary flows & other variations
The text doesnt mention this but FAIRNESS is
another consideration in these sort of procedures
In the case of Whole Foods, other employees are
voting on the probationary employee with a 2/3
majority required while this certainly could work if
the process is set up correctly, it could also be
corrupted easily by allowing unjustified votes or
through use of secret criteria unknown to the
probationary employee that arent really
connected to job performance

emgt6113ch10 16
 Probationary flows & other variations
Selection of active duty Navy SEALs from
applicants in the other Teams into SEAL Team Six
(focused on counterterror missions and advanced
technology beta testing) reportedly includes a
similar process of voting a candidate into Team Six
Current Team Six members vote yea or nay
apparently without having to justify their vote
Clearly the consequences for SEAL Team Six of total
and complete confidence in any new member (critical
mission success, life or death of team members) are
infinitely greater than the consequences of hiring a new
meat department associate at Whole Foods
emgt6113ch10 17
 Probationary flows & other variations
What about applicants who reject job offers? How
does that affect the utility calculations?
Some subtle assumptions here behind the seemingly
common sense logic
The common sense logic stated in the text:
50% rejection rate for offers made to top 20% of applicants is
equivalent to having a 40% SR instead of the desired 20% because
of need to subsequently dip into the next 20% tier
Important when there is a high correlation between the quality of
the applicants and the rejection of offers
AND when there is a larger proportion of rejected job offers
Studies show unadjusted utility formulas could be overestimating
gains by 30-80% if job offer rejection not considered
emgt6113ch10 18
 Probationary flows & other variations
Now the subtle assumptions about offer rejections:
First assumption is a marked difference in applicant
quality between first choices and second choices -
this is sometimes easy to document but not always
Compare (1) admitting students to a college where test scores are
the criterion for quality of applicant to (2) hiring department chairs
in academic departments at universities (or hiring or tenuring new
faculty for that matter) where judgments of quality are much
more subjective & variable and the pool of candidates is much
smaller
Even when you have a heterogeneous applicant pool and
objective criteria, the assumption of quality difference will be
most valid if there is a relatively steep drop off in quality below
a relevant point in the pool (say the top 10% or 20% of applicants)
emgt6113ch10 19
 Probationary flows & other variations
Now the subtle assumptions about offer rejections:
Second assumption is that all applications are
genuine and little self-selection of applicants)
For relatively unattractive employers, there may
always be some outstanding applicants that make the
rest of the pack look bad
The best may be submitting applications as
backups or because they lack accurate info about
the (relatively low) quality of the employer

emgt6113ch10 20
 Probationary flows & other variations
Now the subtle assumptions about offer rejections:
Think of safety schools for high schoolers applying to
college a Missouri HS student with great credentials to
major in engineering might apply to several schools that are
extremely selective: MIT, Stanford, Cal Tech (all private) &
Illinois (out of state tuition), but may also apply to Mizzou,
S&T, and Saint Louis University as safety schools
Conversely, lower quality organizations may also get a
larger proportion of lower quality (say average)
applicants because applicants judge their chances to be
better there
emgt6113ch10 21
 Probationary flows & other variations
Now the subtle assumptions about offer rejections:
Third, must consider self-fulfilling prophecies
Perception may become reality: once someone is
hired, opinions of their actual performance relative to
the imagined performance of ones who got away may
be unfairly negative due to the lowered expectations
and feelings of disappointment of being rejected by the
good people
Buyers remorse may thus be a factor in subsequent
performance evaluations (similar to marriage? )
By the way, some employers wont make offers to candidates
whom they think are too good because they feel the probability
of rejection is too highemgt6113ch10 22
 Probationary flows & other variations
Now the subtle assumptions about offer rejections:
Fourth and finally, what about places with far
more highly qualified applicants than
positions?
Incumbents & managers at the firm will tend to magnify
differences in quality between favored or chosen
candidates and ones they dont make offers to in order to
justify the decisions to themselves & to others in the firm
In these cases, the objective data would suggest there
shouldnt be much of an effect for rejected offers but
pride of incumbents and managers may distort that
reasoning
emgt6113ch10 23
 Probationary flows & other variations
Multiple Selection Devices
Assuming each selection tool provides unique and valid
information, combining several will result in higher
overall validity for the selection process
Thus a process that includes structured interviews; tests;
work samples; biodata from resumes, application forms,
& background checks; assessment centers, etc. will have
higher validity than using any of these tools alone
This means the increased utility of adding one new
procedure may NOT be as much compared with the
overall existing set as it would when simply comparing
to one existing procedure
emgt6113ch10 24
 Probationary flows & other variations
Multiple Selection Devices
A final caveat is that incremental cost of including
additional devices must be justified by the incremental
increase in benefits from the enhanced validity to the
overall selection process
Many organizations use selection procedures that have
never been analyzed formally for validity or
incremental utilities or for basic cost-benefit ratios
Often do what they do because their competitors do it or
because theyve always done it this way or because
some decision maker one time in the past thought it was
a good idea to do it the way they do it
emgt6113ch10 25
 Probationary flows & other variations
Top down selection is best approach
Example given of study of selection of forest
rangers comparing top-down selection (offers to
best candidates) with two other methods
Results consistent with common sense
expectations but value of study is that it
quantified benefit of top-down selection in
terms of differential increases in productivity
from differential quality of employees

emgt6113ch10 26
 Probationary flows & other variations
Cumulative adjustments & variations
Results from a computer simulation study indicate the
following rank order of impact in reductions of utility
estimates from base line
Economic adjustments (variable costs, taxes, discounting)
Multiple selection devices
Departures from top-down hiring
Probationary periods
Separations of high performers
Overall, after accounting for all of these, the adjusted
utility estimates ranged from 9% to 29% as large as the
unadjusted ones (but still positive in 84% of cases)
emgt6113ch10 27
 Probationary flows & other variations
Cumulative effects of all adjustments & variations
Conclusions
Increasing the validity of selection procedures can pay
off for organizations in monetary terms (and those
payoffs can be calculated)
Many factors affect magnitude of the payoff so, as with
any other business investment (e.g., building a new
plant, acquiring another company, launching a new
product or new marketing campaign) careful analysis is
required to crunch the numbers and make a good
decision

emgt6113ch10 28
 Risk & Uncertainty
Coping with risk & uncertainty in business
processes is a large part of what managers are
paid to do and coping with risks & uncertainties
associated with selection procedures is no
exception
Three standard techniques have been applied
successfully to selection processes:
Break-even analysis (from basic financial management)
Monte Carlo analysis (from decision science/statistics)
Confidence intervals (from statistics)

emgt6113ch10 29
 Risk & Uncertainty
Break-even analysis was discussed in Ch. 2 and
its two advantages are repeated here
Improves decisions w/ imperfect information by
deemphasizing precise estimates in favor of ranges
Helps identify whether variations in estimates matter (if
variations in utility estimates are all above or all below
break even threshold, they dont matter)
Also, break-even points for estimates of SDy are
often a relatively small fraction of estimated SDy
Break-even analysis tends to matter more when
comparing HR investments to other investments
emgt6113ch10 30
 Risk & Uncertainty
Monte Carlo analysis requires software to
sample parameter values from a probability
distribution and then calculate the impacts of the
variations in those parameters on a models
output values
Sort of systematic and justified trial and error
but it can be very useful in systems with multiple
parameters that vary considerably according to
known or justifiably assumed distributions
Most decision science/mgmt science texts have a
chapter or appendix that describes the technique
emgt6113ch10 31
 Risk & Uncertainty
Confidence intervals are a basic concept from
statistics that can be very informative
Confidence intervals can be calculated using any
portion (percentage) of a distribution (usually the
normal distribution) by multiplying the
corresponding number of standard deviations
by the standard error of the estimate
The 95% confidence interval is often used so the upper
and lower bounds on an estimate are computed by
adding to (upper) and subtracting from (lower) the
estimate a factor equal to 1.96 times the standard error
emgt6113ch10 32
 Risk & Uncertainty
Confidence intervals are interpreted to mean that (all
assumptions about the distribution being valid) the
true value of the estimate has a 95% chance of
being within the range specified by the interval
Similar in concept to the margin of error given in
poll results although sometimes the format for
margin of error in polls doesnt explicitly state the
percentage chance (e.g., Presidents approval
rating is 54% with a margin of error of +/- 3 % does
not explicitly tell you they are using a 95% interval)

emgt6113ch10 33
 Risk & Uncertainty
When the estimate is of the difference in means
between two groups, if the confidence interval
includes zero, then it is stated that there is no
statistically significant difference between the
groups (or conditions)
When the estimate is of a correlation and the
interval includes zero, it means the correlation is
not statistically significant based upon that
sample

emgt6113ch10 34
 Communicating to Decision Makers
Text offers some advice about how to frame results of
utility analyses for selection processes in ways that will have
the most positive impact on non-specialist decision
makers (e.g., CEOs and other top managers)
Advice is similar to what one should do when making any
persuasive argument about a business process:
Present conservative estimates (no WAGs or SWAGs)
Illustrate choices and their pros & cons
Avoid information overload and excessive technicalities (e.g.,
statistical formulae should be omitted unless requested)
Emphasize the same concerns and use the same terminology
as managers of operating departments do (cycle times, costs,
quality)
emgt6113ch10 35
 Selection & Talent Supply Chain
Leaders may readily use a mental model of supply
chains when considering many operational aspects of an
organization so it is wise to adopt that mental model when
discussing selection processes
Table 10-1 on pp. 276-277 show examples of the
Employee Selection analogues to the major functions
associated with Supply-Chain Management of material
resources:
Demand Planning/Forecasting
Production Planning & Scheduling
Distribution & Logistics
Inventory Management
emgt6113ch10 36
END OF SLIDE SET

emgt6113ch10 37