12 NOVEMBER 2010

Assignment 2
0LM05-Philosophy of management fashion
Tutor 1 – Group 13
Jonet Gaëlle 0755715
Martinez Pinzon Cesar Augusto 0757067
Miroux Florence 0757953

Instructions: The aim of this assignment is to improve a Master’s thesis project. For doing so, we
have first began by describing and analysing it according to the different aspects of the
relevant/rigorous debate and the Shrivastava’s criteria. Secondly, after evaluating the thesis and
identifying that it was more on the relevant side, we make some proposals to improve in order to
close the gap between the relevant and rigorous sides. Our proposal will change Elbers’s thesis and
make it more rigorous.
Introduction

The aim of this assignment is to improve a Master’s thesis project. In this paper, on the one hand we
will describe the existing project. On the other hand, we will make an analysis to decide if the thesis is
more relevant or rigorous. This analysis will be based on the weight of the different aspects of the
relevant/rigorous debate and the Shrivastava’s criteria. Then, we will make some proposal to
improve the thesis in order to close the gap between rigour and relevance. Last but not least, we
will defend our proposal for improvement.

Description of the master thesis

This master thesis is centralized about the theme of transfer of training. Transfer of training is defined as
’the effective applicability of trained Knowledge, Skills and Attitudes (KSA) in the appropriate work
setting and maintenance of these trained KSA’. (p3)

The aim of this master thesis was to diagnose the Vanderlande Industries (VI) problem within transfer of
training process and to propose ways to improve it.

The problematic and main research question of this thesis is the following: ‘How to organize the input
characteristics (i.e. learner characteristics, training characteristics, and work characteristics) in order to
improve the application of learned KSA in the work setting?’

In the one hand, the writer describes what transfer of training is and using his literature review, he
describes a model with the factors that have a direct or indirect influence on transfer of training. In this
model, there are three essentials training inputs: training characteristics, learner characteristics and
work environment characteristics. In the other hand, he focuses on the most important factors which
are relevant for the enterprise he has studied.

Then, he has collected data from two different groups and he has made a longitudinal design with three
data collection points (e.g. questionnaires). Then, he has analyzed the data in order to find the strongest
relations between the variables he has chosen to study and the transfer of training.

The conclusion of this work is three main improvement points to VI in order to improve the transfer of
training in this company.

Page 2
Analysis of the project in terms of rigour and relevance

Now, after having describing the existing project presented in this master thesis, we are going to analyse
the project in terms of rigour and relevance. In the lectures, we have studied five sets of ideas in order
to weigh the rigorous and relevant aspects.

The first point is: general versus specific. It is clear that the master thesis we are studying is specific
because it refers to a specific company. Moreover, the data are collected in this company and so the
results, relations that the writer has found in his project are applicable essentially in this company. At
the end of his study, he has established three changes that can help the VI company to improve the
transfer of training.

The second point is: formal versus informal. It is important to keep in mind that an informal project does
not mean that the project is bad. We have discovered the notion of robustness. A project which is
robust can be formal or informal and a project which is not robust too. The master thesis we are
studying is more formal than informal because the writer in this study is really well structured. He has
used statistical methods to prove the relations between the variables that he has studied. His
methodology is clear. At the beginning of the report, he has pointed out six questions to which he has
answered in the report in stating in which chapter of his work you can find the answers of these
questions. He has used also the van Strien’s regulative cycle (1997) to structure his study.

The third point is: theory-based versus data-based. To justify our point of view, we are going to state
some sentences of the thesis: “empirical testing of the transfer process in a longitudinal and cross-
sectional research design”, “the chosen variables are outlined and investigated to establish the impact
of these variables on the transfer of training” and “transfer of training and all relevant factors *…+ are
evaluated”. These sentences prove that this model starts from the data and is aimed at finding relations
between quantities. We can say that our thesis is data-based.

The fourth point is: descriptive versus prescriptive. The central research question of this master thesis is
the following: “How to organize the input characteristics (i.e. learner characteristics, training
characteristics, and work characteristics) in order to improve the application if learned KSA (knowledge,
skills and attitudes) in the work setting?” This question is prescriptive because the answer of this
question must be something that should or might be done to change reality. However, we can find a lot
of descriptive statements which describes how things are. From the beginning, where the author has
described the transfer of training until the explanations of the results, we can find descriptive parts

Page 3
because he has explained the model he has used, which variables he has considered in this company and
why. The analysis and the results of this study are also prescriptive. Descriptive is related to rigour
whereas prescriptive is related to relevance. We strongly think that the author of this thesis is aware of
this debate about rigour and relevance because the aim of his work is prescriptive but he has tried to
make effort to make a good description.

The fifth and last point is: science versus engineering or research versus design. We can say that the
thesis we are studying is more design-like than research-like. Indeed, design is problem-oriented; its goal
is to contribute to the solution of a practical problem. Nevertheless, some parts of the thesis are
research-like, for example, at the beginning, he has tried to create a model in gathering all the
information that he had learnt from his literature review.

This project does not involve generalization because the recommendations the writer has provided at
the end of his study are particular to the VI company because in his study, he has considered only the
variables that were relevant for the enterprise. In this study, he has also done an idealisation of the
problem because for example, he has not considered the influence of the personality traits on the
transfer of training.

Now that we are aware of the content of the thesis and the different arguments showing the rigorous
and relevant aspects of the work, we will analyze the thesis in a second step based on the Shrivastava’s
criteria.

In his article [1], Shrivastava develops three criteria to assess the rigorousness of research and five
others for assessing the relevance of research. His unit of analysis was at the research program level. We
will use and adapt those criteria at the level of the master thesis project. For each of them, we have
graded the master thesis between 1 and 5, 1 corresponding to the lowest grade when the thesis
succeeded poorly and 5 when it perfectly succeeded.

The first criterion for assessing the rigorousness of the research is the conceptual adequacy of the
framework that guides the thesis work. We graded it with a 3. On one hand, the writer grounded his
work in a conceptual model that he constructed based on a combination of several previous conceptual
models. The integrated conceptual model of transfer of training the author constructed includes

Page 4
training, learner and work environment characteristics that influence the training process. On the other
hand, the writer does not apply a theory, direct knowledge or methods to the VI company from the
transfer of training field.

The second criterion concerns the methodological rigor. We graded it with a 3. Even if subjective and
objective oriented methods can be rigorous, objective methods are commonly judged to be more
rigorous. The thesis uses subjective methods in the sense that the collection of data is based on the
trainee, supervisor and trainers individual points of view. However, the thesis also uses objective
oriented methods like quantifiable data and analytical methods (Cronbachs alpha test, expectation
maximization algorithm, Mahalanobis, Barlett’s test of sphericity, Kaiser-Mayer-Olkin Measure of
Sampling Adequacy, Pearson product moment correlations are techniques used in the analysis of the
collected data).

The third and last criterion for evaluating the rigor of the thesis is the extent of accumulated empirical
evidence supporting the theoretical structures. We graded this criterion with a 3. On the one hand, the
data explored and analyzed are credible for both the writer and the manager and useful for the
decision-making process. On the other hand, the findings and recommendations cannot be directly
applied in other context, thus the thesis recommendations do not essentially contribute to the transfer
of training’s conceptual model and its field of research.

The first Shrivastava’s criterion for judging the relevance of the research work is the meaningfulness,
understandable and comprehensibility of the results. We graded this criterion with a 4. All along the
thesis project, the writer captures and adequately describes the organization reality. Indeed, he only
considered the relevant variables in VI context which influence the transfer of training. However, we did
not grade this criterion with a 5 because the author did give not specific ways to implement its three
recommendations in term of time and human resources.

The second criteria used were the goal relevance of the work. We graded it with a 4 because the thesis
leads the three useful recommendations for the company. Those recommendations are relevant and
respond to the manager needs which were to know which training method plays an influencing role in
transfer of training in VI.

The third criterion used to assess the relevant aspects is the operational validity and actionability of the
results. We graded it with a 4. The given recommendations are concrete and operationalizable in VI
context. For example, when the author of the thesis formulated the recommendation to increase

Page 5
training expectations and emphasize training value for organization, he went throw a detailed level by
giving practical actions per time period (before, during, and after training) and per stakeholder (trainees,
trainers, supervisors and Vanderlande Academy).

The fourth criterion concerns the innovativeness of the research results. We graded it with a 2 because
we perceived the recommendations made rather common and unoriginal. As Shrivastava explains in his
article, new and non-obvious results are more likely to be useful than old and commonsensical. In this
thesis, some of the results (and its corresponding recommendations) were likely to be obtained without
the previous analysis, for instance, providing more feedback and make longer training programs are
commonsense improvements. Because of that, we believe that the recommendations lack of innovation.

Finally, the last Shrivastava’s criterion for evaluating research relevance is the cost of implementation.
Indeed, costly recommendations have lower chances to be implemented because of the risk of low
investment of money. For this thesis, we graded this criterion with a 4. The recommendations are
reasonable both in term on money and time needed for implementation. For instance, in the first
recommendation, the author advises the trainers to provide more feedback to the trainees and to
stimulate then by asking about their transfer intentions. The implementation of this recommendation is
neither times neither money consuming.

After exploring the different aspects of the rigour/relevance debate and evaluating the thesis according
to Shrivastava’s criteria, we conclude that this thesis is more on the relevant side than on the rigorous
side. According to Fransoo’s article *3+, both research driven and design driven contribute to science and
its appliance. Van Aken argues in the same sense *2+: “the potential for cross-fertilization between
Management Theory an Organization Theory”. Hence, in order to improve this thesis, we will formulate
proposals and defend them. According to Shrivastava, “research program in strategic management focus
on generating research that emphasizes practical usefulness in terms of its meaningfulness, goal
relevance, operational validity, innovativeness, and cost of implementation. They attempt to
incorporate research rigor through conceptual adequacy, methodological rigor and by empirical
evidence.” Hence, we will focus on the rigorous criteria to improve the thesis. However, we are
convinced that the thesis’s author were aware of the rigor/ relevant debate and tried to take it into
account all along its work by making it as rigorous as possible.

Page 6
Improvement proposals for the master thesis project

As it was concluded in the previous sections, although the writer of this Master Thesis demonstrated
that he was aware of the issue addressed in the present paper by putting a lot of effort to close the gap
between rigor and relevance, the project is better located in the “relevance” side; for that reason in the
remainder of this document, specific proposals to improve the rigor of the project are made. It should
be clarified that these proposals were thought to be implemented in the framework of single master
thesis, thus, initiatives like trying to get a general model for ‘Transfer of Training’ or conceptual models
for different types of companies or industries were not considered.

Reduction of subjective effects on measurements.

At the first part of the analyzed thesis, the author constructed a conceptual model of transfer of
training, based on previous conceptual models from literature, which is the starting point for the
research and for the final model for the transfer of training at Vanderlande Industries. The former model
has three blocks of characteristics: 1) “Training Characteristics”, 2) “Learner characteristics” and 3)
“Work environment characteristics”; these blocks influence the training process (block) and its outcome
“transfer performance”, being this the dependent variable for the subsequent statistical analysis. Then,
the quantitative qualification of the each-variable’s block was made based exclusively on surveys applied
to the stakeholders (Trainees, internal and external trainers, supervisors) either in the classrooms or by
mail.

Surveys are one of the most common subjective measurements used in research because they are
perfect for assessing people’s opinions and because it is possible to obtain very much information
against low costs. However, subjective measurements have some disadvantages such as, central
tendency to the mean, ‘Halo effect’1, personality trait influence, tendency to complain and differences in
the interpretations of items. This is an issue that has already been addressed in other fields of study,
namely Job Stress, in which “work environment stress models” require to minimize the effects
mentioned above, that is, these models need to use more objective measurements to reach to reliable
conclusions. In concordance, there is a likelihood that these negative effects could affect the results
obtained in the Master thesis, thus, reducing the reliability of the study; furthermore, considering that

1
It is human tendency in which the perception of one trait is influenced by the perception of another trait.

Page 7
the model used has two blocks of possible predictor variables that are external to the individual, 1) and
3) according to the above mention, the importance of reducing the effect of the subjective
measurements is even higher. It is pertinent now to recall that one of the criteria recommended by
Shrivastava is the Methodological rigor which is related to the use of objective data.

Consequently, in order to reduce these effects a couple of recommendations are made:

First, we suggest the use of another source for gathering data from the block of variables in addition to
the surveys as they were used in the original study; this source would be the use of independent experts
that will assess the pertinent variables of the Training characteristics and of the Work environment
characteristics; In this case, one or more experts could be used to asses both block of characteristics,
and a single expert can evaluate either one or both blocks. Since there will be different sources for the
data, a Methodological triangulation is also proposed, technique that will allow to combine both sources
to obtain the values of the independent variables. Triangulation “refers to the use of more than one
approach to the investigation of a research question in order to enhance confidence in the ensuing
findings”*5+. Although the proposal has a practical implication which is that using an external expert
represents a higher investment, we consider that it is still a completely feasible action that can be taken
if the company is interested in obtaining a better outcome.

In the literature of social research it can be found another alternative to reduce the impact of the
objective measurements, the so-called group assessments, which is then, our second suggestion to
reduce the negative effects of the surveys. Group assessments mean that “the scores of job incumbents
with the same job and working in nearly identical workplaces are aggregated into one general score”*7+,
in other words, this corresponds to the group estimates of the respective characteristic. The reason why
group assessments are more objective measurements is because the influence of idiosyncratic
perceptions of each individual is reduced and the unsystematic error variance is reduced; this is
especially useful for the Training and Work environment characteristics, thus, for these block of
variables we recommend applying the group assessments in addition to it has been done and said until
this point.

Page 8
Adding the effect of personality traits.

The second aspect that is proposed to increase the rigor of the research is the inclusion of a personality
traits model. The motivation for this lies in the fact that a sub-block of the learner characteristics in the
conceptual model defined by author, corresponds to the ‘Personality characteristics (Big five)’, however,
the author decided not to include it because it was argued that “training has to be provided to every
type (e.g. personality characteristics) of trainee group”.

The “Big Five” is the common name that has been given to the Five-Factor Model of personality on
which there is a “broad consensus in the scientific community that five personality traits provide a
comprehensive taxonomy of personality characteristics”*8+; its importance is due to the success it has
had in the clarification of inconsistent results in performance researches during the last two decades; in
fact, the author recognized that other authors who had worked on the transfer of training problem,
have recognized the importance role personality plays in transferring the learned KSA to the work.

In addition, revisions of the researches made in this field have shown that the validities of two of the Big
Five traits (emotional stability and conscientiousness) “generalize in prediction of overall performance”,
that is, two of them are good predictors for general occupations and situations, and the other three
personality traits (extraversion, openness to experience and agreeableness) are valid predictors in
specific niches; furthermore, personality traits are rather stable during time and have proved that they
are good predictors of outcomes in the long term, meaning that, the effects and relations encountered
in a point of time associated with personality can be expected to remain unchangeable in the future [9].

Based on the exposed, it can be stated that the addition of the “Big Five” model to the research can
contribute to generate a greater amount of empirical evidence to the research field of Transfer of
Training and to make the result model more generalizable, thus, more useful for future researches. So,
as a partial conclusion, we recommend including the Five-Factor model to encounter the relations with
the transfer of training and evaluate the effects of different compositions of groups in terms of
personality.

Page 9
Conclusion

The reduction of the effects of subjective measurements in order to make the studied thesis more
rigor, can be achieved by the use of a independent expert to evaluate the training and work
characteristics (additional to the surveys used originally) in conjunction with the utilization on
methodological triangulation; in the same path the use of group assessment would also help to reach
this goal.

Other approach that can be used to improve the research in a way that it could be more general and
contribute more to the study of transfer or training, is the inclusion in independent variables of the
“Big Five Model” to explore the effects personality has on the outcome variable.

Page 10
References

*1+ Shrivastava,P., “Rigor and Practical Usefulness of Research in Strategic Management”, Strategic
Management Journal 8 (1987):77-92

*2+ Van Aken, J.E., “Management Research as a Design Science”, British Journal Of Management 16
(2005): 19-36

*3+ Bertrand, J.W.M. and J.C. Fransoo, “Operations Management Research Methodologies Using
Quantitative Modelling”, International Journal of Operations and Production Management 22 (2002):
241-264.

[4] Willem Elbers, “Improving the transfer of training”, Master thesis (2010)

*5+ Bryman, Alan. “Triangulation and Measurement”. Department of Social Sciences, Loughborough
University.
*6+ Olsen, W. “Triangulation in Social Research: Qualitative and Quantitative Methods can really be
mixed”. Developments in Sociology. (2004).
*7+ De Jonge, J., van Breukelen, G.J.P., Landeweerd, J.A. & Nijhuis, F.J.N. “Comparing Group and
Individual Level Assessments of Job Characteristics in Testing the Job Demand-Control Model: A
Multilevel Approach”. Human Relations, Vol. 52, No 1, (1999).
*8+ Bipp, T. “Linking Personality to Work Motivation and Performance: Individual Differences Effects”.
(2009).
*9+ Barrick, M.R. & Mount, M.K., “Yes, Personality Matters: Moving on to More Important Matters”.
Human Performance, 18(4), 359-372, (2005).

Page 11