2012 International Conference on Education and Management Innovation IPEDR vol.

30 (2012) (2012) IACSIT Press, Singapore

The Application of the Delphi Technique in Technical and Vocational Education in Malaysia
Asnul Dahar Minghat1 and Ruhizan M. Yasin2 Amirmudin Udin1
1 2

Faculty of Education, University Technology of Malaysia Faculty of Education, The National University of Malaysia

Abstract: Technical and vocational education (TVE) research often centers around quantitative research
designs. The Delphi Technique provides a structured communication process designed to produce a detailed examination of a topic and /or problem and discussion from the participating group. The contributions of individuals via this tool produce a group perspective not otherwise attainable. This paper provides the foundations of the Delphi Technique, discusses its strengths and weaknesses, explains its use and stages involved, discusses panel selection, and explains how consensus among participants is reached. Examples of how the Delphi Technique has been used in technical and vocational education are also included.

Keywords: Delphi technique, Technical and vocational education, Vocational subjects

1. Introduction
Technical and vocational education (TVE) plays an important role in the development of a countrys human capital. In todays world, TVE has undergone several transformations in fulfilling its tasks to produce excellent, competitive and skilful human capital [1]. On 10 June 2010, our Prime Minister, Datuk Seri Najib Razak, proposed the 10th Malaysia Plan (10th MP, 2011-2015) at the Malaysia Parliment themed A plan of economic prosperity and social justice. It outlines the countrys five year development plan (2010-2015) and allocated 230 billion (Malaysia Ringgit) in development funds. One of the five main strategic cores in achieving the objectives and targets of 10th MP is the development and preservation of excellent human capital. In his discussion of the strategic cores of the plan, the Prime Minister also referred to the enhancement of student quality and also the mainstreaming of technical and vocational education. Parallel to the plan of mainstreaming TVE, the research aspect must also be given emphasis. The International centre for technical and vocational education and training (UNEVOC) outlined seven sustainable strategies for TVE, one out of which stresses on observation, evaluation and continuous research. In the execution of research in Malaysia, especially in the field of TVE, the quantitative method is most commonly used. This paper will discuss a method that is rarely used by TVE researchers i.e. the Delphi technique. The researcher used this method to construct an indicator and VS (Vocational Subjects) sustainability item for application in daily secondary schools.

2. Foundations of the Delphi Technique

The Delphi technique is essentially the use of expert opinion [2]. It was first used at the RAND Corporation by Olaf Helmer and Norman Delkey in the year 1953 in order to examine military problems [3]. The main objective of the Delphi technique is to obtain highly reliable responses towards problems and surveys given to a panel of experts. This is done by furnishing each individual on the expert panel a survey that contains the same questions and also responses from the expert panel of the preceding round [3]. The

Corresponding author. Tel.: + 60137280015; fax: +6075577351. E-mail address: asnuldahar@yahoo.com 259

Delphi technique is usually used to combine and counterbalance opinions from a group of heterogeneous experts based on information that is offered collectively in the judging process [4]. It was used by [5] in the instrument preparation for a research on principals involving principals who were serving in El Paso, Texas, and also effective principals who were serving in universities. This technique is a group process that uses written responses instead of oral group discussion. In general, the Delphi technique possesses three unique characteristics i.e. transparency, controlled response and group statistical analysis [6 & 7]. According to [8], the five main foundations of the Delphi technique are : (i) each sample is chosen individually and is unaware of the answers given by other members of the expert panel, (ii) each sample is only aware of the answers given by the other samples in round 2 where the researcher analyses the data and sends the analysis results to each sample, (iii) samples are not influenced in any way by the other samples or anyone else in determining their survey answers, (iv) data is analysed statistically, and (v) data given by sample is not biased, thus the panel plays its role in finding the solution to the problem. Nowadays, the Delphi technique is used in numerous fields such as public administration, economics, business, resource and environment management, health, health protection, energy policies and various urban and rural development analysis [9] [10]. The technique can be used as an alternative to the conventional discussion approach in order to avoid problems related to power of personalities, peer influence and also the status effect [11]. In the field of educational research, this technique has been widely used to predict and determine needs in education. For example, in clinical education, the Delphi technique has been used variously to predict, plan and develop curriculums [12], [13], [14], [15], [16] and [17]. All of these researchers have conducted studies in the field of social science education.

3. Strengths and Weaknesses

Every type of research has its strengths and weaknesses. [8] states that Delphi is characteristic of a method that structures group communicational processes to become more effective between individuals and the overall group when solving complex problems. The panel of experts is asked to identify, clarify, neutralize and finally achieve a resolution on an issue through multiple rounds of survey [18]. The advantage of this method is that the expert panel need not physically meet. Therefore, an expert is able to deliver an individual opinion without being influenced by his peers ([19]; [20]; [21]; [22]). The identity of the expert chosen will be kept confidential from the others. Besides that, there are other advantages that attract researchers to use the Delphi technique in their studies. These include: (i) the Delphi technique is capable of reaching a wholesome agreement between the experts as they are not aware of each others identities, (ii) a consensus between the experts on the panel are able to be obtained without bias, influence or pressure towards or from anyone, (iii) the expert panel is able to offer consistent opinions relevant to their fields of specialty, (iv) the Delphi technique is suitable for the use of predicting future needs, and (v) the Delphi technique can be effectively used to obtain more opinions on a complex issue. [8] suggest the following as reasons for failure: (i) imposing monitor views and preconceptions of a problem upon the respondent group by overspecifying the structure of the Delphi and not allowing for the contribution of other perspectives related to the problem, (ii) assuming that the Delphi can be a surrogate for all other human communications in a given situation. Another weakness may include the researcher not being able to conceptualize different ways to examine the problem being investigated. As researchers become more creative in perceiving how different individuals may view the same problem in many different ways, this becomes less of a weakness. In addition, if a researcher hastily tries to complete a Delphi study, the problem may not be given thorough consideration and consensus may not be obtained.

4. Use and Stages of the Delphi Technique

The selection of research participants, especially with the use of the Delphi technique, must be done carefully as faultiness may cause unwanted results ([2]; [23]; [24]; [25]).

5. Panel Selection
260

5.1. Participants for Delphi technique

Past researchers have made several opinions regarding the sample size or the number of experts needed in a Delphi research. [19] believes that a panel formed out of 7 to 100 people is most suitable in producing good research results. However, [8] disagree as they are of the opinion that large numbers of experts on the panel make the research more difficult and produce negative implications during field research. They suggest a 5 to 10 person panel as the best number in order to effectively achieve the research goals. On the other hand, [26] and [27] emphasise that a panel consisting of 3 to 5 people is too small and is unable to fully respond to an issue. They suggest that an expert panel should consist of uniform groups, such as being from the same discipline, and that it should be made up of only 10 to 20 people. Meanwhile, [6] suggests that 10 experts are enough to produce good research results when using the Delphi technique. In accordance to this opinion, the researcher has determined that only 12 experts were to be used in the current study. The current Delphi study was executed in four rounds, but the number of rounds can be increased or decreased depending on the information needed[26]

5.2. Selection criteria for the Delphi technique participants

The Delphi technique was used because it was designed to optimize the input obtained from the individuals on the expert panel. Thus, the most important part of the technique is the selection of experts. [28] defines an expert as someone who is well-versed in a particular craft or field. On the other hand, [6] defines an expert as someone who is knowledgeable in a particular field. [29] suggests the following criteria as a guideline in choosing individuals to form the expert panel. These include a good repertoire among his professional peers, number of papers produced and presented to the national or international public, number of publications on the market and also research completed. Based on this opinion, the purposive sampling of the Delphi technique takes these criteria, as well as other criteria discussed earlier in the paper, into consideration. [11] emphasises that selection of individuals for the expert panel depends on the research objective of the Delphi study. In this context, quality is more important than quantity. Meanwhile, [30] states that the expert panel is selected based on their readiness and capability in delivering opinions related to the topic in discussion, as well as possesses a wide experience relevant to the issue. In accordance to the two definitions of expert given above, the researcher has determined that the selection of experts for the panel in this study is to be based on three criteria: (i) possesses a Ph.D in the field of TVE 10-15 years of service for Teacher Training Institute lecturers or 10 years of teaching experience for teachers, (ii) possesses wide experience in matters of curriculum and execution of technical and vocational subjects at university, teacher training institute or at school, and (iii) directly related to the application of VS or Vocational Subjects, especially with the Ministry of Education Malaysia.

6. Rounds and Determining Consensus 6.1. The first round of the Delphi technique (Formation of a sustainability indicator for VS)
During the first round of the Delphi study, the researcher met and interviewed each participant of the expert panel in order to collect qualitative data on the application of VS at daily secondary schools. The researcher also discussed the concept of vocational education, sustainability and sustainable development in the context of Malaysia. This provides better understanding of the phenomenon that the research is concerned with ([31]; [32]; [33]). In the first round of the Delphi research, a structured interview was used and the data was analysed according to a qualitative method that involved reduction of data, verification of data and presentation of data. A total of 20 sustainability indicators for VS in daily secondary schools were successfully formed during the first round. These indicators are knowledge of VS teachers, technical competence of VS teachers, counselling, staff development program, commitment of administration, internships and industrial collaboration, networks and collaborations, articulation, generic skills, competency-based training, teacher teaching methods, entrepreneurship, industrial input, accreditation, teacher creativity, teacher innovation, student interest, student creativity, student innovation and ICT skills. The results were then used to develop items in the survey instrument for each VS sustainability indicator to be used in rounds two to four of the Delphi technique. Items for the sustainability indicator were developed earlier by the researcher based on literature review. However, the researcher has taken into
261

consideration the opinion of [34] who suggests three approaches in the construction of a test or item i.e. the rational approach, the empirical approach and the rational-empirical approach. The rational approach is the classic approach where items are constructed based on thought, belief and rational. The empirical approach refers to the construction of items based on criteria as the key to validity. In this research, the rationalempirical approach was used. The rational approach involved the creation of items based on construct or the VS sustainability indicator, while the empirical approach involves selection of items based on TVE expert panel consensus. Through the combination of these approaches, the validity of the instrument is increased by constructing items that fit the construct. 200 items were developed by the researcher at this stage which will be given to the participants of the Delphi study during the second round.

6.2. The second round of the Delphi research (Formation of a sustainability indicator for VS)
During the second round, the researcher once again met each of the experts on the panel in order to explain the format and method of answering the survey question and to clear any confusion. Then, they had two weeks to interact with the survey instrument by stating their agreement, providing comments and also troubleshooting the indicator and the survey. After all the data from the second round was obtained, it was analysed for use in the third round. Expert consensus towards the VS sustainability indicator items were evaluated based on the least Inter-quartile Range (IQR). Consensus levels (refer Table 1) in this study were taken from [35], [36] and [15] where: High consensus = Inter-quartile Range 0 to 1.00 Moderate consensus = Inter-quartile Range 1.01 to 1.99 No consensus = Inter-quartile Range 2.00 and above.
Table 1 Details of Data Analysis for Rounds Two to Four of the Delphi Research for VS Teacher Knowledge Indicator N o Indicator / Item M VS Teacher Knowledge VS teachers must 4.8 5 0 High 4.9 possess extensive 2 2 knowledge of the field taught IQR = Inter-Quartile Range, Med = Median, M = Min 1 5 0 High 4.9 1 5 0 High Round 2 Me IQ Conse d R nsus Level Round 3 Me IQ Conse d R nsus Level Round 4 Me IQ Conse d R nsus Level

6.3. The third round of the Delphi technique (Formation of a sustainability indicator for VS)
During the third round, the participants of the expert panel were once again asked to respond to items analysed in the previous round without the addition of any new items. A justification is required if the level of agreement goes beyond the majority agreement scale for the previous round. In short, the third round aims to lessen the differences in opinion among the participants of the expert panel.

6.4. The fourth round of the Delphi Technique (Formation of a sustainability indicator for
VS)
The aim of the fourth round is also to lessen the differences in opinion among participants of the expert panel towards the survey analysed in round three. At the end of this round, all the survey items achieved high consensus among the experts. Therefore, the survey rounds for the Delphi technique were halted and the survey items were used for the purpose of survey research. The researchers decision to halt the number of rounds when a high consensus level was achieved is in accordance with the opinions of ([25]; [26]; [27]). They are all of the belief that a good number of rounds conducted in a Delphi research should be between 2 to 10 rounds in order to obtain findings which comply with the research objectives. However, the number of rounds will be halted if the required information has been obtained or if there is an early agreement as to the number of rounds that the researcher needs to conduct.
262

7. Summary
The Delphi technique provides different opportunities to researchers compared to survey research. Essential components of the Delphi technique include the communication process, a group of experts, and essential feedback. The Delphi technique has been used in a variety of ways in government, business and education systems. This paper presents the key principles of the Delphi technique and provides a number of examples of where it has been used in technical and vocational education research. The technique benefits from being a democratic and structured approach that harnesses the collective wisdom of participants. Its most significant strength lies in the ability to garner opinion and seek consensus among a diverse group of participants.

8. References
[1] Mohamad Rizal Azman Rifin. Vocational Education New Version. Berita Harian. 2011. 58, 17-23 February 2011. [2] Helmer, O. Analysis of the future: The Delphi Method. Dlm Bright J.R. (pnyt.). Technological forecasting for industry and government: Method and applications. New Jersey: Prentice-Hall Inc. 1968. [3] Helmer, O. A Guide to Future Research Olaf Helmer. New Jersey: Sage Publication, Inc. 1983. [4] Jones, J.E. A look in future: The 2020 organization. http://www.improve.org.2020org.html [22 Oktober 2009]. 2009 [5] Dunlap, K., Daresh, J., Ganter, M. & Hvizdak, M. View from the field: the principlals Perspective on effective school leadership characteristics. Paper presented at the Annual Conference of the American Educational Research Association, San Diego. 1998. [6] Dalkey, N.C., Rourke, D.L., Lewis, R. & Snyder, D. Studies in The Quality of Life: Delphi and Decision-making. Lexington: Lexington Books. 1972 [7] Armstrong, M.A. The Delphi Technique http://www.pei.int.com/Research/MARKETS/DELPHI.htm [20 Oktober 2009]. 1989. [8] Linstone, H. & Turoff, M. The Delphi Method: Technique and Applications. Massachusetts: Addison-Wesley. 1975 [9] Miller, S.K. The history of effective school research: A critical overview. Paper presented at the American Educational Research Assioation, Montreal: Canada. http://www.forumfyi.org [20 Jan 2010]. 1993. [10] Ali, A.K.. Using the Delphi method to search for empirical measures of local. The Qualitative Report. 10 (4): 718-744. http://www.nova.edu/ssss/QR/QR10-4 /ali.pdf [23 July 2009]. 2005. [11] Thangaratinam, S. & Redman, C.W.E. The Delphi technique. The Obstetrician & Gynaecologist Journa.l 2005, (7): 120-125. [12] Norizan Abdul Razak. Computer competency of in-service esl teachers in Malaysian secondary schools.PhD Thesis. The National University of Malaysia. 2003. [13] Rusilawati Othman. Budaya sains dan teknologi di kalangan murid sekolah dan hubungannya dengan perubahan tempoh pembelajaran sains. Tesis Dr. Falsafah. Universiti Kebangsaan Malaysia. 2007. [14] AbdulSalam Sulaiman Dawood Al-Hadabi. Analysis of the environmental concepts in science textbooks of the basic education stage in the republic of Yemen. PhD Thesis. The National University of Malaysia. 2007. [15] Muhammad Faizal A.Ghani. Pembentukan model keberkesanan dan penambahbaikan sekolah di Malaysia. PhD Thesis. Universiti Malaya. 2008. [16] Zaharah Hussin. Pembinaan kandungan kurikulum pendidikan akhlak untuk latihan perguruan pendidikan Islam. PhD Thesis. The National University of Malaysia. 2008. [17] Norlia T. Goolamally. Pembinaan dan pemeriksaan ciri-ciri psikometrik instrumen pemilihan pemimpin sekolah. PhD Thesis. The National University of Malaysia. 2010. [18] Saedah Siraj. Future Curriculum. Kuala Lumpur: Universiti Malaya Press. 2005.

263

[19] Steward, J., OHalloran, C., Harrigan, P. & Spencer, J.A. Identifying appropriate tasks For the preregistration year: Modified Delphi Technique. British Medical Journal. 1999, 319(7204):224-229 http://bmj.bmjjournals.com/cgi/content/abstract/319/7204/ [18 November 2009]. [20] McGill, J. Developing leisure identity: A pilot project. Brampton, ON: Brampton Caledom Community Living. 1998 [21] Czinkota, M.R., & Ronkainen, I.A. International business and trade in the next decade: Report from a Delphi study. Journal of International Business Studies, 1997, 28(4): 827-844 [22] Robbins, S.P. Management. Prentice Hall, New Jersey. 1996. [23] OHalloran C., Stewart, J., Harrigan P., Spencer J.A. & Barton J.R. Identifying appropriate tasks for the preregistration year. British Medical Journal, 1999, 319, 224229. [24] Landford, H.W. Technological Forecasting Methodologies: A Synthesis. USA: American Management Association. 1972. [25] Martino, J.P. Technologies Forecasting for Decision Making. New York: American Elsivier Publishing Company Inc. 1972. [26] Ludwig, B. Predicting the future: have you considered using the Delphi methodology? Journal of Extension. 1997, 35(5): 233-239 http://www.joe.org/joe/1997october/tt2.html [10 November 2009] [27] Delbecq, A., Van de Ven, A. & Gustafon, D. Group Technique for Program Planning: A guide to Nominal Group and Delphi Process. Glenview: Scott-Foresman. 1975. [28] Kamus Dewan. Kuala Lumpur: Dewan Bahasa dan Pustaka. 2007. [29] Tajul Ariffin Noordin. Pendidikan Satu Pemikiran Semula. Kuala Lumpur: Dewan Bahasa dan Pustaka. 1997 [30] French, P., Psychol, C., Yin-yo ho & Lee, L. S. A Delphi survey of evidence-based nursing priorities in Hong Kong. Journal of Nursing Management . 2002, 10: 265-273 [31] Marohaini Yusoff. Penyelidikan Kualitatif: Pengalaman Kerja Lapangan Kajian. Kuala Lumpur: Penerbit Universiti Malaya. 2001. [32] Bogdan, R.C. & Bliken, S.K.. Qualitative Research in Education: An Introduction to The Theory and Methods. Ed. Ke-4. Boston: Allyn and Bacon. 2002. [33] Patton, M.Q. Qualitative Research and Evaluation Methods. London: Sage Publications. 2002. [34] Murphy, K.R. & Davidshofer, C.O. Psychological Testing. Principles and Applications. Ed. Ke-4. New Jersey: Prentice-Hall. 2005. [35] Boonon, Kasem . The future of teacher education in Thailand: A Delphi application, Tesis Ph.D. University of Alabama, Tuscaloosa, Alabama. Photocopy. Ann Abor, Mich.: University Microfilms International, 1980. 20 cm. 1979. [36] Ibrahem Narongraksakhet. Developing local-based curriculum for Islamic schools in Southern Thailand. PhD Thesis. Universiti Malaya. 2003.

264