Anda di halaman 1dari 8

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful

Projects 29-31 May 2010, Heraklion, Crete, Greece.

AA012 An MCDA Approach for Project Selection in Public Sector


Evangelos Bellos (Researcher, National Technical University of Athens, Greece) Dimitra Voulgaridou (Research & Teaching Associate, University of the Aegean,Chios,Greece) Konstantinos Kirytopoulos (Assistant Professor, University of the Aegean,Chios, Greece) Dimitrios Panopoulos (Researcher, National Technical University of Athens, Greece)

Effective and efficient selection of projects is a crucial issue for every organization and should be aligned with its vision and strategies. The decision making process is highly complex, especially for public organisations, as it encompasses various, often conflicting criteria. Moreover they have to consider a much larger picture than private organisations and many scholars argue that public investment decisions fall into the category of constrained optimisation problems. The present study develops and proposes an approach for the prioritization of public projects, based on the Analytic Network Process (ANP), which is further validated through a real case example concerning the project selection process of a Greek municipality. The results indicate that social and political criteria dominate the decision making process, while the ANP documents the results in such way that they can be communicated to various stakeholders, ensuring public trust and establishing a system of transparency in public administration.

Abstract

Keywords

Project Selection, Multicriteria Decision Analysis, ANP, Public Sector

1. Introduction
Project selection is a long lasting problem in business. Since finite resources exist at each period of time, the senior management of each organisation should decide which projects to undertake out of a set of potential alternatives. Effective project selection may lead to the prosperity or decline of an organisation, thus the problem has attained the focus of both practitioners and academics. In order to come up with a decision, the senior management of an organisation should analyze and evaluate the economical and technical feasibility of each project, evaluate alternatives, secure financing etc. (Fitsilis et al., 2008). The problem is even more interesting and difficult to be solved for a public organization. In this cases, apart from the typical set of criteria such as technical (e.g. maturity of implementation), economic (e.g. investment cost) or those related to sustainability (e.g. environmental impact) the decision makers have also to take into account political criteria (e.g. mission and strategic policies) or social ones (e.g. contribution to employment). Thus the problem is further aggravated due to
- 117 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

the various tangible and intangible characteristics that need to be taken into account. The complexity of the decision environment and the strategic importance for the organisation leads to the need for a multicriteria decision analysis approach (Kirytopoulos and Voulgaridou, 2008). Although the number of MCDM methods is already big and still increases, there is no specific, generally accepted, method for every selection problem as each problem is unique (Kirytopoulos et al., 2008). In this paper the Analytic Network Process (ANP) method is selected for the analysis of decision criteria and alternatives. The scope of this paper is twofold. First the paper emphasises on revealing the criteria that have to be taken into account in public project selection. Second the paper offers a comprehensive approach on how ANP can be used to aid decision making in the aforementioned problem. The rest of this paper is organised as follows. The second section offers a brief literature review on public project selection criteria. The third section is divided in two sub-sections where the analytic network process and its implementation on the public projects selection are elaborated. The paper concludes with the last section where basic findings are discussed, minor limitations of the approach proposed are underlined and further research ideas are offered.

2.

Literature Review

Effective and efficient selection of projects is a crucial issue for the viability of every project-based, private or public organization. Management decisions concerning the prioritization of projects proposals and the determination of the appropriate investment plan are guided by both intrinsic and extrinsic criteria (Mohanty, 1992). The intrinsic criteria concern the resources requirements and availability, the experience of the organization in managing similar projects, the managements perspectives and the time horizon of the projects. The extrinsic criteria concern the risk/return ratio, the target-market needs, the governmental policies and regulations, the socio-economic climate and the legal and technological implications. The selection criteria can be further specified depending on the organization or the project type. As far as public organizations are concerned, the project selection process becomes even more complex. The great number of stakeholders, whose interests may be conflicting, introduces major difficulties into the decision-making phase. Zanakis (1995) argued that political factors related to a project play much more leading role than technical or other more rational considerations. According to Chapman (2006), public project investments involve great capital engagement, uncertainty and long lasting impacts. Furthermore, it is difficult to assess the value of a project, as it is affected by dynamic changes of the surrounding socio-economic environment (Medaglia et al., 2008). When comes to public project selection criteria, apart from targeting at the maximization of net financial benefits or returns to stakeholders, decision makers should also prioritize and select projects through social equity, economic and political criteria (Medaglia et al., 2008). Puthamont and Charoengam (2007) indicated 32 factors that influence public project selection during the conceptual, design and the final approval stage of the selection process. The identified factors are related to the organization strategy, the projects rationale, feasibility, investment analysis, maturity, benefits/impacts and socio-economic and political environment. In Spain, R&D public projects (Santamaria et al., 2010) are appraised through the compliance of projects goals with governmental strategies, the
- 118 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

innovativeness and technological development of a project, quality, applicability of project results etc. In addition, the project selection may be guided by the projects region and the projects size.

3. 3.1

Proposed Approach The Analytic Network Process

The Analytic Network Process is a generalisation of the Analytic Hierarchy Process (AHP) developed by Thomas Saaty (1996). ANP incorporates feedback and interdependent relationships among decision criteria and alternatives and provides a general framework to deal with decisions without making assumptions about the independence of higher level elements from lower level elements or the independence of the elements within a level as in AHP. In fact, the ANP uses a network of elements without need to specify levels (Saaty, 2005). Technically, the model consists of clusters and elements. As in the AHP, the dominance or relative importance of influence is the central concept and judgments are provided from the fundamental scale of the AHP (Saaty, 2005) by answering the question: Given a criterion X, which of the two elements Y,Z is more dominant with respect to that criterion? In order for the influences among the elements to be meaningful at the final stage of the method (synthesis), a specific element, each time, is used to perform the pairwise comparisons. This element is called control criterion. In short, the ANP approach handles interdependence among elements by obtaining the composite weights through the development of a supermatrix.

3.2

The Proposed ANP model for public project evaluation

The proposed model applies the ANP to the problem of evaluating, prioritizing and selecting public projects. The model has been validated through a real case implementation concerning the prioritization of three different public projects for a Greek municipality, within the frame of the development of its annual programs/actions plan. The referred municipality is located in a depressed area near Athens and 10 years ago had suffered from a big earthquake, which caused serious damages in most public infrastructures. The projects, which were examined in order to validate the proposed model, have been selected with the contribution of the technical director of the municipality who was asked to propose the three most important projects that should be implemented during the next year, according to his opinion. In this matter the following projects were examined: 1. Repairs on earthquake-stricken roads and buildings (short name: EARTHQ), referring to specific infrastructures damaged by the earthquake, which have not been repaired yet. Estimated budget 7 million euros, external funding 0.3 million euros, high maturity of implementation. 2. Bioclimatic reconstruction of the central square of the municipality (short name: BIOCL), including a total reconstruction of the square on a basis of bioclimatic design using cool barrier technologies and relevant planting. Estimated budget 0.5 million euros, external funding 0.25 million euros, low maturity of implementation. 3. Development of a web portal for the municipality (short name: PORT), offering e-government services to the citizens. Estimated budget 0.3 million euros, external funding 0.225 million euros, low maturity of implementation, high urgency (open call for proposals).
- 119 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

The purpose of applying the proposed model was to prioritise these projects on the basis of a set of specific criteria, with the use of the Analytic Network Process. The ANP processes were made by using Super Decisions Software created by Saaty (2004) to alleviate the mathematical burden. The framework of the method is illustrated through seven steps: Step 1- Model construction: To develop the public project evaluation ANP model, fist, the authors performed an extended literature review concerning the evaluation criteria in the public sector and then interviewed the public project managers about the critical factors that need to be taken into consideration in order to choose among the different candidate projects. These factors were initially compiled as a long catalogue with overlapping data and redundancies. Afterwards, they were reviewed and categorised into clusters. These final clusters and criteria were communicated to the public project managers, hereafter referred as decision makers, who, after some minor corrections, verified the final model. From a technical point of view, the structure of the ANP model is described by its clusters and elements and the connection between them. These connections indicate the flow of influence among the elements. The resulting model is illustrated in Figure 1 and consists of six clusters: 1. The 1ECONOMIC cluster controls the economic viability of the decision and consists of two elements: the Investment Fundability (representing the chance of allocating sources for funding the project, including the possibility of receiving external funding, e.g. through Operational Programmes of the Greek National Strategic Reference Framework and the Investment Financial Analysis (evaluating financial indexes like ROI, IRR, etc). 2. The 2POLICY/POLITICAL cluster consists of the main elements that determine policy-related criteria such as the compliance of the project to the Organisations Mission and the Strategic Policies, the Urgency of the project (taking into account not only the real urgency for covering a specific need, but also possible deadlines concerning opportunities for ensuring external funding for the project) and the projected Public Perception of each project. 3. The 3SOCIAL cluster is related with the impact of each project to the citizens and the society, consisting of two elements, the first one concerning the possible benefits for the society and the second one concerning the contribution of the project to local employment. 4. The 4SUSTAINABLE DEVELOPMENT cluster encompasses all those elements concerning the environment and the regional development. These elements include: the Environmental Impact of the project (in the local region, as well as in national level) and the general projected contribution of the project to Local Development. 5. The 5TECHNICAL cluster controls the technical characteristics of each project that influence the final decision and consists of the projects Maturity of Implementation (concerning the status/availability of prerequisite factors for implementing the project like relevant studies, resolved legal issues, reassurance of equity capital, land expropriation needs etc), the Availability and Technical Ability of the Human Resources of the Organisation for monitoring the project and possible Regulatory Needs (in the terms of requirements for completing any regulatory/legal actions as prerequisite for implementing the project).
- 120 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

6. Finally, the 6ALTERNATIVES cluster comprises all candidate projects as described previously on this subsection.

Fig 1: The Public Project Evaluation ANP Model

The connections among the clusters and elements are depicted in Figure 1 with arrows. The definition of these connections can only be defined by the decision makers, the public project managers in the present case, who are expert in terms of knowing how the elements interact in the real environment. For example, the arrow from the cluster 2POLICY/POLITICAL to the cluster 4SUSTAINABLE DEVELOPMENT, captures the notion of outer dependence of ANP and is interpreted as follows: some of the elements in cluster 4SUSTAINABLE DEVELOPMENT influence some of the elements in cluster 2POLICY/POLITICAL (Specifically, the Public opinion/perception is influenced by Environmental Impact and Contribution to Regional/Local Development). Finally, the concept of inner dependence of ANP (some elements of a cluster influence other elements of the same cluster is depicted with a feedback loop. Refer, for example to the loop above the 5TECHNICAL cluster, which represents the fact that Maturity of implementation is influenced by both Human Resources (ability & availability) and Regulatory requirements. The volume and the diversity of the criteria identified during this step, reveal the complexity of the decision problem in the public project evaluation environment. Step 2- Clusters pairwise comparisons and Clusters Priority Matrix: After the model was constructed, an additional meeting between the authors and the public project managers took place, during which the managers were asked to respond to a series of pair wise comparisons with respect to a control criterion. The elements in a cluster are compared by applying Saatys 19 scale (Saaty, 2005) according to their influence on an element in another cluster to which they are connected or on elements in their own cluster, as described above. Super Decisions software is able to calculate the inconsistency ratio for each comparison matrix, so the most consistent value for the entries can be determined. The inconsistency measure is useful for identifying possible errors in judgments as well
- 121 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

as actual inconsistencies in the judgments themselves. For example, if A is more important than B and B is more important than C, C cannot be more important than A. In general, the inconsistency ratio should be less than 0.2. Afterwards, the Clusters Priority Matrix is constructed. Its columns consist of the weighted priorities derived during the pair wise comparisons. Step 3- Elements pair wise comparisons: in order to compare the elements of the clusters, the same procedure described in the former step was followed. Step 4- Formation of the SuperMatrix: The priorities of the elements are arranged both vertically and horizontally according to clusters. This matrix is known as SuperMatrix. Each vector taken from a paired comparison matrix is part of the column of the SuperMatrix representing the impact, with respect to the control criterion of the elements of that cluster on a single element of the same or another cluster, listed at the top. Step 5- Formation of the Weighted SuperMatrix: The weighted priorities at the Clusters Priority Matrix are used to weight all the elements in the block of column priorities of the SuperMatrix corresponding to the impact of the elements of that cluster on another cluster. This process is repeated for all the clusters resulting in the Weighted SuperMatrix. Step 6- Formation of Limiting Matrix: The Limiting Matrix and the systems solution derives from multiplying the Weighted SuperMatrix (step 5) by itself, which accounts for variable interaction, until the systems row values converge to the same value for each column of the matrix. This power method process yields the limiting matrix, which provides the relative importance weights for every element in the model (Niemira and Saaty, 2004). The final results, weights of criteria and prioritization (normalized values) of public projects under evaluation as derived from the limit matrix are graphically depicted in Figures 2 and 3, respectively. Step 7- Sensitivity analysis on the final outcome: Sensitivity analysis is concerned with what-if kind of questions to determine if the final answer is stable to changes in the inputs, either judgements or priorities. Of special interest is to see if these changes alter the order of the alternatives. The sensitivity analysis performed with respect to Impact on society, which is the dominant criterion (refer to figure 2), showed that irrespective of its value, the rank of the projects is preserved. Performing similar sensitivity tests for other criteria led to the conclusion that the outcome is very stable and does not change the overall rank.

- 122 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

3Regulatoryrequirements 2Humanresources(availability &ability) 1Maturityofimplementation 2Contributiontoregional/local 1EnvironmentalImpact 2Contributiontoemployment 1Impactonsociety 3Public opinion/perception 2Urgency 1Missionandstrategic policies 2Investmentfinancial analysis 1InvestmentFundability

Fig 2: Final weights of the decision criteria


3Webportaldevelopment 2Bioclimatic reconstructionofcentral square 1Repairsonearthquake strickenroads& buildings 32.18% 26.29% 41.52%

Fig 3: Prioritization of projects

4.

Conclusions

The evaluation and selection of public projects is a difficult endeavour as it takes into account many conflicting criteria, both tangible and intangible. The present study proposed an MCDA approach based on the ANP that is capable of capturing this complex decision environment. The ANP method, as part of this approach, enables the decision maker to visualize the impact of various criteria in the final outcome and documents the evaluation results in such way that they can be communicated to various stakeholders, ensuring public trust and establishing a system of transparency in public administration. Although the case study presented in this paper concerned a limited number of projects, ANP is capable of handling many more alternative but with the cost of complexity. For the case study presented here, the results indicate that the project with the highest priority is the Repairs on earthquake-stricken roads and buildings followed by the Web-portal Development and the Bio-climatic reconstruction of central square (refer to figure 3). Moreover, the most dominant criteria that influenced the decision were found to be the social (e.g. Impact on Society) and the political criteria (e.g. Public opinion/perception, Urgeny) followed by economic criteria (e.g. Investment Fundability). The results indicate that, under the current conditions and according to the public projects managers value system, social and political criteria are favoured over economic criteria, which explains the fact that a project with major socio-political impact dominates over more economic compensative projects. The ANP approach proposed by this paper bears also a few limitations. The most important is considered to be the fact that the outcome of the model depends on the inputs provided by the public managers. Moreover, the formation of the pairwise comparison matrices is a time-consuming and complex task. Finally, inconsistency may also occur, leading to doubtful or wrong results.
- 123 -

PM-05 - Advancing Project Management for the 21st Century Concepts, Tools & Techniques for Managing Successful Projects 29-31 May 2010, Heraklion, Crete, Greece.

This study raises several important issues that could spark further research. The most important is the evaluation of the model and its implementation at additional case studies in related public organisations. Such an endeavour might be beneficial and result in developing a complete system that could support the decision makers and reveal the significance of certain alternatives, criteria and elements.

5.

References

Chapman, C., Ward, S. and Klein, J. (2006), An optimised multiple test framework for project selection in the public sector, with a nuclear waste disposal casebased example, International Journal of Project Management, 24, 373-384. Fitsilis, P., Kirytopoulos, K. Leopoulos, V., Pantouvakis, J.P. & Saridakis, I. (2008), Requirements for assessing the managerial capability of organizations implementing projects of public interest the Greek Standard ELOT-1429, Proceedings of the PM-04 > 4th Scientific Conference on Project Management & 1st IPMA / MedNet Conference, CCI NTUA, Chios 29-31 May, 530-536. Kirytopoulos, K. and Voulgaridou, D. (2008), Sub-Contractor Selection Decisions via a Multicriteria Model, Proceedings of the PM-04 > 4th Scientific Conference on Project Management & 1st IPMA / MedNet Conference, CCI NTUA, Chios 29-31 May, 204-209. Kirytopoulos, K., Leopoulos, V. and Voulgaridou, D. (2008), Supplier selection in pharmaceutical industry: An analytic network process approach, Benchmarking: An International Journal (BIJ), 15(4), 494-516. Medaglia, A., Huethb, D., Mendietab, J-C., Sefaira, J. (2008), A multiobjective model for the selection and timing for public enterprise projects, Socio Economic Planning Sciences, 42, 31-45. Mohanty, R.P. (1992), Project selection by a multiple- criteria decision-making method: an example from a developing country, International Journal of Project Management, 10(1), 31-38. Puthamont, S. and Charoenngam, C. (2007), Strategic project selection in the public sector: Construction projects of the Ministry of Defence in Thailand, International Journal of Project Management, 25, 178-188. Saaty, T. (1996), Decision Making with Dependence and Feedback: The Analytic Network Process, RWS Publications, Pittsburgh, PA. Saaty, T. (2004), Super Decisions Software, RWS Publications, Pittsburg, PA Saaty, T. (2005), Theory and Applications of the Analytic Network Process. Decision Making with Benefits, Opportunities, Costs and Risks, RWS Publications, Pittsburg, PA. Santamaria L., Barge-Gil, A. and Modrego, A. (2010), Public selection and financing of R&D cooperative projects: Credits versus subsidy funding, Research Policy, (article in press). Zanakis, S.H. and Mandakovic, T. and Gupta, S.K. and Sahay, S.and Hong, S. (1995), A review of program evaluation and fund allocation methods within the service and government sectors, Socio-Economic Planning Sciences, 29(1), 5979.

- 124 -