Accomplishment Completion, fulfilment, achievement of specific result by the end of the biennium in comparison to the planned Expected Accomplishment.

It could be less, more or the same as expected accomplishment.

Accomplishment account A summary of a specific subprogramme accomplishment that is based on data collected for the indicators of achievement and other relevant information that serves as the source of reporting on whether the relevant Expected Accomplishment was achieved. Accountability Obligation for a manager of resources to demonstrate that work has been conducted in compliance with the established plans, budgets, rules and standards and to report fairly and accurately on performance results. It includes responsibility for the justification of expenditures, decisions or results of the discharge of authority and official duties, including duties delegated to a subordinate unit or individual. The effective discharge of accountability is predicated on clearly defined responsibilities, performance expectations, limits of authority, and clarity on how the exercise of responsibility and authority will be monitored and assessed. One of the main functions of monitoring and evaluation is to contribute to strengthening accountability by providing objective information on the veracity of a managers reporting. Activity Action taken or work performed to transform inputs into outputs. Additional output Output that was implemented in addition to those initially programmed in the programme budget. They are of two types: Added by legislation Outputs added by a legislative decision taken after the biennial budget was approved by the General Assembly either by the General Assembly itself or by a specialized body; Added at the initiative of the Secretariat Refers to outputs that have been introduced additionally by programme managers by using the resources released from postponements and/or terminations or as the response to request received and funded through extrabudgetary means to enhance the attainment of the objectives and expected accomplishments of the subprogramme. Analytical method A means to process, understand and interpret data. Appraisal An overall assessment of the relevance, feasibility and potential sustainability of a project or other operational exercise. It is an assessment of the overall soundness of the project and a justification for its implementation. Criteria commonly include relevance and sustainability. An appraisal may also relate to the examination of opinions as part of the process for selecting which project to fund. The purpose of appraisal is to enable decision-makers to decide whether the activity is in accordance with mandates and represents an appropriate use of resources. Assumption Hypothesis about risks, influences, external factors or conditions that could affect the progress or

success of a project or a programme. Assumptions highlight external factors, which are important for the success of project or programme, but are largely or completely beyond the control of management. Attribution A causal link between observed (or expected to be observed) changes and a specific intervention. Attribution refers to that which is to be credited for the observed changes or results achieved. With regard to attribution for the achievement of accomplishments/results, evaluations aim to ascertain a credible link between outputs and achieved accomplishments. In assessing this link, attribution takes account of the effects of other interventions that are independent of the effort being evaluated. Audit An exercise to determine if there is an adequate and effective system of internal controls for providing reasonable assurance with respect to: Integrity of financial and operational information; compliance with regulations, rules, policies and procedures in all operations; and safeguarding of assets; The economic and efficient use of resources in operations and identifying opportunities for improvement in a dynamic and changing environment; and Effectiveness of programme management for achieving stated objectives consistent with policies, plans and budgets. Baseline Data that describe the situation to be addressed by a programme, subprogramme or project and that serve as the starting point for measuring performance. A baseline study would be the analysis describing the situation prior to the commencement of the programme or project or the situation following initial commencement of the programme or project to serve as a basis of comparison and progress for future analyses. It is used to determine the accomplishments/results and serves as an important reference for evaluation. Benchmark Reference point or standard against which performance or achievement can be assessed. A benchmark often refers to an intermediate target to measure progress within a given period as well as to the performance of other comparable organizational entities. Beneficiary The individual, group, or organization, whether targeted or not, that benefit, directly or indirectly, from the implementation of a programme, project or output. Best practice Planning, organization, managerial and/or operational practices that have proven successful in particular circumstances and which can have both specific and/ universal applicability. Best practices are used to demonstrate what works most effectively and to accumulate and apply knowledge about how and why they work in different situations and contexts. Bias Anything that produces systematic error in an evaluation finding. Bias may result in over- or underestimating the object of evaluation or assessment.

Budget fascicle Document containing proposed programmatic, financial and resource information of a budget section for the forthcoming biennium and submitted to the General Assembly for approval.

Capacity building The process by which individuals, groups, organizations and countries develop, organize and enhance their systems, resources and knowledge, all reflected in their abilities, individually and collectively, to perform functions, solve problems and set and achieve objectives. Technical assistance in the form of knowledge transfer activities such as training, fellowships, study tours, is used to increase capacity. Carried forward Outputs that have been postponed in the previous biennium and are to be completed in the current one. Case study The examination of the characteristics of a single case (such as an individual, an event, a programme or some other discrete entity). A sample of multiple cases can also be examined to look for commonalities and to identify patterns. Case studies are often used to gather qualitative information in support of findings obtained through quantitative methods. Causal relationship A logical connection or cause-effect linkage ascribed to the relationship between accomplishments/results and efforts to achieve them or between final results and their impact on the target beneficiaries. Generally the term refers to reliably plausible linkages. Client satisfaction The satisfaction of a programmes clients, defined as the organizations or individuals who are affected by that programme, often measured in terms of meeting their needs or expectations. Cluster evaluation An evaluation of a set of related outputs, projects and/or programmes or subprogrammes. Conclusions Conclusions present reasoned judgments based on a synthesis of empirical findings or factual statements corresponding to specific circumstances. Conclusions point out the factors of success and failure of the evaluated projects and programmes, with special attention paid to the intended and unintended results and impacts, and more generally to any other strength or weakness. Conclusions draw on data collection and analyses undertaken, through a transparent chain of arguments. Condition The situation that exists, and that the evaluator seeks to describe and explain. Condition can describe how well a programme is operating or the problem it is facing. Conference services, administration, oversight An output category which comprises meeting services, interpretation, verbatim reporting; translation, editorial services; documentation and publication services (including text processing); library services;

overall administration and management; human resources management; programme planning, budget, accounts; central support services; audits; inspections, evaluations, management reviews. Content analysis A systematic approach to analyzing themes in audio, visual, electronic or print communication. Selected material is reviewed and assessed on the basis of predetermined criteria (such as the reflection of key messages, accuracy, prominence, and reference to sponsoring organization). Control group A selected subgroup of beneficiaries who do not receive the same treatment, input or training, but share characteristics similar to the target group. Thus, differences between the control group and the target group can be measured and evaluated. Contribution The link between the activities carried out by various organizational units to arrive at final products or services delivered to end-users to accomplish a desired result. Cost-benefit analysis A specialized analysis which converts all costs and benefits to common monetary terms and then assesses the ratio of results to inputs against other alternatives or against some established criteria of cost-benefit performance. It often involves the comparison of investment and operating costs with the direct and indirect benefits generated by the investment in a project or programme. Cost-effectiveness Comparison of the relative costs of achieving a given result or output by different means. It focuses on the relation between the costs (inputs) and results produced by a project or programme. A project/programme is more cost effective when it achieves its results at the lowest possible cost compared with alternative projects with the same intended results. Criteria The standards used to determine whether or not a programme or project meets expectations. Data Specific quantitative and qualitative information or facts that are collected. Data collection method The mode of collection to be used when gathering information and data on a given indicator of achievement or evaluation. Collection methods include the review of records, surveys, interviews, or content analysis. The term is one of the elements of the indicator methodology form for reporting through IMDIS. Data collection tool Instrument used to collect information for use in performance assessment, self-evaluation and external evaluation. Examples are mail, telephone, in-person and web-based surveys, direct or participatory observation, interviews, focus groups, expert opinion, case studies, literature search, and content analysis of internal and external records. The data collection tools must be strong enough to support the findings of the evaluation.

Data source The origin of the data or information collected. Data sources may include informal and official records, individuals, documents, etc. This term is one of the elements of the indicator methodology form for reporting through IMDIS. Description of results Succinct statement based on the data collected on the performance measures at the indicator of achievement level. It interprets and articulates such data in a results oriented language. A field is available in IMDIS for reporting on preliminary and final results for each indicator of achievement. Discretionary action A decision, taken by a programme manager on changes made to the programme design and/or implementation, to produce more efficient and effective outcomes. Addition, termination, postponement of outputs can be effected through discretionary measures. Discretionary external evaluation An evaluation requested by programme managers inthe United Nations Secretariat and managed and conducted by entities outside the programme being evaluated such as theJoint Inspection Unit (JIU) or the Office of Internal oversight Services (OIOS), or any external evaluator. In this type of evaluation, the programme manager requests the JIU, OIOS or the external evaluator, to design, manage and/or conduct the evaluation, while the programme manager's role is as the "evaluaee". Discretionary external evaluations are primarily useful for programme managers who wish to improve the performance of their programmes on the basis of objective assessments, though they may also be useful to support decision-making at the governance level. Discretionary internal evaluation/Self-evaluations These are optional, non-mandatory evaluations conducted by programme managers for their own use. They address issues that are over and above those covered by Mandatory Internal Evaluation/Selfassessments. They are normally not required to be reported upon at the intergovernmental level. While manages may contract external consultants and specialists to help with the exercise, these evaluatons are considered internal or self-evaluations because the programme managers remain responsible for the design and/or management of the evaluation, and are ultimately responsible for the quality of the reports and for using the results to improve operations. Such self-evaluations are useful when formulating best-practices and lessons-learned and can cover cross-cutting issues relevant to a number of subprogrammes. Effect Intended or unintended change caused directly or indirectly by the delivery of an output, project or programme. Effectiveness The extent to which a project or programme attains its objectives, expected accomplishments and delivers planned outputs. Efficiency A measure of how well inputs (funds, expertise, time, etc.) are converted into outputs.

End-user Recipient of an output or accomplishment. Evaluability Extent to which an activity or a programme can be evaluated in a reliable and credible fashion. Evaluation A process that seeks to determine as systematically and objectively as possible the relevance, effectiveness and impact of an ongoing or completed programme, project or policy in the light of its objectives and accomplishments. It encompasses their design, implementation and results with the view to providing information that is credible and useful, enabling the incorporation of lessons learned into both executive and legislative decision-making process. Evaluation is often undertaken selectively to answer specific questions to guide decision-makers and/or programme managers, and to provide information on whether underlying theories and assumptions used in programme development were valid, what worked and what did not work and why. Evaluation scope A framework that establishes the focus of an evaluation in terms of questions to address, the issues to be covered, and defines what will be analyzed and what will not be analyzed. The scope defines the parameters of the evaluation and is presented in the Terms of Reference. Evaluation team Group of specialists responsible for the planning and conduct of an evaluation. An evaluation team produces the evaluation report. Evaluator An individual involved in all stages of the evaluation process, from defining the terms of reference and collecting and analyzing data to developing findings and making recommendations. The evaluator may also be involved in taking corrective action or making improvements.

Evidence The information presented to support a finding or conclusion. Evidence should be sufficient, competent and relevant. There are four types of evidence: observations (obtained through direct observation of people or events); documentary (obtained from written information); analytical (based on computations and comparisons); and self-reported (obtained through, for example, surveys). Expert groups, rapporteurs, depository services Output category that includes the following output types: Expert groups Output type that refers to expert groups and teams of experts established by an intergovernmental body to provide assistance or elaborate further studies and prepare proposals on a mandated topic. The services provided have to be related to an intergovernmental mandate; Assistance to representatives, rapporteurs Output type that refers to the substantive assistance provided by the Secretariat to special representatives/rapporteurs designated by intergovernmental bodies under their established mandate. The services provided have to be related to an intergovernmental mandate;

Depository services Output type that refers to the sealing, registering and safekeeping of international agreements signed by Member States and parties to a treaty. Ex-post evaluation An assessment of the relevance, effectiveness and impact of a programme or project that is carried out some time after its completion. It may be undertaken directly after or long after completion. The intention is to identify the factors of success or failure, to assess the sustainability of results and impacts, and to draw conclusions that may inform other projects and programmes.

External evaluation An evaluation performed by entities outside of the programme being evaluated. Generally, it is intergovernmental organs that commission such evaluations and receive final reports on them. As a rule, external evaluation of a project, programme or subprogramme is conducted by entities free of control or influence by those responsible for the design and implementation of the project and programmes. In the United Nations Secretariat, there are two types of external evaluation, namely: (1) Mandatory External Evaluation and (2) Discretionary External Evaluation. (Under exceptional circumstances, the Secretary-General may commission high level external evaluation undertaken by renowned authorities or experts) Expected accomplishment A desired outcome or result involving benefits to end-users, expressed as a quantitative or qualitative standard, value or rate. Accomplishments are the direct consequence or effect of the delivery of outputs and lead to the fulfilment of the envisaged objective. External factor Event and/or condition that is beyond the control of those responsible for an activity but that has an effect on the success or failure of the activity. It may be anticipated in the form of assumptions or they may be unanticipated. Facilitator A person who helps members of a group to conduct an evaluative meeting and achieve the result for which they came together. A Facilitator should have no vested interest in the outcome of the meeting, nor dictate its substance. Feedback A process consisting of the transmission of relevant information from monitoring and evaluation exercises to targetted users so as to facilitate learning and decision-making. Such information usually comprises findings, conclusions, recommendations and lessons learned. Finding Statement about the programme or project based on empirical evidence gathered through monitoring and evaluation activities.

Focus group A group of individuals having some common interest or characteristics, brought together by a moderator, who uses the group and its interaction as a way to gain information about a specific or focused issue. Formative evaluation Sometimes known as interim evaluation, it is conducted during implementation phase of projects or programmes to improve their performance. Formative evaluations may also be conducted for other reasons such as compliance, legal requirements or as part of a larger evaluation initiative. It is intended for managers and direct supporters of a project. Goal The higher-order aim to which a programme is intended to contribute: a statement of longer-term intent. Highlights of programme results The most salient results achieved by a department/office during the biennium based on the results/accomplishments of the various subprogrammes under it. These highlights of programme results are included in the Programme Performance Report for each Budget Section and reflect the most significant achievements towards realizing the programme's objectives. IMDIS The Integrated Monitoring and Documentation Information System is the online tool for planning, monitoring and reporting of results-based programme performance in use at the United Nations Secretariat since 1998. IMDIS enables programme managers as well as the Office of Internal Oversight Services to verify Organization-wide progress throughout the biennium and to extract all necessary information for final performance analysis and reporting. Impact The overall effect of accomplishing specific results. In some situations it comprises changes, whether planned or unplanned, positive or negative, direct or indirect, primary and secondary that a programme or project helped to bring about. In others, it could also connote the maintenance of a current condition, assuming that that condition is favourable. Impact is the longer-term or ultimate effect attributable to a programme or project, in contrast with an expected accomplishment and output, which are geared to the biennial timeframe. Implementation rate The degree to which planned outputs are completed during the biennium. In the biennial Programme Performance Reports, the output implementation rate is usually calculated in three different ways. The first calculation (I1) shows the percentage of mandated outputs (those initially programmed plus those carried forward plus those added by legislation) that were completed. The second calculation (I2) shows the rate of implementation of all outputs, which is the sum of mandated outputs plus those added by the Secretariat. It indicates how much of the overall workload of the biennium was accomplished. Finally, the third figure (I3) is the ratio of all outputs implemented by a particular programme (those programmed, carried over, reformulated and added by legislation and by the Secretariat) to the outputs programmed in the biennial budget. To understand the purpose of I3, it should be recalled that while budgetary resources were provided to deliver the outputs programmed in the biennial budget,

developments during the biennium may have resulted in additions to the workload that had to be implemented within the available resources. The I3 implementation rate shows how much was delivered by a particular programme compared with what had been programmed at the outset of the biennium. In-depth evaluation Refers to the scope of the evaluation and is not to be considered a category or type of evaluation. Both internal and external evaluation can be "in-depth" when they take a comprehensive and broadranging review of the work of a programme or organizational entity. In-depth evaluation can be undertaken by external entities or by programme managers. The objective is to make recommendations that will help the Senior Managers, intergovernmental bodies or the General Assembly to formulate decisions aimed at increasing the overall relevance, effectiveness and impact of the programmes of the United Nations. Indicator A measure, preferably numerical, of a variable that provides a reasonably simple and reliable basis for assessing achievement, change or performance. A unit of information measured over time that can help show changes in a specific condition. Indicator of achievement Used to measure the extent to which expected accomplishments have been achieved. Indicators correspond to the expected accomplishment for which they are used to measure performance. One expected accomplishment can have multiple indicators. Indirect effect The unplanned changes brought about as a result of implementing a programme or a project. Information management system A system of inputting, collating and organizing data that should provide selective analyses and reports to the management, to assist in monitoring and controlling the project organization, resources, activities and results. Internal evaluation Evaluation that is managed and/or conducted by entities within the programmes being evaluated. There are two types of internal evaluation, namely: (1) Mandatory Internal Evaluation (Self-assessmetns) (2) Discretionary Internal Evaluation (Self-evaluation) Input Personnel, finance, equipment, knowledge, information and other resources necessary for producing the planned outputs and achieving expected accomplishments. Inspection Inspections address specific mandates, management issues or areas of high risk and recommend remedial action. Inspection teams also examine procedures set in place by management to ensure adequate preparation, implementation, evaluation and control of the programme of work and the resources appropriated for it.

Interim performance measures Data collected for the indicators of achievement during the biennium. They are used as benchmarks to determine whether progress is being made towards the intended results. Joint evaluation An evaluation in which different programmes, agencies and/or partners contribute and participate. Learning Reflection on experience and results of monitoring and evaluation to identify how a situation or future actions could be improved and then using this knowledge to make actual improvements. This can be individual or group-based the latter would be considered organizational learning when it transcends the group and becomes part of the corporate knowledge base. Learning involves documenting experiences, the process of synthesis and the application of lessons learned to future actions. Legislative mandate A request for action addressed to the Secretary-General or an executive head acting on his behalf in a resolution or decision adopted by a competent United Nations intergovernmental organ. Lesson learned Generalization derived from evaluation experiences with programmes, projects or policies that is applicable to a generic situation rather than to a specific circumstance and has the potential to improve future actions. A lesson learned summarises knowledge at a point in time, while learning is an ongoing process. Logical framework Management tool (also known as a logframe) used to identify strategic elements of a programme or project (objective, expected accomplishments, indicators of achievement, outputs and inputs) and their causal relationships, as well as the assumptions and external factors that may influence success and failure. It facilitates planning, implementation, monitoring and evaluation of a programme or project. Mandatory external evaluation An evaluation mandated by an Intergovernmental body that is managed and performed by an entity outside the programme being evaluated. Mandatory external evaluations are mandated by bodies such as the Committee for Programme and Coordination (CPC), functional commissions, regional and sectoral intergovernmental bodies and other technical bodies. Mandatory external evaluatons are generally managed and conducted by the Joint Inspection Unit (JIU) as part of its UN-wide mandate, and by the Office of Internal Oversight Services (OIOS), as well as by appointed external evaluators. Mandatory external evaluations are primarily useful for oversight and support to decision-making at the governance level, while they also may be useful for programme managers. Mandatory internal evaluation/Self-assessments These are compulsary assessments performed by programme managers once every two years. These assessment are required for all United Nations Secretariat programmes and are framed by the logical frameworks in the approved bienniual programme budget documents. They are reported on through the biennial Programme Performance Report (PPR). They use information generated by measuring the extent to which Expected Accomplishments were achieved through the collection of indicator data and

use of IMDIS tool as a means to record progress and report results and accomplishments. They may also use information generated from other types of evaluation and assessment, as appropriate. Methodology A set of analytical methods and techniques appropriate for evaluation of the particular activity. It could also be aimed at collecting the best possible evidence needed to answer the evaluation issues and analytic questions. Methodology form A set of fields developed and incorporated into IMDIS for each indicator of achievement that allow subprogrammes and programmes to identify and document the elements of the indicator that will be measured during the biennium, identify data sources, determine data collection and verification methods, fix the periodicity of measurements, create or identify a presentation format and identify external factors that could distort or influence measurements. This should be done early in the biennium to ensure proper data collection and reporting of results. Meta-evaluation An evaluation designed to aggregate findings from a series of related evaluations dealing with a common topic. Monitoring A periodic assessment by programme managers and by the Office of Internal Oversight Services, of the progress in achieving the expected accomplishments and delivering the final outputs in comparison with the commitments set out in the programme budget as approved by the General Assembly. It provides assurance that the implementation of a programme or project is proceeding as planned. Monitoring and evaluation The combination of monitoring and evaluation together provide the knowledge required for effective programme and project management and for reporting and accountability responsibilities. Non-recurrent publication An output category of printed or otherwise published document, book, study, bulletin, newsletter, journal and other analytical paper prepared exclusively in a given biennium by the departments in the Secretariat which forecasts, informs or reports on a certain situation or issue. A non-recurrent publication can be mandated or initiated at the discretion of a programme manager. Objective Description of an overall desired achievement involving a process of change and aimed at meeting certain needs of identified end-users within a given period of time. A good objective meets the criteria of being impact oriented, measurable, time limited, specific and practical. The objective is set at the next higher level than the expected accomplishments. Other substantive activities An output category comprising, inter alia, fact-finding missions, human rights and humanitarian assessment and relief missions, promotion of legal instruments, seminars for outside users, recurrent and non-recurrent publications, electronic, audio and visual issuances, exhibits, guided tours, lectures; booklets, fact sheets, wall charts, information kits; press releases, press conferences; special events;

technical material for outside users; substantive servicing of, including documentation for, inter-agency meetings. Outcome In the United Nations Secretariat, outcome is used as a synonym of an accomplishment or a result. Outlier A subject or other unit of analysis that has extreme values. Usually outliers are excluded from statistical tabulations because they are considered unrepresentative or unreliable. Output A final product or service delivered by a programme or project to end-users, such as reports, publications, servicing of meetings, training, advisory, editorial, translation or security services, which a programme is expected to produce in order to achieve its expected accomplishments and objectives. Outputs may be grouped into broader categories. Output category Grouping of outputs, established at the planning stage, in the work programme of departments/offices. For example Other substantive activities or Conference service, administration and oversight See also: Conference services, administration, oversight, Expert groups, rapporteurs, depository services, "Non-recurrent publication, Parliamentary documentation, Recurrent publication, Other substantive activities, Carried forward, Reformulated, Postponed, Terminated Output delivery An output is generally considered to have been delivered when the service is completed or when the planned product is made available to the intended end-users; e.g., in the case of a report or a technical publication, when it has been submitted to the General Assembly or relevant intergovernmental bodies or circulated to Member States or other primary users; in the case of a sales publication, when it is placed on sale. See also: Implementation rate Parliamentary documentation An output category of reports prepared by the Secretariat on relevant issues in response to approved mandates or at the initiative of the Secretariat. It also includes reports of intergovernmental/ expert/other bodies for which the substantial assistance of the Secretariat was provided. Other reports considered parliamentary documentation are United Nations official records; sessional notes and other documents prepared for scheduled meetings; conference room papers; legal and financial/budgetary opinions/statements provided to intergovernmental, expert and treaty bodies. See also: Output, Output category Participatory evaluation A broad term for the involvement of various stakeholders in evaluation. It involves the collective examination and assessment of a project or subprogramme by the stakeholders (programme managers and staff included) and solicits views of end-users and beneficiaries. Participatory evaluations involve reflective, action-oriented assessments of performance and accomplishment which yield lessons learned and instructive practices. See also: Evaluation

Performance The degree to which a programme or project delivers results in accordance with stated objectives, timely and effectively as assessed by specific criteria and standards. See also: Results-based management (RBM) Performance assessment External assessment or self-assessment by programme units, comprising monitoring, reviews, end-ofyear reporting, end-of-project reporting, institutional assessments, and/or special studies. See also: "Self-evaluation, "Self-monitoring" Performance management Measures taken by a programme manager, based on monitoring and evaluation information, to foster continuous improvement. It is supported by performance measurement. See also: Results-based management (RBM), "Learning" Performance measure Specific statistics chosen because they provide valid, practical and comparable measures of progress or level of change towards achieving expected results for the period. Performance measurement A system for the collection, interpretation of, and reporting for the purpose of objectively measuring how well programmes or projects contribute to the achievement of expected accomplishments and objectives and deliver outputs. See also: Indicator, IMDIS, Logical framework, Results-based management (RBM) , Data, Expected accomplishment, Objective, Activity, "Methodology form" Performance monitoring A continuous process of collecting and analyzing data to compare how well a programme, project, or policy is being implemented against expected results. See also: IMDIS, Programme performance report (PPR), "Self-monitoring" Postponed Status of programmed output that is not delivered to intended users during the relevant biennium and is carried forward for completion to the next biennium. The postponement can be by legislative decision or at the discretion of the Secretariat. IMDIS automatically enters postponed outputs in the inventory for the following biennium and they do not need to be re-programmed. See also: Output, Output category Programme Consists of the activities undertaken by a Department or Office together with a coherent set of objectives, expected accomplishments and outputs intended to contribute to one or more organizational goals established by Member States. The programme is guided by t the mandates entrusted to a department/office by the General Assembly. See also: Logical framework, Budget fascicle

Programme manager The Secretariat official responsible for the formulation and implementation of a programme as defined above, normally the head of a major organizational unit. For the purposes of self-evaluation, programme managers comprise those who are involved in the implementation of subprogrammes, namely, the head of a division, branch or section. Programme performance report (PPR) The mandated report of the Secretary-General submitted to the General Assembly biennially reflecting implementation and results for programmes in the Secretariat. The PPR for 2002-2003 was produced both in print A/59/69 and electronic format. See also: IMDIS, Results-based management (RBM) , Monitoring, Self-monitoring Programmed outputs Outputs approved by the General Assembly at the beginning of the biennium for delivery. See also: Output, Terminated, Reformulated, Postponed Project Planned activity or a set of planned, interrelated activities designed to achieve certain specific objectives within a given budget, organizational structure and specified time period. Within the Secretariat, projects are used in technical cooperation activities. See also: Programme Project/Programme cycle management A tool for understanding the tasks and management functions to be performed in the course of a project or programmes lifetime. This commonly includes the stages of identification, preparation, appraisal, implementation/supervision, monitoring, evaluation, completion and lesson learning. See also: Results-based management (RBM) , Work plan Project evaluation Evaluation of an individual project designed to achieve specific objectives within specified resources, in an adopted time span and following an established plan of action, often within the framework of a broader programme. The basis of evaluation should be built into the project document. See also: Evaluation Project document A formal document covering a project, which sets out, inter alia, the needs, results, outputs, activities, work plan, budget, pertinent background, supporting data and any special arrangements applicable to the execution of the project in question. Once a project document is approved by signature, the project represents a commitment of resources. See also: Project evaluation, "Project/Programme cycle management Proxy indicator Proxy indicator is used when it is difficult to identify direct indicators to measure the result. Proxies are indicators that may tell us indirectly whether a result has been achieved. See also: Indicator of achievement, Causal relationship, Indicator, Data

Qualitative data Information that is not easily captured in numerical form (although qualitative data can be quantified). Qualitative data typically consist of words and normally describe people's opinions, knowledge, attitudes or behaviours. See also: Data Quality Assurance Quality assurance encompasses any activity that is concerned with assessing and improving the merit or the worth of a programme, project or an output and their compliance with given standards. Quantitative data Information measured or measurable by, or concerned with, quantity and expressed in numerical form. Quantitative data typically consists of numbers. See: Data Rating system Forming and validating a judgment on the relevance, performance and success of a programme or project through the use of a scale with numeric, alphabetic and/or descriptive codes. See also: Data Recommendation Proposal for action to be taken to enhance the design, allocation of resources, effectiveness, quality, or efficiency of a programme or a project. Recommendations should be substantiated by evaluation findings, linked to conclusions and include the parties responsible for implementing the recommended actions. See also: Finding, Conclusions Recurrent publication An output category of regularly printed or otherwise published document, book, study, yearbook, bulletin, newsletter, journal and other analytical paper periodically prepared by the departments in the Secretariat which forecasts, informs or reports on a certain situation or issue. A recurrent publication can be mandated or initiated at the discretion of the Department/Office. See also: "Non-recurrent publication, Output category Reformulated Status of outputs that were completed and delivered to the intended users during the reporting period/biennium after undergoing revision to the original formulation. An output is considered reformulated if it continues to address the same subject matter of the originally programmed output and to cater to the same intended users. See also: Output Relevance The extent to which an activity, expected accomplishment or strategy is pertinent or significant for achieving the related objective and the extent to which the objective is significant to the problem addressed. See also: Evaluation

Reliability Consistency or dependability of data and evaluation judgements, with reference to the quality of the instruments, procedures and analyses used to collect and interpret evaluation data. Evaluation information is reliable when repeated observations using similar instruments under similar conditions produce similar results. See also: Evaluation Result The measurable accomplishment/outcome (intended or unintended, positive or negative) of a programme or project. In the Secretariat practice, result is synonymous with accomplishment and outcome. See also: Accomplishment, Expected accomplishment, Outcome, Effect, Impact, Resultsbased management (RBM) Results-based management (RBM) A management strategy by which the Secretariat ensures that its processes, outputs and services contribute to the achievement of clearly stated expected accomplishments and objectives. It is focused on achieving results and improving performance, integrating lessons learned into management decisions and monitoring of and reporting on performance. See also: Logical framework, Performance management", Self-evaluation, " IMDIS Revised baseline An adjusted baseline recorded in IMDIS in cases where the actual figures recorded in the previous biennium differ from the estimated baselines in the proposed programme of work. See also: Baseline, Target,, IMDIS Revised target An adjusted target recorded in IMDIS in cases where the estimated baselines and targets in the proposed programme of work have to be adjusted to capture actual achieved results from the previous biennium and realistic achievements for the current biennium. See also: Baseline, Target,, IMDIS Sample The selection of a representative part of a universe in order to assess parameters or characteristics of that universe. Random sampling is the selection of a group of subjects (the sample) from a larger group (the population or universe), so that each individual or other unit is chosen entirely by chance. See also: Universe, Data Self-evaluation Self-evaluation is a systematic assessment carried out by those entrusted with the design and delivery of a programme, the findings of which are used to improve planning, performance and programme results. See also: Evaluation Self-monitoring Ongoing assessment by the head of a department or office of the progress in achieving the expected accomplishments and delivery of outputs. See also:IMDIS, Results-based management (RBM), Monitoring

Stakeholder Agencies, organizations, groups or individuals who have a direct or indirect role and interest in the objectives and implementation of a programme or project and its evaluation. In participatory evaluation, stakeholders assume an increased role in the evaluation process as question-makers, evaluation planners, data gatherers and problem solvers. See also: End-user, Beneficiary Statement of accomplishment/results achieved A succinct synopsis of accomplishments achieved relative to the Expected Accomplishment which is based on and distilled from the Accomplishment Account. It captures the key facts of what was achieved in this regard during the biennium. See also: Accomplishment account, IMDIS, Results-based management (RBM), Programme performance report (PPR) Strategic evaluation An evaluation of a particular issue, often cross-cutting, with significant implications for the major priorities of Member States and the Secretariat with high risks to stakeholders. Its timing may be especially important owing to the urgency of the issue which poses high risks to, and has generated conflicting views from stakeholders. Subprogramme A subprogramme consists of activities within a programme aimed at achieving one or a few closely related objectives as set out in the strategic framework. The subprogramme structure shall correspond, to the extent possible, to an organizational unit, normally at the division level. See also: Budget fascicle Substantive servicing of meetings An output category covering the provision of core counselling, advice and reporting services to UN meetings and consultations, both formal and informal, of intergovernmental bodies such as the General Assembly and its sessional bodies; the Security Council; the ECOSOC and its sessional bodies; the subsidiary organs of the General Assembly, the Security Council and ECOSOC and those of suborgans/working groups of those subsidiary bodies; consultations of the treaty bodies; the other organs/conferences held under the United Nations auspices and their subsidiary bodies; and Pledging conferences. Informal meetings are only those scheduled with interpretation. Technical services such as interpretation and translation are not part of substantive servicing. Servicing of a three-hour meeting is considered as one output. See also: Output category Summative evaluation A study conducted by independent evaluators at the end of a project or programme to measure extent to which anticipated results were achieved; ascertain the effectiveness and relevance of approaches and strategies; indicate early signs of impact; and recommend what interventions to promote or abandon. Summative or Terminal evaluation is intended to provide information about the merit and worth of the project or programme. See also: Evaluation

Survey Systematic collection of information from defined respondents through the administration of a structured questionnaire in person, by telephone, by mail, or by web. A survey can be administered to the universe of all respondents or to a sample of that universe. See also: Sample, Data Sustainability The extent to which the impact of the programme or project will last after its termination; the probability of continued long-term benefits. See also: Impact Synthesis The process of identifying relationships between variables and aggregating data with a view to reducing complexity and drawing conclusions. See also: Data Target A specified objective that indicates the number, timing and location of what is to be achieved. Target group (back to top) The main beneficiaries of a programme or project that are expected to gain from the results of that programme or project. They are closely related to its impact and relevance. See also: End-user, Beneficiary, Stakeholder Terms of reference Written document presenting the purpose and scope of the evaluation or inspection, the methods to be used, issues to be addressed, the resources, schedule, and reporting requirements. See also: Evaluation Terminated Status of programmed outputs not delivered to the intended users during the biennium. Termination can be by legislative decision or at the discretion of the Secretariat. Programme managers are encouraged to exercise their discretion to terminate outputs if they become redundant, duplicative, obsolete, irrelevant, or when extrabudgetary resources are not available to implement them in either the current or future biennium. See also: Output Thematic evaluation An evaluation focused entirely on a single, cross-cutting theme or activity in order to assess cumulative effects of multiple programmes that share common objectives and purposes. It aims to appraise the effectiveness of coordination and cooperation between different programmes of the Secretariat in the thematic area. See also: Evaluation

Triangulation The use of three or more methods to conduct an evaluation or substantiate as assessment. By combining multiple data sources or methods evaluators seek to overcome the bias that comes from single informants and single methods. See also: Data, Evaluation, Bias, Best practice Universe A group of persons (or other subjects of study) that the evaluation wishes to describe or generalize about. Also called population. See also: Data, Sample Validation The process of cross-checking to ensure that the data obtained from one monitoring and evaluation method are confirmed by the data obtained from a different method. See also: "Data, Reliability Validity The extent to which the data collection methods or tests accurately measure what they are supposed to. Valid evaluations are ones that take into account all relevant factors, given the whole context of the evaluation, and weigh them appropriately in the process of formulating conclusions and recommendations. See also: Data Variable In evaluation, refers to specific characteristics or attributes, such as behaviours, age, or test scores, that are expected to change or vary. For example, the level of adolescent drug use after being exposed to a drug prevention program is one variable that may be examined in an evaluation. In monitoring, refers to the elements of the indicator to be measured. For the indicator Number of Member States, with the assistance of the Secretariat, having new or enhanced demand reduction strategies and programmes in place, the variables/elements to be measured would be (a) the number of Member States having new demand reduction strategies and programmes in place with the assistance of the Secretariat, and (b) the number of Member States having enhanced demand reduction strategies and programmes in place with the assistance of the Secretariat. See also: Methodology form Work plan A detailed document stating outputs to be delivered and activities to be carried out in a given time period, how the activities will be carried out, and what progress towards expected accomplishments will be achieved. It contains timeframes and responsibilities and is used as a monitoring and accountability tool to ensure the effective implementation of the programme. The work plan is designed according to the logical framework.