Box 1: Essential Criteria for Performance Indicators

On the selection of Performance Indicators, Diana and Daraio (2010) propose six essential
criteria:

1. Indicators should be strictly related to the objective of the evaluation exercise

2. The data needed to collect them should be available

3. Analytical procedures to compute them should fit the data (for example, adequately treating
non-metric or qualitative information)

4. They should consider elements that can be directly or indirectly influenced by the subject on
which the evaluation is carried (public transport operator, transit agency, policy maker), so that
they are effectively responsive to those actions that can be taken by the subject itself

5. They should be robust, so that they can be used at different points in time to monitor trends,
but also across different services to draw comparisons.

6. They should have some intuitive meaning and they should be easy to interpret; in particular
they should be unambiguous, so that their variation in a given direction can be safely considered
as good or bad.

Monitoring and Evaluation Theory

Monitoring and evaluation theory is a broad area which covers a wide number of practices and
aspects. Monitoring is, in essence, collecting data on variables which are of interest. The key
questions relating to monitoring may be: how much should be monitored, what should be
monitored, and how to develop and choose indicators. Monitoring theory deals with questions
such as categorising different types of variables which may be collected, and how best they can
be managed given their (i) resolution (how frequently a sample is taken - every week, month,
year etc.), (ii) latency (time before the information can affect policy) and (iii) diversity (types of
information being collected (euros, kilometres, passengers, etc.)). Finally, the theory of change is
a central area in monitoring and evaluation theory; this is a conceptual exercise where a
programme planner defines programme objectives, and systematically maps out how those
objectives will be achieved. A theory of change demonstrates the pathway of how to get from
here to there (i.e. what is needed for goals to be achieved). It requires underlying assumptions
to be detailed out in a way that they can be tested and measured and puts the emphasis first on
what the organization wants to achieve rather than on what the organization is doing. In short
developing a theory of change forces the analyst to engage in the how of a programme, as
opposed to the what.