The Manual of Best Practice Performance Indicators for Wastewater Servicesprovides guidelines for the establishment of a management tool for wastewater utilities based on the use of performance indicators. The publication comprises the text and a CD-ROM with the SIGMA Lite WWsoftware, developed by Instituto Tecnologico del Agua (ITA), Valencia Polytechnic University, Spain. The focus is on those performance indicators considered to be the most relevant for the majority of wastewater utilities, to be used routinely at management level and potentially for metric benchmarking practices. A set of three comprehensive appendices includes a glossary of technical terms, specifications of each parameter required to assess the performance indicators and an introduction to the software with tips for use and an example of application. This product will be an invaluable reference source for all those concerned with managing the performance of wastewater services including customer groups, utility managers and policy-makers, regulators and other stakeholders. Contents Structure of the wastewater PI-System Data reporting Context Information Performance Indicators Implementation Strategy for the PI-System Appendix 1 - Glossary of technical terms Appendix 2 - Data Definition and Processing Rules Appendix 3©SIGMA Lite WW
Read an Excerpt
Increasing diversity in the provision of services for water supply and wastewater disposal, together with greater accountability for the quality of those services, necessitate scrutiny using assessment systems that are consistent, transparent and auditable. Whilst a number of countries worldwide have well defined and regulated systems for evaluating the performance of water service providers, this is by no means global. It is within this context that the International Water Association (IWA) is developing systems of performance indicators (PIs) for water supply and wastewater services. In July 2000 a task force from the Operations and Maintenance Specialist Group of IWA produced the IWA Manual of Best Practice "Performance Indicators for Water Supply Services" (Alegre et al., 2000). The original document is available from IWA Publishing, however, on-going testing with some 70 organisations worldwide has resulted in a number of changes to the approach recommended and the PI system. The lessons from the testing of the water supply PI system have been used to inform the development of this manual, an equivalent document which deals with Performance Indicators for Wastewater Services. The approach to the specification, selection and usage of the PIs is generic to both water supply and wastewater services. Although the particular PIs recommended for use for performance assessment of each of the two systems are not the same. The IWA PI systems are intended to provide objective and comprehensive management tools for undertakings and other stakeholders involved in any aspect of water and wastewater service provision. The PIs have been developed independently from the level of economic development, or type of Institutional system in which the undertaking operates, and allow globally diverse economic, demographic, cultural and climatic characteristics to be acknowledged. The PIs cover a wide range of activities comprising: management, personnel, financial, physical, operational, environmental and quality of service aspects. The PIs for wastewater systems and associated procedures presented in this manual are expected to undergo further development through wide-scale pilot testing.
1.2 Why use performance indicators
Irrespective as to whether it is a private company or a public municipal service supplier, any undertaking needs to strive for high degrees of efficiency and effectiveness to achieve its management goals. In addition, other stakeholders, such as regulators or customers require assurance that the undertaking is performing appropriately. With increasing diversity in the way in which water services are provided, a recent state-of-the-art review, covering more than 16 countries, concluded that some form of standardised performance assessment has become essential (Merkel, 2002).
Performance Indicators (PIs) may be considered as providing key information needed to define the efficiency and effectiveness of the delivery of services by an undertaking (Deb & Cesario, 1997). Efficiency is the extent to which the resources of an undertaking are utilised to provide the service, for example, maximising service delivery for the minimum use of available (possibly natural) resources. Effectiveness is the extent to which declared or imposed, objectives, such as levels of service, (specifically and realistically defined) are achieved. PIs may also be considered as providing information for 'metric' benchmarks - quantitative comparative assessment of performance (Larsson et al., 2002). The actual comparison of performance between similar service provision is undertaken via 'process' benchmarking - examining business processes, comparing the activities of different organisations and seeking to identify best practices. A performance indicator may thus be used as a quantitative (or in some cases qualitative) measure of a particular aspect of an undertaking's performance or standard of service. PIs may be used to compare performance historically, or against some pre-defined target (Alegre et al., 2002a; Matos et al., 2002b). PIs may be used by a wide range of stakeholders in evaluating the performance of the undertaking, including internal evaluation within the undertaking itself.
As an example of PI use, in 1995 the six-Cities Group in Scandinavian countries (Copenhagen, Gothenburg, Helsinki, Malmo, Oslo, and Stockholm) initiated the development of a coherent performance benchmarking system for water and wastewater services (Stahre & Adamsson, 2002). They have developed a set of performance indicators that may be considered as a standardised reference language necessary for making consistent system comparisons. Annual benchmarking exercises are carried out and the results presented in two forms, in summary, to the Managing Directors of the undertakings and in more detail to the operational staff. Comparative performance between the six undertakings and trends looking back five years are also highlighted. The metric benchmarking processes have also been supplemented by process benchmarking whereby differences in performance between the organisations are investigated.
Precise definition of the PIs to be used by each stakeholders and within stakeholder subgroups is not straightforward. The use of indicators in general is problematic, and has received a lot of attention with regard to attempts to define sustainability (e.g. Ashley et al., 1999). A major difficulty is where the PIs are used in a judgemental way. One example of this is the way in which the Regulators assess the economic performance of the water undertakings in England and Wales. The performance objectives and targets are produced jointly by the Water Companies, the Government and the Water industry economic Regulator, the Director General of Water Services and Office of Water Services (OFWAT). Targets are set in association with other bodies, NGOs, consumer representatives and the Environment Agency. In the UK sector the three critical areas of performance judged by the regulatory bodies are value for money, service delivery and environmental and social impacts. The UK regulatory approach is essentially judgemental and comparative to assess the performance both between companies and against pre-determined targets. The actual PIs in each specific application that are used reflect this perspective (Ashley & Hopkinson, 2002).
Recently, a number of regulatory approaches have addressed asset serviceability. This differs from the traditional levels of service concepts, by being concerned with the longer-term service provided by capital assets. This necessitates capital maintenance in order that assets may continue to deliver services to customers (Reynolds, 2000). There is a need for this approach where current customers are receiving a satisfactory service (as defined by current performance in terms of levels of service), but where there appears to be no incentive for longer-term investments by the undertaking for future customers. In the UK, the approach to PIs to assess asset serviceability is now risk based (Ewan Associates & Mott MacDonald, 2001). In this, asset serviceability is defined as the ability of an asset to deliver a defined service to customers and safeguard the environment. There are four levels of PI. At level 1, the base data are used to reflect the state of the undertaking's assets and the impact on customers and the environment. The higher level indicators require further analysis or combinations of the base PIs in order to assess trends in serviceability, i.e. as an indication of a potential declining trend in serviceability. In the IWA PI manuals the indicators provide the base information equivalent to the UK Level 1 serviceability indicators.
Whilst the IWA PIs are intended to be comprehensive (covering all forms and contexts for water service provision), they are meant to be informative rather than judgmental. It is expected that much of their usage will be within organisations for self-improvement with regard to specific aspects of the undertaking's activity and of the system behaviour. This may also be informed by self-comparison (benchmarking) with other similar water service providers operating with similar constraints and contexts.
Given that as yet it is not possible to define what may or may not be a sustainable system, moves toward greater sustainability in water services can only be achieved using criteria and indicators, such as the PIs described here, that inform judgement as to the best way of providing the service (Ashley et al., 2002). Hence the approach being used is a key component in building systems that are likely to be as sustainable as current knowledge allows. In this context sustainability means both the traditional definition (including social-environmental-economic criteria) and the undertaking definition, which is that the service provider remains in business (Foxon et al., 2001).
Usage of performance indicators
Performance indicators can be of use to a wide range of stakeholders (e.g. Audit Commission, 2000).
However, it is envisaged that the PIs presented in the IWA manuals will be utilised primarily by the service providers. The following outline the reasons why various stakeholders may use the PIs.
For water supply and wastewater undertakings:
To assist with strategic and structured planning;
To facilitate better quality and more timely responses from managers;
To allow for simpler and more structured monitoring of the effects of management decisions;
To provide key information to support a more pro-active approach to management, rather than reacting to apparent system or organisational malfunctions;
To highlight strengths and weaknesses of individual departments, or regional sectors of the organisation, demonstrating where there may be a need to improve productivity, procedures and routines;
To assist with the implementation of a Total Quality Management regime, as a way of improving all-round quality and efficiency throughout the organisation;
To facilitate the implementation of benchmarking routines, both internally, for comparing the performance at different locations or for sub-systems, and externally, for comparison with other similar undertakings, thus providing key information to promote future performance improvements;
To provide sound scientific, technical, financial and personnel information for auditing the workings of the undertaking and to help to predict the likely effects of any recommendations made as a result of an audit.
For national or regional policy-making bodies and Regulatory Agencies:
To provide a consistent framework for comparing the performance of wastewater undertakings and identifying areas of activity where improvement is required;
To support the formulation of policies for the sector, ideally within a perspective for the integrated management of water resources, including environmental protection, resource allocations, investments, and the development of new regulatory tools.
To provide key monitoring information to help safeguard customer interests where service is provided by a monopoly, particularly to monitor compliance with contracted service level goals.
To provide evidence of adequate environmental performance by comparison with pre-defined standards.
For financing bodies:
To assist in assessing investment priorities, risks, project selection, realisation and audit.
For consumers and their representative bodies:
To provide wide-ranging standardised information about the performance of the undertaking in a way as transparent as possible.
For Quality Certifying organizations:
To provide key information for quality assurance purposes.
For Auditors and Economic Regulators:
Use as part of a consistent framework for auditing asset value, condition, and financial and related performance.
For multi-lateral organisations:
To provide a tool for identifying the main differences in performance of services between regions, and hence to inform future strategies for improvement and the allocation of resources.
The wastewater PIs may be used by these various organisations to evaluate performance historically, regionally or nationally, against previous time periods or comparable organisations. Trends in PIs with time may show historically improving or deteriorating performance in time for remedial measures to be taken before major problems occur in service delivery. Where new systems are being implemented, PIs may show whether these are achieving effective and efficient performance compared with other organisations. This is particularly important in developing countries. PIs require a lot of data; hence they can also provide the incentive for good monitoring, data recording and processing, and help decision makers focus scarce resource allocations into the key areas where data needs to be collected.
The main forms of using performance indicators are (adapted from Alegre, 2002b):
Exclusively within the undertaking - managers may use PIs to monitor the evolution of performance, comparing the results achieved at a given time with those achieved in previous periods. Another form of use is comparing the results actually achieved with preset targets or with reference values that have been published from other organizations. For the former, the undertaking may define internal indicators, whereas for the latter, the undertaking has to adopt standardized definitions to avoid non-commensurate comparisons.
In the framework of benchmarking initiatives - the internal use of PIs can be complemented with metric benchmarking initiatives; these can be held within a group of undertakings that can agree among them a set of indicators to be adopted by all the group members, based on a common set of definitions. The analysis, interpretation and eventual publication of the results must always take into account these definitions.
As part of a regulatory framework - the worldwide growth of private-sector participation in the management of water supply and wastewater systems requires added vigilance from economic regulators. For example, the adoption of PI systems may be used as part of a surrogate for true commercial competition among water service providers. Where the performance of an undertaking is being evaluated against another the use of standardised definitions for PIs is essential.
As part of contractual agreements - traditionally, contract agreements between the municipalities and private operators are very detailed in terms of the financial arrangements, but may be less specific in terms of the quality of service to be provided to the customer. The use of PIs provides a means whereby specified and scheduled targets may be defined in an objective and auditable way, protecting the customers' interests. Whilst the use of customised PIs for this is feasible and sometimes necessary, however, the use of standardised PIs provides a more objective, independently accredited and comparable foundation on which to base agreements.
As part of Quality Certification Systems - monitoring of the undertaking's internal processes to attain ISO 9000 standards or other certification can be assisted by using PIs. Utilisation of standardised PIs provides a more robust foundation for certification than self-selected PIs.
To provide public domain statistical reports - PIs may provide the baseline statistics for citizen information, or to be made available collectively by organisations such as Waterworks Associations, regionally or internationally (e.g. World Health Organisation, Asian Development Bank, World Bank). Traditionally, this type of performance monitoring has been limited to statistical data and economic indicators, but recently other types of indicators are also being presented, dealing for example with environment. Unfortunately information presented in these reports is usually inconsistent, and varies considerably between different undertakings (e.g. Hopkinson & Whitaker, 1999). It may also be of doubtful value due to a lack of precise definitions for the indicators. The use of consistent and standardised PIs is therefore very important in these applications.
Excerpted from "Performance Indicators For Wastewater Services"
Copyright © 2003 IWA Publishing.
Excerpted by permission of IWA Publishing.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
Table of Contents
List of Figures, vii,
List of Tables, vii,
Foreword by Peter Stahre, viii,
Measurement Units and Symbols, xii,
Other Conventions, xii,
1. Introduction, 1,
2. Development of a manual for wastewater service PIs from the water supply manual, 9,
3. About this manual, 13,
4. The wastewater PI system, 15,
5. Implementation strategy of a PI system, 33,
6. Data reporting, 43,
7. Performance Indicators, 47,
8. Context information, 91,
9. References, 105,
10. About the authors, 107,
Appendix 1 - Glossary, 111,
Appendix 2 - Variables definitions, 121,
Appendix 3 - Example of significance levels assignment for PI, 165,
Appendix 4 - SIGMA Lite WW, 171,