IN THE PACIFIC ISLAND REGION Monitoring and Evaluation Initiative
PROGRESS REPORT No.2
(DRAFT REVIEW OF MONITORING AND EVALUATION SYSTEMS AND INFORMATION MANAGEMENT AND PRELIMINARY EVALUATION OF ROUNTABLE AND ACTION STRATEGY TARGETS AND INDICATORS)
Philip Tortell, Consultant
Acknowledgements and Invitation
This report represents a blend of what has been gleaned from various documents and websites together with the results of consultations both electronic and during country visits. I am grateful to all those who met with me during the country visits, replied to the questionnaire and shared with me their views and opinions or helped with obtaining information for me. I also wish to thank all those kind people who made travel arrangements, set up appointments and helped in any other way to ensure the effectiveness of my country visits. I wish to make particular mention of Ms Kate Brown-Vitolio of SPREP who has spared no effort to ensure that my assignment is as successful as it can be. While reflecting what I have been told and what I have found out, this report is ultimately my own perception of the situation I am reporting on and does not represent the views or opinions of the Roundtable or any of its constituent organizations.
The report is meant to generate discussion and I would sincerely welcome comments and reactions to what it says and what it contains.
Philip Tortell, Consultant
Environmental Management Limited
PO Box 27433 Wellington, NEW ZEALAND
Tel +64-4-384 4133, Fax +64-4-384 4022, Email
CONTENTS 1 INTRODUCTION 4
1.1 Background 4
1.2 Methodology 5
1.2.1 Documents and websites reviewed and consulted 5
1.2.2 Country visits 7
1.2.3 Questionnaire 8
2 RESULTS : INFORMATION MANAGEMENT 9
2.1 M&E Approaches of Selected Donors and regional organizations 9
2.1.1 Global Environment Facility (GEF) 9
2.1.2 United Nations Development Programme (UNDP) 10
2.1.3 United Nations Environment Programme (UNEP) 11
2.1.4 The World Bank 11
2.1.5 Asian Development Bank (ADB) 12
2.1.6 Secretariat of the Pacific Regional Environment Programme (SPREP) 12
2.1.7 Secretariat of the Pacific Community (SPC) 13
2.1.8 Pacific Islands Forum Secretariat (PIFS) 13
2.1.9 South Pacific Applied Geoscience Commission (SOPAC) 14
2.1.10 Council of Regional Organizations in the Pacific (CROP) 14
2.2 Other Regional Approaches to M&E 15
2.2.1 Environmental Vulnerability Index (EVI) 15
2.2.2 Millennium Development Goals (MDGs) 16
2.2.3 Pacific Regional Information System (PRISM) 17
2.2.4 The Pacific Plan 17
2.3 National and Regional Environmental Monitoring Systems 18
2.4 National and Regional Environmental Databases 20
3 RESULTS : VIEWS ON ROUNDTABLE 23
3.1 Awareness and familiarity with Roundtable 23
3.2 Perceived strengths of the Roundtable 23
3.3 Perceived weaknesses of the Roundtable 24
3.4 The Working Groups of the Roundtable 25
3.5 Relationship between the Roundtable and the Governments/Administrations in the region 26
3.6 Relationship between the Roundtable and SPREP 27
4 RESULTS : ACTION STRATEGY 29
4.1 Goals 29
4.2 Objectives and Targets 30
4.3 Indicators 31
4.3.1 Indicators and the Roundtable 31
4.3.2 Guidance on developing indicators of success 32
4.3.3 Outline plan for developing a suite of indicators for the Roundtable 36
5 DISCUSSION, CONCLUSIONS AND RECOMMENDATIONS 37
5.1 Information management 37
5.2 The Roundtable 38
5.3 The Action Strategy 39
Annex 1 Travel Schedule
Annex 2 Edited Notes from meetings in Apia, Suva and Noumea
Annex 3 Questionnaire and Collated Replies
1 INTRODUCTION1.1 Background
The Objective of this Assignment is to develop a monitoring and evaluation process for the Action Strategy for Nature Conservation for the Pacific Islands Region1 and use it to analyse the achievement of the 2003-07 targets across the region.
The Assignment involves the review of a number of key documents, extensive consultations (including at country level), and the design of an Environmental Indicators System operationalized at country level. The consultant is also required to assume the lead reporting/facilitating role in a number of regional meetings over the 2.5 years of the consultancy. The legacy of the Assignment will be an enduring, simple, but effective monitoring system.
The Terms of Reference go on to say that to achieve the Objective, the consultant will :
Conduct an initial desktop study on the current monitoring and evaluation systems in the Region such as SOE, EVI, MDGs indicators and PRISM. This includes a review of the M&E approaches of bilateral and multilateral aid donors, the GEF and Pacific regional institutions.
Review and confirm data sources and data sets around the region – by country – and their accessibility and availability.
Evaluate the achievement of the Action Strategy targets and outcome indicators across Pacific Island Countries and Territories.
Promote and advise on the Action Strategy and facilitate engagement of PICT agencies and PIRT members in the regional coordination mechanism represented by the Roundtable and the Action Strategy. Specifically this will involve assisting countries (through NBSAPs) to develop their national M&E systems and data-sets to serve the NBSAPs or similar national plans and to assist Roundtable members with their M&E and data collection.
Propose adjustments to the 5-year targets and indicators and develop proposed achievable and measurable 5-year target and indicators for possible amendment of the Action Strategy during its review.
Develop a user friendly sustainable monitoring and evaluation process or mechanism within the Round Table for long term monitoring of progress of the Action Strategy
Provide important input into the 9th Pacific Islands Roundtable Meeting to be held in July 2005, the Roundtable Management Group Meeting to be held in January 2006 and the 8th Pacific Islands Conference on Nature Conservation and Protected Areas which will take place in July 2007.
Provide advice on how this process can assist reporting against relevant areas of the Pacific Plan and possibly tailor relevant indicators for Regional MDG monitoring.
According to the Work Plan, the first four tasks above were targeted in this first Phase of the Assignment.
This report brings to a close Phase One of this Assignment. It comprises the results of the review of current monitoring and evaluation activities of key multilateral donors, selected regional organizations and other M&E systems (ToR 1). It then reports on information management but not on a country level since the information was not forthcoming (ToR 2). The views obtained on the Roundtable and the Action Strategy (ToR 3) through the limited responses available, are covered next and the report concludes with a broad discussion drawing conclusions and recommendations. In view of the poor response to the questionnaire and the need to canvas widely before reaching final conclusions on the attainment of AS targets and indicators, this is seen as only a preliminary report and further consideration will be given in the next Progress Report on completion of Phase Two. The RT and AS were promoted (ToR 4) but the opportunities to date were limited and this will be a continuing activity. Likewise, assistance to countries to develop M&E systems to serve their NBSAP will be provided as requested and as opportunities arise.
1.2 Methodology The work commenced with desk reviews of various documents and including WWW research. Country visits were undertaken to Samoa, Fiji and New Caledonia mainly to RT members, regional organizations, in-country SPREP focal points, NBSAP Coordinators, and Government organizations responsible for environmental management. Electronic consultations were also attempted through an emailed Questionnaire.
1.2.1 Documents and websites reviewed and consulted Publications and other Documents A number of publications and other documents such as unpublished internal or meeting reports, were obtained from the organizations visited. Other documents were downloaded from websites visited following searches conducted as described below. The following documents have been reviewed and/or consulted :
ADB (2002) Technical Assistance to the Islamic Republic of Pakistan for Strengthening Portfolio Performance and Monitoring. Asian Development Bank, Manila.
ADB (2004) Pacific Regional Environmental Strategy, 2005-2009. Vol 1: Strategy Document; Vol 2: Case Studies. Asian Development Bank, Manila.
ADB (2005) Annual Evaluation Report : 2005 Annual Evaluation Review. Asian Development Bank, Manila.
Anon (1998) Guidelines for Monitoring and Evaluation for Biodiversity Projects. Global Environment Division. Informal World Bank publication.
Anon (2004) Action Strategy for Nature Conservation in the Pacific Islands Region, 2003-2007.
Anon (2005) EVI: Description of Indicators – 20 December 2004. SOPAC and UNEP
Beanland, Ruth and Beat Huser (1999) Integrated Monitoring – A Manual for Practitioners. Prepared for Environment Waikato, Hamilton.
Bubb, P, Martin Jenkins and Valeria Kapos (2005) Biodiversity Indicators for National Use – Experience and Guidance. UNEP, Nairobi.
Froude, Victoria (1998) Environmental Performance Indicators : An Analysis of Potential Indicators for Terrestrial Biodiversity. Ministry for the Environment, Wellington.
Froude, Victoria (1998) Environmental Performance Indicators : An Analysis of Potential Indicators for Marine Biodiversity. Ministry for the Environment, Wellington.
Helming, Stefan and Michael Göbel (1997) ZOPP – Objectives-oriented Project Planning. A Planning Guide for New and Ongoing Projects and Programmes. Unit 4 Strategic Corporate Development, Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) GmbH, Eschborn.
Hughes, A.V. (2005) Strengthening Regional Management : A Review of the Architecture for Regional Cooperation in the Pacific. Consultative Draft. Report to the Pacific Islands Forum.
Ministry for the Environment (1996) National Environmental Indicators : Building a Framework for a Core Set. Ministry for the Environment, Wellington.
Ministry for the Environment (1998) Environmental performance Indicators – Proposals for Terrestrial and Freshwater Biodiversity. Ministry for the Environment, Wellington.
Pacific Islands Forum Secretariat (2005) The Pacific Plan – for strengthening regional cooperation and integration. Pacific islands Forum Secretariat, Suva.
Smith, SE and Alejandra Martin (2000) Achieving Sustainability of Biodiversity Conservation – Report of a GEF Thematic Review. Monitoring and Evaluation Working Paper 1. GEF, Washington.
South Pacific Applied Geoscience Commission (2003) SOPAC Pacific Islands Regional Ocean Information System. Project Proposal – prepared for donor countries and organizations. SOPAC, Suva.
South Pacific Community, with UN and CROP (2004) Pacific Islands Regional Millennium Development Goals Report 2004, Goal 7 : Ensure Environmental Sustainability. South Pacific Community, Noumea.
SPREP (2005) Working with Pacific Communities for our Environment. The 2004 Annual Report. Secretariat of the Pacific Regional Environment Programme, Apia.
State Committee for Nature Protection and UNDP (2005) Guideline Principles on the Application of Environmental Indicators to Monitor the State of the Environment in Uzbekistan. State Committee for Nature Protection, Tashkent.
UNDP Evaluation Office (2002) Handbook on Monitoring and Evaluation for Results. UNDP Evaluation Office, New York.
UNDP, UNEP, World Bank & World Resources Institute (2004) World Resources 2002-2004. World Resources Institute, Washington, D.C.
UNEP (2005) UNEP Project Manual: Formulation, Approval, Monitoring and Evaluation 2005. United Nations Environment Programme, Nairobi.
UNEP (CBD - SBSTTA) (2005) Indicators for assessing progress towards, and communicating, the 2010 target at the global level. Paper, Item 5.4 of Provisional Agenda for Tenth Meeting of SBSTTA, Bangkok.
Ward, Trevor, Fanaura Kingstone and Suliana Siwatibau (1999) Indicators of Success for the
South Pacific Biodiversity Conservation Programme. Vol 1 – Technical Report. Draft for Discussion.
Yetter, Carl (undated) Development of Delaware’s Coastal Zone Environmental Indicators Decision Support System. From – http://gis.esri.com/library
Websites Google was used as the search engine. Combinations of keywords such as “monitoring”, “evaluation” and “biodiversity” were used in the initial broad searches. In more specific searches focused on a known organization, the search commenced by entering three keywords : the name or acronym of the known organization, the word “monitoring” and the word “evaluation”. The first 50 links/entries that resulted from the search were then scanned for likely interest and the most relevant sites were visited and followed through. The following websites have been visited and consulted.
Reference website http://gis.esri.com/library
United Nations Statistical Division, Environment Statistics http://unstats.un.org/unsd/environment
South Pacific Regional Environment Programme www.sprep.org.ws
European 2010 Biodiversity Indicators http://biodiversity-chm.eea.eu.int/information/indicator
IUCN Evaluation Office – Methods and Tools http://www.iucn.org/themes/eval/methods.htm
Pacific Islands Forum Secretariat (PIFS) http://www.forumsec.org.fj/
New Zealand Ministry for the Environment http://www.mfe.govt.nz/
1.2.2 Country visits
My participation in Roundtable 9 in Alotau provided me with an excellent opportunity to familiarize myself with the RT membership, establish contacts, confirm my assignment prescription, and present my draft Work Plan for the assignment.
During the planned country/territory visits, the aim is to meet with SPREP Focal Points; in-country representative offices of the consultancy funders; Roundtable members; Coordinators for National Biodiversity Strategy and Action Plans (NBSAP), or equivalents; Ministries and Departments of Environment, Nature Protection, Conservation and related organizations; international, multilateral and bilateral aid organizations; environmental NGOs; research and technical regional organizations; any other potential sources of data/information; and repositories/managers of data and information.
Annex 1 carries the Schedule for the visits to Samoa, Fiji and New Caledonia and Annex 2 carries notes made during the various meetings. Over the 19 days of travel, some 40 meetings were carried out with 48 individuals from the following 32 organizations :
Bilateral donors 3
Multilateral donors 4
Local NGO 2
International NGO 7
Regional Organization 7
It is interesting to note that out of the 48 individuals, only 5 are considered to be either RT members or very closely associated with the RT; while a further 7 are considered to have a slightly more remote association with the RT.
It was not possible to contact some individuals (including RT members) to arrange meetings, and some scheduled meetings did not take place for a variety of reasons.
The original Questionnaire is in Annex 3, and so are the collated responses. The Questionnaire was comprised of four substantive parts as follows –
1 Personal details of the person filing the response and his/her organization
2 Views on the Roundtable and the Action Strategy
3 Information on current Monitoring Programmes
4 Information on existing Databases and Datasets
The objectives of the Questionnaire were to obtain views and opinions on the strengths and weaknesses of the Roundtable and the Action Strategy, and record current monitoring and information management activities.
The first batch of questionnaires was sent out to 102 persons and included all those who had attended the Alotau meeting of the Roundtable, all NBSAP Coordinators, all individuals met during my visits to Apia, Suva and Noumea and some other individuals who were known to the Roundtable and expected to have an interest in this exercise. By the target response date, only one response had been received so the target date was extended and all those on the list were circulated once again. Some more responses trickled in, especially when reinforced by an email from Kate Brown exhorting all to reply. Some said they had been travelling and apologized for the late response; others needed the questionnaire sent again since they had misplaced the original email; but the majority were totally silent. In the event, 21 responses were received and these came from :
International Organizations 4
NGOs (international and domestic) 6
Independent Consultant 1
Countries/Territories 7 (2 countries and NC which contributed 5 replies)
Core members of the Roundtable contributed two replies; while those considered in the outer circle of RT membership contributed a further five replies.
The areas of interest/responsibility of the home organizations of respondents ranged across a number of disciplines as follows :
Biodiversity protection and species management 16
Nature conservation, protected areas management 13
Nature conservation monitoring 10
Data/Information and Databases management 13
Sustainable development planning 13
Land use planning and management 4
Coastal zone planning and management 9
Other areas of interest/responsibility were :
Governance, community development, livelihoods, health, disaster preparedness, community resource management
Mapping, remote sensing and GIS
Community development & livelihood support
Teaching Environmental Management and Conservation
Research on Conservation-related activities
Economic Management and Planning, Rural Development, Agriculture, Hazard Risk Management
To reduce poverty and hunger by improving fisheries and aquaculture (includes sustainable resource use)
2 RESULTS : INFORMATION MANAGEMENT In an effort to review and confirm data sources and data sets around the region, by country, and their accessibility and availability, as required by the Terms of Reference, a Questionnaire was sent out as detailed above. The Questionnaire respondents identified 19 Environmental Monitoring Programmes and reported 15 Databases. However, 9 (43%) respondents did not provide a reply on monitoring programmes and 2 made reference to other potential sources of information; 8 (38%) did not reply to the question on databases.
2.1 M&E Approaches of Selected Donors and Regional Organizations
2.1.1 Global Environment Facility (GEF) The GEF has a strong commitment to monitoring and evaluation. As noted in its Monitoring and Evaluation Policies and Procedures2, the purposes of the GEF M&E are “to monitor, evaluate, and disseminate GEF project-related information and lessons on: the performance of projects as well as adequacy of policies and procedures; the changes in country capacities for addressing global environmental issues; the changes in policies affecting the global environment; the global environmental benefits of projects and programs; and the adequacy of GEF guidelines and procedures on project cycle management”. The GEF has identified five specific monitoring and evaluation criteria which are the following :