Review of monitoring and evaluation systems and information management and preliminary evaluation of rountable and action strategy targets and indicators

Download 0.68 Mb.
Size0.68 Mb.
  1   2   3   4   5   6   7   8   9   10


Monitoring and Evaluation Initiative



Philip Tortell, Consultant

January 2006

Acknowledgements and Invitation
This report represents a blend of what has been gleaned from various documents and websites together with the results of consultations both electronic and during country visits. I am grateful to all those who met with me during the country visits, replied to the questionnaire and shared with me their views and opinions or helped with obtaining information for me. I also wish to thank all those kind people who made travel arrangements, set up appointments and helped in any other way to ensure the effectiveness of my country visits. I wish to make particular mention of Ms Kate Brown-Vitolio of SPREP who has spared no effort to ensure that my assignment is as successful as it can be.
While reflecting what I have been told and what I have found out, this report is ultimately my own perception of the situation I am reporting on and does not represent the views or opinions of the Roundtable or any of its constituent organizations.

The report is meant to generate discussion and I would sincerely welcome comments and reactions to what it says and what it contains.

Philip Tortell, Consultant

Environmental Management Limited

PO Box 27433 Wellington, NEW ZEALAND

Tel +64-4-384 4133, Fax +64-4-384 4022, Email


1.1 Background 4

1.2 Methodology 5

1.2.1 Documents and websites reviewed and consulted 5

1.2.2 Country visits 7

1.2.3 Questionnaire 8


2.1 M&E Approaches of Selected Donors and regional organizations 9

2.1.1 Global Environment Facility (GEF) 9

2.1.2 United Nations Development Programme (UNDP) 10

2.1.3 United Nations Environment Programme (UNEP) 11

2.1.4 The World Bank 11

2.1.5 Asian Development Bank (ADB) 12

2.1.6 Secretariat of the Pacific Regional Environment Programme (SPREP) 12

2.1.7 Secretariat of the Pacific Community (SPC) 13

2.1.8 Pacific Islands Forum Secretariat (PIFS) 13

2.1.9 South Pacific Applied Geoscience Commission (SOPAC) 14

2.1.10 Council of Regional Organizations in the Pacific (CROP) 14

2.2 Other Regional Approaches to M&E 15

2.2.1 Environmental Vulnerability Index (EVI) 15

2.2.2 Millennium Development Goals (MDGs) 16

2.2.3 Pacific Regional Information System (PRISM) 17

2.2.4 The Pacific Plan 17

2.3 National and Regional Environmental Monitoring Systems 18

2.4 National and Regional Environmental Databases 20


3.1 Awareness and familiarity with Roundtable 23

3.2 Perceived strengths of the Roundtable 23

3.3 Perceived weaknesses of the Roundtable 24

3.4 The Working Groups of the Roundtable 25

3.5 Relationship between the Roundtable and the Governments/Administrations in the region 26

3.6 Relationship between the Roundtable and SPREP 27

4.1 Goals 29

4.2 Objectives and Targets 30

4.3 Indicators 31

4.3.1 Indicators and the Roundtable 31

4.3.2 Guidance on developing indicators of success 32

4.3.3 Outline plan for developing a suite of indicators for the Roundtable 36

5.1 Information management 37

5.2 The Roundtable 38

5.3 The Action Strategy 39


Annex 1 Travel Schedule

Annex 2 Edited Notes from meetings in Apia, Suva and Noumea

Annex 3 Questionnaire and Collated Replies


1.1 Background

The Objective of this Assignment is to develop a monitoring and evaluation process for the Action Strategy for Nature Conservation for the Pacific Islands Region1 and use it to analyse the achievement of the 2003-07 targets across the region.
The Assignment involves the review of a number of key documents, extensive consultations (including at country level), and the design of an Environmental Indicators System operationalized at country level. The consultant is also required to assume the lead reporting/facilitating role in a number of regional meetings over the 2.5 years of the consultancy. The legacy of the Assignment will be an enduring, simple, but effective monitoring system.
The Terms of Reference go on to say that to achieve the Objective, the consultant will :

  1. Conduct an initial desktop study on the current monitoring and evaluation systems in the Region such as SOE, EVI, MDGs indicators and PRISM. This includes a review of the M&E approaches of bilateral and multilateral aid donors, the GEF and Pacific regional institutions.

  2. Review and confirm data sources and data sets around the region – by country – and their accessibility and availability.

  3. Evaluate the achievement of the Action Strategy targets and outcome indicators across Pacific Island Countries and Territories.

  4. Promote and advise on the Action Strategy and facilitate engagement of PICT agencies and PIRT members in the regional coordination mechanism represented by the Roundtable and the Action Strategy. Specifically this will involve assisting countries (through NBSAPs) to develop their national M&E systems and data-sets to serve the NBSAPs or similar national plans and to assist Roundtable members with their M&E and data collection.
  5. Propose adjustments to the 5-year targets and indicators and develop proposed achievable and measurable 5-year target and indicators for possible amendment of the Action Strategy during its review.

  6. Develop a user friendly sustainable monitoring and evaluation process or mechanism within the Round Table for long term monitoring of progress of the Action Strategy

  7. Provide important input into the 9th Pacific Islands Roundtable Meeting to be held in July 2005, the Roundtable Management Group Meeting to be held in January 2006 and the 8th Pacific Islands Conference on Nature Conservation and Protected Areas which will take place in July 2007.

  8. Provide advice on how this process can assist reporting against relevant areas of the Pacific Plan and possibly tailor relevant indicators for Regional MDG monitoring.

According to the Work Plan, the first four tasks above were targeted in this first Phase of the Assignment.

This report brings to a close Phase One of this Assignment. It comprises the results of the review of current monitoring and evaluation activities of key multilateral donors, selected regional organizations and other M&E systems (ToR 1). It then reports on information management but not on a country level since the information was not forthcoming (ToR 2). The views obtained on the Roundtable and the Action Strategy (ToR 3) through the limited responses available, are covered next and the report concludes with a broad discussion drawing conclusions and recommendations. In view of the poor response to the questionnaire and the need to canvas widely before reaching final conclusions on the attainment of AS targets and indicators, this is seen as only a preliminary report and further consideration will be given in the next Progress Report on completion of Phase Two. The RT and AS were promoted (ToR 4) but the opportunities to date were limited and this will be a continuing activity. Likewise, assistance to countries to develop M&E systems to serve their NBSAP will be provided as requested and as opportunities arise.

1.2 Methodology
The work commenced with desk reviews of various documents and including WWW research. Country visits were undertaken to Samoa, Fiji and New Caledonia mainly to RT members, regional organizations, in-country SPREP focal points, NBSAP Coordinators, and Government organizations responsible for environmental management. Electronic consultations were also attempted through an emailed Questionnaire.

1.2.1 Documents and websites reviewed and consulted
Publications and other Documents
A number of publications and other documents such as unpublished internal or meeting reports, were obtained from the organizations visited. Other documents were downloaded from websites visited following searches conducted as described below. The following documents have been reviewed and/or consulted :

  • ADB (2002) Technical Assistance to the Islamic Republic of Pakistan for Strengthening Portfolio Performance and Monitoring. Asian Development Bank, Manila.

  • ADB (2004) Pacific Regional Environmental Strategy, 2005-2009. Vol 1: Strategy Document; Vol 2: Case Studies. Asian Development Bank, Manila.

  • ADB (2005) Annual Evaluation Report : 2005 Annual Evaluation Review. Asian Development Bank, Manila.
  • Anon (1998) Guidelines for Monitoring and Evaluation for Biodiversity Projects. Global Environment Division. Informal World Bank publication.

  • Anon (2004) Action Strategy for Nature Conservation in the Pacific Islands Region, 2003-2007.

  • Anon (2005) EVI: Description of Indicators – 20 December 2004. SOPAC and UNEP

  • SPREP, Apia.

  • Beanland, Ruth and Beat Huser (1999) Integrated Monitoring – A Manual for Practitioners. Prepared for Environment Waikato, Hamilton.

  • Bubb, P, Martin Jenkins and Valeria Kapos (2005) Biodiversity Indicators for National Use – Experience and Guidance. UNEP, Nairobi.

  • Froude, Victoria (1998) Environmental Performance Indicators : An Analysis of Potential Indicators for Terrestrial Biodiversity. Ministry for the Environment, Wellington.

  • Froude, Victoria (1998) Environmental Performance Indicators : An Analysis of Potential Indicators for Marine Biodiversity. Ministry for the Environment, Wellington.

  • Global Environment Facility (2002) Monitoring and Evaluation Policies and Procedures. Global Environment Facility, Washington.

  • Helming, Stefan and Michael Göbel (1997) ZOPP – Objectives-oriented Project Planning. A Planning Guide for New and Ongoing Projects and Programmes. Unit 4 Strategic Corporate Development, Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) GmbH, Eschborn.
  • Hughes, A.V. (2005) Strengthening Regional Management : A Review of the Architecture for Regional Cooperation in the Pacific. Consultative Draft. Report to the Pacific Islands Forum.

  • Ministry for the Environment (1996) National Environmental Indicators : Building a Framework for a Core Set. Ministry for the Environment, Wellington.

  • Ministry for the Environment (1998) Environmental performance Indicators – Proposals for Terrestrial and Freshwater Biodiversity. Ministry for the Environment, Wellington.

  • Pacific Islands Forum Secretariat (2005) The Pacific Plan – for strengthening regional cooperation and integration. Pacific islands Forum Secretariat, Suva.

  • Smith, SE and Alejandra Martin (2000) Achieving Sustainability of Biodiversity Conservation – Report of a GEF Thematic Review. Monitoring and Evaluation Working Paper 1. GEF, Washington.

  • South Pacific Applied Geoscience Commission (2003) SOPAC Pacific Islands Regional Ocean Information System. Project Proposal – prepared for donor countries and organizations. SOPAC, Suva.

  • South Pacific Community, with UN and CROP (2004) Pacific Islands Regional Millennium Development Goals Report 2004, Goal 7 : Ensure Environmental Sustainability. South Pacific Community, Noumea.

  • SPREP (2005) Working with Pacific Communities for our Environment. The 2004 Annual Report. Secretariat of the Pacific Regional Environment Programme, Apia.

  • State Committee for Nature Protection and UNDP (2005) Guideline Principles on the Application of Environmental Indicators to Monitor the State of the Environment in Uzbekistan. State Committee for Nature Protection, Tashkent.
  • UNDP Evaluation Office (2002) Handbook on Monitoring and Evaluation for Results. UNDP Evaluation Office, New York.

  • UNDP, UNEP, World Bank & World Resources Institute (2004) World Resources 2002-2004. World Resources Institute, Washington, D.C.

  • UNEP (2005) UNEP Project Manual: Formulation, Approval, Monitoring and Evaluation 2005. United Nations Environment Programme, Nairobi.

  • UNEP (CBD - SBSTTA) (2005) Indicators for assessing progress towards, and communicating, the 2010 target at the global level. Paper, Item 5.4 of Provisional Agenda for Tenth Meeting of SBSTTA, Bangkok.

  • Ward, Trevor, Fanaura Kingstone and Suliana Siwatibau (1999) Indicators of Success for the

South Pacific Biodiversity Conservation Programme. Vol 1 – Technical Report. Draft for Discussion.

  • Yetter, Carl (undated) Development of Delaware’s Coastal Zone Environmental Indicators Decision Support System. From –

Google was used as the search engine. Combinations of keywords such as “monitoring”, “evaluation” and “biodiversity” were used in the initial broad searches. In more specific searches focused on a known organization, the search commenced by entering three keywords : the name or acronym of the known organization, the word “monitoring” and the word “evaluation”. The first 50 links/entries that resulted from the search were then scanned for likely interest and the most relevant sites were visited and followed through. The following websites have been visited and consulted.

  • Reference website

  • United Nations Statistical Division, Environment Statistics

  • South Pacific Regional Environment Programme

  • European 2010 Biodiversity Indicators

  • IUCN Evaluation Office – Methods and Tools


  • Samoa Biodiversity Database

  • IPPC Portal – Phytosanitary Metadatabase

  • Niue Biodiversity Website

  • WRI Earthtrends: The Environmental Information Portal - Country Profiles - Biodiversity

  • USP/Forum/SPC/SOPAC Pacific Islands Marine Resources Information System

  • WorldFish Centre

  • The SPC MDGs website

  • The Pacific Regional Information System (PRISM)

  • The GEF Monitoring and Evaluation policies and procedures

  • The UNDP Evaluation Office

  • The UNDP Evaluation Office
  • The UNEP World Conservation Monitoring Centre

  • The World Atlas of Biodiversity

  • The CROP Charter

  • CROP Home page

  • The Pacific Plan

  • World Bank, MDGs

  • The UNEP Programme Coordination and Management Unit

  • The WCMC Proteus Project

  • The World Bank Independent Evaluation Group

  • The ADB Operations Evaluation Department

  • ADB Website – News & Events - ADB's Support for Monitoring and Evaluation of Development Projects

  • Environmental Vulnerability Index


  • Pacific Islands Forum Secretariat (PIFS)

  • New Zealand Ministry for the Environment

1.2.2 Country visits

My participation in Roundtable 9 in Alotau provided me with an excellent opportunity to familiarize myself with the RT membership, establish contacts, confirm my assignment prescription, and present my draft Work Plan for the assignment.

During the planned country/territory visits, the aim is to meet with SPREP Focal Points; in-country representative offices of the consultancy funders; Roundtable members; Coordinators for National Biodiversity Strategy and Action Plans (NBSAP), or equivalents; Ministries and Departments of Environment, Nature Protection, Conservation and related organizations; international, multilateral and bilateral aid organizations; environmental NGOs; research and technical regional organizations; any other potential sources of data/information; and repositories/managers of data and information.
Annex 1 carries the Schedule for the visits to Samoa, Fiji and New Caledonia and Annex 2 carries notes made during the various meetings. Over the 19 days of travel, some 40 meetings were carried out with 48 individuals from the following 32 organizations :
Bilateral donors 3

Multilateral donors 4

Government 9

Local NGO 2

International NGO 7

Regional Organization 7

It is interesting to note that out of the 48 individuals, only 5 are considered to be either RT members or very closely associated with the RT; while a further 7 are considered to have a slightly more remote association with the RT.
It was not possible to contact some individuals (including RT members) to arrange meetings, and some scheduled meetings did not take place for a variety of reasons.

1.2.3 Questionnaire

The original Questionnaire is in Annex 3, and so are the collated responses. The Questionnaire was comprised of four substantive parts as follows –

1 Personal details of the person filing the response and his/her organization

2 Views on the Roundtable and the Action Strategy

3 Information on current Monitoring Programmes

4 Information on existing Databases and Datasets

The objectives of the Questionnaire were to obtain views and opinions on the strengths and weaknesses of the Roundtable and the Action Strategy, and record current monitoring and information management activities.
The first batch of questionnaires was sent out to 102 persons and included all those who had attended the Alotau meeting of the Roundtable, all NBSAP Coordinators, all individuals met during my visits to Apia, Suva and Noumea and some other individuals who were known to the Roundtable and expected to have an interest in this exercise. By the target response date, only one response had been received so the target date was extended and all those on the list were circulated once again. Some more responses trickled in, especially when reinforced by an email from Kate Brown exhorting all to reply. Some said they had been travelling and apologized for the late response; others needed the questionnaire sent again since they had misplaced the original email; but the majority were totally silent. In the event, 21 responses were received and these came from :

  • International Organizations 4

  • NGOs (international and domestic) 6

  • USP 3

  • Independent Consultant 1

  • Countries/Territories 7 (2 countries and NC which contributed 5 replies)

Core members of the Roundtable contributed two replies; while those considered in the outer circle of RT membership contributed a further five replies.

The areas of interest/responsibility of the home organizations of respondents ranged across a number of disciplines as follows :

  • Biodiversity protection and species management 16
  • Nature conservation, protected areas management 13

  • Nature conservation monitoring 10

  • Data/Information and Databases management 13

  • Sustainable development planning 13

  • Land use planning and management 4

  • Coastal zone planning and management 9

Other areas of interest/responsibility were :

  • Governance, community development, livelihoods, health, disaster preparedness, community resource management

  • Mapping, remote sensing and GIS

  • Fisheries management

  • Community development & livelihood support

  • Teaching Environmental Management and Conservation

  • Research on Conservation-related activities

  • Economic Management and Planning, Rural Development, Agriculture, Hazard Risk Management

  • To reduce poverty and hunger by improving fisheries and aquaculture (includes sustainable resource use)

In an effort to review and confirm data sources and data sets around the region, by country, and their accessibility and availability, as required by the Terms of Reference, a Questionnaire was sent out as detailed above. The Questionnaire respondents identified 19 Environmental Monitoring Programmes and reported 15 Databases. However, 9 (43%) respondents did not provide a reply on monitoring programmes and 2 made reference to other potential sources of information; 8 (38%) did not reply to the question on databases.

2.1 M&E Approaches of Selected Donors and Regional Organizations

2.1.1 Global Environment Facility (GEF)
The GEF has a strong commitment to monitoring and evaluation. As noted in its Monitoring and Evaluation Policies and Procedures2, the purposes of the GEF M&E are “to monitor, evaluate, and disseminate GEF project-related information and lessons on: the performance of projects as well as adequacy of policies and procedures; the changes in country capacities for addressing global environmental issues; the changes in policies affecting the global environment; the global environmental benefits of projects and programs; and the adequacy of GEF guidelines and procedures on project cycle management”. The GEF has identified five specific monitoring and evaluation criteria which are the following :

Share with your friends:
  1   2   3   4   5   6   7   8   9   10

The database is protected by copyright © 2019
send message

    Main page