Assessment and Learning that Values the ‘Golden Goose’
Cutting across these key players are action points in which all have roles to play to carry forward the challenge of assessing and learning that strengthens social change. Many questions are still on the table calling for more precise insights, nitty-gritty dilemmas remain to be thrashed out, and thus practical hands-on work needs to be undertaken. The principles outlined in this document need to be fleshed out for different organisational set-ups, capacity levels and social change processes. For all those involved – activists, intermediaries, evaluators, donors – generating practical ideas and sharing inspiring examples is essential. This means investing in:
Concrete efforts to systematise and review the respective benefits and limitations of different grounded case studies that have enabled critically reflective learning and assessment.
Training efforts for social change organisations around the idea of how to assess social change, based on existing stock of experiences and approaches plus recognition of core non-negotiable principles and purposes.
Peer support opportunities for those in social change organisations to ask and receive ideas for addressing dilemmas and challenges on assessment and learning processes.
Seeding experimentation with particular combinations of approaches and methods with detailed documentation of the processes.
A new model of assessment and learning is needed that places developmental social change at the heart, rather than myopically focusing on the interim steps. SPARC refers to development as ‘the golden goose’ (Patel 2007) and urges a model of assessment and learning that places the goose at the centre, rather than its golden eggs. Assessing and learning about development as a process of social change means charting the ‘golden eggs’ that can be discerned, in the form of processes that multiply and serve increasing numbers, building capacities and provoking shifts of thinking in government as well as among the poor. However, by valuing only the eggs, the goose is in danger of serious neglect. Sheela Patel cautions: “With few insights about how to understand it and measure its level of maturity and sustainability, external assessment processes are too rigid to understand these dynamics. Sadly, the goose is often killed due to lack of understanding”. A model of assessment and learning that builds on the reflections in this document would be more effective at strengthening social change that tackles the persisting injustices about which all of and everyone in development should, in theory, be concerned.
ASC. 2005. Second Round of Discussions on Theme 1 - ‘Understanding Social Change’.Assessing Social Change E-discussion, April 2005.
ASC. 2006. Summary of Discussions on Theme 3 - Understanding and dealing with different actors in the assessment processes (ourselves and funding agencies). Assessing Social Change E-discussion, May 2006.
Batliwala, S. 2006. Measuring Social Change: Assumptions, myths and realities. Alliance. 11:1 (12-14).
Batliwala, S. undated. From Evaluation to Learning in Social Change. The challenges of “Measuring Development, Holding Infinity”. http://www.justassociates.org/bio.htm#srilatha
Burke, B. 1998. Evaluating for a change: Reflections on participatory methodology. In: Understanding and practicing participatory evaluation. E. Whitmore. San Francisco, Jossey-Bass. 80: 43-55.
Byrne, A. D. Gray-Felder, J. Hunt and W. Parks. 2005. Measuring Change: A Guide to Participatory Monitoring and Evaluation of Communication for Social Change. Communication for Social Change Consortium, London. http://www.communicationforsocialchange.org/pdf/measuring_change.pdf (16 pages)
Church, M. M. Bitel, K. Armstrong, P. Fernando, H. Gould, S. Joss, M. Marwaha-Diedrich, A. L. de la Torre and C. Vouhé. 2002. Participation, Relationships and Dynamic Change: New Thinking on Evaluating the Work of International Networks. Working Paper No. 121. Development Planning Unit, University College London. http://www.ucl.ac.uk/dpu/publications/working%20papers%20pdf/WP121.pdf
Coates, B. and R. David. 2003. Learning for change: The art of assessing the impact of advocacy work. In: Development and the Learning Organisation. 373-387. J. Pettit, L. Roper and D. Eade (Eds). Oxfam, Oxford.
Dlamini, N. 2006. Transparency of process. monitoring and evaluation in learning organisations. CDRA Annual Report 2005-2006. Community Development Resource Association, Cape Town. Pages 20-31. http://www.cdra.org.za/AnnualRep/CDRA%20Annual%20Report%202006.pdf
Earl, S., F. Carden, and T. Smutylo. 2001. Outcome mapping: Building learning and reflection into development programs. Ottawa, IDRC. http://www.idrc.ca/openebooks/959-3/
Edwards, M. and D. Hulme (Eds.). 1995. Beyond the Magic Bullet: NGO Performance and Accountability in the Post-Cold War World. Kumarian Press, New York.
Estrella, M., J. Blauert, D. Campilan, J. Gaventa, J. Gonsalves, I. Guijt, D. Johnson and R. Ricafort (Eds). 2000. Learning from change: Issues and experiences in participatory monitoring and evaluation. Intermediate Technology Publications, London.
Eyben R., T. Kidder, A. Bronstein and J. Rowlands. Forthcoming (2007). Thinking about Change for Development Practice: A Case Study from Oxfam GB. Development in Practice.
Eyben R. and S. Ladbury. 2006. Building Effective States: Taking a Citizen’s Perspective’. http://18.104.22.168/docs/publications/drc_general/Report/drccitizensperspective.pdf. Institute of Development Studies, United Kingdom.
Foresti, M., A. Evans, and A. Hudson. 2006. Evaluation of Citizens’ Voice and Accountability: Evaluation Design and Development Framework. Project Inception Report. Overseas Development Institute, London.
Gasper, D. 1997. Logical Frameworks: A Critical Assessment. Working Paper 264. ISS, The Hague.
Gaventa, J. 2005. Reflections on the Uses of the ‘Power Cube’ Approach for Analyzing the Spaces, Places and Dynamics of Civil Society Participation and Engagement. CFP evaluation series 2003-2006: no 4. MBN Secretariat, The Hague. http://www.partos.nl/uploaded_files/13-CSP-Gaventa-paper.pdf
Guijt, I. 2007. Critical Readings on Assessing and Learning for Social Change. A Review. Institute of Development Studies (UK) and Learning by Design (Netherlands). http://www.ids.ac.uk/ids/Part/proj/socialchange.html
Guijt, I. (Ed). 2007. Negotiating Learning. Collaborative Monitoring for Forest Resource Management. Resources for the Future, Washington DC.
Guijt, I. 2007. Facilitating Learning for Social Change: Ways Forward for ICCO’s Support. Insights and ideas emerging from discussions with the Steering Group, December 13-15, 2006. Internal report. ICCO, the Netherlands.
Guijt, I. 2004. ALPS in Action: A Review of the Shift in ActionAid towards a new Accountability, Learning and Planning System. Report written for ActionAid International. July 2004.
Hope, A. and S. Timmel. 2000. Training for Transformation: a handbook for community workers. Practical Action, London.
IIRR. 1998. Participatory Monitoring and Evaluation: Experiences and Lessons. Workshop Proceedings. International Institute of Rural Reconstruction, Cavite, Philippines.
Jordan, L. and P. van Tuijl. (Eds). 2006. NGO Accountability. Politics, Principles and Innovations. Earthscan Publications, London.
Joseph, S. 2005. Rwanda Ubudehe. Local Collective Action. Update June 2005.
Just Associates, 2007. Making Change Happen 3: Power. Concepts for Revisioning Power for Justice, Equality and Peace. Just Associates, Washington DC. http://www.justassociates.org/index_files/MCH3red.pdf .
Mosedale, S. 2005. Assessing Women’s Empowerment: Towards A Conceptual Framework. Journal of International Development 17, 243–257.
Mott, A. 2003. Strengthening Social Change Through Assessment and Organizational Learning. Report on The Gray Rocks Conference - An International Exchange with Grant Support from The Ford Foundation. September 2003. Community Learning Project, Washington DC. http://www.communitylearningproject.org/docs/Gray%20Rocks%20Conference%20Report.pdf
Mwasuru, M. 2007. Assessing Social Change Through Participatory Action Research: The Case of Kasighau Small-Scale Miners. Case study produced for the ‘Assessing Social Change’ initiative. http://www.ids.ac.uk/ids/Part/proj/socialchange.html.
Pankaj, V. 2007. Measuring Social Change: Lessons from the Field. June 2007. Transforming Evaluation II (2). http://www.innonet.org/client_docs/File/newsletter/2007-06/transforming_evaluation_june2007.pdf
Patel, S. 2007. Reflections on Innovation, Assessment and Social Change. A SPARC case study. Case study produced for the ‘Assessing Social Change’ initiative. http://www.ids.ac.uk/ids/Part/proj/socialchange.html
Reeler, D. 2007. A Theory of Social Change and Implications for Practice, Planning, Monitoring and Evaluation. Community Development Resource Association, Cape Town, South Africa.
Ringsing, B. and C. Leeuwis. Forthcoming 2007. Learning about advocacy: A case study of challenges, everyday practices and tensions. Submitted and revised for Evaluation.
Reilly, M. 2007. An Agenda for Change n the USA: Insights from a Conversation about Assessing Social Change in Washington, DC. Case study produced for the ‘Assessing Social Change’ initiative. http://www.ids.ac.uk/ids/Part/proj/socialchange.html
Samba, E. 2007. Sauti Ya Wanawake. The Role of Reflection in Women’s Social Change Work. Case study produced for the ‘Assessing Social Change’ initiative. http://www.ids.ac.uk/ids/Part/proj/socialchange.html
Schmelzle, B. 2005. New Trends in Peace and Conflict Impact Assessment. In: Dialogue Series No. 4: New Trends in Peace and Conflict Impact Assessment (PCIA).
D. Bloomfield, M. Fischer and B. Schmelzle (eds.) 2005. http://www.berghof-handbook.net/std_page.php?LANG=e&id=24&parent=5
Sjöblom, L. 2006. Transcript of presentations and discussions 29th-31st August 2006. International Dialogue Conference. After the Paris Declaration.
Wilson-Grau, R. and M. Nunez. 2007. Evaluating International Social Change Networks: A Conceptual Framework for a Participatory Approach. Development in Practice. 17 (2): 258-271.
Win, E. 2006. ALPS and Change. PowerPoint Presentation by EverjoiceWin, ActionAid International, Zimbabwe given at Money and Movements International Meetings, November 9-11, 2006, Mexico. http://www.awid.org/go.php?pg=mm_resources (PPT)
Woodhill, J. 2007 (in press). M&E as learning: Rethinking the dominant paradigm. In: J. de Graaf et al., eds. Monitoring and Evaluation of Soil Conservation and Watershed Development Projects. World Association of Soil and Water Conservation
Zuidberg, L., E. van Walsum, and M. Nugteren. 2006. Final Report. Lot 7 Gender Study for Evaluation of the Theme-based Co-financing Programme. Ref. 2005 / S 4-003592. Volume 1: Main Report. 2nd Draft.
1 Mott 2003, p5.
2 This paragraph is based on Woodhill, forthcoming 2007.
3 An emergent property becomes apparent when several simple entities or processes operate in an environment but form more complex behaviours as a collective. Certain properties emerge that the entities/processes do not have themselves.
4 See http://www.kenyalink.org/sucam/
5 This section is largely based on a note written by C. Clark, V. Miller. S. Musyoki and L. VeneKlasen ‘Theme 2 Part One: Methods, Tools and Processes for Assessing Social Change’ to kick start the second thematic discussion of the ASC group.
8 ‘Complex’ is not the same as ‘complicated’. A complex system has many elements that can interact with each other and their environment. Complex systems display a level of organization without any external organizing principle being applied. Part of the system may be altered and the system may still be able to function. In complicated systems, parts have to work in unison to accomplish a function. A key defect in a critical part brings the entire system to a halt.
9 See http://www.ifad.org/evaluation/guide/annexd/index.htm for 34 M&E methods.
10 See http://www.idrc.ca/evaluation.
12 See http://www.philodialogue.com/dialogue.htm.
13 With thanks to Iñigo Retoloza Eguren for these insights.