Building recognition of these challenges into social change processes is neither an easy task nor one that is undertaken particularly well. Allowing these features to shape practice is not undertaken systematically or often enough. Those involved in the ASC discussions described current practice as ‘ad hoc’.
The challenge of balancing predictability and flexibility is one of the main issues for organisations involved in delivering interventions. Some say that social change cannot be predicted (cf Reeler 2007) and may be counter-developmental. Others predict too much and request clear lines of cause and effect. ask for a perfectly linear, logical, cause and effect description of predicted social change. Ritu Shroff suggests that what appears to work best is facilitating communities themselves to articulate desired changes and anticipate the drivers and inhibitors of such changes. The most effective individuals engaged in social change efforts are those with in-depth contextual understanding and highly skilled and motivated to facilitate and listen. Effective interventions always demonstrate much nimbleness and flexibility, she stresses.
Stereotypical and simplistic assumptions can hinder the emergence of a more appropriate and integrated use of assessment and learning in social change contexts. On the one hand are activists keen to act and do, who consider stopping to reflect a relative waste of time (Reilly 2007). On the other hand, are those who may appreciate the merits of reflection but are embedded in conventional development and M&E thinking and do not understand how to (or want to) create reflective processes within a social change paradigm. Both groups stereotype assessment as either a vague, non-threatening, learning jaunt or as an excessively scientific, objective, numbers-driven process. Marta Foresti, ASC group member, urges a return to a basic understanding: “What about viewing assessment as ‘formulating a responsible judgement’ or even a ‘plausible explanation’ of why what happens actually happens, why it does not, why it does not happen in any other way?”. Another myth that scares organisations into resorting to stereotypes and going overboard in terms of rejecting or uncritically accepting donor-driven M&E is a perception of donors sitting somewhere waiting for ‘the verdict’ so that they can take money away. The reality is that money is usually not (re)allocated based on genuine efforts to assess social change but for other reasons. Hence there may well be more room for manoeuvre and proposing an alternative assessment framework than many might think.
Rethinking how assessment and learning should happen does not have to be complex. Batliwala (undated) suggests four questions to help transform existing approaches. First, is the process involving and empowering the desired constituencies? How is it changing personal frameworks about development and practice? What is the new learning about change that it produces? And finally, how are is learning being transformed into new theory/knowledge?
Notwithstanding the simplicity of such guiding questions, many factors will affect the value that assessment efforts can bring to change processes. The quality of planning that has gone into the effort being assessed and not reducing complex processes to a series of activities will facilitate or hinder subsequent reflections on implementation. The quality of critical thinking, group leadership and facilitation that guides the learning process is, of course, paramount. The overall organisational or group context and dynamics determines whether reflection, learning, and transparency are fostered. Is there personal commitment and passion for learning, and what about the necessary skills and preparation of those facilitating the process and using assessment methods? The cultural appropriateness and accessibility of methods and concepts will also determine how engaged local people can be. The question of who is asking for the evaluation must be considered. Is it the government, overseas donors, grassroots groups themselves? This influences the purpose, process and methods can and will be used. These factors are critical for effective assessment of social change efforts and must be considered in designing an appropriate process (see next section).
Frameworks, Concepts and Methods: Towards a Purpose-built Assessment Process
When confronted with the limitations of existing mainstream M&E approaches, many in the development sector seek solace in methodological alternatives. They hope that somewhere there is an approach that will overcome the paradigmatic tensions, enable clarity of analysis, prove effectiveness, and strengthen people’s organisations. While there are some relatively innovative approaches emerging, assessment and learning requires more than a method. To be effective, frameworks, values and skills need to merge with a method to construct an appropriate assessment process. This section discusses the elements needed to create a purpose-built assessment and learning process.
The Emergence of Alternative M&E Practice5
The spread and evolution of participatory approaches in development such as Participatory Rural Appraisal6, participatory education, political theatre and REFLECT7, has contributed to the interest and desire to move beyond appraisal and planning to use participatory methods for monitoring and evaluation. The generic term used for these methods is Participatory Monitoring and Evaluation (PM&E). The term and processes of PM&E became very popular in the mid 1990s and are still in use, although similar but less known processes have been used for years. Compared to mainstream M&E thinking, more participatory forms of M&E aim to find to engage multiple stakeholders on many sides of the development process (donors, communities, governments, etc.) in participating in and implementing M&E processes, separately and collaboratively (Estrella et al 1997, Burke 1998, Byrne 2005).
Increased attention to and experience by some large and influential development organisations with more participatory forms of assessment and learning have helped to draw attention from a purely upward accountability orientation to a more two-way process. It is a significant shift and challenge for large development organisations to consider their own accountability to less powerful stakeholders, such as marginalized people. However, in practice, few have yet taken far-reaching examples, with ActionAid International (AAI) still being almost the only international NGO cited as implementing the practice. Nevertheless, discussions about the importance of downward accountability are contributing to the methodological opening up of the M&E field, including in Oxfam Great Britain, which has now designed a new learning and assessment approach. This has stimulated changes, such as greater emphasis on assessment that fosters learning and capacity building so as to equip stakeholders and help them function better
As social change groups, particularly those involved in rights-based initiatives, begin to grapple with issues of power more directly as part of a repoliticisation trend in development thinking, they are also looking beyond conventional tools and techniques to the experiences of social movements over the years. For example, an often less known source for implementing and assessing social change is popular education, which has influenced much participatory education work. This methodology for promoting critical and collective consciousness and thus, a link between new understanding and action, was first developed in the 60s and 70s, and has been adapted over the years to deal with identity and private dimensions of power. As part of the renewed interest in more effective ways of understanding and measuring social change, a rediscovery of popular education in recent years is evident as a way of unpacking the change process and thus defining how to assess it.
The recent interest and engagement of many development organisations in rights and social justice work has at least advanced the demand for innovations. There has been an increased request to monitor and evaluate more fundamental, abstract development concepts such as ‘empowerment’ (Mosedale 2005), ‘voice and accountability’ (Foresti et al 2007), ‘power’ (Gaventa 2005), and ‘realization of rights’, among others. But while the development field has tried and tested methods for the M&E of mainstream development, none seem fully adequate or appropriate for assessing social change in terms of such concepts. This has created much frustration by those using existing methodological options that were developed primarily for tangible, countable development changes.
Despite the growing demand for alternatives and increasing attempts to develop new methods or approaches processes, innovations are needed to meet three specific needs. The ASC discussions indicated that three types of methods are critical: those that help to embed assessment within the social change process; those that assess social change and citizen participation as a rights-based process, and those that encourage critical reflection. Central to the latter category are ways to help clarify the often tacit theories of social change that guide efforts. Progress in these three areas will help meet the needs and challenges set out in section 2.2.