In 1997, an international group of some 70 development professionals debated issues related to participatory monitoring and evaluation, a significant ancestor of the ‘assessing and learning for social change’ focus in this paper. Striking at the time was a non-questioning acceptance of an indicator-based approach to M&E that sought to prove whether the objectives of a linear, project-mode of development were being met. Then, the agenda for action focused on the need to experiment with and build capacity in participatory indicator development, impact assessment that listened to local voices, and alternative information management processes (IIRR 1998, 46-59) – and the role of donors, NGOs, CBOs and government agencies to make this possible. Relatively little focus was given to the central concerns of locally-relevant learning and the questioning of development models.
Ten years on, the ASC discussions refer to a far more politicised understanding of development as social change rather than as projects delivered through the mainstream development system by NGOs and international donors. Critical reflection in strategic alliances with unlikely partners, articulating theories of change, and the role of stories to clarify and convey the complexity of transformation are part of a new emerging discourse and practice. Social movements and evaluators are considered part of the agents of change, to join donors and NGOs. The received wisdoms of monitoring and evaluation are being challenged in more fundamental ways based on a different understanding of development itself as complex, emergent, and transformative. However, these shifts in thinking are slow and much is needed to come to assessment and learning processes that strengthen social change rather than hinder it. This final section sets out an agenda for action for key players and on critical issues.
Notwithstanding the use of a discourse that refers to ‘critical reflection’ and ‘learning, donors, by and large, favour a mode of M&E that is rooted in fears of non-compliance of agreements based on a development model that is considered predictable. Assessments are undertaken to meet financial procedures, to legitimize existing or future investments, and to set conditions for compliance. Favouring hands-off and objective assessments denies the learning potential of the work that they support.
Yet donors are critical partners and can make all the difference in development processes that recognise the value of local social change. Important lessons from North American non-profit organisations (Box 26) echo emerging insights from the development sector. “It requires them to loosen their focus on pre-planned interventions that lay out years ahead of time what is to be achieved, how and by when. It requires them to open their minds to the possibility of change happening in non-linear and unpredictable ways, and that social change occurs perhaps more slowly than they thought. It means allowing trust in the underlying principles of a methodology and a partnership to guide funding arrangements through bumpy patches.” (Reilly 2007)
Box 26. Eight Lessons from North American Non-profit Organisations (Pankaj 2007)
In order for social change to be measured accurately, grantmakers need to provide enough resources to cover not only specific interventions but also operational needs.
Community organizations need to be sustainable over a long period of time to have an impact on social change.
Funders need to have a realistic perspective on what types of change can occur within a given grant period.
Funders need to share lessons learned from their funding experiences to promote general field building.
In designing a multi-year funding partnership, flexibility is essential, with an understanding and willingness on both sides to adjust the process en route, rather than waiting to the end to find out what strategies did not work.
If a funder is interested in developing common outcomes among multiple grantees, this requires an understanding that consensus building takes time yet is vital to overall buy-in of the initiative and its results.
Measuring social change requires the long-term commitment of both grantmaker and grantee.
In practical terms, donors need to rethink the principles on which they base their models of evaluation and learning. Important principles would include (based on Patel 2007):
A learning orientation that is rooted in continuity of relationships. Those working in funding agencies have to feel excited by the development processes they help make possible. They need to want to be active participants in reflections and learning. This requires continuity in relationships to construct a shared understanding of strategy, results, and opportunities.
Acknowledging the need for risk-taking and related methodological implications. Social change as development for empowerment and an emergent process involves taking risks, funding innovations, setting precedents. This mode of development does not come with guarantees – it is not the same as funding polio vaccinations with children with known outcomes. Understanding the role of donors, including in relation to assessment and learning, when development is seen as experiment and as risk requires time and opportunity for frank discussions and clarifying what procedures are useful.
Understanding the real architecture of scale. All donors must show impact at scale to their constituencies. Yet is there clarity about how it is produced and sustained and who drives it? More exchanges and debates are needed that build a grounded understanding of impact at scale. Assessment processes are imminent opportunities for such learning to occur.
Understanding the deep politics of development and change. The management systems currently propagated by donors cannot produce the types of transformation that social movements and social change processes seek. Building community capacities to undertake these processes and manage them demand different strategies from professional NGO-managed projects. More support is needed for different strategies and for intermediaries to be allowed to take on different roles.
Creating opportunities to hear stories from social movements and the mouths of the poor. Reports rarely convey the reality of transformations that is taking place in poor communities. Can donors identify and fund alternative forms to hear these realities by inviting direct testimonies and the telling of stories?
Amidst what might seem like a daunting agenda, one action point merits special attention, that of consistency - donors must align their espoused values with the systems they use. Donors consider they are pro-poor, think they appreciate that development is complex and context-specific, and certainly see the need to support a diversity of efforts at different levels. Many even recognise the deeply political nature of the work they support. Yet their procedures and protocols do not align with these values. If there is only one task that is taken up by donors, then it lies in the creation of far greater consistency than is currently the case between the formal development goals they uphold and the mechanisms and processes they have created to support the realisation of such goals.