Four short examples provide a flavour of how methodologies and frameworks can be useful for organisations that support and value social change. Mama Cash and ActionAid International use frameworks that allow for great flexibility and yet provide a general direction based on the values they consider important for development. SPARC’s story from India and that of CTA-ZM in Brazil provide insights into how local organisations deal with the need for flexibility and rigour in learning and assessment.
The Merits of a Framework – Mama Cash
Mama Cash is a Dutch organisation, a women's fund that finances social change initiatives conceived by women with first-hand experience14. Mama Cash focuses on funding strategic work related to women’s rights (Zuidberg et al 2006). It differs from other grant-making institutions in its small-scale support to groundbreaking women’s rights initiatives. Mama Cash defines groundbreaking as “transformative or change oriented initiatives in women’s rights, before these initiatives are mainstreamed”. The average annual grant provided by Mama Cash is between € 500 and € 15,000. In 2005 Mama Cash funded 284 projects valuing about € 2.3 million.
Since 2004, Mama Cash has been developing an approach called ‘Making the Case’ to measure social change resulting from projects implemented by grantees. It has been extensively tested and is now being introduced more systematically. It funds and some grantees. It has been shown to have a stimulating effect on the users by helping them to:
provide a theory of change and framework for measuring success
build the story, the message and the evidence (for learning, for mobilizing resources)
enables aggregation and collective learning about:
the dimensions of change on which women’s groups work, key inhibitors and accelerators;
Collective outcomes per country, region, globally;
Evidence based communications for leveraging more support for women’s rights work and women’s funds.
Making the Change asks for evidence related to five dimensions of change.
Shifts in definitions/reframing – The issue is viewed differently in the community or larger society;
Individual and community behaviour – People are behaving differently in the community or larger society;
Institutional and policy changes – An institutional policy or practice has been changed;
Maintaining/holding the line – Earlier progress has been maintained in the face of opposition.
The methodology asks grantees to identify relevant dimensions of change, by considering the question ‘To what extent is the issue viewed differently because of your work?’. They then tell the story of change in terms of baseline information, Goals, Strategies, Evidence (indicators, quantitative and qualitative), External Accelerators, External Inhibitors, Internal Accelerators, Internal Inhibitors and any Unexpected results.
The ‘Making the Case’ approach is promising but needs further refinement (Zuidberg et al 2006). Some methodological ambiguities persist, such as about time and geographical scales of social change projects, and how to deal with the attribution versus contribution issue. It needs further flexibility for translation to local conditions, in terms of language and cultural, social and economic categories.
ActionAid International’s15 Global Framework
ActionAid International introduced its ‘accountability, learning, planning system’ (ALPS) in 2000. It has led to sweeping changes in the way learning and accountability are perceived and implemented throughout the organisation. In 2006, further refinements were articulated and implemented following a stock take of initial experiences and an external review(Guijt 2004).
ALPS is a framework that sets out the key accountability requirements, guidelines, and processes in the global organisation. It recognises that principles (see Figure 1), attitudes and behaviours shape the quality of assessment and learning processes and outputs. It sets standards for what to do but critically, also, how to do it. It does this by articulating a set of core considerations that have driven the new approach. For example, it stresses that poor people must define agenda and own ‘indicators’ (used in a broad sense) of what change looks like and that rigid frameworks are less important than process and relationships. Central in the process is clarity about the question of who wants to know what – and why it matters.
Figure 1. ALPS principles ( Win 2006)
At a global level, the ‘Global Monitoring Framework’ asks for all levels of the organisation to look at and comment on four key questions and consider four areas of change that focus on power:
What did we do?
In pursuit of which right(s)?
Resulting in what changes? For whom?
Critical consciousness, capacity, action of rights holders
Tangible changes in people’s lives/material conditions
Organisation, action and movement building/growth
Policies and practices of states & other duty bearers
With what impact on power and power relations?
The methods that are encouraged and used to provide the evidence that is then analysed, documented and shared are: storytelling16, participatory review and reflection processes17, written reports, external reviews and peer reviews18.
Society for the Promotion of Area Resource Centres, India19
SPARCis a registered non profit society which began work on urban issues in 1984. SPARC works in 21 cities throughout India via alliances with other organisations. It is based in Mumbai, working with communities to improve their homes, neighbourhoods and employment opportunities. Its founding objective was ‘to establish area resource centres that serve the needs and priorities of local inhabitants (especially the poorest)’. SPARC works closely with two other organisations in ‘the Alliance’: the National Slum Dwellers Federation and Mahila Milan.
The Alliance uses a mix of different methods for assessment and learning – group exchanges, self evaluation, stakeholder feedback, individual professional development, commissioned external perspectives, action learning, and action research. However, such a list does not do justice to the core approach. The essence lies in the simplicity of issues that simply seeks to respect people’s ideas and keep donors at arm’s length to give people the space to act on these ideas. The poor must be organised, and therefore they are the ones who need to develop skills; hence it becomes essential to create a physical, emotional and social space for people to pool their human resources and facilitate learning.
This basic conviction and focus has led to an ‘embeddedness’ of assessment and learning so that reflection and evaluation are built into action mechanisms (see Box 17) by creating environments for different types of reflection and learning. For example, in the savings and credit groups to which millions of women belong, they share struggles and triumphs, leading to innovative ideas for action. This is a breeding ground for a continual supply of leaders and self-confidence among members. The surveying/mapping process focuses on learning about strategic planning and offers a baseline for impact assessment, while precedent-setting pilot projects and housing-related training have a more practical focus but also generate new leaders. Assessment and learning cannot be pulled apart and identified as specific stages or learning outcomes.
Box 17. The power of learning and scaling up through exchange visits
The Alliance started work with pavement dwellers in one area with five communities and today covers 550,000 households in 9 states and 70 small medium and large cities in India. That scaling has occurred from a process of community exchanges. Learning and mutual support are shared through a process of exchanges - visits to each other's communities so that experiences can be shared. Increasingly such exchanges also include public officials and other professionals, encouraging their exposure to the way in which organisations of the urban poor perceive, analyse and respond to the issues that they prioritise within their local contexts. It is through exchanges that new institutional relationships are frequently created. A conventional strategy of meetings and workshops designed and managed by the NGO were not acceptable to the community leadership. Seeing women on pavements managing complex advocacy negotiations inside and outside their neighbourhoods was more powerful than being told about it. It highlighted a critical issue of who in the processes leads the reproduction of strategy in the alliance - women and men took the initial risks to explore change.
Exchanges start by encouraging communities to reflect on their own situation. Together, neighbours identify their problems and explore possible solutions; they then either visit a group close by or invite them to their own settlement. Within the city, these exchanges occur rapidly and informally. The first few visits are facilitated by more experienced core trainers of the local federation, then people organise their own exchanges until spontaneous visits occur. Two types of exchanges occur: with core trainers travelling to assist city level groups and local community leaders, now confident and capable, visiting nearby settlements. Most exchanges involve groups of four or five women and two men. Members of recently organised communities meet leaders and/or visit established community organisations to share experience and frustrations. The more established groups begin the process of assisting new settlement organisations. The exchange process helps community leaders feel comfortable about participating in change. They gain this through interaction with peers and by understanding the change process in other settlements.
The embedded nature of assessment and learning also instils a shared clarity of understanding at the community level of what the process is about and a commitment to the long term process. People know what they mean by ‘social change’ even if they use different words to articulate it. As one woman said “We learned to love each other more.” This is theory embedded in practice. Learning occurs for everyone. For example, SPARC has learned about the merits of different strategies from the people, for instance, that an adversarial approach to authority would not serve their interests.
SPARC has strongly and explicitly resisted ‘NGO-ization’, keeping structures small and acting as a procedural buffer to donors. Most formal organisations get caught up in the assumed need for procedures and structures to guide assessment and learning, in the process forgetting relationships and principles. For example, SPARC maintains unstructured financial resources to anticipate emerging experimentation. They are constantly surprised with what was needed to be done, how much it would cost, and all attempts to systematize it only forced deeper subversions of allocated funds. Now almost 50% of the budget is for ‘precedent setting’.
Centro de Tecnologias Alternativas – Zona da Mata, Brazil20
CTA-ZM is a local NGO that has been active since 1987 in the state of Minas Gerais, Brazil, working with 20 municipalities. It focuses on developing viable alternatives with rural workers unions and their smallholder members. Currently, their activities span four programmes of work, in all of which participatory methodologies are embedded - diagnosis, planning, monitoring, evaluation, and systematization. CTA-ZM has changed its focus and role in line with changing needs – from gaining more knowledge of the region and sensitising farmers and their groups to more focused thematic programmes with a long term perspective. More recently, the NGO has taken on a more advocacy and facilitating role, documenting and disseminating its work with the farmers and municipalities to inspire other municipalities and NGOs in Brazil.
Since about 2002, CTA-ZM has given increasing attention to its own learning processes and information flows. This has involved everyone in the wider institutional set-up, including municipal partners, the Executive Committee/Council/General Assembly of CTA-ZM, and the technical team. CTA uses a mix of approaches (see Table 2 below). From a very basic and numbers-focused monitoring system, CTA has developed a wide range of mechanisms and processes that feed its need for insights at all levels. The learning process is continually evolving as the governance structures that drive the work evolve and, therefore, decision-making and information needs shift.
Table 2. Overview of mechanisms for learning within the organisation and its wider network of partners
Regular contact between team members and other key actors, esp. farmer leaders