Funded by the Governance and Civil Society Program, Ford Foundation (usa) 1Table of Contents

Issues of Scale in Assessment and Learning

Download 405.53 Kb.
Date conversion08.11.2016
Size405.53 Kb.
1   ...   7   8   9   10   11   12   13   14   15

Issues of Scale in Assessment and Learning

Working effectively on a small island of social change is not enough to achieve societal transformation of the scale needed to sustain improvements. This requires change at different levels, in which remaining connected – from local levels to larger scales of impact and from higher scales of strategy to local levels of implementation – is important. The challenges of scaling up and scaling or ‘translating’ down (see Box 21) are different, with a common concern being how to maintain integrity of the social change values across the levels. This section outlines the challenges of scaling up and down of social change and some of the implications for assessment and learning.

Two Directions of ‘Scale’ to Bear in Mind – Scales

An often automatic association with the term ‘issues of scale’ is that of scaling up of impact. At the local level, it is relatively easier to hear stories of personal stories of transformation and observe changes. The scope of action and actors involved is relatively restricted, hence assessment processes appear more manageable. However, complications of scale quickly arise when seeking to implement social change at a larger geographic or population scale that can lead to dilution of original principles or strategies, exclusion of significant groups, high costs or long time frames due to the desire to ensure participation.

Box 21. The different directions of ‘scale’

Scaling up refers to the increase in scope or coverage of activity or impact, or to take up lessons derived from a specific context into another context or level.

Scaling down refers to the process of translating down a generic intent (policy, programme strategy, indicator set, funding stream) and contextualizing it and making it locally relevant.

Another set of issues arises in trying to scale up lessons from a specific context. How can lessons be shared meaningfully in other contexts, what are the pitfalls in conveying and taking up lessons from elsewhere? Sheela Patel explains that SPARC decided not to scale up the organisation itself in order to achieve greater impact but to take the experience to others and see what they make of it in their own contexts. This required finding ways to explain their model of working to other NGOs and thus making possible scaling up of their experiences and lessons.

The central challenge in scaling up is how to stay true to the original vision and processes despite the greater numbers involved and larger diversity of experiences. Working at a more aggregate scale than the local one, which means operating in relationships that cross geographic and institutional boundaries, may require adding some water to the wine in terms of intensity of contact, reducing the intensity or nature of citizen participation, working through representatives rather than directly with people, etc. It also means examining issues of linkages between advocacy efforts, policy change and changes in people’s lives, as these lie at the heart of many dilemmas about what change to assess at what level and how to do this. Can integrity of principles and focus be maintained across levels, with social change staying locally relevant? Can assessment and learning stay grounded in local endeavours, despite larger scales of analysis, thus informing and inspiring them? Assessment and learning will need to take different shapes depending on the level at which it is occurring. This leads to others questions, including: What can learning and assessment in which the poor and marginalised remain central look like at different levels; what procedures and mechanisms and learning style are effective? How can gaps between levels be bridged to ensure that learning does not stay captive within its own level? What is needed to create a learning web, not simply a learning organisation?

Interconnectedness also encompasses the challenges of scaling down. Recently debates about development among bilateral aid agencies and the international financial institutions has fuelled an almost obsessive focus on the ‘national level’. This includes debates related to the Paris Declaration with its technical, mechanical indicator set and target list (see Box 22) that has left civil society on the sidelines. The Paris Declaration speaks a technical language while dealing with what is essentially political – resource allocation and use. As part of the so-called ‘new aid modalities’, the trend is one of focusing on national level budgetary, planning and M&E processes. This trend appears echoed by the focus on national level advocacy work of so many (I)NGOs that are striving to achieve change at the national or international level. But what about local governance and local accountability? Who is paying attention to this and to ensure that efforts to assess change at national level have local relevance?

Box 22. Indicators to administer not assess development effectiveness(from Sjöblom 2006)

Brian Pratt, of INTRAC (Oxford, UK) says of the Paris Declaration 12 indicators: “the Paris Agenda is not a theory of development … it is about how one administers aid .. not whether aid has any effectiveness…. If you look at the indicators, they are not indicators of effectiveness; they are indicators of whether things have been reasonably managed. It does not mean that managing it well means a greater impact and greater effectiveness. …. One of the possible outcomes of all this is slightly better administered, bad aid as opposed to badly administered, bad aid and that is one of the outcomes we really have to expect might well be a success story. …. monitoring and evaluation has been downgraded and many of big donor agencies have pushed this question aside or it has been reduced to these very technical, administrative indicators, rather than real indicators of impact. There is a loss of interest in learning about quality.”

A critical factor in scaling down of social change efforts concerns ensuring citizen engagement in development processes that originated from higher levels of generality and abstraction (see Box 23). The emphasis on national policies and ownership often implies that only formal or national level civil society groups and organisations are involved in these processes. These are then assumed to represent ‘civil society’ nationally, which is often not the case. Sometimes intermediary or umbrella organisations are created in urban centres with the sole aim of representing civil society at national levels, and often find it difficult to maintain strong links with their original constituency. In the process, community-based organisations can lose out, asymmetric power relations among civil society organisations are aggravated, and different knowledges and networks are excluded.

A national or international derived process or policies contains risks for assessment and learning in terms of who participates, who facilitates, and the focus of the learning. The emphasis on (inter)national processes and policies fails to address local needs and reality. The subsequent danger is that (inter)nationally conceived assessment strategies and protocols fail to appreciate the contextual specificities of social change that could foster deeper learning about development. The real risk of generalisations that is run can create notions of citizens’ voice and ‘civil society’ that do not reflect realities on the ground.

Box 23. Larger level social change processes and the disconnect from people (Mwasuru, pers.comm.)

In Kenya, efforts were undertaken to implement government policy on land distribution, an explosive political issue and one that touches profoundly on ‘social change’ in a transformative sense. Overseas consultants and the World Bank were involved who were interested in seeing models of land reform. But local groups complained that the process was being described by government officials who were unable to represent their interests well. In some cases, local people were consulted (in one case they invaded the process), but even then the potential for listening was compromised. The distances are too great, the organisational arrangements too difficult, and the time limits too pressing to allow for genuine representation. Delegates ended up representing themselves and the process was not even consultative. The government called it ‘participatory’ but it was a sham.

A slightly more effective example from Kenya involves the ongoing constitution-making process. The secretariats organised consultations across the country to identify concerns that needed discussing. All consultations were documented, with the outcomes considered almost a ‘draft’ of the constitution, to be discussed by delegates. However, almost half of these delegates were members of parliament. The process was captured by their short term interests and lost sight of local concerns. Eventually a draft was produced by only one faction. Nevertheless, when it was finally put to a referendum, people rejected it. The government was back at square one and are still negotiating. Although the successful consultation was hijacked at higher levels, it had already expanded local awareness sufficiently to stimulate the subsequent rejection. If people had not been involved from the start, they might not have been critical of the draft product.

International donors are tending to support the strengthening of state bureaucracy but do not always sufficiently value and invest in what is needed for citizens to help build effective states. This has led donors funding CSOs to police the state, keeping an eye on donor money, to the detriment of funding work for and with marginalised groups. But is it about a well-functioning government apparatus or about vocal citizens, innovating with local change? Both are, of course needed, and monitoring the state can, if guided by a vision of social change, build important capacities. If citizens do indeed ‘build the state’ (Eyben and Ladbury 2006), then donors need to see effective states as those with capable and mobilised citizens and ‘state building’ as constructing relationship between citizens and the state. And to invest accordingly. Emphasising national processes versus local social change has far-reaching implications of what is considered valid evidence of development and also for the role of citizens in assessing development as a process of social change.

1   ...   7   8   9   10   11   12   13   14   15

The database is protected by copyright © 2017
send message

    Main page