Document(s) 2 of 18

Knowledge shared equals results shared.

At the onset of the twenty-first century, the concept of sharing enjoys little currency among elites and the dominant media; our airwaves, newspapers, and websites are crammed with stories about the folly and the tragedy of domination, selfishness, and greed. Yet human beings have always shared with one another, and continue to do so. In every community on the planet, remarkable acts of solidarity and mutual aid enrich and enliven everyday life. Our species is oriented to sustaining the collectivity as well as to meeting our own individual needs.

Sharing is central to successful development cooperation. Within poor and disempowered areas, amid scarcity and want, resources must be pooled and channeled into common, achievable undertakings. In development cooperation, the most effective interventions are carried out by coalitions of interests in both the South and the North: villagers and barrio residents, local development workers, government officials, local nongovernmental organization (NGO) staff, Northern development specialists, and donor agency personnel. Sharing information, resources, and common objectives, such coalitions can influence, even drive, all phases of the intervention cycle: planning, implementation, and evaluation.

Evaluation involves the production of knowledge about the effectiveness and efficiency of development interventions. Traditionally, most evaluations have been donor driven and professionally controlled; they have been top-down exercises in which the sharing of knowledge has occurred too little and too late. However, many years of development practice have established beyond doubt that local citizens possess valuable information and analytical capacity to assess the achievements and constraints of development processes. Participatory evaluation strategies can help communities and development agencies mobilize and share local knowledge in combination with the expertise of outside specialists. The shared knowledge that emerges through this process is more accurate, more complex, and more useful than knowledge that is produced and deployed by professionals alone. It is precisely by sharing the different types of knowledge they bring to the evaluation process—and the new knowledge they create together—that citizens and professionals can generate analysis that will render interventions more capable of yielding significant and lasting results.

Shared knowledge is the essence of participatory evaluation. Shared knowledge better serves the interests of both the local beneficiaries and development agencies. By its very nature, participatory evaluation democratizes and enriches the assessment of development. At the same time, participatory evaluation enhances the capacity of interventions to achieve impacts that benefit the stakeholders engaged in the process. Drawing on experience from all parts of the world, this book examines the practice of participatory evaluation: its limits and its potential, what works and what doesn't work, the issues that frame and shape it, and the diversity of methods used to implement this approach to evaluation.

The Diverse Identities of Participatory Evaluation

Participatory evaluation has been given varied and multiple meanings by practitioners around the world. The profile of participatory evaluation takes on different forms, depending on the conceptual and methodological framework emerging out of a given context and a particular set of circumstances. Such a framework is determined by the varying degrees of emphasis or priority given to one or more of the following main characteristics of participatory evaluation. It is a process that:

• Supports and extends participatory models of development more generally;

• Empowers communities, organizations, and individuals to analyze and solve their own problems;

• Values the knowledge and experience of local citizens in analyzing their economic, political, social, and cultural reality;

• Uses learning and education to promote reflection and critical analysis by both project participants and development workers;

• Serves the purpose of improving the program and organization in a given development intervention, in the interests of the beneficiaries;

• Involves the active participation of project beneficiaries, who play a decisive role in the entire evaluation process;

• Promotes the beneficiaries' ownership of a development program;

• Uses participatory methods of obtaining data and generating knowledge, employing a wide range of predominantly qualitative methods, sometimes in combination with quantitative methods; and

• Is participatory and collective and that creates better, more in-depth, and more accurate knowledge of the performance and impacts of a development intervention.

After more than two decades of practice and reflection, participatory evaluation has now acquired four broad identities: as a development intervention in its own right, as a project management tool for sustainable development, as a source of obtaining qualitative data, and as a challenge to the conventional ways in which donor agencies undertake the evaluation of their development assistance. In relation to this last identity, some practitioners of participatory evaluation in recent years have been testing approaches to better serve the needs of both donor agencies and beneficiaries. This is important work.

For the purposes of this book, we propose the following working definition of our subject:

Participatory evaluation is a process of self-assessment, collective knowledge production, and cooperative action in which the stakeholders in a development intervention participate substantively in the identification of the evaluation issues, the design of the evaluation, the collection and analysis of data, and the action taken as a result of the evaluation findings. By participating in this process, the stakeholders also build their own capacity and skills to undertake research and evaluation in other areas and to promote other forms of participatory development. Participatory evaluation seeks to give preferential treatment to the voices and decisions of the least powerful and most affected stakeholders—the local beneficiaries of the intervention. This approach to evaluation employs a wide range of data collection and analysis techniques, both qualitative and quantitative, involving fieldwork, workshops, and movement building.

This introductory chapter highlights the major critical issues and themes— development-related, epistemological, and methodological—as they have emerged in the evolution of the theory and practice of participatory evaluation over the past two decades.

Development Issues

Starting with Empowerment

The term participatory evaluation first appeared twenty years ago in development cooperation literature in the context of increasing interest in the paradigm of participatory development. Case studies and manuals for practitioners were published and circulated in the 1970s and 1980s by engaged scholars, development practitioners, and some policymakers in the large donor agencies. Participatory evaluation techniques were applied around the world, at the local level, through the various disciplinary or sectoral "lenses," including adult education, anthropology, sociology, primary health care, water supply and sanitation, and rural and community development. At the heart of all this work, and, in effect, what binds this literature together, lies the conviction that evaluation should and can be used to empower the local citizens to analyze and solve their own problems. In this sense, participatory evaluation has distinguished itself from conventional forms of aid evaluation. Conventional approaches have relied heavily on outside professional experts to "objectively" assess the technical and management effectiveness and efficiency of development interventions with reference to project plans, logical frameworks, and work breakdown structures. Conventional evaluation has not challenged power relations in the development process.

Advocates of participatory evaluation staked out their ground vigorously and often aggressively, arguing on many levels in favor of the right of local citizens to define and shape their world, and that such a process yields more accurate and more socially just and equitable development strategies. Northern and Southern practitioners alike were influenced by the work of Southern educators, intellectuals, and activists such as Paulo Freire and Orlando Fals-Borda and later by Kamla Bhasin, Rajesh Tandon, Anisur Rahman, Francisco Vio Grossi, and many others. Inspired by Southern practitioners and stimulated by the experience of working in the South, many Northern practitioners also began to articulate the theory and shape the practice of participatory evaluation (see Selener 1997). They included Marja-Liisa Swantz, Budd Hall, John Gaventa, Marie-Thérèse Feuerstein, and Robert Chambers. A host of nongovernmental and other organizations around the world adopted participatory research and evaluation as their modus operandi. These organizations included the International Council for Adult Education and its regional affiliates, CUSO in Asia, Oxfam-UK, the Dutch agency NOVIB in Latin America, and many units of UNICEF, to name only a few.

As participatory evaluation began to be used more widely in the 1980s, it was found that, in some instances, it degenerated into a narrowly conceived technical application of a "toolbox" of methods. Advocates cautioned that participatory evaluation is not merely a technical exercise. They reminded the development community that the central mission of participatory evaluation is to empower individuals and communities, not merely to mobilize their labor or ideas.

During the 1990s, the limitations of local participation have increasingly been recognized. Practitioners became aware that the participation process must be reconceptualized to include other stakeholders involved in development interventions. This did not mean abandoning preferential treatment for the participation of local citizens and, in particular, the poor. But it did call for greater recognition and precision in analyzing whose participation and how these various participations interrelate (Rebiens 1995). Consequently, the rhetoric of people's justice and revolution of the 1970s has been ratcheted down and, for better or worse, a more pragmatic strategy for involving all stakeholders is now more frequently built into participatory evaluation. We believe that this is for the better.

The 1990s have also seen a much greater priority and visibility accorded by the large donor agencies, particularly the World Bank and some bilaterals, to participatory development. This is ironic, because such institutions for the past two decades have aggressively promoted structural adjustment programs, the very antithesis of citizen participation. Perhaps the end of the cold war and the collapse of communism have made participation a "safer" endeavor. In any case, the participation theme has now been widely "bought," for at least a while, by some interests within the large donor agencies, and this circumstance has created new space for activism by advocates of participatory evaluation and other forms of participatory development.

From Empowerment to Stakeholder Interaction

Rebiens (1995) noted that often the rhetoric of empowerment in participatory evaluations far exceeds the reality achieved on the ground. This is an important and valid observation. He also observed that there is a danger that outside professionals can, in an expression of solidarity with local citizens, put words into the mouths of project participants. This, too, is a valid concern. How the outsider manages his or her own agenda in the participatory evaluation process has always been a challenge to practitioners and is much discussed in the literature.

In response to these concerns, Rebiens argues that participatory evaluation should incorporate more centrally the framework and methods of "fourth-generation" evaluation (Guba and Lincoln 1995). In this type of evaluation, "the issues to be looked into are defined by stakeholders and where the evaluation knowledge cannot be objective, but emerges out of interaction between evaluator and evaluee" (Rebiens 1995, 9). In this sense, the concept of participation is replaced by the concept of stakeholder interaction, which offers perhaps less rhetorical attraction but more methodological achievability.

Preferential Treatment for Decisive Stakeholders

However, in participatory evaluation, some stakeholders are created more equal than others. In a full-fledged participatory evaluation process, there are decisive and non-decisive participants. "Decisive participants are those who are central to the process that is being evaluated. These are the activists, the animators, the local people. The non-decisive participants are those who have stakes in the evaluation process but, are not central to it. These are the intermediary organizations, the donors and other stakeholders" (Chaudhary, Dhar, and Tandon 1989, 9).

In other words, a bias must be built into the participatory evaluation process in favor of the poorest interests and their allies. The powerful and elites can participate, but their voices cannot be permitted to dominate. This tests the will and skill of the outsiders involved in the evaluation, who are likely being contracted and paid by the powerful rather than by the less powerful.

This fundamental commitment to a bias in favor of the least powerful constituencies in the evaluation exercise sets participatory evaluation apart from other collaborative forms of assessment. Beneficiary assessment, for example, treats the perspectives and values of local program participants as but one component of a larger effort that includes detailed analysis of the views of program managers and executives in both the North and the South (see Casely and Kumar 1987; Salmen 1987; Kumar 1993; Valdez and Bamberger 1994). When local participants' views represent only one of a menu of perspectives, there is every likelihood that the proximity, persuasiveness, and power of the Northern aid professionals and/or Southern elites and professionals will exert a disproportionate degree of control over the evaluation's findings and conclusions. While they may well participate and mobilize for action, such participation and action may not be relevant, and may even be in contradiction, to the interests of the poor targeted by the development intervention under study.

Several of the chapters in the present volume illustrate the countervailing forces acting against the priorities, analyses, and solutions of what should be the decisive participants in the evaluation process (see Chapters 5, 10, and 13).

Participation at Different Points on the "Aid Chain"

Development cooperation is an international enterprise with many stakeholders spanning a range of nations, classes, and cultures. Social and organizational relations in development interventions can be conceived of as a chain linking identifiable components. Among the various components of this "aid chain" are Northern taxpayers, Northern (government) donor agencies, Northern implementation agencies (private firms, educational institutions, NGOs), Southern governments, Southern implementation agencies (line ministries, national NGOs, universities), local development organizations (village development committees, neighborhood development groups), and Southern citizens who are intended to benefit, ultimately, from the intervention. These various stakeholder groups possess different levels of wealth and power, different cultures, access to different information, and different missions. Their involvement in a development program or project requires that they form, in essence, a coalition for the purpose of achieving the common objectives of the intervention. Stakeholders interact continuously along the aid chain, held together by the project coalition.

In conventional aid evaluations, power and influence are typically concentrated in the hands of the sponsors of the evaluation—the donor agencies—and their consultants. Donor accountability and management concerns are weighted heavily; the time and resources devoted to beneficiary involvement are typically minor.

In contrast, the participatory evaluation approach seeks to reverse this power relationship and devote extensive resources and time to local participants as authentic stakeholders. To do otherwise, as Guba and Young (1989, cited in Rebiens 1995) argue, is unfair and discriminatory, since these stakeholders have much to risk in the evaluation. At the same time, however, the participatory evaluation process seeks to ensure that other stakeholders on the aid chain are able to participate fully, especially project field staff and regional project managers in the intervention area, as well as stakeholders at other points on the chain.

Participation Continuum or Cycle of Reflection and Action?

Most literature on participatory development in general and participatory evaluation in particular includes the concept of a continuum of participation. At one end of this continuum, project beneficiaries are passive recipients of inputs and activities. At the other end of the continuum is total self-management by the beneficiary group. In between, moving from less to more participation, the other stages are consultation, consensus building, decision making, sharing of responsibilities, risk sharing, and partnership. The continuum is sometimes depicted as consecutive steps in a ladder (Beaulieu and Manoukian 1994). In general, full participation is full power sharing among the major actors.

Still, some commentators express dissatisfaction with the continuum concept. The key to understanding participation, they argue, is the cycle of problem identification, information gathering, goal setting, choosing options, and taking action. "At the point of action, the process starts over again. Life in any community is made up of a great many sequences which interpenetrate to produce a pattern of life in the community" (Smith 1995, 64). In fact, both concepts are useful. The continuum sheds light of the nature of negotiations and transactions among stakeholders. The reflection-action cycle highlights the nature of the problem-solving process used by specific groups to assert their interests.

Epistemological Issues

Macro-Level Deconstructionism

Epistemology is the study of how knowledge is produced. At the macro level, that is, at the level of societies and of national political discourse and struggle, participatory evaluation is part of the tradition of deconstructionism.

The postmodernist critiques suggest important implications for participatory evaluation in the relationship of power and knowledge and in the politics of research and knowledge. Postmodernist writers, such as Foucault (1973, 1977) and Lyotard (1984), challenge the universal validity of the overarching explanatory theories or the "grand narratives" that have shaped the politics and guided social change in the modern period. Postmodernism, recognizing a multitude of perspectives and approaches, seeks to "deconstruct," or pull apart, the grand themes by engaging in an analysis of power relationships related to specific situations and social issues.

Viewed in the context of postmodernist critiques, participatory evaluation represents an attempt to deconstruct the dominant research and evaluation paradigms. In particular, participatory evaluation attempts to change the power relations in the creation and use of knowledge. At the same time, this reconception of power and power relations addresses the larger issues of poverty, inequality, and oppression.

Micro-Level Constructionism

At the same time, recent theoretical work has more clearly identified the epistemological parameters of participatory evaluation. In particular, at the micro level, participatory evaluation is now becoming associated with constructionist epistemology, in which various stakeholders bring their perceptions and analysis of reality "to the table" to create a negotiated reality, from which flow recommendations for action (Rebiens 1995, 5). This epistemological tradition is also associated with "fourth-generation" evaluation, which is defined as an interpretative approach to evaluation based on and guided by issues identified by all stakeholders (Guba and Lincoln 1995).

Research in the education sector indicates that the process of stakeholders socially constructing their reality through participatory evaluation enhances organizational learning significantly (Cousins and Earl 1992). This is consistent with the work of Senge (1990) and others on organizational learning in other sectors. In the field of education, research committees, advisory committees, work groups, and administrative councils have all been found to be effective vehicles for cooperative stakeholder inquiry (Cousins 1996).

The Continuing Rationalist-Objectivist Challenge

Notwithstanding the growing recognition that, at the micro level, participatory evaluation is part of the constructionist paradigm, there remains a serious challenge from the rationalist-objectivist tradition. Advocates of this tradition claim that in fact an objective reality does exist, can be measured accurately, and stands apart from the subjectivity of constructionism (Mathie 1995). One source of this challenge is the dominant discourse in the mainstream evaluation field in the North, which is rooted in a rationalist-objectivist epistemology. Another is the dominant discourse in the field of development cooperation, where concerns with accountability, value for money, and results have been heightened by budget cuts and public scrutiny.

Gender and Knowledge Production

There is also a gender dimension to how knowledge is produced. Leading practitioners such as Bhasin (1992, 1994), Maguire (1987, 1993), Waring (1990), and others have shown how women are systematically excluded in most societies from knowledge-production processes that are dominated by men—even ones that claim to be participatory. Further, they have shown that, overall, women's style of creating knowledge tends to be more holistic and collective than male-dominant forms.

Participatory evaluation has, very imperfectly, begun to confront the structural implications of gender relations. Sometimes special methods are used to engage the participation of women in the evaluation process. Over the past decade, an array of gender-sensitive approaches to evaluation and impact assessment has been developed in the North (Maguire 1987) and the South (Ellis 1997).

However, too frequently, even after many years of practice, women's voices are muted and their priorities remain invisible in participatory evaluations. "Women do not have the power necessary to represent personal concerns publicly and, by default, have to conform to the categories of concern given in advance," writes Mosse (1994, 515). Scholars and practitioners of participatory evaluation everywhere must work harder to overcome such barriers to a gender-equitable process of knowledge production.

Ecology and Knowledge Production

There is an ecological dimension to knowledge production in participatory evaluation, as well. Vio Grossi argues that an ecological society, in its ideal form, is a society in a permanent process of decentralizing and distributing power, rather than concentrating it in the hands of the privileged. An authentic ecological society, he writes, is characterized by an ethic of diversity and decentralization in all aspects of life: biological, economic, political, and cultural. In the sphere of education, an ecological society must ensure equitable distribution among the population to participate in the creation of knowledge, in the process of actively learning and generating a diversity of critical, heterogeneous, and imaginative ideas (Vio Grossi 1995). Among marginalized groups of people everywhere, women, in particular, have a special role in the ecological production of knowledge (Bhasin 1994).

Methodological Issues

Relationship to Other Forms of Critical, Collaborative Inquiry

Methodologically, participatory evaluation shares much in common with other forms of critical, collaborative inquiry. In particular, participatory evaluation is closely related to what is known as participatory research and participatory action research. Growing out of the work of Paulo Freire, Francisco Vio Grossi, Rajesh Tandon, Patricia Ellis, Kamla Bhasin, Marie-Thérèse Feuerstein, and the International Council for Adult Education, participatory research, like participatory evaluation, "links social investigation to education and action" (Hall, Etherington, and Jackson 1979, 5) and relies on committed, activist outside evaluators to promote the community's right to know and control the knowledge creation process (Fernandes and Tandon 1981; Chaudhary, Dhar, and Tandon 1989).

Participatory action research is closely associated with the work of Orlando Fals-Borda, Anisur Rahman, Susanta Tilakaranta, and many others and seeks to enable marginalized groups in society to construct countervailing power to that of their oppressors through the acquisition of serious and reliable knowledge. With its roots in sociology and anthropology, participatory action research pays special attention to methods that involve collective research, value folk culture, recover indigenous history, and produce and diffuse new knowledge (Fals-Borda and Rahman 1991).

A term that is currently receiving prominence in some evaluation literature is empowerment evaluation. According to Fetterman (1996, 4), empowerment evaluation "is the use of evaluation concepts, techniques and findings to foster improvement" among citizens and the programs intended to serve them. Emerging from community psychology and community development, this approach to evaluation is "attentive to empowering processes" (Fetterman 1996, 4) and uses self-evaluation and reflection by program participants to collectively help themselves and improve their programs. Outside evaluators act as coaches or facilitators in these processes, in which training, advocacy, and action all are essential elements. Although not as overtly or ambitiously political as participatory action research or participatory research, empowerment evaluation is clearly related to participatory evaluation as well.

Qualitative and Quantitative Methods

Participatory evaluation employs a wide range of qualitative and quantitative research techniques. The use of some of these techniques is demonstrated in the case studies presented in this book. Among the qualitative methods used in these processes are community evaluation committees, community workshops, self-directed focus groups, popular theater, community radio, transect walks, wealth ranking, and many others. Participatory evaluations may also make use of quantitative methods that rely on questionnaires, household interviews, and survey sampling techniques, as well as computerized statistical analysis. With today's powerful notebook computers, sophisticated quantitative data analysis can be carried out around the campfire at night in rural areas. In any case, the choice of methods used in any particular evaluation project will depend on local conditions and the comfort level, skills, and interests of various stakeholders. The ability to choose from a diversity of techniques, observes Tandon (1990, 100), is "important because some constituencies may feel more comfortable using stories, drawings, role-plays, theatre, puppetry, and similar other forms of data collection and analysis, while others may feel more familiar and comfortable with questionnaires, in-depth interviews, surveys and the like" (see also Marsden, Oakley, and Pratt 1994; Marsden and Oakley 1990; Feuerstein 1978, 1986, 1988).

Practitioners have begun to turn their attention to developing, in conjunction with communities, qualitative and quantitative indicators of effectiveness, efficiency, and impacts. Considerable emphasis in this work is placed on devising indicators or indices that measure the capacity of communities to manage the development process (see, for example, Chapter 4).

Partnership: Shared Values and Equality

Participatory evaluation can also be viewed in the context of the development paradigm of partnership in international development cooperation. Among other elements of a genuine partnership relationship between Northern and Southern development organizations, participatory evaluation contributes to building relationships based on the values of equality, sharing, mutual trust, and transparency. Seen from this perspective, participatory evaluation can be one of the key interventions that helps to overcome the donor-driven and control-oriented approaches that predominate the field of development cooperation (see Gariba, Kassam, and Thibault 1994).

In the context of the dynamics of a genuine partnership, and given sufficient will on the part of Northern donors to engage in a partnership relationship, participatory evaluation can both strengthen recipient ownership and accommodate donor accountability requirements. In addition, since the concept of partnership is integrally linked with the process of capacity building, participatory evaluation contributes to the building of research and evaluation capacities of the Southern partners.

In terms of relations between implementing agencies, there are good reasons for partnerships in participatory evaluations to be Southern led rather than, as is usually the case, being led by the North. Southern-led partnerships are guided by an in-country project team with the authority, knowledge, and skills to move the project ahead efficiently and effectively. Foreign resources should be mobilized only when necessary, set clearly within the context and priorities established by the Southern team. There are considerable cost savings to be accrued in view of the lower fee or salary structures of Southern as opposed to Northern development professionals and the fewer mistakes and less wasted time and resources resulting from the decisions of an on-site, knowledge-intensive team (Gariba and Jackson 1993).

The Importance of Facilitation Skills

Clearly, the facilitation skills of participatory evaluators are central to making the process work successfully. Whether they are Northern or Southern, outsiders or insiders, those coordinating the effort must create a process and an environment that permit each of the stakeholder representatives to speak freely and to learn productively. In particular, the facilitator must create a safe environment where stakeholders will not fear retribution and where the usual hierarchies are decisively leveled. The facilitator must understand in detail the political, cultural, gender, and organizational dynamics that may prevent representatives from speaking or that may permit them to register their views assertively and clearly.

Manuals and Tool Kits

The past decade especially has witnessed the production of an array of handbooks and "toolboxes" for practitioners in participatory evaluation. In the areas of health and social development, a seminal work in this regard was Marie-Thérèse Feuerstein's 1986 book Partners in Evaluation, as well as subsequent articles and reports by Feuerstein (1988). In water and sanitation, Deepa Narayan of the World Bank has been a leader (Narayan and Srinivasan 1995; Narayan 1993; Narayan-Parker 1991). In community forestry, the Food and Agriculture Organization has published practitioner guides on participatory evaluation (for example, Davis-Case 1990). A superb bibliography developed by the Institute of Development Studies at Sussex University and published by the International Institute for Environment and Development, based in London, abstracts manuals and tool kits produced by such Northern NGOs as Actionaid, World Vision International, the World Resources Institute, PACT, Enfants du Monde, and Save the Children U.K. (Gosling 1993); Southern NGOs such as the Bangladesh Rural Advancement Committee; and donor organizations, including the International Fund for Agriculture Development and the U.S. Agency for International Development (McPherson 1995). The United Nations Development Programme has published a new handbook on participatory evaluation, as well (UNDP 1997).

One of the issues highlighted by this and other bibliographies (see also Johnson 1994) is the richness of methods available in different traditions related to participatory evaluation, particularly participatory rapid appraisal (known in some quarters as participatory learning for action), NGO self-evaluation, beneficiary assessment, and so on. However, practitioners have begun to caution one another on the pitfalls of mixing potentially incompatible methods. This issue deserves closer scrutiny.

In any case, a productive dialogue on methods continues. There are many vehicles for this dialogue. PRA/PLA Notes is published regularly by the International Institute for Environment and Development. Convergence, the journal of the International Council for Adult Education, regularly publishes case studies of participatory research and evaluation, as do the Community Development Journal, the Rural Extension Bulletin, and World Development, among other journals concerned with development cooperation.

Challenges

Can the Powerful Change Their Behavior?

Participatory evaluation, in its fullest, most developed form, can take place only if the powerful change their behavior. They must, in Chambers's view, give things up. "For the rich to give up their wealth, without being forced by countervailing power, is difficult and improbable; but [for the powerful] to give up dominance at the personal level, putting respect in place of superiority, becoming a convenor, and provider of occasions, a facilitator and catalyst, a consultant and supporter, is less difficult; for these roles bring with them many satisfactions and non-material rewards" (Chambers 1995, 42). The task, and the opportunity, writes Chambers, is to enable large numbers of Northern aid executives and managers and senior government officials in the South to experience these satisfactions and rewards directly—personally.

The critical point here is that for less powerful interests to become authentically involved in participatory evaluations, the powerful must concede control of the process. They must place their trust in stakeholders further along on the aid chain. There is no natural contradiction between ensuring accountability and facilitating participation. In fact, as all the chapters in this book show, participation can maximize rather than limit results in development cooperation.

There is increasing clarity on the kinds of skills development officials require in order to help make participatory development happen. Rather than directing and controlling, project managers must learn to perform such new roles as initiating, facilitating, participating, sharing expertise, navigating, and nurturing the development process (World Bank 1996). These skills must be taught, and rewarded.

From Local Action to Global Change

But even if the powerful change their behavior, aren't participatory evaluations too "micro" in nature, too local, almost by definition, to exert a significant impact? Aren't the macro forces of globalization, transnational corporate strategy, and structural adjustment far more powerful? Certainly, individual participatory evaluation efforts on their own cannot change the macro context. It is essential, therefore, that networks of practitioners, scholars, agencies, and communities be established to exchange experience and aggregate lessons learned. It is also essential that communities and professionals engage in broader social movements and political activities that can build greater leverage to challenge the power of global forces and institutions.

Promoting Accountability and Results

Recent years have seen a major push among Northern donor agencies toward greater accountability and results in the delivery of development cooperation projects. Falling aid budgets and deficit-conscious legislatures have prompted calls for more value for money in development programming and have driven a move away from activity-based management regimes toward results-based management systems. Accordingly, the evaluation and monitoring functions in development cooperation—now more frequently referred to as performance review and measurement—are being "reengineered" to emphasize the assessment of outputs and impacts rather than, as in previous eras, of inputs and activities.

Can participatory evaluation promote accountability and results, or is it too "soft," too qualitative, and too process oriented? The answer is that participatory evaluation can advance accountability and results in international development. No one has a greater stake in optimizing results than project beneficiaries on the ground. This is precisely the group whose views and decisions can find strong expression in participatory evaluation. Furthermore, accountability and results are advanced when the broader network of stakeholders interacts, evaluating together the outputs and impacts of what has been accomplished, identifying obstacles to progress, and formulating joint action to improve the intervention. Accountability is an issue that involves more than Northern treasuries and auditors-general. A poor farmer or an illiterate microentrepreneur calling a central government official or a foreign project manager to account in a participatory evaluation workshop is an even more important type of accountability in a developmental sense.

Blending participatory evaluation with results-based management demands that local project participants, as well as other stakeholders, become engaged in a meaningful way in defining the results to be achieved by a project, as well as the indicators and methods to be used to assess performance. Because participatory evaluation will produce better knowledge than conventional forms of evaluation—that is, analysis that is more accurate and locally appropriate, often backed by a consensus among key stakeholders—it will also help produce better development results.

Confronting the Economics of Participation

One of the obstacles to widespread adoption of participatory evaluation is the perception that it is more costly and time-consuming than more conventional, expert-driven evaluation. In fact, at the front end, it is. Participatory evaluation processes take time while stakeholder groups engage in the process, define their positions, and revise those positions in dialogue with other parties. It takes time to listen, negotiate, and take action when a plurality of parties is involved in evaluation. It takes money, too.

Certainly, it is much faster and cheaper to ask a consultant to simply design and conduct a more conventional evaluation alone, managing his or her own views and schedule rather than those of others. The problem with this is that conventional approaches can result in self-centered, inaccurate analysis. Equally important, nonparticipatory evaluations do not engage the stakeholders in building a common plan of postevaluation action. Shared analysis and shared action can lower downstream program costs, and increase downstream benefits, of subsequent development interventions.

Those who favor participatory approaches in development cooperation often claim that participation is a process and, as such, cannot be quantified. This position is no longer tenable. Cost-aggressive managers in donor agencies, Southern governments, and NGOs frequently block participatory evaluation proposals with economic arguments. And such arguments can be refuted only with hard data.

Some relevant literature exists in this area. In particular, Isham, Narayan, and Pritchett (1996) transformed qualitative data on participation in 121 World Bank water projects into quantitative data suitable for statistical analysis. Based on earlier work by Narayan (1995), they reported "strong statistical findings that increasing participation directly causes better project outcomes" (Isham et al. 1996, 196). Another study of World Bank projects revealed that participatory projects, overall, did not take more time to plan and implement than nonparticipatory projects (Rietbergen-McCracken 1996).

We need a new round of economic research on participation that involves professionals and citizens in all sectors of the development-cooperation enterprise. Taxpayers in the North and poor households in the South have much to gain from such an exercise if it is done with integrity and rigor and communicated to the general public in accessible and transparent form.

Spreading and Improving the Practice of Participatory Evaluation

As a process that promotes both development results and democracy, participatory evaluation deserves to be spread and applied more broadly worldwide in the future. For this purpose, an organized global network on participatory evaluation would be of great assistance.

But "spread" alone is not enough. Advancing the quality of participatory evaluation practice, promoting its continuous improvement, is an equal priority. Both spread and improvement must be encouraged at the same time.

Recent years have seen a number of organizations in the South and the North step forward to lead a global renewal of energy and activism in participatory evaluation. Among these organizations are Participatory Research in Asia (New Delhi), the International Institute of Rural Reconstruction (Quito), the Institute of Development Studies at Sussex University (Brighton), the Participatory Action Research Network at Cornell University (Ithaca), the International Development Research Centre (Ottawa), the United Nations Development Programme (New York), and the World Bank (Washington, D.C.), as well as many other NGOs and practitioner networks around the world (see the list of organizational resources and useful websites at the end of this book). A wide range of historical and methodological works (Selener 1997; Smith, Williams, and Johnson 1997), project assessments (Rietbergen-McCracken 1996; Narayan 1995), manuals (United Nations Development Programme 1997; World Bank 1996), and bibliographies (Mebrahtu 1997; McPherson 1995; Johnson 1994) has recently been added to the literature on participatory evaluation and development. Because of these efforts, the velocity of the spread and improvement of participatory evaluation is accelerating, and its application is expanding rapidly.

Organization of the Book

This book is intended to make a contribution in both spreading the practice of participatory evaluation and stimulating further work on improving the quality of participatory evaluation practice. The following sections summarize the chapters contained in this book.

Issues, Strategies, and Methods

In an important overview chapter, Jim Freedman points out both the simplicities and the complexities of participatory evaluation. He demonstrates the simplicities and common sense of participatory evaluation by making brief references to experience in the management and implementation of sewage and water projects in Ecuador, a marketing cooperative project for fishers in Brazil, and water and health projects in Kenya. However, when viewed in the context of a radical critique of development expertise, participatory evaluation takes on the political and other complexities of development processes.

Scott Clark and John Cove then examine the critical issue of ethics in participatory evaluation. Based on their training and experiences in social anthropology, and drawing on the many decades of struggles by anthropologists that led to the development of codes of ethics in anthropology, they offer stimulating perspectives on the question of ethics in participatory evaluation. They conclude that the evaluator has ethical responsibilities to three parties: project beneficiaries, the funding organization, and the evaluator's discipline.

In relation to the fairly recent shift that has occurred among the major donor agencies in the field of development cooperation from activity-based to results-based management systems, Edward Jackson examines the compatibility of participatory evaluation with results-based management. He makes the case that participatory evaluation can serve the interests of results-oriented development interventions, and vice versa. He cites several specific tools that are emerging in development practice that can enhance the interaction between participatory evaluation and results-based management.

Sulley Gariba analyzes an attempt to use a participatory impact assessment process to foster village-level capacity building in poverty alleviation programs. Based on the experience of evaluating an integrated rural development project in northern Ghana, the analysis concentrates on the process by which an evaluation exercise has been used as an integral part of the development intervention activity, while satisfying the primary objective of assessing impacts. It also reviews the use of tools created by local professionals and community members to assess impacts of development interventions on a continuous basis.

Case Studies

In a pioneering initiative, Kamla Bhasin documents a workshop discussion in India among seven grassroots rural development workers on how to evaluate the process of participatory development that is intended to help the poor collectively analyze the socioeconomic, political, and cultural structures that keep them poor and prevent them from getting organized to challenge these structures.

Marie-Thérèse Feuerstein reviews a participatory evaluation process and its impact on the community in a health project in Patna in the poor, heavily populated state of Bihar in India. She focuses on the role of, and techniques used by, the community health and development team that facilitated this evaluation process.

Yusuf Kassam presents a case study of the combined use of qualitative participatory methodology and quantitative survey methods in evaluating the training impacts of a large, bilateral rural development project involving the landless poor in Bangladesh. He argues that, although the necessity and importance of statistical and quantitative evaluation are not to be denied, qualitative participatory evaluation has an important role to play in producing a body of unique and illuminative data on qualitative development impacts.

Sheila Robinson and Philip Cox describe an alternative evaluation methodology, which they call "process evaluation," and how it was used in a large-scale health development project in Nepal. They provide details of the evaluation methodology and its underlying concepts and discuss lessons learned in the use of this methodology for those involved in international development.

Through a case study of a complex development project involving research and training centers in Southeast Asia and partner Canadian universities and colleges, Gary Anderson and Deborah Gilsig address the question of forms, purposes, and levels of participation by different stakeholders in the aid chain in an evaluation process. This case study illustrates some of the methods and issues of "fourth-generation" evaluation involving interaction by different stakeholders.

Bonnie Mullinix and Marren Akatsa-Bukachi share their experience in Kenya of developing an NGO training program to provide field-workers with the skills and experience to facilitate participatory evaluation with women's groups. This training initiative grew out of the authors' concern that assessments of program impacts are dominated by the needs and voices of donor agencies and project implementers. They believe that the marginalization of program beneficiaries results in the loss of crucial information in development interventions.

Andrew Livingstone summarizes the approaches, methods, and matrices used by a team of Ghanaian and Canadian development specialists in an internal participatory monitoring and evaluation of a water and sanitation project in Ghana. The emphasis of this project was on creating and supporting community water boards as local decision-making structures.

Patricia Ellis analyzes the participatory methodology that was used by community members to evaluate a pilot project for the integration of women in rural development in a small community in Rose Hall in the eastern Caribbean island of St. Vincent. This project, initiated by the Women and Development Unit (WAND) of the University of the West Indies, was itself a participatory, bottom-up development project.

Elizabeth Whitmore recounts a case of conducting a participatory evaluation involving a dairy-goat farmers' cooperative in a small, poor village in Mexico. In light of her experience as a contracted evaluator in this project, she reflects on a number of issues and dilemmas inherent in participatory approaches to evaluation, such as the tension between process and product, the role of the outside consultant, conflicting interests among different stakeholders, the effectiveness of short-term site visits, and gender issues.

Continuity, Commitment, and Hope

It probably takes half a century for an idea to make a truly significant impact on the world. The past twenty years have seen participatory evaluation play a role of increasing importance in international development. Given current trends in the world at large, in development cooperation, and in this field of evaluation practice itself, there are strong indications that participatory evaluation may play an even more significant role in the next thirty years. For this to happen, the involvement of a new generation of young practitioners, scholars, and policymakers is necessary. The continuity and commitment demonstrated in past work on participatory evaluation must be carried forward.

References

Beaulieu, R., and V. Manoukian. 1994. "Participatory Development: A Brief Review of CIDA's Experience and Potential." Policy Branch, Canadian International Development Agency, Hull.

Bhasin, K. 1992. "Alternative and Sustainable Development." Convergence 25 (2): 26–36.

———. 1994. "Let Us Look Again at Development, Education and Women." Convergence 27 (4): 5–14.

Casley, D., and K. Kumar. 1987. Project Monitoring and Evaluation in Agriculture and Rural Development. Baltimore: Johns Hopkins University Press.

Chambers, R. 1995. "Paradigm Shifts and the Practice of Participatory Research and Development." Pp. 30–42 in Power and Participatory Development: Theory and Practice, edited by N. Nelson and S. Wright. London: Intermediate Technology Publications.

Chaudhary, A., S. Dhar, and R. Tandon. 1989. "Report of International Forum on Participatory Evaluation." International Council for Adult Education and Society for Participatory Research in Asia, New Delhi.

Cousins, J. B. 1996. "Consequences of Researcher Involvement in Participatory Evaluation." Studies in Educational Evaluation 22 (1): 3–27.

Cousins, J. B., and L. M. Earl. 1992. "The Case for Participatory Evaluation." Educational Evaluation and Policy Analysis 14 (14): 397–418.

Davis-Case, D. 1990. "Community Forestry: Participatory Assessment, Monitoring and Evaluation." Food and Agriculture Organization.

Ellis, P. 1997. "Gender-Sensitive Participatory Impact Assessment: A Caribbean Perspective." Special issue of Knowledge and Policy: The International Journal of Knowledge Transfer and Utilization 10 (1/2), edited by Harry Cummings, William Found, and Terry Smutylo. International Development Research Centre, Ottawa.

Fals-Borda, D., and M. A. Rahman, eds. 1991. Action and Knowledge: Breaking the Monopoly with Participatory Action-Research. New York: Apex Press.

Fernandes, W., and R. Tandon, eds. 1981. Participatory Research and Evaluation—Experiments in Research as a Process of Liberation. New Delhi: Indian Social Institute.

Fetterman, D. M. 1996. Empowerment Evaluation: An Introduction to Theory and Practice. Pp. 3–46 in Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability, edited by D. M. Fetterman, S. J. Kaftarian, and A. Wandersman. Thousand Oaks, Calif.: Sage.

Feuerstein, M.-T. 1978. "Evaluation in Education—An Appropriate Technology for a Rural Health Programme." Community Development Journal 13: 101–5.

———. 1986. Partners in Evaluation: Evaluating Development and Community Programmes with Participants. London: Macmillan.

———. 1988. "Finding the Methods to Fit the People: Training for Participatory Evaluation." Community Development Journal 23 (1): 16–25.

Foucault, M. 1973. The Order of Things: An Archaeology of the Human Sciences. New York: Vintage Books.

———. 1977. The Archaeology of Knowledge. London: Tavistock.

Gariba, S., and E. T. Jackson. 1993. "Building Capacity in Monitoring and Evaluation in Africa: The Potential of North-South Partnerships—A Case Study from Ghana." Presented to the annual conference of the Canadian Association for the Study of International Development, Carleton University, Ottawa.

Gariba, S., Y. Kassam, and L. Thibault. 1994. "Report of the Study of Partnership Institutional Strengthening." Prepared by E.T. Jackson and Associates Ltd. and GAS Development Associates, Ltd. Partnership Africa Canada, Ottawa.

Gosling, L. 1993. "Assessment, Monitoring and Review Toolkits." Save the Children Fund, London.

Guba, E. G., and Y. S. Lincoln. 1995. Fourth Generation Evaluation, 2d ed. Newberry Park, Calif.: Sage.

Hall, B., A. Etherington, and T. Jackson. 1979. "Evaluation, Participation and Community Health Care: Critique and Lessons." Presented to the American Public Health Association Meeting, New York.

Isham, J., D. Narayan, and L. Pritchett. 1996. "Does Participation Improve Performance? Establishing Causality with Subjective Data." World Bank Economic Review 9 (2): 175–200.

Johnson, S. 1994. "Participatory Research: A Selected Annotated Bibliography." International Development Research Centre and Norman Paterson School of International Affairs, Carleton University, Ottawa.

Kumar, K., ed. 1993. Rapid Appraisal Methods. Washington, D.C.: World Bank.

Lyotard, J.-F. 1984. The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press.

Maguire, P. 1987. Doing Participatory Research: A Feminist Approach. Amherst, Mass.: Center for International Education.

———. 1993. "Challenges, Contradictions, and Celebrations: Attempting Participatory Research as a Doctoral Student." Pp. 157–76 in Voices of Change: Participatory Research in the United States and Canada, edited by P. Park, M. Brydon-Miller, B. Hall, and T. Jackson. Westport, Conn./Toronto: Bergin and Garvey/OISE Press.

Marsden, D., and P. Oakley, eds. 1990. Evaluating Social Development Projects. Development Guidelines No. 5. Oxford: Oxfam UK.

Marsden, D., P. Oakley, and B. Pratt. 1994. Measuring the Process: Guidelines for Evaluating Social Development. Oxford: Oxfam UK.

Mathie, A. 1995. "Evaluation in a Result-Oriented Agency Environment: The Case of CUSO." Presented to the International Evaluation Conference, Vancouver.

McPherson, S. 1995. "Participatory Monitoring and Evaluation: PRA Bibliography." PRA/PLA Notes. International Institute for Environment and Development, London.

Mebrahtu, E., ed. 1997. "Participatory Monitoring and Evaluation: An Introductory Pak." Institute of Development Studies, University of Sussex, Brighton.

Mosse, D. 1994. "Authority, Gender and Knowledge: Theoretical Reflections on the Practice of Participatory Rural Appraisal." Development and Change 25: 497–526.

Narayan, D. 1993. "Participatory Evaluation: Tools for Managing Change in Water and Sanitation." World Bank Technical Paper No. 27. World Bank, Washington, D.C.

———. 1995. "The Contribution of People's Participation: Evidence from 121 Rural Water Supply Projects." Environmentally Sustainable Development Occasional Paper Series No. 1. World Bank, Washington, D.C.

Narayan, D., and L. Srinivasan. 1995. "Participatory Development Tool Kit: Training Manuals for Agencies and Communities." World Bank, Washington, D.C.

Narayan-Parker, D. 1991. "Participatory Evaluation: Tools for Managing Change in Water/Sanitation." PROWESS/UNDP/IBRD/UNICEF/WHO, New York.

Rebiens, C. 1995. "Participatory Evaluation of Development Interventions: The Concept and Its Practice." Working Paper No. 4, Department of Intercultural Communication and Management, Copenhagen Business School, Denmark.

Rietbergen-McCracken, J., ed. 1996. Participation in Practice: The Experience of the World Bank and Other Stakeholders. Washington, D.C.: World Bank.

Salmen, L. A. 1987. Listen to the People: Participant-Observer Evaluation of Development Projects. Oxford: Oxford University Press.

Selener, D. 1997. Participatory Action Research and Social Change. Ithaca, N.Y.: Cornell Participatory Action Research Network, Cornell University.

Senge, P. 1990. The Fifth Discipline: The Art and Practice of Organizational Learning. New York: Doubleday.

Shiva, V. 1993. Monocultures of the Mind. London: Zed Books.

Smith, D. 1995. First Person Plural: A Community Development Approach to Social Change. Montreal: Black Rose Books.

Smith, S. E., D. G. Willms, with N. A. Johnson, eds. 1997. Nurtured by Knowledge: Learning to Do Participatory Action-Research. Ottawa/New York: International Development Research Centre/Apex Press.

Tandon, R. 1990. "Partnership in Social Development Evaluation: Thematic Paper." Pp. 96–101 in Evaluating Social Development Projects, edited by D. Marsden and P. Oakely. Oxford: Oxfam UK.

United Nations Development Programme. 1997. Who Are the Question-makers? A Participatory Evaluation Handbook. New York: UNDP, Office of Evaluation and Strategic Planning.

Valdez, J., and M. Bamberger, eds. 1994. Monitoring and Evaluating Social Programs in Developing Countries. Washington, D.C.: World Bank.

Vio Grossi, F. 1995. "La sociedad ecologica que nace con nosotros." El Canelo 64 (June): 30–33.

Waring, M. 1990. If Women Counted: A New Feminist Economics. San Francisco: HarperCollins.

World Bank. 1996. The World Bank Participation Sourcebook. Washington, D.C.: Sustainable Development Department, World Bank.

Document(s) 2 of 18