Document(s) 5 of 18

Is participatory evaluation compatible with results-based management? This chapter makes the case that participatory evaluation can serve the interests of results-oriented development interventions, and vice versa. Furthermore, there are some specific tools that are emerging in development practice that can enhance the interaction between participatory evaluation and results-based management.

The Shift to Results-Based Management

Recent years have witnessed a shift among the major donor agencies in the field of development cooperation from activity-based management systems toward results-based management systems. In turn, donors are obliging their partner agencies—Southern governments and nongovernmental organizations (NGOs) and private firms in both the North and the South—to adopt results-oriented management practices, as well. This change in approach has been prompted by calls for improved accountability and value for money in foreign-aid spending by deficit-oriented Northern legislatures and electorates, which are making similar demands on other areas of public spending.

While results-based management, or RBM, is only now being applied to development cooperation, its fundamentals are clear. Rather than designing, managing, monitoring, and reporting on inputs and activities of development programs and projects, donor agencies and their fundees and contractees will plan, implement, and assess interventions in terms of the extent to which they achieve their projected results. How these results are achieved is less important than what is, in fact, achieved.

Earlier versions of parts of this chapter were published in 1997 in the special participatory evaluation issue of Knowledge and Policy: The International Journal of Knowledge Transfer and Utilization, Vol. 10, No. 1/2, under the sponsorship of the International Development Research Centre, Ottawa.

Categorizing Types of Development Results: A Matrix Approach

Progress is being made in categorizing the types of results that can be generated by development interventions. RBM generally dictates that there are three essential types of results: outputs, outcomes, and long-term impacts.

An output is the most immediate, tangible result of an activity. An output could be, for example, the number of persons trained in a course. Outputs can usually be achieved within the period of one month to one year. An outcome is a medium-term result that is the logical consequence of the intervention achieving a combination of outputs. For instance, an outcome might be the application of new knowledge and skills by participants following their training course. Outcomes may take one to five years to achieve.

An impact is a long-term result that is the logical consequence of the intervention achieving a combination of outcomes and outputs. Carrying our example further, the new knowledge applied by trainees to their work might result in improved quality of life (for example, reduced incidence of disease, improved housing) for the local citizens served by the trainees. The achievement of an impact may require from five to twenty-five years.

The output-outcome-impact "chain" of results is useful in distinguishing among types of development results. However, results can also be defined by the level of intervention. Results may be achieved at the macro, or policy, level; at the meso, or institutional, level; and at the micro, or community, household, or individual, level. Successful development interventions are often characterized by mutually reinforcing activities and results "up" and "down" these different levels (see Beaulieu and Manoukian 1994).

Taken together, then, results can be conceived of as a matrix, with outputs, outcomes, and impacts across the horizontal axis and macro, meso, and micro levels down the vertical axis. Variations on this type of matrix are being used by the Canadian International Development Agency (CIDA) and its partner NGOs in evaluating, planning, and managing basic human needs, human rights, and poverty-reduction projects. Other development organizations employ variations on this matrix.

At the same time, development agencies are modifying other tools, such as the logical framework analysis, or logframe, from an activity orientation to a results orientation (CIDA 1997). This work continues.

Results-Based Management and Participatory Evaluation: Conflict or Convergence?

While there is growing (though sometimes grudging) recognition that results-based management may, in fact, be a helpful approach for development professionals operating in large bureaucracies, is it compatible with participatory forms of development practice? It would appear, perhaps, that the technocratic instruments of RBM represent the antithesis of the approaches used by social-justice activists and engaged scholars advocating participatory interventions.

But think again. When you sit with villagers under a tree, or with barrio residents in a church basement, and discuss their hopes and aspirations for themselves and their children, they usually frame their objectives for the future in terms of concrete results. They want better housing, less disease, a cleaner environment, more education, increased income, or greater political decision making. Progress toward these results can be tracked and evaluated, quantitatively and qualitatively. In fact, the ultimate beneficiaries of a development intervention—the poor, the disadvantaged, the disempowered—can, and should, lead the effort among stakeholders to define the results to be achieved by a given intervention. And they should be the leaders in reviewing performance on these results, as well.

"Customer-Driven" Development

It may be that the RBM–participatory evaluation combination can provide a vehicle whereby the "customers" of development interventions actually exert substantial influence over the process of defining and evaluating results. Customer-driven product development and service delivery enjoy wide success in both the public and private sectors around the world today. "Ask the customer," exhort prominent management books. While development cooperation is not merely a business or a service, it must, nevertheless, succeed. By using this language, however, practitioners need not abandon their values of social solidarity and the common good.

Building in the Bias: Enabling Participants to Define and Evaluate Results

There is an array of participatory methods that can be used to enable project participants to define and evaluate development cooperation results. The key to all participatory planning and assessment initiatives is ensuring a mechanism for shared control by project beneficiaries and other stakeholders. This mechanism might be a study team, research committee, working group, or task force. So powerful are the resources and skills of other stakeholders, though, that a bias must be built into the structure and process to ensure that authentically shared control prevails. This bias must be in favor of the poorest, least literate project participants. Gender differences, class differences, language, medium of communication, meeting times, meeting location, cultural norms and forms, and seasonal rhythms and responsibilities should all be considered in biasing the structure and process in favor of the most marginalized participants.

Once these arrangements have been made, the most appropriate combination of data collection and analysis methods should be selected. As Figure 3.1 shows, one basic decision concerns the mix of qualitative and quantitative methods to be employed. Another decision involves selecting the appropriate combination of qualitative data collection and analysis techniques. Field-based techniques, workshop-based techniques, self-assessment tools, and cultural methods are all possible choices on the methods "menu."

Participatory Tools for Defining and Evaluating Results

A number of tools by which project participants, working with allied professionals, can both define and assess development results of the interventions they are associated with have been developed over the past decade. One group of tools is workshop-based in nature. Workshop-based techniques seek to create a learning atmosphere and a safe environment in which all parties feel free to participate. The workshops usually involve a series of activities designed to build consensus on problem analysis and action to be taken.

One of these tools is ZOPP, which was developed by the German technical agency GTZ and its partners in Africa and elsewhere. ZOPP is actually a project planning process that is undertaken through a series of stakeholder workshops. Participants, who include local government officials as well as community representatives from the project area, collectively generate a problem tree that describes relationships among the issues constraining local and regional development. Then, by reversing the problem tree, participants generate an objectives tree, which sets out the actions needed to remove these constraints. The workshops focus on which elements in the objectives tree will be addressed in the project being planned. A project planning matrix or logical framework is then developed by the participants, and the project is sent to each of the stakeholder groups for official approval (World Bank 1996). A variant of the ZOPP approach is PC/TeamUp, which places emphasis on team building. PC/TeamUp also uses a special computer software package to assist teams in planning projects. This approach makes use of tree analysis, logical frameworks, SWOT (strengths, weaknesses, opportunities, threats) analysis, bar charts and other scheduling tools, and total-quality management approaches (World Bank 1996).

Another workshop-based technique is social gender analysis (SGA). Developed originally by the Coady International Institute of St. Francis Xavier University in Canada, in cooperation with its overseas partners, SGA takes project stakeholders through a series of exercises that analyze the disadvantages faced by various groups in the project area and the resources and strategies required for them to achieve social sustainability. Class, gender, access to and control of resources, benefits from participation, and links between local and national or global levels are all key elements in SGA workshop discussions. The SGA process results in a project strategy that includes clear objectives, an implementation plan, and measures for monitoring and evaluation. SGA is a gender-sensitive tool, but its

Figure 3.1 Possible Methods of Participatory Research and Evaluation

use is not limited to gender concerns. Rather, it is concerned more broadly with social sustainability (see Plewes and Stuart 1991).

Another group of tools involves field-based techniques. Perhaps the best known of these techniques is participatory rural appraisal (PRA), which evolved from rapid rural appraisal. Originally developed for rural areas and farming systems by the Institute of Development Studies in Sussex, England, PRA has spread around the world to many other sectors, including projects involving the urban poor. PRA employs group animation techniques and problem analysis exercises to promote sharing of information and joint action among stakeholders. To this end, PRA uses semi-structured interviews, focus groups, wealth ranking, community meetings, needs assessment, mapping, transect walks, and other participant-observer techniques (Chambers 1993, 1994a, 1994b, 1995; Nelson and Wright 1995; Gallardo, Encena, and Bayona 1995; Okali, Sumberg, and Farrington 1994; Scoones and Thompson 1994; Kumar 1993).

Another approach is participatory research or participatory action research (PR/PAR), representing quite a different tradition. PR/PAR involves both workshops and fieldwork, but with two elements that make it distinct from the other approaches. First, PR/PAR takes a relatively longer time to implement—months, even years, rather than days or weeks. Second, PR/PAR emphasizes issues of power more than most other methods, in both its analysis and its action. PR/PAR uses a more oppositional framework vis-à-vis elites than other participatory methods, which tend to try to co-opt elite involvement and acceptance.

Apart from these distinctive features, PR/PAR makes use of a full range of workshop facilitation techniques, field observation methods, and, particularly, group dialogue methods. Projects in this tradition have also employed computerized data analysis, oral history, popular theater and other media, and many other techniques. The insider-outsider dialectic is monitored carefully in PR/PAR, with special efforts made to give voice to the insiders who are on the margins of local communities. The same obstacles faced by other approaches— obstacles related to class, gender, ethnicity, wealth, and power—must be continually addressed by PR/PAR, as well. PR/PAR is often carried out by social movements and frequently has a longer-term goal of broader, structural transformation (see Park et al. 1993; Fals-Borda and Rahman 1991; Manoukian 1996).

Still another cluster of self-assessment tools also uses both workshops and PRA techniques but focuses the analytic work on communities developing their own indices of development for planning and impact assessment. In northern Ghana, for example, a North-South monitoring team on a large rural development project developed a tool entitled the Village Development Capacity Index (VDCI). Funded by the CIDA and the government of Ghana, this index was designed to rank communities in the project area in terms of their performance on poverty indicators and on village development capacity indicators. Poverty indicators, for which data were collected through household interviews, included safety of water sources, literacy rate, food security, and household expenditure patterns. The tool permits other relevant indicators to be added, as appropriate to the project context.

Village development capacity indicators, the data for which were collected through key-person interviews, field observation, and focus groups, included status of village organization, previous experience of the village in managing development interventions, level and types of village cooperation and mutual aid, and the status of infrastructure and assets under village control, both collectively and by individual residents. Stakeholders participated in the process of allocating scores on each of these indicators to the villages under study. Village representatives then reviewed the findings of the VDCI exercise in a workshop, provided feedback, and discussed action to be taken through the overall development project to redress weaknesses identified in the communities. Scores on the VDCI for each village were then tracked by monitoring teams over time in order to assess progress (see Chapter 4).

The same research team then developed a matrix for measuring the extent and nature of partnership between African and Canadian NGOs. A partnership and institution-building matrix was constructed, based on the experience of ten case study partnerships from several subregions in Africa. This matrix included indicators of the compatibility of the two partner-agencies, operational principles, operational mechanisms, commitment, support modes, intervention modes, outcomes, and sustainability of the partnership. Data for these indicators were collected through key-person interviews, document review, field visits, and stakeholder workshops and meetings. Further work on this index has been carried out for additional Canadian-African NGO partnerships (Gariba, Kassam, and Thibault 1995; Kassam, Gariba and Mothenbesoane-Anoh 1996).

Work on both the VDCI and the partnership matrix built on earlier efforts to construct regional and community-based self-assessment tools for use in underdeveloped parts of Canada. In the 1980s, research sponsored by the Economic Council of Canada had resulted in the construction of a development index to measure disparities among subregions. This index was based on more than fifty statistical indicators of economic performance (employment, income), capacity (labor force skills, access to capital), energy (new business start-ups, competitiveness), and policy measures (Lamontagne 1994). Along similar lines, the Canadian Association of Single Industry Towns developed a vulnerability checklist that enables community leaders to assess the extent of economic diversification in the local economy and of existing organizational and material resources on which to build a broader-based and more sustainable economic strategy before a local plant or mine is forced to shut its doors by decision makers far away (Decter 1993).

A more recent generation of this work appeared in aboriginal development organizations in Canada in the early 1990s. Aboriginal leaders and economic consultants, working in a national committee, created a guidebook for aboriginal communities to gather their own data for development planning purposes. The guide helps communities assess the community situation, set development priorities and goals, identify activities to meet those goals, implement development activities, and monitor and evaluate the results. The guide includes advice on selecting indicators to track development progress and worksheets that link strategic goals with specific activities. Most of the indicators suggested by the guide are output indicators, such as community revenue by source and value of loans provided by the local development corporation. Impacts on household incomes and quality of life are not emphasized in the guide but could be easily accommodated. The key here, as with all self-assessment instruments, is that the process calls for the community to choose its key indicators itself (Lamontagne 1994).

Developing Appropriate Results Indicators

Participatory planning and evaluation efforts demand simple, reliable indicators of development results that are agreed to by participants and other stakeholders. In the case of antipoverty, human rights, and basic needs projects, the most crucial results indicators relate to money and power. It is also important that these indicators of results be identified at different levels: macro, meso, and micro. Moreover, impact indicators should be emphasized. Although they are the most difficult and complex indicators to achieve, they are, ultimately, the most important tests of whether a development intervention has succeeded.

Micro-Level Impact Indicators

Much work has been done on impact indicators of individual, household, business, and community gains in income by the poor. One finding from this work is that, even at the micro level, there are multiple levels—or sublevels— of impact. In rural development projects, for example, improved grain varieties and expanded irrigation works (inputs) may result in increases in dry-season rice production (output). Such increased production is then sold, generating new income for participating households (impact) (see, for example, van Dusseldorp 1993). In turn, this new income (an impact at another level) is used to purchase rice-milling equipment, start a small kiosk, or construct a new well (impacts), all of which could further influence the quality of life of the household. These potential "onward" impacts must be tracked in the future, as well. Such income gains may also be used by households to pay user fees for village services such as schools, health clinics, or marketing cooperatives, thus boosting infrastructure assets at the community level—and serving as inputs to those services.

Microcredit programs also involve multiple levels of impact. Loans and technical training provided by such programs (inputs) can result in increased owner drawings through improved profitability, expanded sales, and commercial sustainability in participating microbusinesses (enterprise-level impacts). In this way, very small enterprises create employment income for individuals and households (inputs), which generates multiplier effects in the community when households purchase other goods and services and pay taxes or user fees (impacts). The growth of participating microenterprises may also foster the establishment of new businesses by other households to supply the successful enterprises in the program (ACCION/Calmeadow Foundation 1988; Ashe 1992, 1995; Otero and Rhyne 1994). And revenue accruing to each community in which such programs are operated can reinvest taxes and user fees in new infrastructure, services, and training, thereby further building the capacity of the community to develop itself.

Like all development interventions, however, enterprise support programs must assess costs as well as benefits. Fiscal cost-benefit studies seek to assess the return on taxpayer investment. Costs include tax credits and deductions as well as grants associated with the program under study. A percentage of these costs can be allocated to the specific enterprises assessed. Among the benefits calculated for this type of analysis are increased tax revenues from the operations of the enterprises that are attributable to program support and avoided income-support costs (unemployment insurance, welfare). In addition, tax revenues from the firms' suppliers and employees attributable to the support of the program are estimated. Comparing fiscal costs and benefits thus enables evaluators to estimate a payback period for overall public expenditure on the enterprises under study and, if desired, on the program as a whole (Jackson and Lamontagne 1995). Although this approach is obviously more appropriate to Northern contexts, it can be adapted to developing country situations as well.

The work of Moser (1989) and others (Plewes and Stuart 1991) in the field of gender and development has generated useful micro-level impact indicators on power in particular. The gender and development approach is based on an assessment of women's practical and strategic needs, especially as indicated in gender roles in both productive and reproductive work in the household and in managing the community. Practical gender needs relate, for example, to women's access to adequate water supply, health care, and employment. Strategic gender needs relate to the legal and property rights of women, access to credit, equality of wages, freedom from domestic violence, and women's control over their bodies. As the impacts of development interventions yield positive impacts in these areas, women's power increases at all levels: in the household, in the community, in the nation.

Meso-Level Impact Indicators

Increasing the income of the poor also demands impact assessment at the meso, or institutional, level. Institutions promoting poverty alleviation—ministries, NGOs, social movements, and donor agencies—must build their own capacity to assist the poor in achieving gains in income and power. Following Lusthaus, Anderson, and Murphy (1995), all institutional assessments must include analysis of the external environment, organizational motivation, organizational capacity, and organizational performance. With respect to organizational performance, some key impact indicators in antipoverty efforts include financial sustainability of the organization as a whole, self-sufficiency or sustainability of poverty alleviation programs run by the institution, and the percentage of communities assisted by the organization that have moved from being categorized as poor to being categorized as less poor or moderately well-off. Other relevant indicators can include income per capita, unemployment rates, and labor force participation in the area served by the institution, as measured over time.

The relationship between institutional capacity and performance, on the one hand, and community development capacity, on the other, must be assessed in detail. The perceived relevance of the services of the institution to its clients (or "customers") is also a crucial factor deserving careful study. Further, gender-disaggregated data must be collected for all indicators to permit an assessment of the comparative gains made by men and women as a result of the institution's efforts to strengthen itself.

In the field of social development, Oxfam-UK has suggested that indicators of participatory management structures and processes are especially relevant. Such indicators include:

• Evidence of shared decision making among participants and staff;

• Signs of commitment among participants to the group's goals and activities;

• Evidence of shared leadership;

• Signs of solidarity and cohesion;

• Capacity for self-reflection and critical analysis; and

• Capacity to take action in relation to problems identified.

Such indicators may be applied to test the performance of development institutions, NGOs, project management teams, and local-partner organizations (Marsden and Oakley 1990; Marsden, Oakley, and Pratt 1994).

Macro-Level Impact Indicators

All macro-level income gains by the poor can be measured by conventional indicators over time, particularly gross domestic product (GDP) per capita. Other key indicators, some of which are used to calculate the United Nations Development Programme (UNDP) Human Development Index, include share of national income of the lowest 40 percent of households; percentage of the population in absolute poverty; and public expenditures on social programs, health, and education as a percentage of GDP. Additional relevant indicators include daily calorie supply, access to health services, and the prevalence of radios, telephones, and motor vehicles (see UNDP 1994, 1995).

Other indicators can be assessed at the macro level as well. In some cases, new policies and legislation in favor of the poor can be attributed to the demonstration effects of antipoverty projects or to the lobbying of NGOs, government ministries, donor agencies, and social movements on behalf of the poor. Such programs may confer new economic benefits (for example, food prices, credit availability) or civil rights (for example, the general right to organize, women's rights to own land or business assets). These impacts, in turn, will serve as "inputs" that can be transformed, at all levels, into further gains in money and power for the poor. Assessing whether and how such transformation occurs is an important task.

At the national level, research indicates that one year of schooling for girls or young women can reduce the fertility rate by 5 to 10 percent. Reduced fertility boosts economic growth rates on a per capita basis. In addition, increased access to credit for women has been found to do more to reduce poverty and spur investment than does increased credit provision for men (World Bank 1995). At the macro level, educational attainment, fertility rates, and access to credit are all important indicators of gains in power and money by poor women and by women in general.

Useful work had been done on basic indicators of gains (or losses) in human rights in development projects. Norway's Madsen (1991) developed guidelines for project design and evaluation at the micro level that test project performance against international conventions of the Intentional Labor Organization (ILO). The specific rights that Madsen recommends be tested in this way relate to forced removals and resettlements of peoples, land rights, the right to organize, child labor, forced labor, gender discrimination, conditions of employment (including health and safety, as well as worker remuneration), and the rights of participation. The ILO and some Scandinavian aid agencies have included protections for some of these rights in project agreements with Southern government ministries and Northern consulting firms. The World Bank, the Asian Development Bank, and the Organization for Economic Cooperation and Development's Development Assistance Committee published specific guidelines intended to protect project participants against forced resettlement.

Madsen argues, as do most advocates of human rights generally, that project-level—that is, usually micro-level—performance on human rights indicators must be tested against the standards of international conventions and, if relevant, national laws. Thus, human rights impact assessment must necessarily be multilevel in nature.

Furthermore, it is likely that, among marginalized peoples whose rights have been systematically denied for centuries, impact assessment practitioners will need to educate project participants about their rights while the research is being carried out. There is a moral obligation to do so, as well as a program-matic or professional obligation. Such an educational process can be facilitated greatly by participatory assessment techniques, especially the direct representation of project beneficiaries on assessment teams.

South House Exchange (SHE), a consulting firm specializing in human rights and development, has authored a training curriculum based on case studies of NGO interventions in this field. One case study is about a project involving Guatemalan refugees living in refugee camps in Mexico who are planning their return home. Trainees using the case study are asked to draw up a plan for the refugees' return; the plan must set out the roles of the refugees, a local women's NGO, and foreign partner NGOs. Trainees are also asked to identify "indicators that the plan's elements were working" (SHE 1995, 5). Almost by definition, such indicators must:

• Relate to gains or losses in human rights by the refugees and the local NGO, measured through the application of international standards;

• Disaggregate data by gender;

• Assess the effectiveness of the intervention at the micro (individual) and meso (institutional) levels in the South and at the meso level in the North; and

• Pay special attention to impacts related to NGO capacity-building and North-South partnerships.

Such multilevel, multidimensional impact assessment can be significantly enhanced by participatory methods. The participation of women refugees in a Southern-led project evaluation process would yield especially rich insights and also build additional project commitment and capacity among participants.

Moving the Practice Forward

Clearly, the potential (and the limitations) of the relationship between participatory evaluation and results-based management is only beginning to be understood. It will be through collective action and reflection, globally and locally, that this area of development cooperation practice can be advanced. There will be pitfalls and complexities. But substantive gains can be made, especially if engaged practitioners are able to regularly exchange views, experiences, and techniques at the country, regional, and global levels. Networks, newsletters, monographs, and case studies would contribute much to this learning process.

Participatory evaluation can be compatible with results-based management. Whereas advocates and practitioners of RBM are not necessarily interested in participatory development approaches, advocates and practitioners of participatory evaluation can find productive ways of blending RBM into their work. There is already an array of practical tools and indicators that can be mobilized to this end. The convergence of RBM and participatory development promises to be a rich and creative site of development cooperation practice in the years ahead.

References

ACCION International and the Calmeadow Foundation. 1988. An Operational Guide for Micro-Enterprise Projects. Toronto: Authors.

Ashe, J. 1992. "Microlending Programs." Pp. 559–69 in The Social Investment Almanac, edited by P. D. Kinder, S. D. Lyndenberg, and A. L. Domini. New York: Holt.

———. 1995. "Strategies for Financing Micro-Enterprises: The Working Capital Model." Working Paper, Centre for the Study of Training, Investment and Economic Restructuring, Carleton University, Ottawa.

Beaulieu, R., and V. Manoukian. 1994. "Participatory Development: A Brief Review of CIDA's Experience and Potential." Policy Branch, Canadian International Development Agency, Hull.

Canadian International Development Agency. 1997. "The Logical Framework: Making It Results-Oriented." Performance Review Branch and the Bilateral Branches, CIDA, Hull.

Chambers, R. 1993. Challenging the Professions: Frontiers for Rural Development. London: Intermediate Technology Publications.

———. 1994a. "The Origins and Practice of Participatory Rural Appraisal." World Development 22 (7): 953–69.

———. 1994b. "Participatory Rural Appraisal (PIA): Analysis of Experience." World Development 22 (9): 1253–68.

———. 1995. "Paradigm Shifts and the Practice of Participatory Research and Development." Pp. 30–42 in Power and Participatory Development: Theory and Practice, edited by N. Nelson and S. Wright. London: Intermediate Technology Publications.

Decter, M. B. 1993. "What We Can Do for Ourselves: Diversification and Single Industry Communities." Pp. 136–200 in Regional Development from the Bottom Up, edited by M. Lewis and D. McNair. Vancouver: Westcoast Development Group and the Centre for Community Enterprise.

Fals-Borda, O., and M. A. Rahman, eds. 1991. Action and Knowledge: Breaking the Monopoly with Participatory Action-Research. New York: Apex Press.

Gallardo, W. G., V. C. Encena, and N. C. Bayona. 1995. "Rapid Rural Appraisal and Participatory Research in the Philippines." Community Development Journal 30 (3): 265–75.

Gariba, S., Y. Kassam, and L. Thibault. 1995. "Report of the Study of Partnership Institutional Strengthening." Partnership Africa Canada, Ottawa.

Jackson, E. T., and F. Lamontagne. 1995. "Adding Value: The Economic and Social Impacts of Labor-Sponsored Venture Capital Corporations on their Investee Firms." Canadian Labor Market and Productivity Centre, Ottawa.

Kassam, Y., S. Gariba, and E. Mothenbesoane-Anoh. 1996. "Strengthening Partnerships for Community Health: Report of the Evaluation of the Impact of Partnership for Capacity Building in Canada's International Immunization Program." Canadian Public Health Association, Ottawa.

Kumar, K., ed. 1993. Rapid Appraisal Methods. Washington, D.C.: World Bank.

Lamontagne, F. 1994. "Development Indicators and Development Planning: A Case Study." Pp. 208–22 in Community Economic Development: Perspectives on Research and Policy, edited by B. Galway and J. Hudson. Toronto: Thompson Educational Publishing.

Lusthaus, C., G. Anderson, and E. Murphy. 1995. Institutional Assessment: A Framework for Strengthening Organizational Capacity for IDRC's Research Partners. Ottawa: International Development Research Centre.

Madsen, H. L. 1991. "Towards Human Rights Assessments of Development Projects." Working Paper, Program of Human Rights Studies, Chr. Michelson Institute, Fantoft, Norway.

Manoukian, V. 1996. Guest lecture on participation. Presented to the graduate course on Project Management in Developing Countries, School of Public Administration, Carleton University, Ottawa.

Marsden, D., and P. Oakley, eds. 1990. Evaluating Social Development Projects. Development Guidelines No. 5. Oxford: Oxfam UK.

Marsden, D., P. Oakley, and B. Pratt. 1994. Measuring the Process: Guidelines for Evaluating Social Development. London: David Bruman Associates.

Moser, C. 1989. "Gender Planning in the Third World: Meeting Practical and Strategic Needs." World Development 17 (11): 1799–1825.

Nelson, N., and S. Wright, eds. 1995. Power and Participatory Development: Theory and Practice. London: Intermediate Technology Publications.

Okali, C., J. Sumberg, and J. Farrington. 1994. Farmer Participatory Research: Rhetoric and Reality. London: Intermediate Technology Publications.

Otero, M., and E. Rhyne, eds. 1994. The New World of Microenterprise Finance. West Hartford, Conn.: Kumarian Press.

Park, P., M. Brydon-Miller, B. Hall, and T. Jackson, eds. 1993. Voices of Change: Participatory Research in the United States and Canada. Toronto/Westport, Conn.: OISE Press/Bergin and Garvey.

Plewes, B., and R. Stuart. 1991. "Women and Development Revisited: The Case for a Gender and Development Approach." Pp. 107–32 in Conflicts of Interest: Canada and the Third World, edited by J. Swift and B. Tomlinson. Toronto: Between the Lines.

Scoones, I., and J. Thompson, eds. 1994. Beyond Farmer First. London: Intermediate Technology Publications.

South House Exchange. 1995. "Guatemalan Refugees: Preparing for the Return Home— A Case Study." SHE, Ottawa.

United Nations Development Programme. 1994. Human Development Report 1994. New York: Oxford University Press.

———. 1995. Human Development Report 1995. New York: Oxford University Press.

van Dusseldorp, D. B. W. M. 1993. Projects for Rural Development in the Third World: Preparation and Implementation. Wageningen, Netherlands: Wageningen Agricultural University.

World Bank. 1995. "Toward Gender Equality: The Role of Public Policy." World Bank, Washington, D.C.

———. 1996. World Bank Sourcebook on Participation. Washington, D.C.: World Bank.

Document(s) 5 of 18