Document(s) 3 of 18

Most development projects in developing countries funded by development assistance agencies, especially those large-scale projects implemented by the governments of the recipient countries, involve external and internal evaluations that are predominantly quantitative in nature involving a lot of statistics. The evaluation of the developmental impacts of these projects is also portrayed in numbers and statistical configurations. Very little attention is paid to the qualitative empowering impacts of development processes that require the use of a participatory evaluation methodology.

Several reasons can be cited to account for the neglect of undertaking participatory evaluation and obtaining qualitative data:

• The institutional demands to justify the significant investment of large sums of money and to lubricate the chain of accountability make it imperative to obtain the so-called hard data on project performance.

• Participatory evaluation in terms of its value and methodological validity is either not well understood or not fully recognized by the donors.

• The bureaucrats in the funding agencies and other stakeholders working under considerable pressure and stress do not have enough time or patience to plow through the long and detailed texts generated by participatory evaluation methods. They tend to prefer "at-a-glance" information presented through statistical tables.

• In many quarters, development work is still perceived predominantly as a technical and mechanistic exercise rather than as a complex and dynamic process of transformation.

This chapter was first published in Knowledge and Policy: The International Journal of Knowledge Transfer and Utilization, Vol. 10, No. 1/2, under the title "The Combined Use of the Participatory Dialogue Method and Survey Methodology to Evaluate Development Projects: A Case Study of a Rural Development Project in Bangladesh." Reprinted by permission of Transaction Publishers, New Brunswick, New Jersey.

• The constraints of evaluation budgets inhibit the use of participatory evaluation, which is time-consuming and therefore more costly.

• Many consultants hired to carry out evaluations are either not well versed in or do not recognize the legitimacy and value of participatory evaluation approaches and techniques, according to their norms of what constitutes a "scientific, objective, and empirical" inquiry. It is argued that participatory evaluation produces knowledge that is "subjective, soft, and impressionistic."

While the necessity and importance of statistical and quantitative evaluation—especially of large-scale projects involving a large number of beneficiaries—are not denied, qualitative participatory evaluation has a complementary role to play in producing a body of unique and illuminative data that cannot be produced by the conventional research methodology. Participatory evaluation produces insights and perceptions that, at the very least, represent value added. Knowledge produced by participatory evaluation transcends the statistical silhouette of reality and presents a project's "flesh and blood," as it were, thereby giving a more intimate feel of the "pulse" of a project.

The combined use of survey and participatory evaluation methodologies produces macro- and microknowledge of reality, each informing and enriching the other. It enables one to see, so to speak, both the canopy of a forest as well as the individual trees, plants, and creatures underneath that canopy. What is often and sadly not realized by the project stakeholders is that their evaluation needs and objectives are better served and enriched by undertaking an evaluation that combines both traditional and participatory methodologies.

This case study is an example of an internal evaluation of the training component of a massive, long-term, and multimillion-dollar rural development project in Bangladesh conducted by the author and his Bangladeshi counterpart (Kassam and Kamal 1992) that combined both the traditional survey and participatory evaluation methods. The evaluation was greatly facilitated by the project's Canadian resource team based in Dhaka, as well as by the Bangladesh Rural Development Board, the project's implementing agency.

The Project

The project, named RD-12, is a rural development project for the assetless rural poor (the bittaheen), funded by the government of Bangladesh and the Canadian International Development Agency (CIDA) and implemented by the Bangladesh Rural Development Board (BRDB), a governmental agency. It started in 1988 as a five-year project—a continuation and expansion, with some modifications, of a previous phase of the project called RD-2 and funded by CIDA as well. The project was extended to June 1997.

The purpose of the project is, first, to assist assetless rural men and women by providing them with skills, training, and credit necessary for income generation. In addition, the project is intended to strengthen the capacity of BRDB to plan, implement, and sustain development among the rural poor.

The project promotes the creation of organizational structures of and for the rural poor designed to foster local leadership and reduce exploitation, enabling participation of the rural poor in local affairs and markets and thus helping to ensure an adequate supply of inputs and services to them. At the same time, the project contributes to institution building within the government of Bangladesh by continuing to improve awareness of the needs and constraints of the rural poor, the institutional requirements for addressing these needs, and the ability of the BRDB and key government ministries to meet these requirements.

E.T. Jackson and Associates Ltd., an Ottawa-based management consulting firm, has served as the Canadian Executing Agency (CEA) for the project. The firm supports the delivery of credit, training, and organizational services by the BRDB. CIDA provides $53 million toward this bilateral project, including $13 million for a revolving loan fund, and the government of Bangladesh contributes $3 million.

By the end of the initial five-year period ending in June 1994, RD-12 had mobilized 16,366 village-based Bittaheen Cooperative Societies in seventeen districts (six greater districts) of the project area. An estimated 500,000 members of the new bittaheen societies benefit directly. Almost 3 million household members benefit indirectly. Three-quarters of the society members have borrowed funds to start income-generating activities or microenterprises, and the average loan recovery rate has been about 94 percent. Between 1990 and 1994, RD-12 delivered two million person-days of training to the project participants across Bangladesh. BRDB staff received over 70,000 person-days of training during the same period. The large numbers of people trained in RD-12 make this project the largest human resource development project that CIDA has supported.

Through the management of the project by Jackson and Associates as the CEA, gender equity and social development strategy have remained important priorities for the project right from the inception mission. The firm contracted Bangladeshi and Canadian specialists to provide training for BRDB in gender and social analysis (GSA) and has promoted employment equity at all levels of the project.

Over 70 percent of RD-12's borrowers are now women. More than two-thirds of about 11,500 loan societies organized by the project staff are women's societies. The repayment rate for the women's societies outpaces that of the men's groups. Women members also mobilize higher average savings than their male counterparts. Furthermore, 47 percent of the 2,500 field staff of the BRDB are women, a percentage that has set a new standard for employment equity in the government of Bangladesh.

The firm's commissioned study that assessed progress on GSA found that RD-12 has achieved much in terms of promoting the integration of women in the development process but still could do more to facilitate attitudinal change on gender issues among men at all levels of the project.

Project Evaluation Methodology

The methodology used to carry out the evaluation of the training component of RD-12 consisted of documentary review, survey methods, and participatory evaluation methods. The survey methodology included an interview questionnaire that contained both closed- and open-ended questions, and this instrument was used with a sample of 2,104 beneficiaries and 126 field functionaries. The survey was conducted by a team of twenty Bangladeshi researchers with an equal gender balance. In addition, case studies of four men's and four women's cooperative societies were prepared, focusing on their development, achievements or failures, and problems encountered.

Under participatory evaluation, dialogues were conducted with a random sample of ten beneficiaries (members of cooperative societies), five women and five men. The dialogue method was used in order to gain a deeper and qualitative understanding of the dynamics of social and economic transformation among the beneficiaries, including the psychosocial and other developmental impacts on the beneficiaries of the project.

The Use of the Dialogue Method and Its Conceptual Context

In addition to the purpose of illuminating the qualitative developmental changes that occurred among the beneficiaries, the dialogues were intended to complement, supplement, and enrich the quantified data obtained through the interview questionnaire, case studies, and file review. In this way, the dialogues provided "flesh and blood" to the quantified findings on the impacts of training and the overall project performance that could not possibly be obtained by conventional research instruments. Furthermore, arising out of the priority placed on the gender dimension of the project, the dialogues were also intended to give a voice to the women beneficiaries.

The dialogues were conducted by two Bangladeshi researchers (one woman and one man), who were given a special orientation on how to conduct the dialogues. Using open-ended and nonleading questions, the dialogues were taped and transcribed verbatim. In conducting the dialogues, what was crucial was not only to record the participants' own thoughts and feelings but to do so in their own words and idiom and in their own style of expression. In transcribing the dialogues, they were not tampered with in any other way except for minimal editing for linguistic errors. After a dialogue was taped, it was played back to the participant.

The dialogue method is part of qualitative and participatory research and evaluation methodologies (see Kassam and Mustafa 1982) based on the work of, among others, Paulo Freire (1970b), Orlando Fals-Borda (1977), Budd Hall (1975), W. Filstead (1970), Kathleen Rockhill (1976), M. Parlett and D. Hamilton (1972), Michael Pilsworth and Ralph Ruddock (1975), R. Chambers (1978), Peter Oakley (1986), M. Patton (1987), H. Richards (1985), and C. Weiss (1972).

The investigation of psychosocial and qualitative changes in people's lives can best be illuminated by adopting an anthropocentric approach, which involves the interpretation of reality exclusively in terms of human values and experience (Kassam 1979). The dialogue method helps to portray the uniqueness of individual perceptions of and experiences in a development process. Whereas the information obtained through the use of conventional evaluation instruments such as questionnaires, interview schedules, or checklists superimposes the description of empirical social reality on to a predetermined framework of that reality, genuine dialogue eliminates most of the preconceived and preconstructed elements of the traditional evaluation process. Epistemologically, therefore, the dialogue provides a more accurate and authentic reflection of social reality. The dialogue makes it possible for social reality to be described on a "clean slate," as it were, and fills in the silhouette produced by the use of quantitative survey methodology. The dialogue helps to capture and portray the dynamics of the social and economic transformation among the beneficiaries of a development project.

The dialogue method is used not to quantify, verify, or predict the personal and qualitative social impact of a development program but rather to "illuminate" it (see Parlett and Hamilton 1972). While the dialogue method makes it possible to get a more accurate glimpse of the total human context with all its complexities and social interactions, the very process of dialogue serves a number of other important purposes. Through dialogue, the people participate actively in the evaluation process, whereas the conventional methods and instruments treat people as those who are "researched upon" and as mere sources of information. Through dialogue, the participants of a development process are treated as central subjects and actors of that process and are given the opportunity, to use Freire's words, to "name the world." The dialogue very often serves as a liberating experience for the participants.

An Example of a Dialogue

The following dialogue was conducted with Ms. Mosammat Jainab Bibi, the manager of Shahapur Bittaheen Women's Cooperative Society in the district of Jamalpur.

Ms. Bibi joined the society in 1984 and is now the manager. She has studied up to Class III and has one son and two daughters. Her son attends Class X and one daughter is married. She is involved in paddy-husking and poultry-rearing activities. She received training on members' education, cow rearing, and poultry. She is also attending the manager's training regularly.

"I joined the society in 1984. Mr. Tara, the local upazila official, had distributed fifteen wheat feeding cards among fifteen vulnerable female villagers. One day he told us to mobilize another fifteen women to form a society. We did it and he helped us to form a BRDB society. We deposited Tk. 1 per week as savings. We were not united then. We did not know each other. When the other fifteen women joined us we held a weekly meeting. We continued it and Mr. Tara would also attend. We generated a little fund and Mr. Tara and we deposited it at BRDB office. With our consent he formed a BRDB society for us. We deposited Tk. 1 or 2 as savings in 1984. We did everything by ourselves like raising savings, depositing them at the bank, issuing verity vouchers, taking receipts from bank, etc. We registered our society on 30.3.85. It is nearly seven or eight years that we have been running our society.

"Look, we are poor. We had no dignity in the local area. We worked in others' houses. At that time they helped us know the path of life. They invited us to receive training from BRDB so we could run the society smoothly. Our husbands were very cruel to us then. They threatened us in many ways. They challenged us saying what sort of law the government had established that all the women should have to hold meetings neglecting all their household work. Not only that—the rich also taunted us and ostracized us.

"We requested the BRDB to help us with credit support so that we could husk paddy. My husband works all day. We thought if we could husk paddy by taking a loan we could deposit Tk. 1 or 2 as savings besides repaying the loan. Considering our request, they provided each of us with Tk. 500 as credit support. We bought 2 maunds of paddy each and husked it. We repaid the loan installments and deposited Tk. 1 or 2 as savings from the profits we earned by paddy husking. We had no poultry so we bought some through the profit.

"We take a loan every year and husk paddy which provides us with some profit. We spend a little of that for the education of our children. Previously the Railway School was completely reluctant to admit our children. The directors of BRDB asked us once: 'How many are you?' We answered we were forty-six. They replied that means at least forty-six children and advised us to go and admit our children in the Railway School and gave us hope that they would help us. When we went there, the teachers were in panic. We asked them: 'Why do you not want to admit our children—because we are poor? Since we have no clean clothes? Why do you admit rich children?' Then the teachers agreed to admit our children.

"They told us to pay Tk. 10 for each boy or girl as an admission fee. We had protested earlier but realized very soon that we had to pay Tk. 10 because it was compulsory for everyone. They gave us seven days to collect the money. We collected the money and admitted our children in the school. That's how we overcame that problem.

"One of our members lost her husband. They were poor though they had a rich neighbor. The whole day had passed but she could not perform the funeral ceremony due to lack of money. She requested the rich neighbors to help her but they did not respond. Having no alternative, she came to us. We called all forty-six members and held a meeting. We took a decision to contribute according to our ability, whether it was Tk. 5 or 10. We raised Tk. 200 and purchased cloth for Tk. 110 and spent the rest for soap and other things. Ten of us read the Holy Quran and buried the dead body after the Jumma prayer. That was another problem we had overcome.

"The rich persons took the matter negatively. They were in fear that their prestige would suffer by our united movement. They expressed their bitter opinions: 'These women worked in our houses earlier. What is happening to them? Why are they educating themselves? We would like to help them to bury the dead man, but those women do the job without taking any help! Our prestige will sink!' They held a meeting afterward.

"One man had a shop on a pucca [paved] road where members dry their paddy. The elites wanted to stop us drying paddy on the road but our members ignored them. They said the government is the owner of this road and if government puts any objection then we will not dry our paddy on the road. A son of a member went to that shop to buy molasses. But the shopkeeper did not want to sell to him. He asked the grocer: 'Why do you not sell to me? I have money!' The grocer stopped him and punched him on the chest. The boy returned home crying. His mother informed the society about the incident and sought justice. We went to Mr. Ibrahim, our Union Member, and appealed for justice. He advised as to go to the shop. We went there along with other poor villagers. The grocer taunted us and said: 'How dare these women not keep purdah! Why are they holding a meeting in my shop? I am a Mondal [his family name]! Why have these shameless beggars come here?' We replied that our son had been assaulted by you and we want justice. The Upazila Member came and called the grocer. He reproached him severely and asked him to beg pardon. At last the grocer gave us Tk. 300 for medicine for the injured boy and another Tk. 200, totally Tk. 500 as penalty.

"Through the Society I have received skill development and membership education training. I have conducted weekly meetings of the members. I have received poultry and cow rearing training and I cultivate fish and vegetables. I have learned how to rear poultry and how to keep the poultry shed. Shock sick birds should be kept separately. Dead birds should be buried in a hole. Rani khet is a disease of poultry birds. I have told all these thing to the members in our weekly meetings. I have recommended they rear poultry and to sell their eggs to generate savings. Fowls can be sold for Tk.100. Cloth can be purchased from it. Thus we have been trained. The symptoms of Rani khet are lime-like stools and drowsiness of the birds with a high temperature.

"We cultivate fish collectively. We have no pond but we requested one old man to provide us with his pond for fish culture. 'We will cultivate fingerlings or young fish in your pond. We will sell the fish after two to three months regularly and the rest will be yours,' we said. We took the pond under this condition and we earned Tk. 880 in two months. We have been cultivating fish for five to six years and earn Tk. 500–600 each two to three months. We maintain the pond and take care of it, catch the fish, and sell them. We do not get the help of any men. We have utilized our training fully.

"We also rear cows. Presently there are two or three cows which provide two or three sheer of milk daily. We know from our training that straw mixed up with urea and potash is a good feed for cows. We provide it to them and the milk quantity has increased four to five times.

"There are no conflicts in our society. Unity prevails among us. Before we were in misery. Now we are working unitedly and our distress is disappearing.

"Previously we worked as maid servants grinding spices. Our hands swelled. We would be the poorest if BRDB had not supported us with loans. These loans saved us from going to others' houses for work. We have taken loans six times from RD-12. We have taken loans twice earlier, both for Tk. 5,000 and have bought cows with these loans. I sold my first cow since it became sick. I bought a cow again and reared it according to the training I received. We need more education. We make mistakes due to insufficient education. Training helps us to increase our income. We could earn more if we were provided with more training. For example, they advised us to apply lime after drying the pond—1 sheer of lime in 1 decimal area of a pond. Urea also should be applied.

"We hold weekly meetings in addition to monthly meetings. We do everything collectively such as fish culture and vegetable cultivation. We gain profit by selling bran. Everyone has planted a palm tree in their homesteads. Other members suggested we do that. The age of the society and the palm tree is the same and all of those trees are bearing fruit. The society improved us a lot. We had no house and no tin roof. We had a roof made of straw. I have taken a Tk. 500 loan from the savings deposit and bought a betel leaf shop for my husband. He is earning Tk. 20–25 daily from that shop. I have saved the money and built this house. At the initial stage we deposited Tk. 1 and later Tk. 2. We worked and have developed our status. Now we deposit Tk. 5 and are trying to increase it.

"The society is our future. My son is studying at school. Savings deposits help us in time of need. All forty-six members pay attention to each others' problem. If ten members sign the resolution book, we can get the money. But we rarely do it. We are depositing the money for our future.

"Presently I am involved in paddy husking and poultry rearing. I am educating my children. I have given my two daughters into marriage with the help of the society without which it would have been difficult for me. In earlier days we starved day after day. Now we can save Tk. 1,000 to 1,500 after all household expenditures. Husband and wife together are running our family. When my husband ran our family alone, we were in severe trouble. We could not even buy a pen for our children. It stopped them from going to school. Now I husk paddy, sell the rice and bran and pay the educational expenses for our children. I have raised savings by selling chicken eggs. I gave my two daughters into marriage with this extra income. My son is a student of Class X. I deposited money by selling rice bran. I could not buy cloth in the early days. Now I can buy suti sharee [cotton sari]. When my husband earned for family, we could not even eat. According to the advice of BRDB I have started working, rearing poultry birds and purchasing cloth by selling them.

"Thank God our condition has improved. Now we need more loans and training which will be more useful. I read up to Class III. Training of BRDB enriched us. I have one son and two daughters. I have stopped having more children. Family planning training taught us this. One boy and one girl are enough. We teach the members to have only one or two children through the society.

"Now my life is good enough. I had no house before—now I own a house and I husk paddy. I could not speak in those earlier days. I was afraid of the sir [authorities]. People taunted me. They called me shameless. My relations from my husband's side also taunted me. But I encouraged myself with the thought: 'What would it be like if we the poor worked together?' I have observed improvement. I have received loans. I have undertaken activities with this loan assistance. It has improved our condition. The people who taunted us earlier now pay us proper respect. Now we do not go to the rich people for help. The Society has provided us with a house. Rich people can see it. They thought they would take loans for house building. But the chance comes to us. It is an intolerable matter to the elites that the resourceless people are now becoming resourceful.

"Now we can take meals three times a day. We can purchase clothes and educate our children. Due to dowry we could not give our daughters into marriage. Now we have taken a decision that we will not take dowry and will arrange dowryless marriages.

"We know about nutrition—that vegetables, pulses, and eggs provide calcium which keeps our health good. We drink tubewell water although we had no tubewell in previous days. Diarrhea will appear if we do not drink tubewell water. We have come to know these things through training. Every member accepts family planning methods. No one has more than two or three children. In case of any problem, first we try to solve it by ourselves. We do not go to other person if the problem is solved by us. If it is not, then we seek others' help. In case of any disease of our livestock or poultry, we go to the veterinary doctors. Our Organizer Madam and others help us.

"Training has taught us how to speak. We hope to receive training on sewing. Members are interested to work on block printing; paddy husking is not so suitable. We can print clothes if we have a printing factory, something our members saw at BRAC [Bangladesh Rural Advancement Committee, a large national NGO].

"Now we are not afraid of rich and powerful persons. In previous days they had taunted us. But now they don't have the temerity to taunt us. If we want justice from the Upazila Member, he gives top priority on our case even it is the case of one of our members. They are afraid of us because we move collectively and unitedly.

"At the primary stage my husband was unwilling to allow me to attend the meeting of the society. Now he realizes that besides the husband, the wife can also help run the family. So he has become enthusiastic. He is of the opinion that the women should work collectively. It is good we are improving. We also make them understand that we are poor. BRDB is providing as with advice. Let us see what comes of it.

"We will need no loans after two to four years. We needed work since we were poor but now we are going to overcome poverty. Now we know that capital can be formed by rearing poultry. We help each other. We help other members to marry their daughter. We give gifts also. If a member cannot give her installment for two to four weeks due to any ailment, we collectively help her to pay the installment. We help each other in illness by raising funds at Tk. 1 or 2 per person.

"We made and provided saline when one of us was attacked with diarrhea. Most of the members have bought one or two decimals of land. I have bought five decimals of land. For this I sold the calf which I bought by loan and added my savings. Everyone has two to four hens. Altogether we have 500 hens presently. I took Tk. 100 from profits from paddy trading and bought three hens. Today I have fifteen hens and many fowl.

"I have formed another three societies in the neighboring areas. They are running well.

"We have been awarded the national prize. It is a shield of silver. The society is good. Savings and shares are satisfactory. Prime Minister Khaleda Zia gave us a prize last December as we are the number one society in Jamalpur. Besides that we got a cassette recorder from Jamalpur administration.

"The name of this society is Shahapur MBSS. We hope we will win more prizes in the future."

A Brief Analysis of the Dialogue as a Source of Qualitative Data

A brief analysis of the dialogue quoted above is presented here to illustrate the significance of the information contained in the dialogue as a source of qualitative data.

The dialogue revealed how one beneficiary and her cooperative society, on whose behalf she spoke, benefited and were empowered by their participation in the project. The dialogue describes how the society members tackled the different problems and issues that faced them, such as how they confronted the school authorities to admit their children, how they asserted their right to dry their paddy on a paved road, how they sought justice from the grocer who assaulted the son of one of the society members, and how they used their entrepreneurial skills in negotiating the use of a pond from a community member for starting their fish culture project. The dialogue also describes how they used their acquired knowledge on health, nutrition, and family planning in their daily living; their awareness of their basic human rights; their unity, solidarity, and self-reliance; their collective and group decision-making actions in dealing with other social problems and events; the management of their income-generating activities; the loans they took; the savings they made; the ways in which they used the income that they derived; the kinds of further investments they made from their incomes; their increased awareness of the importance of the education of their children; and how they have overcome their fear of the "authorities."

These examples of how the society members managed their lives, their families, and their communities, as demonstrated by both the substance and the tone of the dialogue, showed a powerful and profound impact on the social and economic development of the members of the cooperative society.

At the level of social development, the changes in their self-perceptions and other psychosocial changes resulted in their social empowerment. At a broad level, this empowerment included their sense of feeling liberated from their former conditions of marginalization and oppression, their acquisition of more control over their lives and destiny, and their capacity to resist exploitation and injustice by the rich and powerful. Other elements of their empowerment relate to the achievement of self-confidence, self-assertiveness, moral courage, group solidarity, collective and democratic decision making through the society, and higher aspirations for themselves and their children.

In terms of economic development, their economic empowerment included such elements as the acquisition of income-generating skills, the ways in which they manage their income-generating projects, the reinvestment of the money earned from their projects, the accumulation of their savings and shares, and the development of entrepreneurial skills.

All these elements of the beneficiaries' social and economic empowerment constitute many of the necessary conditions for achieving the sustainability of developmental impacts.

The findings on the social and economic impacts of the project obtained through the dialogue corroborated the findings from the survey questionnaire. In addition, they enriched and provided more intimate qualitative details and insights that interfaced with the bare-bones statistics and their analysis. Such an in-depth, qualitative, and more comprehensive body of knowledge on the development process could not be elicited from the survey questionnaire. More importantly, the developmental impacts were described and viewed from the beneficiaries' own perceptions, perspectives, and frames of reference, and in their own words, idiom, and style—not those of the researcher. The beneficiaries' own perceptions and perspectives represent an important and more authoritative evaluation feedback to the project's stakeholders.

Lessons Learned

In using the qualitative and participatory dialogue method in combination with the survey methodology in carrying out the evaluation of the developmental impacts of this program, some major lessons have been learned concerning approaches, methods, and presentation of findings that affect how evaluation is perceived, carried out, used, and valued. These lessons include the following:

1. The combined use of survey and participatory evaluation methodology certainly enriches the evaluation findings of a development project, particularly its qualitative social impacts, and makes the reading of the evaluation report more interesting and stimulating, as was intimated by the project's stakeholders.

2. One of the strengths of this evaluation study was that it treated and presented the verbatim transcriptions of the dialogues as an integral part of the main findings of the study. They served as a key body of self-contained information, standing on their own rather than being tucked away in the shadow of appendices. The content of the dialogues is at least as substantive, valid, and authoritative as the quantified and statistical data. In addition, some relevant excerpts and quotations from the different dialogues were incorporated, along with the quantified data, into the body of a key chapter called "Impacts of Training."

3. The qualitative findings obtained through the dialogues were instrumental in making BRDB officials understand and recognize the importance and significance of the qualitative social impact of the project on the beneficiaries. Before the study was conducted, BRDB tended to be almost exclusively preoccupied with the credit performance and economic impact of the project, as indicated by the number of income-generating activities initiated, level of loan disbursements, loan recovery rate, frequency of loans taken, increase in income, and so forth. Only peripheral attention was paid to those project interventions that were intended to promote social development of the beneficiaries.

As part of the follow-up of this evaluation study, the author was then able to build a more convincing case (based on the profound social impacts as revealed by the dialogues) that, while economic development is certainly one of the critical goals of the project, a development process needs to be viewed more holistically. The argument for this case was made along the following lines: For the credit component of the project to succeed, training for social development is equally as important as the training for economic development. It was recognized that credit was the backbone of the project. However, access to and productive utilization of credit should be seen as a means to an end, namely, economic and social development that culminates in empowerment. If economic development is to become sustainable, it cannot occur without social development, and sustainable social development cannot occur without economic development. The dynamics of a sustainable development process require that economic and social development be perceived in an interactive manner and not be dichotomized as an either-or situation. Failure or reluctance to recognize the complementariness of economic and social development will, in the long run, undermine the success of the credit program. The mutual reinforcement of economic and social development will create the necessary conditions and environment that will enhance the prospects of achieving sustainable economic and social development.

4. The use of the dialogue method, like any other participatory evaluation method, has its own limitations and constraints. The researchers have to devote much more time in conducting the dialogues than in administering interview questionnaires. Consequently, it is more costly to conduct an evaluation that uses such methods. However, in the case of this evaluation study, since only ten beneficiaries were involved in the dialogues out of a total sample of 2,104 beneficiaries, the extra cost was negligible.

5. The researchers assigned to conduct dialogues need special orientation in the development philosophy underlying the dialogue approach and training in the skills of using the dialogue method effectively, and they also need to be quite sensitive and sophisticated in establishing a rapport with the participants.

6. The other limitation with using the dialogue method is the representativeness of the participants with whom the dialogue is conducted, since the participants selected have to be articulate. This factor tends to delimit the use of random sampling in favor of a more selective sampling of those participants who are more self-confident, more assertive, and willing to describe their experiences and express their views without any inhibitions.

7. The dialogue method also imposes a constraint on the size of the sample chosen, which again has implications for its representativeness. Because the use of the dialogue method is time-consuming and produces long texts, the sample needs to be much smaller than a sample selected for a survey questionnaire.

8. It would have been value added if the project could have found a mechanism to share the dialogues with the other beneficiaries of the project. In this way, the sharing of self-evaluated development experiences with other beneficiaries can become an educational process through which the beneficiaries can get an opportunity to discuss and reflect on developmental changes and impacts. Such a process would give them the opportunity to get some idea of their own potential for development, and promote a feeling of solidarity. The text of the dialogues could also be used to serve as highly motivating and relevant literacy follow-up reading materials in the same project.

References

Chambers, R. 1978. Rural Poverty-Oriented Monitoring and Evaluation. Brighton: Institute of Development Studies, University of Sussex.

Fals-Borda, O. 1977. "For Praxis: The Problem of How to Investigate Reality in Order to Transform It." Paper presented at Symposium on Action Research and Scientific Analysis, Cartagena.

Filstead, W. J. 1970. Qualitative Methodology. Chicago: Markham.

Freire, P. 1970a. Cultural Action for Freedom. Middlesex: Penguin Books.

———. 1970b. Pedagogy of the Oppressed. New York: Herder and Herder.

Hall, B. 1975. "Participatory Research: An Approach for Change." Convergence 8 (2): 24–32.

Kassam, Y. 1979. The Voices of New Literates from Tanzania. Dar es Salaam: Tanzania Publishing House.

Kassam, Y., and M. Kamal. 1992. "Report of the Evaluation Study of the Training Component of RD–12." Ottawa: E.T. Jackson and Associates Ltd.

Kassam, Y., and K. Mustafa, eds. 1982. Participatory Research: An Emerging Alternative Methodology in Social Science Research. Toronto: International Council for Adult Education.

Oakley, P. 1986. "Evaluating Social Development." Journal of Social Development in Africa (1).

Parlett, M., and D. Hamilton. 1972. Evaluation as Illumination: A New Approach to the Study of Innovatory Programmes. Edinburgh: Centre for Research in the Education Sciences, University of Edinburgh.

Patton, M.Q. 1987. How to Use Qualitative Methods in Evaluation. London: Sage.

Pilsworth, M., and R. Ruddock. 1975. "Some Criticisms of Survey Research Methods in Adult Education." Convergence 7 (2): 33–43.

Richards, H. 1985. The Evaluation of Cultural Action. Basingstoke: Macmillan.

Rockhill, K. 1976. "The Uses of Qualitative Research in Adult Education to 'Enlighten, Enoble, and Enable.'" Paper presented at the International Conference on Adult Education and Development, International Council for Adult Education, Dar es Salaam.

Weiss, C. 1972. Evaluation Research. Englewood Cliffs, N.J.: Prentice-Hall.

Document(s) 3 of 18