Document(s) 8 of 18

A community health and development team working in the poor, heavily populated Indian state of Bihar has found new direction and a new level of commitment from the community as a result of incorporating participatory evaluation into its expanding program.

On the banks of the Ganges River at Patna, northeastern India, stands the Kurji Holy Family Hospital. It is a 275-bed teaching hospital serving Patna, the capital of Bihar, the second most populous state of India. For more than thirty years, the hospital has run a community health program with the aim of reaching out to the poorest in the surrounding area. An estimated two-thirds of the population of Bihar are living below the poverty line. Twenty percent of the state's population in 1988 comprised tribal people and Harijans or "untouchables," who now refer to themselves as Dalits.

The poverty in Bihar has little to do with the quality of the land. Bihar is made up of fertile plains, and the southern plateau of the state provides 40 percent of India's minerals. The lack of development has more to do with the meager industrial development and the neglect of the state's infrastructure. Farmers have to manage without roads and power supplies, let alone the benefits of modern agricultural technology.

The unequal distribution of land is another serious barrier to progress. Most of the cultivated areas produce zero or low growth due to poor irrigation and lack of modern inputs and extension services. Wealthy landlords continue to own huge tracts of land but offer little support to those who work on their estates. The remaining cultivable land is divided into plots that are often too small to be efficient. Most of the tribal and Dalit families are landless. They are forced to hire themselves out, mainly as farm laborers, on a daily basis.

Community Health

The Kurji Holy Family Hospital decided to start using a room in the hospital as a community health department in 1959. The aim was to familiarize staff

This chapter was first published in Contact 32 (August 1993).

with the conditions of the poor living in periurban areas of Patna. By 1968, the department had its own Urban Health Centre in a separate building on the hospital grounds.

A year later, a request from a priest working in a particularly poor, rural area twenty-one kilometers from the hospital heralded the start of an outreach program. He requested weekly clinics for the people of Maner, an area extending westward along the Ganges. The work in Maner flourished, and in 1978, the Maner Community Health Centre was built by the Catholic Medical Missionaries.

Since then, the program has increasingly emphasized social development. A number of community workers are employed in what is now a multidisciplinary team.

The Evaluation Process

Deciding to Evaluate

Prior to the participatory evaluation, there had been several earlier efforts in studying the progress of the program. One was undertaken in 1976 by a hospital management team, and another by the Voluntary Health Association of India (VHAI) in the early 1980s. VHAI recommended better definition of target areas, more preventive and promotive health services, such as for tuberculosis and leprosy, and a greater emphasis on maternal and child health (MCH) care. At that time, men—as the workers and income earners—received considerably more attention from the health services than did their wives and children.

A few years later, two members of the community health department staff attended a seminar on social analysis and began to feel that the program needed to focus even more intensively on the poorest people in the communities. The participatory approach interested these two individuals. It was an approach that made reaching the poorest a priority, and it involved health workers and the community in making their own evaluation and their own recommendations for adjustments to the program.

However, to most of the members of the health staff team, the idea that they were to evaluate the program for themselves seemed ambitious. They had very little baseline data, and some of the community health workers (CHWs) and community development workers (CDWs) were unable to read or write.

Finally, the team invited Marie-Thérèse Feuerstein, a facilitator in participatory evaluation, to come and visit them to explain the process. The visit was made possible through funding from the program's partner, Misereor, Germany.

In Patna, the facilitator described to the health team how each of them—whatever their background—would be able to participate in an evaluation of their program. They soon became convinced that they would like to adopt such an approach and, together, set dates for her return.

Defining the Objectives of the Evaluation and Choosing Evaluation Methods

Six months later, in October 1988, the evaluation facilitator returned to Patna to join the twenty staff members working on the community health and development team. Half of them were based at the Kurji Urban Health Centre and the other half at the Maner Community Health Centre. The intention was to evaluate the progress of the past four years (1984–1988).

The first event was a six-day training workshop to plan and prepare for the evaluation. The first task was to define the objectives of the evaluation. All were agreed that the main objective of the program as a whole was to help people meet more of their basic needs. The problem then was how to measure progress toward this objective.

The facilitator asked the team to think about the life conditions that were influencing the health and social development of families in the communities they aimed to serve. From a primary focus of looking at family needs, participants then "scaled up" to look at community needs. They drew up a list of factors affecting health development, including education, food, housing, and so on.

When discussion moved specifically to health needs, each requirement that was mentioned was drawn on the blackboard, forming a primary health care circle around a family group. The team discussed the links between achieving the components in the primary health care circle and achieving other basic needs.

The list of conditions affecting people's health and lives provided indicators, or markers, for measuring progress in different areas. For example, an increase in the percentage of people living in good-quality homes would constitute progress in living conditions.

In order to measure changes in the indicators of progress, the team realized that there were key questions that needed to be answered in order to establish whether components of the program were successful. For example, how extensive was their health program, and had it improved the health of school-children? Questions were decided upon for the four main activity areas of the program: health activities, social support activities, program organization and management, and training.

Next, the team worked out how to collect the information. They decided on nine main evaluation methods:

• Analysis of records and documentation

• Survey of MCH from a sample of women aged fifteen to forty-nine

• Mid-upper-arm circumference measurement of children aged one to four

• Flash cards, weight-by-height chart, and puppetry for schoolchildren

• Village meetings and focus group discussions

• Special staff meetings

• Group questionnaire on community health to nursing and midwifery students

• Observations

• Visits to key informants

The team also had to delegate responsibilities; decide the sequence in which evaluation preparation, implementation, and analysis would take place; and make plans about how to present and use the evaluation results. In all, it took the team two and a half days of the six-day workshop to complete the detailed plan.

Preparing and Pretesting the Evaluation Methods or "Tools"

Another two days of the workshop were devoted to training and designing and developing the evaluation tools. The health center staff, including CHWs and CDWs, and students were trained in interview techniques. Everyone also had to learn how to organize focus group discussions and to understand how to develop the "tools," such as the model for the survey forms.

To obtain the additional MCH information needed, a three-page questionnaire was prepared for the survey. The questionnaire was then pretested on a random sample of mothers attending the community health department's Urban Health Centre at the hospital site in Kurji.

It was decided that focus group discussions should be held with those involved in milk cooperatives, for example, to find out what had been achieved. The focus group discussions involved a team member acting as facilitator to steer a discussion. He or she prepared several key questions in advance, and the discussion provoked by the questions would identify factors that had contributed to or impeded success.

Preparation of a number of evaluation tools was necessary for the evaluation of the primary school program. For example, in order to check for physical development, Save the Children Fund weight-for-height charts were made more durable by sticking red, green, and yellow insulating tape onto the different bands. Strips of X-ray film had to be cut and strategically colored to measure the mid-upper-arm circumference of the children. In order to evaluate the health education program in the schools, team members produced sets of hand-drawn and painted flash cards. These cards were used to test how much the children understood about the links between health and hygiene as a result of the health education they had received. Staff also made puppets that they used to entertain the children and communicate specific health information after a particular evaluation session had been completed.

Collecting New Information Using the Evaluation Tools and Selecting Existing Information (Data) to Evaluate the Program

As mentioned earlier, the four components of the program were health activities, social support activities, organization and management, and training. Each member of the evaluation team took responsibility for collecting the data in one of these four areas of activity. The questions relating to the different components had already been discussed and agreed upon during the evaluation workshop.

Health Activities

Maternal and Child Health. Many of the activities of the Urban Health Centre at Kurji and the Maner Community Health Centre in the rural area catered to mothers and small children. There was, therefore, a considerable amount of information (data) available. For example, analysis of some of the program records indicated that few women were receiving antenatal care. At the evaluation planning workshop, all were agreed that the most important questions in the evaluation would relate to achievements and shortcomings in MCH.

The team decided to use a survey as a major evaluation tool. This was not initially a unanimous decision. Some team members, and even the facilitator herself, would not necessarily have chosen to undertake a survey as the major evaluation tool. The facilitator felt that the process could be too time-consuming within the overall evaluation, and that more active participation of villagers with poor literacy skills might be achieved by focus group discussions. However, other team members felt that a survey would not only raise awareness in the villages but also help team members themselves to strengthen their own survey and evaluation skills.

The survey sample included a total of 441 women aged fifteen to forty-nine. They were interviewed in the health centers, in the villages, and at work in surrounding fields. Over half the women were aged twenty-two to thirty, and approximately three-quarters of them were living in the target area for program activities.

The interviewers reported that they were well received by most of the village women. Although some women were reluctant at first to answer questions because there were no free handouts, most offered their time willingly. Some women even said that they liked being asked questions. They said that it gave them an opportunity to think about aspects of their lives that they had not considered before. For example, they were particularly interested in talking about the dowry system and about their own experiences of marriage and pregnancy.

The interviewers reported that carrying out the survey had helped them get to know the women better. "I was surprised that village women were prepared to answer questions," said one member of the team. "Even our male interviewers found that the women were cooperative and very willing to discuss openly."

School Health. Of the twenty-nine schools involved in the program's health activities, three were selected for evaluation—two in urban areas and one in a rural area. The sets of hand-drawn flash cards were used by the evaluation team to assess the children's knowledge of three common health problems, namely, diarrhea, scabies, and eye infections.

The team also decided to build new skills into the school health evaluation "package." They therefore introduced aspects of the child-to-child approach as part of the evaluation. In the child-to-child method, older children spread health messages to younger children, peers, families, and communities. Teachers helped to identify students who would work with the evaluation team and become key actors in the evaluation. These older children weighed and measured the younger pupils, using the weight-for-height chart.

The final part of the school health evaluation provided an opportunity for the children to improve their health knowledge. Members of the team used their handmade puppets to present a story. The puppet characters were based on those the children had seen in the flash cards. In telling the story, the puppets answered the questions the children had been asked during the evaluation. In this way, the children enjoyed themselves and had the opportunity to learn all the right answers to the questions they had been asked while being shown the flash cards.

Social Support Activities

A second area of program activity was the social development program that had started years earlier. The activities included assistance to local people in taking advantage of government welfare schemes, support for cooperatives, lobbying work with bonded laborers, and encouragement for youth drama and women's activities.

During the evaluation planning, the team had decided to use focus group discussions to evaluate these activities. This decision was made partly because focus group discussions allowed further participation of those who could not read and write and partly because there were no baseline income data on which to base a survey.

Government Welfare Schemes. Focus group discussions with villagers during the evaluation revealed that, in the words of some villagers: "Government tries to give people good quality—but they end up getting bad quality."

The main problem appeared to be that even when people did manage to receive food or livestock through the schemes, they were sometimes of poor quality or inappropriate to the family's needs. For example, the grain given in return for work on roads, construction of schools, and social forestry was often substandard. The goats supplied on loan were often sick or producing very little milk. Families were not always trained to handle the animals or items they received, for example, a horse and cart. Some families felt that they had actually become poorer, as they now had the added burden of loan repayment. This perception was particularly true in cases where their animals had died.

Another problem was that obtaining a loan was made difficult by the corruption among local officials and local bank clerks. Families were often asked for bribes of 10 percent or more of the loan, and the bank officials receiving the advance would take their own share before releasing the money.

Milk Cooperatives. One of the three milk cooperatives established through the program produced considerable benefits for the fifty-two members involved. Focus group discussion revealed that average family income of the members had risen well above the average. Motivation and literacy had increased through having to keep accounts and write business letters. Having the status of a registered body also made it easier for the members of the milk cooperative community to apply for government schemes. As a result, they planned to start poultry farming, a fair-price shop, and a preschool for young children. However, two other milk cooperatives failed. They ran into financial and management problems because too few people had adequate business skills.

Bonded Laborers. During the focus group discussion, it emerged that 600 bonded laborers had organized a rally about their situation to present their case to state-level officials. The program had started to support the activities of these men because of the extreme deprivation of life as a bonded laborer.

Youth Drama. Evaluation of the youth activities of the program was hampered because the youth group had not yet been re-formed. Unfortunately, a misappropriation of funds had occurred, leading to a loss of public support and eventually to the group being disbanded.

Women's Activities. With funds from a government scheme, the program had helped twenty-five Dalit women to attend a three-month training for self-employment. The evaluation revealed, through focus group discussions, that although some women had initially succeeded in self-employment schemes, such as making and selling fans, they had later run into difficulties in buying raw materials.

Program Organization and Management

The process of evaluating the program's organization and management was comprised mainly of drawing together existing information (data) that would be needed for the overall evaluation analysis. For example, it was necessary to prepare information about target areas, details of program costs, and arrangements for program monitoring and networking.

Program Training

Although the program had trained a diverse range of personnel, including dais or traditional midwives, community health workers, and government workers such as kindergarten teachers, the evaluation focused on the community training of student nurses and midwives and of the novices—young women who had been received into the house of Medical Mission Sisters but who had not yet taken their vows.

A number of different evaluation tools were used, including both individual and group interviews and focus group discussions. The students and novices participated actively, particularly in the group interviews about their training for community health. They answered key questions while a facilitator arranged their answers in table form on the blackboard.

Analyzing the Data Collected, Reaching Conclusions, and Producing Recommendations

With the questionnaires of the MCH survey completed, the long and arduous task of pulling together and analyzing the results began. Working in rotation, a team of ten in the urban health center and of six in the rural community health center each took more than two days to count the results and to present them in the form of tables. One member of each team drew empty "dummy" tables on the blackboard to receive the data and totals. Both teams used pocket calculators to work out percentages, averages, and ratios, as necessary.

The team at the rural health center in Maner had eight deliveries to attend to while the data analysis was taking place. Fortunately, all births were normal, and despite the interruptions and the long working hours, the team completed the tasks of analyzing data, reaching conclusions, and making recommendations.

By the time all the results were tabulated, most team members considered the survey to have been worthwhile. In fact, those staff members who had been most against it at the start were among those who quoted the survey findings most frequently.

Maternal and Child Health Care

Some of the survey findings were surprising and, in some cases, even shocking. For example, among 217 women interviewed in the rural area, more than 85 percent had received no antenatal care. A similar proportion had never used family planning (see Table 6.1).

Results such as these caused anxiety. "If this is the situation in the villages where we have been working, the situation must be much worse in other villages," said one concerned team member. The school health evaluation provided some small-scale but useful data for the evaluation. For example, it was clear that some children were seriously underweight. The evaluation also showed that the children's knowledge of the causes of common health problems was poor. For example, half of the children in the rural schools did not connect flies and contaminated food with diarrhea.

Although pleased with the achievements of the school health evaluation, the team working in the two schools in the urban area were concerned by the absence of girls in the classrooms. Girls growing up in urban areas are often expected to care for younger children and livestock or to become "rag pickers," collecting waste paper, plastic, or tins to sell by the kilo. It was therefore decided that some out-of-school activities should be planned to reach the children who do not attend school.

Milk Cooperatives

The conclusion of the evaluation of the government welfare schemes was that although the activities were helping to link people with the government schemes, more attention needed to be given to preventing bribery and illegal bank practices and to understanding how those families living in extreme

poverty might be better able to benefit from the schemes.

One conclusion of the evaluation of the work with milk cooperatives was that more training in bookkeeping, writing business letters, and leadership should be made available so that a larger network of people could be drawn upon to develop milk and other cooperatives.

Bonded Laborers

The focus group discussion with bonded laborers revealed that, as a result of new awareness, the landlords were more willing to release workers. However, this did not mean that these men could necessarily find jobs and homes. What was needed was more support for these men on their release. Otherwise, some were forced to return to the very landlords from whom they had been set free.

Youth and Women: A Future Priority

Even though the youth drama and the women's activities programs had faced problems, both were considered to be very important areas for the continuing and future success of the overall program.

Program Organization and Management

As part of the process of analysis and recommendation, discussion of program organization and management revealed the need to incorporate additional monitoring and evaluation procedures into future program activities. Program recordkeeping would have to be partly redesigned, and monthly monitoring meetings would be held.

Staff also recognized that there was a need to redefine appropriate target areas. For example, the Maner Community Health Centre was especially active in fifteen villages, with a population of between 1,000 and 3,000 in each. The large size of this population compromised the program's ability to follow up all the social support activities. Management and organization of the health activities, however, appeared to be working well.

During the evaluation, analysis of program costs and expenditures also took place, and the team decided that greater contact with other health and development programs in India would be useful.

The "Social Cement" of the Program

Although several criticisms of the community training were voiced, such as rapid student turnover, most felt that it was the presence of the students and novices in the villages during training that contributed "social cement" for the entire community health program. Although the majority of the general nurses would not subsequently be working outside a hospital setting, it was felt that their community training would enable them to give better and more realistic patient care in hospitals. At the community level, villagers in the program did not generally know who were staff and who were students.

The commitment and affection shown toward the villagers by the novices, who spent several months actually living with Dalit families, were singled out for special mention. During their community training, the novices took their turns in fetching water, carrying out household tasks, and harvesting rice.

Both teams produced a list of recommendations. The urban group produced fifteen recommendations and the rural group eighteen. They were short and practical and emerged directly from analysis and discussion of the evaluation findings. However, each set of recommendations took a considerable length of time to secure because it was essential to achieve a consensus. Without full agreement, the group commitment needed for implementing the recommendations would be weak.

Even before the evaluation process was over, members of the team were exhibiting a new enthusiasm toward their program. "This experience has helped us see that our work is bearing fruit," one member of the evaluation team said. They also felt that the participatory evaluation approach had stimulated interest in the community itself and created a new closeness between the program staff and those it was trying to reach. "I feel that I now know the village women in a deeper way," said another team member during one of the closing sessions. At the conclusion of the evaluation, the team's hope was that the new closeness and understanding would strengthen the continuing program.

Preparing the Report in Forms Suitable for Sharing with Various Groups

Each section of the report was prepared by different individuals or groups from within the evaluation team. Most members of the team had done nothing like that before. The facilitator assisted by carrying out some basic editing, but she made every effort to keep as many of the original expressions and styles as possible.

In retrospect, the evaluation team felt that it would have been better to have allowed more time for them to produce charts, posters, and other visual aids to accompany their own part of the report. However, the full sixty-eight-page document was eventually typed, stenciled, and bound in attractive, locally produced covers that had been prepared in advance.

Sharing the Findings of the Report

Community members were invited to come and hear about the findings of the evaluation. The interviewers in the urban area had specifically invited the women respondents to hear about the results. The response was overwhelming. More than 100 women arrived at the meeting room, some followed by protective fathers-in-law who sat on a mat at the back of the hall.

The meeting was extremely lively and lasted three hours. Thanks to extensive preparation by members of the team, activities included awareness-raising games and songs, role play, and picture graphics—all of which included messages from the survey findings. In this way, the statistical results of the survey were turned into pictures and actions.

For example, one member of the team had prepared "flannelgraph stories." From flannel material, she had cut out shapes of women in saris and stuck them on the board. In one story, three women were in red saris and seven women in green, showing that only 30 percent of village women were receiving antenatal care.

Some of the students had also composed an "Antenatal Song," which was sung loudly and with great enthusiasm by the women and members of the evaluation team. The lyrics of the song encouraged women to think about their needs during pregnancy, and especially about the need to seek antenatal care.

The evaluation team presented a mimed role play about the difficulties of a village woman during pregnancy and childbirth. This was the first time that a drama had been presented in this form. Many of the women discussed at length what they had seen in the role play. Others remained silent, knowing that they were not allowed to speak in the presence of fathers-in-law who had accompanied them to this unaccustomed outing.

A final three-day workshop brought together the evaluation teams from the urban and the rural centers plus senior hospital officials and invited guests. The purpose was to share and analyze all the results of the evaluation and to decide which recommendations should be for short-term, and which for long-term, action. It was also an opportunity to make a general plan of the program's work for the coming year and to decide on a schedule for the next evaluation.

Putting the Recommendations into Practice in the Ongoing Program

Follow-Up

This section is based on a report from Sister Grace Pullumakal, who wrote to CMC in 1993 with reports from the Urban and Maner Community Health Centres.

In January this year, hospital administrator Sister Grace Pullumakal reported that the hopes of those involved in the participatory evaluation were being realized. Today, the community health program is flourishing. "More and more people are fighting for their rights. They are now very aware of the need for education and immunization, for example," she says.

Sister Grace says that a clear sign of the new community orientation at the urban clinic is the fact that it has been renamed. "We now call it the Community Health Centre Kurji instead of the Urban Health Centre."

Focus on Women

The changes go far deeper than the change of name. In both Kurji and Maner, the greater recognition of the need for participation has created a new focus on women's development. It is now recognized that without specific efforts to support women, little progress can be made in increasing community participation. Maner has opened three women's literacy centers, and at Kurji, health education sessions are included in regular sewing classes.

A priority at both of the health centers is the immunization program. Tremendous strides have been made since the evaluation in 1988. That year, the survey revealed that in the Maner catchment area, only 17 percent of the children aged six months to five years were fully immunized. The follow-up report, written in 1993, showed that two-thirds (66 percent) of the children under five years old were fully immunized.

It has proved much more difficult to increase attendance at antenatal and postnatal clinics. Instead, the health staff at both Kurji and Maner make a special effort to give attention to the mothers when they come to the clinics with their children. In Maner, there is an additional scheme in which two or three people in each village are asked to keep an eye on pregnant and lactating women and to report any cases that might need follow-up by health staff.

The school health programs in both areas continue to be very successful. In Kurji, the aim is now to extend the child-to-child approach to all schools in the catchment area. The program has found that the interest and cooperation of the school principal are very important to their efforts.

The youth drama program is back in full force. Last year, there were 150 performances in Maner, many of which helped people to understand their situation better and to seek solutions. In Kurji, the theater's message has concentrated on the need for education. Drama performances and meetings in one area led people of four villages to get together to build a straw room for a school. Afterwards, they hired a teacher and are now collectively paying his salary.

Schools for "Rag Pickers"

The continuing social support activities include organizing discussion groups about government welfare schemes and support for a milk cooperative, as well as new projects in community participation for safe water and nonformal education for women and children. In Kurji, there are now five nonformal schools for dropouts and for the "rag pickers" who otherwise miss school because they have to do their work in the mornings. Parents value these non-formal schools not only because of the education their children receive but because it helps keep the children from becoming involved in drinking, drug taking, and other addictive habits.

The program is associating more with other voluntary groups. The centers have worked with UNICEF on their immunization programs, and with several women's organizations.

Much has also been achieved in the area of training. For example, at the Kurji Community Health Centre, there is now a better-planned program for the nurses and midwives during their time spent in the community. The students become actively involved in surveys, case studies, and street dramas, and also in the village meetings where the important plans and decisions are made.

Lalti's Story

Finally, an individual example of personal success since the participatory evaluation in 1988 is the experience of a Dalit woman called Lalti. She had been a very enthusiastic participant during the evaluation that took place in her home village of Binteoli. She was identified at that time by members of the evaluation team as a possible future leader for women in her own community.

With her follow-up report, Sister Grace told us that ever since Lalti took part in the evaluation and feedback session, she has been very active in stimulating community initiatives in her own village. Her involvement in the community health program has helped her grow in self-confidence and ability. Lalti now has regular employment as a dai in private practice. Sister Grace says that Lalti's employment takes her enthusiasm for community participation and development beyond the limits of her own village. Sister Grace concludes: "With her home-visiting, Lalti now reaches out to many."

Document(s) 8 of 18