Document(s) 13 of 18

The Ghana Water and Sewerage Corporation (GWSC) Assistance Project commenced in 1990, funded by the Canadian International Development Agency (CIDA) and the government of Ghana (GOG). The project's scope is to establish community and district-based management of small urban water supplies throughout northern Ghana, concentrating upon fourteen towns in the first phase, 1990 to 1998. The main objectives of the project are to establish and train fourteen effective local-level water management boards; rehabilitate the fourteen small urban water supplies, emphasizing low-cost appropriate technology; and strengthen GWSC and increase its capacity to promote this shift to community management of water supplies.

The project is implemented by GWSC, a large, relatively centralized parastatal agency of the GOG. Technical assistance is provided by a Canadian Executing Agency (CEA), Wardrop Engineering Inc., using expatriate and Ghanaian advisory personnel.

External monitoring by CIDA of the project's activities has taken place since inception, with a two-person team visiting the project, usually twice a year. Through staff interviews and observation of project reports and publications, the external monitors have identified deviations from the project's planned implementation schedule, issues to be addressed to mitigate obstacles and delays, and constraints affecting project implementation. External monitoring has essentially served to measure compliance of the project's performance to planned activities and outcomes.

Project management realized that many of the project's activities are breaking new ground in northern Ghana. There are no precedents for community management of small urban water supplies upon which to draw. District-level management structures are at an early stage of development, and recently introduced decentralization policies by GOG demand considerable capacity building at the local level. Women's roles in water supply management decision making, while significant in some areas, are generally proscribed. The GWSC is under considerable pressure to transform itself into an economically viable agency, managed on a commercial basis.

Faced with this large degree of uncertainty, and subject to rapid and often unpredictable changes in conditions, project management identified the need early on to establish an effective internal monitoring and evaluation mechanism. The desire was to learn by doing: measure the appropriateness, effectiveness, and sustainability of key project activities; and identify modifications to activities and new initiatives needed to reach optimal outcomes. It was realized that, to achieve a reliable measure of project performance, internal monitoring and evaluation would need to be a fully participatory exercise, involving the project and all stakeholders at all levels.

Methodology

Internal monitoring and evaluation are planned to take place annually, in the March–April period each year from 1993 to 1998. A work plan is developed to determine which project activities to focus on annually. The work plan comprises four distinct stages: data collection at the community level; data collection at the institutional level; data analysis and report preparation; and presentation of a report and discussion of the recommendations with project management to prepare an action plan for the coming year.

The monitoring methodology is based on internal monitors conducting participatory discussions and semistructured interviews with key individuals and groups at both the institutional and the community level. During these interviews, open-ended questions are asked relating to the perceived effectiveness of project activities. Specific questions are targeted according to the individuals or groups being interviewed. Responses to questions are recorded to highlight the key elements of each response, positive and negative opinions and impressions, and suggestions and questions raised by the interviewees. In 1993, fifty-eight persons were interviewed at the community/district level, and forty persons at the institutional level, providing a rich source of data.

Data collected from interviews and from secondary sources are analyzed using a qualitative matrix for each objective. Each matrix consists of a set of criteria of effectiveness, posed as questions, which are established in reference to the activities being monitored. In each matrix, monitoring data are used to assign a rating against each criterion, depending on the positive or negative responses to the question posed. Comments are provided to clarify responses, constraints, and sources of enhancement to effectiveness.

Changes in effectiveness from one monitoring period to another will be identified by comparing the ratings assigned in each matrix for each objective. Changes of a positive nature will be reflected by higher scores or ratings, and conversely, changes of a negative nature will be reflected by lower scores or ratings. Activities that are stalled, with no change from one evaluation to another, or with a consistently ineffective score or rating, will also be identified.

The knowledge gained from the analysis of data is integrated to develop recommendations and short-term action plans to improve project effectiveness.

A baseline report resulted from a five-week initial mission conducted in March and April 1993. The mission was conducted by a four-person team.

Objectives

It was decided to focus upon three main areas of project work that are the primary determinants of sustainability and effectiveness. These areas are community management, human resource development, and gender development.

Internal monitoring and evaluation objectives were established

• To evaluate the appropriateness and sustainability of the project's community management strategy;

• To evaluate the effectiveness of the project's training activities; and

• To evaluate the sensitivity of the project to gender equity issues.

An annual work plan is developed for internal monitoring and evaluation. Initially, the 1993 work plan concentrated upon establishing baseline parameters for future comparison. The case study presented here is primarily a description of that baseline. In subsequent years, annual work plans will concentrate upon measuring changes in appropriateness, effectiveness, and sustainability against the baseline data.

Process

Three matrices were developed against which project performance in the three areas identified in the objectives could be gauged.

Data were collected primarily from participatory discussions and semistructured interviews at the community/district level and at the institutional level by a four-person monitoring team. The team was chosen to reflect not only the subject areas of monitoring (such as training, community management, and gender issues) but also a gender and perspective (Canada/Ghana) balance.

During a three-week period, qualitative data were collected from the community and institutional levels by a participatory process and from secondary data sources, such as project reports and papers. The team then processed and analyzed the data to collectively formulate responses to the questions posed by the normative criteria. Once consensus within the team was reached, scoring of the matrices was undertaken.

Scoring, while being subjective and qualitative, is internally consistent, in that it indicates the collective perception of project performance from all data sources. The scoring system used is based upon Table 11.1.

The total score for each of the three matrices then reflects the overall performance of the community management strategy, the training activities, and the sensitivity of the project to gender equity issues.

Results of 1993 Baseline Mission

The matrices used for the preparation of the 1993 internal monitoring and evaluation baseline report are presented in Figures 11.1 through 11.3. Analyses were conducted for the major elements that constitute the main project component being evaluated.

Community Management

Under the community management strategy, representativeness, decision making, communications, planning capacity, interrelationships, and sense of achievement were analyzed. Overall, the project's community management strategy has been partly appropriate and sustainable to date and scored 68 within a possible range of 21 to 105.

There is a coherent understanding and agreement among Water and Sanitation Development Boards (WSDBs), district government, and GWSC on the principles contained in the strategy. WSDBs have a clear vision of their role in enabling the process of community management, and individual WSDB members generally have a clear picture of their specific roles and responsibilities. Linkages and interactions between WSDBs and GWSC have improved substantially, and GWSC's customer relations have benefited considerably. To date, planning water supply rehabilitation has been the main achievement of WSDBs. Plans prepared reflect community needs, desired service levels, and affordability and are generally appropriate and potentially sustainable. Significant consideration has been given to operation and maintenance requirements in the preparation of rehabilitation plans.

Although a framework for operation and maintenance involving both GWSC and WSDBs has been developed, both GWSC staff and WSDB members need a clear definition of their roles and responsibilities in operation and maintenance. GWSC's ability to effectively participate in community-managed operation and maintenance is constrained by weak accounting and billing procedures and by inadequate financial management and control systems.

Figure 11.1: Matrix for the Evaluation of the Appropriateness and Sustainability of the Project's Community Management

Training

With the project's training activities, analyses were performed on both GWSC in-house training and community/district-level training, linkages developed, and collaborative arrangements forged. Overall, the project's training activities have been partly effective to date and scored 72 within a possible range of 25 to 125.

With the development and approval of a commercial optimization program, which includes a comprehensive training plan for GWSC financial and commercial staff, there is a degree of optimism that this component of the project is now receiving attention. WSDB management training has generally been effective and well organized and has resulted in more positive interactions between GWSC staff as trainers and WSDB members. Public education, delivered primarily by WSDB members and community groups, has been well received, although it has only recently been initiated. Increased support for WSDBs and increased rates of operation and maintenance deposit collection as a result of public education have been reported.

Technical and administrative training appears to be weak. GWSC lacks an effective training database, and there is little or no evidence of job, task, or skill analyses being performed. There does not appear to be any established procedure for selecting GWSC trainees, and no on-the-job training performance indicators are in place. Overall, the training-of-trainers exercise has been poorly understood by GWSC staff at all levels, and no overall technical and administrative training plan exists to provide guidance. At this point, there is little chance of institutionalizing on-the-job training within GWSC, and therefore it is not considered to be a sustainable effort as currently formulated.

Gender Sensitivity

Concerning gender equity sensitivity, analyses were performed for the achievements with GWSC and the project staff, and with community/district and other government staff and individuals. Overall, the project is partly sensitive to gender equity issues and scored 40 within a possible range of 13 to 65.

Women, primarily through WSDB membership, have participated effectively in water supply rehabilitation planning and design. Women WSDB members have received significant training in financial management, administration, water utilization, hygiene, and sanitation. The district government is very supportive of women's involvement on WSDBs. GWSC staff training has made a conscious effort to include women participants, particularly for management and supervisory training sessions.

WSDB decision making does not always reflect community input, and as such does not necessarily reflect adequate prior consultation with community women. Regional and district GWSC staff are generally unaware of gender equity issues, especially the impact of water tariff, billing, and collection practices upon community women. Women's participation in project activities is not being comprehensively monitored.

Issues

Following analyses of the data and scoring of the matrices, the internal monitoring and evaluation team identified an array of issues that had emerged up to 1993:

1. WSDB members should be fully involved in any work being done by GWSC on the community water system, no matter how small the work is. This level of involvement will increase the credibility of WSDB members and give the assurance that rehabilitation of the system has started.

2. Objectively verifiable achievements should be established for project activities in community management. These would include the ability of a WSDB to mobilize and prepare for community management, preparing a successful rehabilitation design, collecting and managing funds for the operation and maintenance deposit, and sustaining the management of operation and maintenance.

3. The project training coordinator should concentrate on establishing a training database by conducting a job analysis for each staff grouping, developing an inventory of the tasks done and skill levels currently held by staff members, and comparing the gap that exists between the two. Based on the training needs identified, a training course could then be developed for each staff member.

4. The job descriptions for GWSC trainers should be modified to recognize their increased job responsibilities for providing training.

5. Quarterly job task performance evaluations should be developed, for both trainers and trainees, to monitor actual outcomes of increased skill levels achieved by on-the-job training.

6. The project should continue to provide ongoing skills training for GWSC trainers, based on training needs identified in these quarterly evaluations.

7. Training on the community management process should be provided at the district and community level for GWSC staff.

8. A communications strategy must be developed to more effectively transfer information between the project and field-level and operations staff.

9. A plan should be developed to communicate the overall project training program to project staff, trainers, GWSC staff, and WSDB members.

10. Within the communications strategy, a public relations plan should be developed, for both internal and external audiences, to link the commercial/financial sector to the continuing viability and maintenance of an effective water system.

11. The project's training plans must be transparent and communicated widely to increase the understanding that training can occur in nonformal as well as formal learning.

12. Project data should continue to be compiled in a gender-disaggregated manner so that the degree of women's and men's participation in the project can be monitored and evaluated from the perspective of both practical and strategic gender needs.

13. Project staff should consult more frequently with women's groups in order to increase staff understanding of gender roles in the community and to assist in identifying appropriate gender-awareness training activities.

14. Women could be trained and/or hired to fill the water sales/customer relations positions within GWSC.

15. Gender differences in payment for water should be monitored and evaluated on a continuous basis to ensure that the financial inputs required at the community level do not have a differential and/or detrimental impact on women.

16. A gender training session should be developed that examines the role of women and men on the WSDBs and on the role of women and men in the supply of water to the community.

17. Mechanisms to include community women in the decision-making process (such as the active involvement of women's groups or meeting with women separately) must be developed and monitored to ensure that community women's water needs are incorporated into the water systems.

18. Public education programs should be designed to influence men's and women's attitudes with regard to the roles men could play in the health and hygiene activities within their families and in the community generally.

19. Women and/or women's groups must be encouraged to assume the responsibility for management of operation and maintenance fund collection, tariff collection, and all financial management matters of the WSDBs.

20. Any training provided by the project to district and community organizations should incorporate awareness of gender issues and skills in gender analysis.

21. The project should continue to expand opportunities for women's participation in decision making and leadership and to seek methods to increase men's participation in the areas of health and hygiene.

Recommendations and Action Plan

These issues were presented to project management, and a round-table discussion then formulated the following recommendations for action:

Project Management

• Improved communications are needed between project management and project/GWSC regional and district staff concerning project strategies, objectives, plans, and activities.

• More information is required on the project's technical training and management/supervisory training plans and activities.

Community Management

• Improved linkages are needed between WSDBs and district government.

• Strengthening of district governments is required, especially in the areas of community management, gender development, and planning.

• Strengthening of WSDBs is required, especially in the areas of gender development, decision making, and communications.

Interim Stabilization

• Improved responsiveness to technical water supply problems in the communities is required.

Technology Demonstration

• A better understanding of appropriate, low-cost technology is needed by the WSDBs that will manage this technology.

Technical Training/Upgrading

• Training coordination must concentrate on revitalizing the training of trainers, establishing and maintaining a database, developing individual staff training plans, and monitoring trainer and trainee performance.

Financial/Commercial Upgrading/Training

• The sense of alienation from the project expressed by regional financial and commercial staff must be overcome.

Management/Supervisory Upgrading/Training

• A clearer explanation of the formal and nonformal components of the management and supervisory upgrading and training activities is required.

• Gender development training must be incorporated into management and supervisory training activities.

• Management training plans for individual staff should be developed.

A tangible result of these recommendations, developed by consensus between the monitoring team and the project management, was an action plan for 1993–94 that aimed at both modifying existing activities and identifying new or additional activities. This action plan was then incorporated into project work plans at all levels.

The internal monitoring and evaluation process is to be continued until the end of the project. The 1993 baseline report produced by the internal monitoring and evaluation team provided a clear picture of the performance and effectiveness of the project to date. The three main activities of the project—community management, training, and gender equity promotion—had been only partly appropriate, effective, and sustainable so far. The need to and the means to improve project activities were clearly delineated, in a broadly participatory manner.

Modified and new project activities subsequently initiated by the 1993 action plan were evaluated in 1994 as to their appropriateness, effectiveness, and sustainability, using a similar methodology and following a similar process as in the 1993 exercise.

Document(s) 13 of 18