Outcome mapping looks at outcomes, for PM&E, in the context of achieving developmental goals rather than focus on pure impacts.

Outcome Mapping in the NEPED Project Context

The Nagaland Empowerment of People through Economic Development Project (NEPED) was Nagaland's first donor-supported project. The implementing team lacked capacity in management and reporting. There was a single-minded focus on executing field activities without allowing for monitoring or evaluation of outputs. As a result, while work in the field looked picture perfect, there was absolute confusion in the office with data, information and experiences finding no apparent parking space. Lack of knowledge and capacity to fit these into an M&E format that was poorly understood was frustrating, to say the least. With the end-of-project evaluation fast approaching, the frustration turned to desperation. The work had been done well, downstream partners corroborated the quality of outcomes and achievements, and yet there was little to show on paper in quantitative or qualitative terms.

Why Outcome Mapping

The following section describes the application of outcome mapping in the NEPED Project within its two phases, focusing on the challenges addressed.

Challenges Addressed in Phase 1

Lack of Baseline Data at Inception

Given the vast reach of NEPED having worked in 1,056 villages in Nagaland, it would have been impossible to conduct a baseline study and survey. Outcome mapping was used as a simulation exercise to go backwards in memory to realistically generate data from the last five years to be used in lieu of baseline data to evaluate performance.

Retrospective M&E for Impact Assessment

NEPED underwent the throes that come with any first-time project. Frequent changes in management, reporting procedures, management information system (MIS) and M&E formats, strategy, etc. disoriented the project team and caused perplexity. Through OM, the team was able to identify major accomplishments, gaps (and not omissions) in the first phase, around which the second phase is designed.

Even as each activity set was viewed as successfully performed when looking from the bottom-up, no clear-cut linkages could be established when looking at the project as a whole. In other words, there was much to show as outputs, but not much to link them to outcomes. By using OM, these linkages were established through a road map.

An interesting aspect was the role played by the body of "local experts", NEPED's "human data bank". The local experts were a group of 10 highly respected and influential farmers drawn from various tribes. They had a vast repository of indigenous knowledge on best practices and provided invaluable inputs to research, monitoring and field implementation on a continuous basis. They were conduits between the farmers NEPED worked with and the project team. They were included in the OM process to bring in "voices from the field", especially in the area of on-farm trials and farmer innovations.

To Reflect as a Team for Self-Evaluation

The project team had gained considerable experience in implementing the project at the field level, with varied levels of success that needed to be collated and analyzed for its own internal self-assessment. By using the organizational practices tool and developing success markers, even the minutest details, that would otherwise have remained forgotten, emerged with solid quantifiable data on best examples, clearly establishing how the team as an organization had evolved along with its partners.

To Share and Record Experiences

During the course of implementing the project, the team's field journals had references to stray bits of information which seemed useful, but remained unreported as it did not fit into the logframe-based reporting format, e.g., farmer innovations, best practices, feedback, indigenous knowledge, local tweaks, etc.

The innovations and tweaks emerged from the data on organizational practices and progress markers. Interestingly, most of these examples would have been called weaknesses, deviations or challenges under conventional project evaluation. OM reflected them as achievements. With the use of outcome mapping, the external evaluation team was saved the trouble of 'digging' out data at the field level, having only to validate the findings with user groups.

Challenges Addressed in Phase 2

Having realized the utility and flexibility of OM as a tool, there was no hesitation in using it again, this time for design, planning, assessing performance and M&E. The strategy maps were used to increase the project's sphere of influence over boundary partners and even to monitor one of them, the state government.

Interestingly, the project team would be reporting to, and be evaluated on, the donor specified Logical Framework Analysis (LFA)-based M&E format – and yet chose to use both. OM would be used to supplement, enrich and feed data into the donor M&E format. This format is designed in pure quantifiable terms, hence OM would also be used to seek out both qualitative and quantitative data, and to help inform and re-evaluate future activities. OM is being used as a tool to engage with communities to understand the impact and effectiveness of the project, and information generated is being fed into another format to meet reporting needs. OM's flexible and participatory approach better enables the team to discuss results with the communities and with other 'boundary partners' and to assess and redesign strategies for work ahead.

For Project Design

After being asked to imagine who/what would change if NEPED was to be "wildly successful" at the end of the project period we created the vision - mission statement, identified who would need to change (our boundary partners) and created outcome challenges for these partners.

The team had been actively involved in the preparation of the project proposal and hence had in depth knowledge of all project components. This really helped in weaving the vision-mission-outcome challenges around the components to come up with progress markers, strategy maps and the organizational practices. Validation was carried out by adapting elements of Participatory Rural Appraisal (PRA) tools like semi-structured interviews, focus group discussions and key informant feedback that fit in very well with outcome mapping.

For Monitoring

The team then moved on to identifying the monitoring priorities and putting together the outcome, strategy and performance journals for the next two years in order to inform itself. The team is using progress markers to monitor behavioral change induced by the project in three boundary partners: farmers, village authorities and Self-Help Groups, who are directly within its sphere of influence. Change in the remaining boundary partner, the state government, is also being monitored by assessing the quality of responses from it to the strategies adopted for it by the project.

For Evaluation

An evaluation plan has been prepared within the given framework, which is the last stage of the outcome mapping process. This was not an easy task. The second phase of NEPED is very broad-based and complex with activities ranging from revolving micro credit and agri-marketing to agroforestry and research. The ranges of partners are wide and the audience even wider.

What was Gained from Outcome Mapping

It provided the team conceptual clarity on the project. The team graduated from the stereotype input-activity-output supervisors and is able to link these around outcome level issues. They can now 'map' their progress into the future, recognizing the major actors and forces, predict possible gaps/threats and strategically plan for it in advance.

It brought in a human dimension to the project. Outcome mapping works on the premise that development brings about change in people, especially their behavior. An OM map presents real-life pictures of human beings - their perceptions, their aspirations, their environment and the challenges to realizing them.

It was empowering. Outcome mapping is highly participatory and consultative. Although discussions are initiated within the project team thinking as a unit, validation by partners enables broadening of the canvas, acknowledging the presence of each as an indispensable unit of the whole process of development. In NEPED, there were cases where farmers deviated from the prescribed menu of activities on their own, without supervision, to come up with more cost and labor-effective innovations, being aware of what was the desired output and the long-term plan. This feeling of empowerment produces positive energy towards successful and sustainable implementation of the project.

It increased capacity and skills. One of the challenges faced in using outcome mapping was validating with boundary partners, especially at the village level. This was so different from the top-down government-sponsored development that was thrust upon Naga villagers. When consulted, it created suspicion and took a while to break down these barriers and to gain their trust and give them the feeling that they were indeed partners. The team had to be trained in formal PRA and facilitation skills. With government officials, who consider participation a threat to their authority, the approach had to be different.

It brought cohesion in the team and partners. In creating the OM map the active participation of all enabled everyone to see the 'big picture' and define roles and contributions within this. Moreover, validating M&E findings with downstream partners provide invaluable inputs from farmers and village groups and help in constantly redesigning project delivery targeted at sustaining the program. This also enhances the feeling of ownership over the project.

Some Challenges in the Use of Outcome Mapping

Ever so often, programs are caught in the trap of accounting for the resources used to achieve quantitative outputs that the "development agenda" remains forgotten. Outcome mapping enables a program to articulate its performace story, to measure and attribute the development process to the various actors involved and to take credit for achivements or outcomes. After all, development is by, and for, the people and a program can only influence, but not control, change.

Contributed by:
Raj K. Verma
Email: raj@leadindia.org