Developing Evaluation Capacity in ICT4D

Context: Why is this project important?

Evaluation that is utilization-focused and outcomes-oriented plays a critical role within Information and Communications Technology for Development (ICT4D) research projects. In December 2007, IDRC’s Acacia, Pan Asia Networking (PAN) and Connectivity and Equity in the Americas (CEA) programme initiatives, along with the Centre’s Evaluation Unit, hosted a 2-day workshop on utilization-focused evaluation (UFE). The modular and interactive workshop included sessions on UFE, as well as a selection of different UFE-based evaluation approaches, such as Most Significant Change (MSC), Outcome Mapping (OM) and Gender Evaluation Methodology (GEM).

Based on demand stemming from the workshop, a decision was made to support a 2-year research project to further develop evaluation capacity amongst select IDRC research partners, both those that were at the workshop and those that were not.

The Project: How does this initiative address the development problem?

The overall objective of this project is to build utilization-focused and outcomes-oriented evaluation capacity among select IDRC-ICT4D project partners in order to increase the quality and utility of evaluation. Accordingly, the project provides training and mentoring to boost researcher’s evaluation knowledge and skills, as well as introduces regional evaluators to utilization-focused evaluation (UFE) concepts and practices. Furthermore, the project gathers evidence in order to examine to what extent a focus on ownership, utility and outcomes actually affects the application of evaluation findings and learning within projects as surmised:

Ownership and UFE share the notion that a close alignment between the interests of the research partner and the focus of an evaluation will lead to findings that are used.
A utilization-focused evaluation approach can benefit from an outcomes orientation by focusing the user’s attention on those changes that are palpable and to which the project can contribute as opposed to the many other, longer-term impacts that cannot be directly linked to a project.
Ownership and outcomes come together when the primary users of the evaluation are given the opportunity to determine the desired outcomes and the signposts (or what some would call markers) to track change that resonates with and is relevant to them. The emphasis on outcomes responds to the tendency to place attention on instrumental outputs and on long-term impacts. The first are of limited interest in terms of project objectives. The latter do not become evident during the life of most projects and are affected by multiple other factors beyond the project’s control. Outcomes can focus on, among other dimensions, what people do differently as a result of interacting with the services and technology offered by a project. They make changes visible that would otherwise not be observed or reported.

Additionally, the project develops UFE curricula, documents evaluation studies and shares project-level and meta-learning with other regions and other thematic research areas using channels such as research/journal publications.

Objectives:

The DECI project incorporates several unique features:

Partners take ownership of the project evaluation agenda
The intended process is one of mutual learning between the project partners, the regional evaluators working with the partners, and IDRC staff.
Evaluation capacity is developed among partners who are implementing research projects.

The UFE approach is expected to shift the researchers’ evaluation concerns away from the conventional focus on whether objectives were reached, to a broader perspective including: process (e.g. to what extent were the methods or case studies appropriate?); context (e.g. to what extent were the findings context-specific); and relevance (e.g. to what extent were the original research questions appropriate, what better questions could have been asked?).

Development Impact

This project intends to disseminate its results broadly in two forms. The first will consist of the dissemination of materials via the Internet, papers at conferences, and publications for wider audiences. This includes the development of learning tools to share this learning process with the broader ICTD and evaluation communities. The second will be the design of targeted events to discuss project results with particular user groups.

Outputs & Website

All publications and materials related to this project will be available as the project progresses