11 - Innovation Systems Perspective: From Measuring Impact to Learning Institutional Lessons: International Development Research Centre

International Development Research Centre (IDRC)

Web Archives > Publications > IDRC Books > All our books > PARTICIPATORY RESEARCH AND DEVELOPMENT FOR SUSTAINABLE AGRICULTURE AND NATURAL RESOURCE MANAGEMENT: A SOURCEBOOK >

Topic Explorer

People

ID: 85054

Added: 2005-07-18 15:13

Modified: 2007-03-25 19:14

Refreshed: 2012-02-10 18:30

Click here to get the URL for the RSS format file

11 - Innovation Systems Perspective: From Measuring Impact to Learning Institutional Lessons

Document(s) 14 of 34

Andrew Hall, V. Rasheed Sulaiman, Norman Clark and B. Yoganand

While perspectives recognizing the institutional context and research have occupied only a modest amount of attention in the international agricultural research policy community, the perspective has come to dominate the policy debate and practice in other research and economic sectors. It is surprising to find that concepts that are informing international agricultural research policy were superseded a decade ago in this wider science and technology policy arena.

The contemporary debate from this parallel policy literature now takes it as given that the linear model of innovation and its neo-classical economics is of little value in evaluating and planning research and development (R&D). There has been a shift in the role of policy from examining the determinants and consequence of research, to a capacity development role where emphasis is on strengthening networks of users and producers of knowledge (Velho, 2002).

Underpinning this shift of perspective over the last two decades has been a deepening understanding of the nature of innovation as a process and the accompanying realization that neo-classical economics alone cannot explain the dynamics of economic systems.

Adapted from:
Hall, A., V. Rasheed Sulaiman, N. Clark and B. Yoganand. 2003. From Measuring Impact to Learning Institutional Lessons: An Innovation Systems Perspective on Improving the Management of International Agricultural Research. Science Direct Agricultural Systems 78 (2003) 213-241. http://www.sciencedirect.com.

The innovation system concept serves to draw different ideas together including the idea of a "national system of innovation". At its simplest, the concept recognizes that innovations emerge from systems of actors. These systems are embedded in an institutional context that determines how individual actors behave and how they interact with other elements of the system. Learning and the role of institutions are critical components of such systems. Learning is an interactive and thus socially-embedded process, which cannot be understood without reference to its institutional and cultural contexts (Lundvall, 1992). Successful systems are characterized by:

continuous evolutionary cycles of learning and innovation

combinations of technical and institutional innovations

interaction of diverse research and non-research actors

shifting roles for information producers, information users and transfers of knowledge dependent on a need basis

an institutional context that supports interactions, learning and knowledge flows between actors

The application of this concept of a national system of innovation in the agricultural research sector is gaining ground (see Hall et al., 2002.; Clark et al., 2003). At the heart of this framework is the contention that R&D is always embedded in social, political and institutional contexts and that unless the influence of this environment is accounted for by decision makers, the evaluation and planning of R&D will be incomplete.

Innovation Systems in Planning and Evaluation Processes

What does this mean for the evaluation and planning process? Some of the principles that are required to relate R&D to institutional context include the following.

An inventory of Innovation Actors

The framework provides a starting point for identifying the full range of actors relevant to a particular innovation system. While many of the normal public-sector actors are present in the conventional policy schema, closer investigation reveals a wider range of individuals and organizations from other sectors.

System Competency

Once a full inventory of actors has been established, it is then possible to examine the extent to which relationships exist among actors. The existence of relationships will depend on the policy context and the wider institutional environment. For example, strong public-private partnerships may have emerged through a liberal policy towards germplasm access. Alternatively, weak linkages may be a result of restrictive personnel polices for public sector scientists that prevent them from undertaking contract research for the private sector. Hence, analysis has the effect of directing the focus of evaluation and planning on linkages that need to be developed and on potential policy changes.

Actor Roles

Part of the relationship analysis concerns the importance of multiple roles played by some actors and the different types of relationship these roles imply. For example, an agricultural university may be both a source of information on regional variety trials, as well as a recipient of improved breeding lines from a crop improvement center. Both types of roles are important for an effective innovation system, and the evaluation and planning process needs to understand their separate but linked existences. Actors with important roles that are excluded from existing arrangements need to be recognized. Technology users and product consumers from poor communities are examples.

Cultural Context

The types of relationship that develop in a particular innovation system reflect the national context as well as different organizational cultures. For example, the national context may have a strongly paternalistic public sector culture with a mistrust of private sector enterprise. Or the public sector may have a strongly hierarchical culture, whereas the NGO sector may have a more decentralized, participatory culture. Partnerships between public agencies and NGOs will not necessarily lead to more participatory approaches because of the organizational culture of the former. The evaluation and planning process needs to account for these contextual features.

Relationship Dynamics

The importance of the nature and dynamics of relationships between the entire range of actors, from the innovation systems point of view, is that their analysis reveals that such relationships are often strongly asymmetrical, preventing interactive learning. For example, partnerships between international and national agencies are often skewed by more favorable access to resources on the part of the former, by historical patterns of interaction, and by professional and cultural norms that value "outsiders" at the expense of "locals". Local political processes, interest groups, ethnic communities, and social hierarchies will all contribute to the political economy of the innovation process. The evaluation and planning process will benefit from an awareness of these dynamics.

Reflection and Institutional Learning

The innovation systems framework regards reflection on process and institutional learning as key elements for success. For example, systems in which there is clearly a gulf between policy rhetoric and research practice have a weakness with regard to institutional learning.

Other indicators of weak institutional learning may be a reluctance to admit mistakes and confront failure and its causes, or even a reluctance to revisit key assumptions about roles or ways of working. In contrast, an organization in which senior management encourages and rewards reflection and learning and where self-evaluation is undertaken regularly, demonstrates a tendency to possess a higher capacity for continuous institutional learning and innovation. The evaluation and planning process could benefit from recognizing the importance of a learning culture within public-sector research organizations and their partners (Watts et al., 2003).

This philosophical shift towards institutional learning and change entails practical changes in international agricultural research organizations. These include the following:

Moving the focus of impact and evaluation from examining changes in technology user groups to including changes in the way the research community operates as well as its interaction with other organizations and institutional (including political) contexts.

Introducing institutional changes that provide incentives to formalize learning as part of the practice of research organizations. This requires changes among donors and senior managers of research organizations and probably within professional bodies relevant to the international agricultural research community.

Recognizing capacity development as an important outcome and purpose of research.

Accepting the need to explore behavioral changes in innovation systems as a way of monitoring progress and learning, as well as a way of promoting critical institutional lessons to wider audiences in the R&D community.

Recognizing the systems nature of capacity development so that evaluation becomes a task that needs to be done collectively with partners as well as at the individual organizational level.

Accepting the need to embed evaluation as learning in the day-to-day procedures of research stations and administrators and acknowledging the skill and resource implications of this. This implies the need for greater numbers of social scientists in international agricultural research organizations, but with a hands-on role of facilitating learning in addition to disciplinary research contributions. It also implies the need to build learning skills among all partners and to allocate time within the research process for collective learning and reflection.

The innovation systems framework is not presented as a panacea for improving the performance of agricultural research. The aim is to draw to the attention of planners, evaluators and research managers to the need for (and the possibility of) thinking about agricultural research in a more holistic and evolutionary fashion.

A Case Study: Learning as a Way of Dealing with the Institutional Context of Research

This case study discusses how the crop post-harvest program of the Department for International Development (DFID), the UK government's international development assistance agency, has gradually recognized the need to pay more attention to the institutional context of the research it was sponsoring and how it responded with an approach that is attempting to embed institutional learning in conventional technology-development projects.

The program is one of the 10 natural-resources research programs. These were originally established by DFID in 1995 as a way of exploiting the UK science base in support of international development. The programs were conceived in the problem solving framework of the project cycle with the "logical framework" used as the key program and project planning and evaluation tool. This was supplemented by monitoring indicators used to judge progress along a notional output pathway. The translation of technical outputs into poverty/development impacts was dealt with as a logframe assumption about the existence of "target institutions" (meaning, in this instance organizations) and functioning "up-take pathways".

As projects progressed the Crop Post Harvest Program started to recognize that process and institutional issues were having serious consequences for the success of its research initiatives. For example, in a series of project commissioned in India to provide technical backstopping to parts of the export horticulture sector, it became apparent that the real problem was one of mobilizing the different parts of the public-sector research system to act in a concerted fashion. Collaboration was particularly important for export development because of the need to deal with quality management issues in an integrated production and post-harvest supply chain. In addition, the broad range of stakeholders in the supply chain, including farmers, whose agendas and circumstances provided the context for developing these solutions, made it difficult for the research organizations to respond effectively, given their prevailing way of working with stakeholders.

At this point of program management team decided to gain a systematic understanding about the way this institutional context was affecting its research. The learning process built up slowly. First there was a pilot project that continued its focus on export horticulture, but which included simultaneous technical and institutional analysis. This highlighted the need to identify a conceptual framework to help understand the wider contextual issues that were affecting the research process. It was at this point that the program started to explore the innovation systems framework.

The exploration began with a policy project in India to examine how the innovation systems idea could be used in the evaluation and planning of R&D. This project was undertaken with a view to drawing both project and program management level lessons. It was contingent on the wider program portfolio of projects in India which in effect acted as case studies. This approach allowed the program in South Asia to experiment with the innovation system idea, while allowing conventional projects to proceed. It became apparent that the arrangement was not ideal. Notably the institutional lessons that the policy project was gathering from the rest of the portfolio could not be used to redirect these projects as the portfolio was not structured in a truly action-research framework. It soon became apparent that the individual technical projects needed to concentrate on generating their own process and institutional lessons, for project management purposes as well as to gain insights of value to the wider program. However, it was difficult for projects that had been commissioned to deliver a narrow set of outputs to accommodate this expanded role.

Nevertheless, the program was able to identify and document a series of research management lessons. These included the following:

There is a need to build stronger and more consultative linkages between public sector science and other actors in the innovation system.

Successful projects were those that focused specially on establishing a coalition of local actors around a particular problem area.

These actors included scientists, but not exclusively, and not necessarily as the lead actor. Moreover, roles may evolve over time.

The selection of the most appropriate actor grouping was very often an empirical issue that could not realistically be resolved at the outset of a project.

There was a tendency, reinforced by the output-oriented, problem solving framework of the conventional project cycle, to under-report process lessons associated with technological success (or failure). These lessons were often complementary to new technological knowledge.

The relative degree of poverty focus was related to the agendas of different project partners and the dynamics that determined how these agendas were promoted in the wider arena of the project.

Needs assessment and participatory approaches were much less important in ensuring a poverty focus than the agendas of the stakeholder involved in projects.

The program consolidated these types of lessons through a program-commissioned formative review (Biggs and Underwood, 2001). The review was principally concerned with providing a basis to argue for changes in the program logframe. Specially, there was good reason to challenge the need to monitor direct poverty impacts at the project and program level (even though in the long-term the program and DFID would be accountable for these outcomes). A more pragmatic approach appeared to be to track behavioral (and therefore institutional) changes that the program was stimulating among project partners as milestones toward reducing poverty. The key leading indicator thus became the extent to which systems capacity to innovate in a pro-poor fashion was being developed. The review recommended that to contribute to the development of this capacity, the program needed to:

shift to an innovation systems approach because the emphasis had to move from a problem-solving framework to a learning framework

shift to action research protocols rather than the project cycle management tools

develop projects that involve groupings of local partners (coalitions), where identifying partners becomes part of the research task

use stakeholder analysis to make agendas transparent

monitor partner and stakeholder roles and interests to maintain a poverty focus

These broad principles have informed program strategic plans for 2002-2005. As the program works through some of the wider implications of this shift, it and its project partners will have to continue to use institutional learning as a core research management tool.

(For further details, see Hall and Sulaiman, 2002)

References

Clark, N, G., A.J. Hall, V. Rasheed Sulaiman and N. Guru. 2001. Research as Capacity Building: The Case of an NGO Development Post-harvest Innovation System for the Himalayan Hills. World Development. Vol. 31, No. 11. pp. 1845-1863.

Freeman, C. 1987. Technology and Economic Performance: Lessons from Japan. Pinter, London.

Gibbons, M., C. Limoges, H. Nowotny, M. Troww, P. Scott and Schwartzman. 1994. The New Production of Knowledge. Sage, London.

Hall, A. J. and R. V. Sulaiman. 2002 Application of the Innovation Systems Framework in North-South Research Collaboration. The International Journal of Technology Management and Sustainable Development. Vol. 1 No. 3 pp. 195-212.

Hall, A.J., M.V.K. Sivamohan, N. Clark, S. Taylor and G. Bockett. 2001. Why Research Partnerships Really Matter: Innovation Theory, Institutional Arrangements and Implications for the Developing New Technology for the Poor. World Development 29 (5), 783-797.

Hall, A.J., V. Rasheed Sulaiman, N.G. Clark, M.V.K. Sivamohan and B. Yoganand. 2002. Public-Private Sector Interaction in the Indian Agricultural Research System: An Innovation Systems Perspective On Institutional Reform. In: Byerlee, D. and R. G. Echeverrý´ a. (eds.). Agricultural Research Policy in an Era of Privatization: Experiences from the Developing World. CABI, Wallingford.

Hall, A. J., R. V. Sulaiman, N. G. Clark, B. Yoganand. 2003. From Measuring Impact to Learning Institutional Lessons. International Agricultural Research. Agricultural System 78: 213-241.

Horton, D. 1998. Disciplinary Roots and Branches of Evaluation: Some Lessons from Agricultural Research. Knowledge and Policy: The International Journal of Knowledge Transfer and Utilization 10 (4), 32-66.

Horton, D. and R. Mackay. 1999. Evaluation in Developing Countries: An Introduction. Knowledge, Technology and Policy 11 (4), 5-12.

Lundvall, B.A. (ed.). 1992. National Systems of Innovation and Interactive Learning. Pinter, London.

Maredia, M., D. Byerlee and J. Anderson. 2000. Ex -Post Evaluation of Economic Impacts of Agricultural Research Programs: A Tour of Good Practice. Paper presented at the Workshop on "The Future of Impact Assessment in CGIAR: Needs Constraints and Options", Standing Panel on Impact Assessment of the Technical Advisory Committee, Rome, 3-5 May, 39 pp.

Nelson, R.R. and S.G. Winter. 1982. An Evolutionary Theory of Economic Change. Harvard University Press, Cambridge.

Velho, L. 2002. North-South Collaboration and Systems of Innovation. The International Journal of Technology Management and Sustainable Development 1 (3).

Contributed by:
Andrew Hall, V. Rasheed Sulaiman,
Norman Clark and B. Yoganand
E-mail: Hall@intech.unu.edu

Document(s) 14 of 34

guest (Read)(Ottawa) Login