Georgina Smith / CIAT

On what (and how and when) to measure when measuring impacts of agricultural research for development

Iain Wright is ILRI’s Deputy Director General for Research, and was a plenary speaker at the recent SPIA and PIM conference on Impacts of International Agricultural Research: Rigorous Evidence for Policy. The following blog post draws on his presentation, and has been crossposted from ILRI News.

We in CGIAR have committed ourselves to tackling some of the greatest challenges that the human population has ever faced.

How do we feed a growing population not only with calories but also with nutrients essential for good health, and do so in the face of climate change? We know that proteins, vitamins and minerals are essential not only for growth in children but also for children’s intellectual development, their cognitive and learning ability. We know that malnutrition can not only stunt children permanently but also damage a nation’s long-term economic development.

Agriculture forms, and will continue to form, the basis of economic development in many part of Africa. Agriculture is the route by which millions of people will escape poverty, not just through improvements to the livelihoods of individual farmers but also through commercialization of smallholder agriculture, which generates employment in farm input services and in the production of value-added products along agricultural value chains.

Although agriculture is often viewed in industrialized countries as harmful to the environment, farming holds the key to effective natural resource management and the provision of essential environmental services, such as reduced intensities of greenhouse gas emissions achieved via well-managed rangelands and trees that store significant amounts of carbon, or absolute reductions in greenhouse gas emission levels achieved through increased agricultural productivity and more efficient use of farm inputs. 

While those of us who work in agricultural research recognize its importance, we must persuade the rest of the world of the case because agricultural research can deliver benefits such as these only if there is sufficient investment in this research. A few decades ago, agriculture received about 15% of official development assistance (ODA). Today, agriculture receives just 4% of total ODA (and of that 4%, agriculture’s livestock subsector receives just 4%, despite the fact that the livestock subsector contributes an average of 40% of the agricultural gross domestic product of developing countries).

Many studies show high rates of return to agricultural research but we need more specific evidence on what investment in agricultural research actually delivers. As donor organizations are under increasing political pressure to change their investment priorities, to better address, for example, domestic issues or the refugee crisis, it is timely to consider the role of impact assessment in CGIAR. 

Agricultural research for development deals with complex agro-socio-ecological systems generating complex problems as well as benefits. We use complex research methodologies to solve the complex problems.

A major challenge in assessing the impacts of our research is having sufficiently robust methods to generate robust evidence. While we have methods to assess rates of uptake of a given technology or the welfare benefits of adoption of that technology, not all research is focused on a single technology. How do we assess the impact of research that delivers a mix of new technologies that are likely to be adopted and adapted in different ways by different farmers? In the livestock sector, for example, improved livestock genetics will have little impact if not accompanied by better livestock feeding strategies and health services, which themselves will require new institutional and marketing arrangements, which in turn will be effective only where there are policy environments conducive to such novel arrangements. In such cases, how do we discern what impacts our research is having?

Where CGIAR research focuses on influencing decision-making, the effects of such research on the complex political processes involved are often difficult to assess. Twelve years ago, I was at a workshop on the interface between research and policy organized by the chief scientist at the Scottish Government Rural Affairs Department, who at that time was Maggie Gill, now chair of the CGIAR ISPC. One participant presented a list of things a minister has to consider when devising a new policy. Technical or scientific evidence was only 1 of 23 things on that list. How do we know what impact our research is having on the other 22 factors being taken into account?

As we consider here impact assessment work in CGIAR, let us also continually ask ourselves how we can best deal with complex questions about impact. This will help us avoid focusing only on things easy to measure.

To meet the global challenges that CGIAR is researching, we will need not incremental but rather transformational change in smallholder agriculture.

“If we focus on things that are easily measured, we will fail to provoke those transformational changes.”

Do we have the tools and methods needed to measure the impacts of complex solutions to complex problems? I believe we need more methodological development of quantitative and qualitative impact assessments. I believe we have much to learn from other sectors, including public health and education.

So as we delve into impact assessment work this week, let us look not only at what we have achieved in the past but also at how we will demonstrate our achievements in the future.

Read more