Measuring Income of Smallholder Farmers: Some Insights from Practitioners

Many ICCO programs aim to break the cycle of poverty and food insecurity by increasing income and well-being of poor households. So does the STARS  program, by facilitating access to financial markets and agricultural services to smallholder farmers.
In cooperation with Mastercard Foundation, 210.000 smallholder farmers are targeted in Rwanda, Ethiopia, Burkina Faso and Senegal. But the research poses certain challenges. How to determine the income of the target group in a program like STARS? And how to measure the changes in income due to your intervention?

Income proxies
Now, here it gets complex: realizing the multiple factors and strong fluctuations that play a role in a household’s income and the sensitiveness of the topic in general. Measuring income directly turns out to be a very tricky and delicate task. One potential way to address these challenges are indirect income measurements, so-called “income proxies”. To measure the economic potential of target groups, ICCO has found  the Poverty Probability Index (PPI) to be a statistically sound and simple-to-use tool that is based on a 10-question household survey. In addition, ICCO developed three supplements to the PPI: the ‘plus’ part, called PPI+, involving farm productivity and sales, aspirational and actual spending and actual assets.These supplements are generic and need to be contextualized to specific program contexts. The STARS program is actively using the PPI tool and supplementing it with selected components of the PPI+ on land size and livestock, household and farm assets that were contextualized and specified in STARS. Our aim: to better understand the base- and endline of our target group from the perspective of their income level and economic status.

But does that all work in practice? There is no short answer. Below we present four key lessons to point out what it actually means to use PPI and PPI+ supplements in a project and discuss the potential and limitations of these tools.

1.     Income can change quickly. Poverty doesn’t.
When looking at the income of a smallholder farmer (or also of many other target groups in context of developing countries) it is important to realize that a household’s income includes both cash and in-kind components, and it is usually generated from multiple activities by several members of a household. Thus, income can change from one day to the other, and it strongly fluctuates over time. Poverty on the other hand goes far beyond just income including a multitude of dimensions such as social status, education, assets, family size, economic prospects, legal rights, etc. To capture and measure this complexity, PPI estimates the proportion of households that live below a certain poverty threshold – the so-called poverty rate for a population or target group. It offers a holistic view on poverty by including a locally relevant set of these different poverty dimensions. PPI data can show whether or not, and to what degree, the targeted households are actually experiencing economic poverty [2]  at a given point in time. The tricky thing: changes in the poverty rate of a target group are of course highly relevant, but often slow to occur and therefore most likely exceeding what we can observe within a project’s life span. During the STARS baseline study we found that the information captured by the PPI (for example relating to family size and literacy level) is not very sensitive to short-term changes. However, the tool did generate valuable insight into the people who are reached. Hence to capture changes that occur within a project life span, we need tools that allow us to measure short-term changes.

To address this challenge, the STARS program approached income also from another angle [3] – ‘the greater the cash income, the more you can spend’. Therefore, we incorporated additional expenditure-oriented metrics in the baseline survey in order to be more sensitive to short-term changes. Keeping the project realities in mind, it was decided to focus on assets that can easily be tracked and verified. Thus, we included a mix of productive and consumptive assets on the household level such as land size, livestock (based on Tropical Livestock Unit TLU), household and farm assets.

When using this approach, it is important to realize that an increase in asset ownership can also result from other income generating activities. Besides that, the approach followed in the STARS program is based on the assumption that an increase in income can be gauged from a small set of assets (excluding other important elements like increased savings, increased expenditures, etc). So results on income remain ‘indirect’, and economic empowerment is covered to a limited extent only.

2.    Qualitative and quantitative data. We need both for actionable insights.
These indicators used  in STARS (land size, TLU, the asset score, PPI) are quantitative measures. By combining them, we are able to triangulate between the results, giving stronger evidence regarding the program impacts. However, these results that are based on the quantitative measures do not generate deep actionable insights into the what, how and why questions that can be used by program staff to design interventions on economic empowerment. Thus, the results of the PPI and PPI+ analyses per se will not deepen our understanding of how income is composed, what strategies households follow, and how different income streams and their cash flows affect them. However, the PPI metrics are useful in answering individual underlying questions regarding particular elements of economic empowerment. For example the PPI data can contain data on mobile phone ownership, whereas the asset survey contains information on the ownership of particular productive assets like farm tools. However, for a stronger learning component and actionable insights, we clearly see the need for complementing them with tools that can capture qualitative aspects.

3.    Tools are only as good as our reasons and methods for using them.
The PPI and the supplements measure different aspects and you have to be aware of the applicability of each tool, for integrating them better in your monitoring. Also, rolling out and analysing a PPI survey and supplement is more than just sending out enumerators with a questionnaire. It is a sequence of working steps that are all important to ensure that you get reliable and useful results. Be it the preparation and fine-tuning of the questionnaire, or the field-testing of the survey, they will help to improve practical implementation. Of course, also a good selection and training of the enumerators is key. And here again, field-testing can be a great opportunity to deepen the enumerators trainings with some hands-on experience before sending them out for the main survey.

In a nutshell: ‘garbage in, is garbage out’. There are a number of practical details to be taken into account throughout the survey and enumerator preparation, data collection, data analysis and interpretation and one needs to be ready for that, if you want to use these tools.

4.    Accountability counts!
Yes, there are problems and limitations, but here are some of our positive observations: What we see is that the PPI tool complemented with assets enumeration can be applied in large-scale surveys using a statistically representative sample size. The tools are relatively easy to implement and analyze for accountability purposes. They can generate robust data which can be conveyed in tables in annual  or donor reports. When you have a good sample selection (including specific target groups) and when relevant additional data is collected on respondents (including things like age or the use of particular products), you will be able to generate good results on real differences between subgroups. This is certainly how it worked for the STARS program.

Next steps
What we propose as a next step is to test alternative approaches that have a stronger component of learning and generating actionable insights, specifically on economic empowerment. These include the use of Smallholder Diaries [4] or the Household Economy Approach [5]. Such approaches may offer a valuable addition to the quantitative metrics that we tested. All that is needed is an economic empowerment program that is willing and able to try this out. To start with, we would like to develop these approaches further together with an enthusiastic master student hosted in the respective program. Are you interested? Then please get in touch with Sandra Nicolics s.nicolics@icco.nl or Dieneke de Groot d.degroot@icco.nl at ICCO’s Global Office.

Authors: Marco Dekker, Sandra Nicolics and Heidi Lampen

[1] Strengthening African Rural Smallholders (STARS)
[2] http://www.impactmanagementproject.com/latest/signpost-ppi/
[3] That expenditures are a proxy for income.
[4] For example see http://www.cgap.org/publications/financial-diaries-smallholder-families
[5] See http://www.heawebsite.org/about-household-economy-approach