Effectively Assessing Household Food Security Status
Blog written by: Marco Dekker, Marijke de Graaf, Martijn Marijnis.
The three authors work at ICCO Cooperation: Marco Dekker is Monitoring, Evaluation and Learning coordinator STARS program, Marijke de Graaf is the Food and Nutrition Security expert and Martijn Marijnis is specialist Monitoring and Evaluation.
There is growing commitment at global level to improve food and nutrition security, reflected by SDG 2. This has gone hand in hand with growing demand for evidence on what policies and programs work best, how and at what cost. Good quality measurement tools are key to collect data on output, outreach, outcome and impact levels, for researchers and practitioners to draw timely lessons and to generate reliable evidence on what works well. In addition this is paramount because “Establishing outcome indicators for food systems can guide policy makers in fostering nutrition-friendly and sustainable food systems while also helping citizens hold their governments accountable for their policy choice” .
HFIAS, a balanced tool?
In order to identify and fight hunger the development community faces the challenge to obtain reliable data on occurrence and severity of food insecurity, with limited human and financial resources. Therefore robust and cost-efficient tools are necessary. The use of the Household Food Insecurity Access Scale (HFIAS) tool in multiple settings and countries since 2008 has given ICCO Cooperation a unique opportunity to compare findings, systematize experiences and improve practices in all stages of the monitoring, evaluation and learning mechanisms. By March 2018 HFIAS data has been gathered in 17 countries, using tools like Akvo Flow, ODK, KOBO ToolBox and CSPro, providing food security data including geo-location through mobile devices. We perceive the HFIAS as a tool that strikes a good balance between practical field-level applicability and reliability of the ensuing data.
HFIAS. How does it work
There is broad experience with measuring food access and utilization, such as income levels, food related expenses, production levels and caloric content of diets. However, these turn out to be technically difficult, data-intensive and costly to collect. The same goes for measuring nutrition status based on anthropometric data, i.e. weight and height, at individual level. Given these challenges the USAID supported Food and Nutrition Technical Assistance (FANTA) project developed a number of tools, among which HFIAS.
The FANTA project identified a set of questions that distinguish the food secure from the insecure households across different cultural contexts. These questions represent three universal domains of household food insecurity and can be used to assign households along a continuum from food secure to severely food insecure, as shown in figure 1 below:
Figure 1: Adapted from Voices of the Hungry
http://www.fao.org/in-action/voices-of-the-hungry/fies/en/
The HFIAS questionnaire consists of nine standardized questions asking respondents to identify behaviors, attitudes and psychological manifestations, that relate to the various domains of the food insecurity experience, as shown above. The questions address the situation of all household members (without distinguishing adults from children or adolescents) over the previous 30 days or during the last month. The questionnaire was validated in several countries across different cultural contexts, and it was concluded that the information generated by HFIAS can be used to assess the prevalence of household food insecurity, as well as to detect changes in the household food insecurity situation of a population over time. For a brief overview of how HFIAS data are analyzed, see HFIAS data processing and analysis
Field experiences of HFIAS
Based on the application of the HFIAS tool in baseline as well as end-line surveys in abovementioned 17 countries we identified an issue with over-reporting the occurrence as well as severity of food insecurity in a population. This is supported by a study done by Maxwell et al. who calculated the correlation between six broadly used food insecurity measures, including the HFIAS. This revealed that the HFIAS presents higher rates of food insecurity compared to other metrics.
ICCO has done further investigation on this issue, comparing HFIAS scores (0 – 27) with HFIAS categories (1 – 4), drawing on the results of the following studies:
- A baseline survey done in Rwanda in May 2017 with 1.063 smallholder households
- A baseline survey done in Senegal in July 2017 with 1.206 smallholder households
- A small pilot study done in Senegal in November 2017 with 100 smallholder households
Figure 2 presents a count of households for each of the possible HFIAS scores in Rwanda. The data show a large group (12%) with a score of 0, meaning they indicated to have no problem in accessing sufficient food. The remaining 88% form a curve with few smallholders that have low or high scores, and most having an intermediate score. The median value in the dataset is 11, which is also where the curve has its peak.
Figure 2: Count of household HFIAS scores, data from Rwanda (n=1.063)
In addition we determined the HFIAS category for each of the households, ranging from 1 (food secure) to 4 (severely food insecure). Figure 3 shows the relation between the HFIAS scores and HFIAS categories. Looking at the different HFIAS categories it becomes clear how wide the range in underlying HFIAS scores is. Even very low scores are assigned to the highest food insecurity category. Households with a low HFIAS score of 2 (on a range of 0 to 27) can be classified as category 2 (mildly food insecure), 3 (moderately food insecure) or even 4 (severely food insecure).
Figure 3: HFIAS prevalence scores in relation to household scores, data from Rwanda (n=1.063)
Issues encountered with the food security categorization
One can expect that with increasing food insecurity, the total HFIAS score increases. More questions would be answered positively with a higher reported frequency, leading to a higher total score. We find however that many high HFIAS categories occur at low HFIAS scores, as shown in Figure 3. This depends on which of the nine questions are answered positively and how the categorization is done. Such a strong influence of single questions would indicate that the categorization formula is very sensitive, and a small variation in input gives rise to a large variation in output.
Data collection through a household survey is always challenging and might yield measurement errors. Our experiences show that this more easily occurs in the following cases:
- When the HFIAS tool is part of a long questionnaire, especially when the respondent is asked to pause his work to take part in the survey, this might lead to respondents quickly giving an answer without much deliberation (called satisficing).
- When questions are perceived as socially sensitive and respondents are too proud to admit to experiences of food insecurity. This is confounded when the social distance with the interviewer is large (like urban interviewers versus rural farmers). Both cases will lead to people misrepresenting their actual situation.
- When questions, concepts and words in the questionnaire are unclear or complicated, or do not translate well into the local language, leading to subjectivity and variability in answers.
When such measurement errors are confounded with a categorization system that is sensitive, this could lead to spurious findings and misclassifications and therefore needs due attention both during collection as well as processing of the data.
Recommendations to improve HFIAS
To address the above identified challenges with HFIAS data collection and processing and to improve the application of the tool the following two steps were explored: 1) reducing the measurement error and 2) reducing the sensitivity of the categorization.
Step 1: Reducing measurement error
First of all the HFIAS questions need to be contextualized to the local situation, and the timing of the survey needs to be aligned to the lean season. The survey should additionally be preceded by a good training and pilot testing with enumerators before going to the field for data collection.
Contextualization
To ensure the validity of the HFIAS tool, the nine questions need to be contextualized. The approach proposed by Coates et al (2007). is elaborate and significantly raises expenses and time needed so instead we organized a number of interviews with smallholder farmers (n=30) asking about their dietary habits, and we used the results to contextualize the questionnaire (mainly by adding examples to the question to make them less generic and abstract). We then piloted the contextualized questionnaire in a small study (n=100) in Senegal.
Results from applying this contextualized questionnaire compared to the original questionnaire, were not very different. Responses to individual questions appeared somewhat less variable, but this may be partly due to the small sample size of the pilot study. Nevertheless we feel that contextualization offers an important improvement to the HFIAS tool. The main benefits of contextualization lie in reducing social sensitivity and it thereby reduces non-response (which did occur on our baseline survey in Senegal) as well as misrepresentation (giving socially acceptable but untrue responses). Making interview questions more specific also relieves the burden on respondents as they do not have to mentally translate an abstract question into a specific answer (and this will also reduce variability between respondents). The modest level of improvement does mean that contextualization can be kept at a practical and affordable level, for example to combine it with enumerator training and pilot testing.
Timing data collection to the lean season
A household’s food security situation is not constant throughout the year. The problem with doing HFIAS measurements outside the lean season is that it can yield relatively positive results outside of the lean season while missing the food shortages that people can experience during the lean season, see blog ‘The importance of timing’.
The baseline survey in Senegal was carried out in July (in the lean season), whereas the pilot study was carried out in November (outside of the lean season). The effect of timing proved to be strong, as expected. During the lean season we found that 13% of the respondents had a household score of 0 and the highest household score encountered was 24 (out of a maximum of 27). Outside the lean season however we found that 44% of the respondents had a household score of 0, with the remaining households not having a household score higher than 16 (out of 27). These data point to a strong improvement of food security after the lean season when new harvests are coming in from the fields. From this it becomes clear there is an absolute need to collect HFIAS data during the lean season, during the hunger months.
To additionally gain insight in the annual fluctuations in food shortages, also outside of the lean season and thus to better understand the overall food security situation of target groups, we propose to combine the HFIAS tool with the MAHFP tool (Months of Adequate Household Food Provisioning). This indicator accounts for seasonality and provides a broader picture of household food access over a period of one year. The indicator consists of only 2 dependent questions:
- Were there months in the past year in which you did not have enough food to meet your family’s needs?
- If yes, which were the months during which you did not have enough food to meet your family’s needs?
Figure 4 presents MAHFP data showing the months when people face food shortages, i.e. identifying the lean season(s). In most cases this coincides with the rainy season and thus the planting season. The figure clearly demonstrates that food security significantly varies throughout the year, explaining the drop in food insecurity that we observed going from July to November.
Figure 4: Count of people indicating food shortages, data from Senegal (n=1206)
Training enumerators
A thorough training of enumerators can help in collecting better data. A training would address the following aspects:
1. The context of the program for which the survey is done. When enumerators understand and can explain the program to respondents and why it is important to participate in the survey, this will help to get buy-in and prevent misunderstandings about possible payment. When the enumerators feel part of the program and when a respondent is motivated to participate, this will more likely generate accurate responses.
2. Understanding the overall logic and individual questions of the HFIAS model. A good understanding of the HFIAS tool will help enumerators engage in a conversational style interview, and apply probing questions to ensure that dietary adaptations are caused by resource scarcity and not for example the seasonal availability of food items on the local markets. This is also a good opportunity to discuss the correct translation of questions in the local language.
3. The social distance between enumerators and respondents. It needs to be well understood that any displays of wealth, education or even religion or ethnicity can create a large social distance with the target group to be surveyed. Enumerators need to be instructed to not wear jewelry or expensive clothes, and refrain from wearing religious or ethnic items that may cause friction with the respondents.
4. Pilot testing. A field test with comparable respondents is important to gain experience with the tool and share experiences and insights afterwards. This can focus on understanding of and reactions to the survey questions by respondents, or the effects of mixed gender interviews (women interviewing men or vice versa). Based on a debrief afterwards, questions and work planning and division can still be adapted.
5. Contextualization of survey questions. We mentioned before that we feel that contextualization can be kept at a practical level, for example by including it in enumerator training and pilot testing. By first discussing the HFIAS questions with enumerators (who preferably would have a good understanding of the local context and target group), the proper translation and relevant wording of questions is already the first step in the process of contextualization. After pilot testing in the field the questionnaire can be revisited, and together with program staff, field supervisors and enumerators the definitive wording of questions can be decided on in order to maximize their correct understanding by respondents. Careful wording can also prevent people from refusing to answer particular questions, which in our case led to the loss of 209 interviews.
Step 2: Reducing the sensitivity of the prevalence scores
As a second step we propose that the prevalence scores are calculated differently, in a way that is less sensitive to small variations in input. We stay within the original thinking in which there are blocks which represent increasing food insecurity (see HFIAS background article for the original way of categorization), but we reduce the overriding importance of questions regarding the adaptation of quantity of the food consumed.
Based on the total household score ranging from 0 to 27, we classify households according to the table below:
According to this classification method, households that are considered food secure are those that have a household score of 0 or 1. Mildly food insecure are those households that have a score from 2 to 8. Moderately food insecure are those households that have a score from 9 to 16. And severely food insecure are those households that have a score from 17 to 27.
Applying this new categorization scheme we find that results become less erratic, see Figure 5. This new approach builds on the premise that higher food insecurity is given by a higher HFIAS score. This takes out the dependence on single questions of the old prevalence score, and prevents the possible spurious finding of households with very low scores being classified as severely food insecure.
Figure 5: HFIAS categories in relation to scores following the old (top) and new (bottom) categorization approach, data from Senegal (n=997)
As a result, the distribution of food insecurity in the population follows a more natural curve, see Figure 6. Compared to the old approach the new approach results in an natural and right-skewed frequency distribution.
Figure 6: Histogram of prevalence categories, data from Senegal (n=997)
Conclusion
With this article we shared our hands-on experiences with a standardized tool as well as recommendations to make the data more reliable. We strongly believe that increased coordination and harmonization on the use of standardized metrics will facilitate sharing of results and lessons learned as well as the establishment of benchmarks. Subsequently this will contribute to more effective interventions towards eradication of malnutrition in all its forms. As illustrated above HFIAS is ‘fit for purpose’ as a standardized approach for the measurement of household food security at project level. For a comparison with the recently developed Food Insecurity Experience Scale (FIES), please go to the article: HFIAS vis-a-vis FIES.
About the data used
Data used for this article were collected in the STARS program (Strengthening African Rural Smallholders). This is a 5-year program running from 2016 to 2020, aiming to increase food security and income of 210,000 smallholder farmers in Rwanda, Ethiopia, Senegal and Burkina Faso by facilitating access to financial markets and agricultural services. ICCO Cooperation implements the program in collaboration with ICCO Terrafina MicroFinance and in partnership with The MasterCard Foundation.
