Evidence from two generations of evaluations
A social cash transfer revolution is taking place in Sub-Saharan Africa (SSA). While most large cash transfer programmes (CTs) in the region date from the mid-1990s, the last 10 years have seen a rapid expansion of targeted CTs across the continent.
Sub-Saharan Africa is characterised by a much greater level of risk and vulnerability than other regions; the devastating effect of HIV/AIDS finds its global epicentre in Southern Africa.
In the last few years, rigorous impact evaluations of CTs have been carried out in no fewer than 12 African countries. The evidence indicates that CTs increased investments in agricultural assets and livestock and decreased negative coping strategies such as begging and pulling children out of school.
The targeting performance is better than the average CT programme around the world. The evaluations found that community-based targeting can be combined with other targeting methods to be effective in reaching poor populations in Africa.
A social cash transfer revolution is taking place in Sub-Saharan Africa (SSA). While most of the largest cash transfer programmes (CTs) in the region date from the mid-1990s – for example, old-age pensions in South Africa, Botswana and Namibia – the last 10 years have seen a rapid expansion of targeted CTs across the continent.
In the last few years, rigorous impact evaluations – both experimental and non-experimental – have been carried out or commissioned on government-run CTs in no fewer than 12 African countries.
There are a number of unique features in sub-Saharan Africa that have shaped the nature of their CTs. The region is characterised by a much greater level of risk and vulnerability, in large part due to the devastating effect of HIV/AIDS, which finds its global epicentre in Southern Africa.
The region also suffers from more generalised levels of poverty, less developed markets and greater political instability. At least in the short term, livelihoods and exit from poverty are inextricably linked to smallholder agriculture and the informal economy, and less to engagement with the formal wage economy.
Moreover, public institutions tend to be weaker in sub-Saharan Africa. Governments have less fiscal space available for social protection, and consequently, aid agencies play a much stronger role in terms of promoting and determining policy – including requiring impact evaluations of pilot programmes. Lack of government ownership and control over the policy-making process complicates programme design and implementation, as competing donors often have conflicting ideas as to the types of social protection interventions to pursue.
This is exacerbated by a considerable lack of consensus among many sub-Saharan Africa governments, and particularly officials in ministries that control the national purse, as to the efficacy of CTs. In general there is also a weaker capacity to implement programmes, and a weaker supply of complementary government services in health, education and nutrition. All of this complicates the design and implementation of CTs and of their impact evaluations.
Three important programme design differences distinguish the programmes in sub-Saharan Africa.
First, the community plays a much stronger role in verifying and selecting beneficiaries, although proxy means tests are increasingly gaining acceptance in combination with community-based targeting, as well as other methods.
Second, conditional transfers are the exception rather than the rule, although most of the unconditional programmes still have strong messages about why households receive the transfer and their responsibilities.
And third, public works programmes are much more prominent; two countries – Ethiopia and Rwanda – combine public works for households with able-bodied labour with direct CTs for eligible households that are labour-constrained or with high-dependency ratios.
The first generation of impact evaluations reflects the nature of the CTs themselves. Concern about vulnerable populations in the context of HIV/AIDS has driven the objectives and targeting of many of these programmes, leading to the emphasis in terms of target population on the ultra-poor, the labour-constrained, and/or caring for orphans and vulnerable children (OVC).
Most of these programmes focus on food security, health, nutritional and educational status, particularly of children, and so, as would be expected, the accompanying impact evaluations concentrate on measuring these dimensions of programme impact.
What does the first generation of impact evaluations show?
The first generation of CT impact evaluations include programmes in Kenya, Malawi, and Mozambique. They produced a lot of useful data and could also draw upon other existing datasets, including national household surveys, to better understand and contextualise results.
The evaluation of the Kenya Cash Transfer for Orphans and Vulnerable Children (CT-OVC), focused on the impacts on household expenditure patterns and investment in children’s education. Overall, the programme was found to have significant positive impacts in expenditure on health and nutritious food, including cereal, meat and dairy, and decreased spending on inferior goods like tubers, alcohol and tobacco.
Large impacts were found on current enrolment of secondary school children older than 12 years. Significant impacts are also found for grade progression among older children; the results indicate that the CT-OVC reduces the probability of older children dropping out and increases their probability of returning to school.
Although overall impacts at the primary level are much smaller, which is expected given free access to primary school, the programme has stronger impacts precisely for those households and children that are more likely to face financial obstacles in accessing primary school.
The impact evaluation of the Malawi Social Cash Transfer programme examined some of the potentially unanticipated impacts of CTs, such as on livelihood activities. Given that the intended Malawi programme participants are extremely poor and labour-constrained households, a common assumption among policy-makers is that these types of CTs do not contribute to productive economic activities.
The findings challenge this assumption. The evidence indicates that programme participants increased investments in agricultural assets and livestock compared with the control group. Also, programme households significantly decreased their participation in low-wage work on others’ land and in other negative coping strategies, such as begging and pulling children out of school.
The targeting performance of the three CTs – in Kenya, Malawi and Mozambique – is found to be better than the average CT programme around the world. The evaluations found that community-based targeting can be combined with other targeting methods (for example, geographical) to be effective in reaching poor populations in Africa.
Results from simulations run on existing data and programmes in Malawi and Ghana further indicate that CTs with social objectives can have similar productive impacts to an input subsidy programme specifically targeted to agricultural households. This does not imply that CT programmes should serve as agricultural policy, but rather that CT programmes have important implications for agriculture, and for rural development, and should be considered as such in the policy process.
What do we have to look forward to?
The second generation of impact evaluations currently underway are exploring several new research questions, and in some cases using innovative techniques to do so.
First, to examine the potential for CTs to mitigate HIV risk, the next round of the Kenya CT-OVC evaluation captures information on sexual debut, partner characteristics, marriage, and pregnancy. Similarly, the upcoming Malawi, Zambia and Zimbabwe evaluations include measures of sexual activity among OVC.
The second new evaluation question relates to psycho-social status and mental health. Both the Kenya and Zambia evaluations include two mental health scales. Preliminary analysis from Kenya suggests a strong and important impact of the programme on reducing depression among OVC.
The third evaluation question relates to the issue of conditionality. While for a variety of reasons, many practical, most programmes in sub-Saharan Africa are unconditional, the issue is still important. The World Bank tends to support conditional CTs, while DFID and UNICEF, tend to support unconditional CTs.
Some of the new World Bank-supported evaluations, including Burkina Faso and the new Kenya CT-OVC expansion, are explicitly examining the issue of conditionality and qualitative field work in a number of countries will examine the perceptions of beneficiaries in terms of expected behaviour.
The final set of new questions relates to the overarching issue of whether and how CT programmes can contribute to overall economic growth. New evaluations in Lesotho, Zambia, Malawi, Zimbabwe and Ghana plus the four-year follow-up in Kenya all contain detailed information on household income-generation activity, both on-farm and off-farm, including use of inputs, production, and sales. Moreover, each of these evaluations is also collecting a separate business enterprise survey in order to model the local economy effects of the programmes.
As new data emerge from these second-generation impact evaluations, there is substantial opportunity to enrich the evidence on the impacts of CTs in sub-Saharan Africa and to better understand the effectiveness of design and implementation variations in the region. Furthermore, this second generation promises to advance the types of evidence available on CTs globally and contribute to new evaluation methodologies.
This is a summary version by the author and the NAI Forum editor of Davis, B., Gaarder, M., Handa, S. and J. Yablonski (2012), Evaluating the impact of cash transfer programmes in sub-Saharan Africa: an introduction to the special issue, Journal of Development Effectiveness, Vol. 4, Issue 1, 1-8.