What are school statistics worth?

In a new UNECE report, we review and apply several methods to measure the return-on-investment of statistics. The benefits of statistics for development are often evident but hard to measure, which makes investments in statistics difficult to justify. The need to measure the value of official statistics has become ever more important in times where National Statistical Offices (NSOs) face tight budget constraints, in particular in developing countries. To maintain relevance, NSOs need to convincingly communicate their added value. The UN’s push for “data for development” is a helpful step in this direction but there is still scepticism: The Economist, in its Jan 24th issue, quotes the Copenhagen Consensus Centre’s Bjorn Lomborg saying that “gathering data is hugely expensive” compared to other “value for money” investments for the SDGs and Jerven (2014) concludes that the benefits of producing the data to monitor all SDG targets are not likely to outweigh the cost.

Measuring the return-on-investment of statistics is ridden with methodological challenges ranging from data generally having multiple users and uses (so it is often not even clear where to expect impacts) to the practical impossibility of running rigorous RCTs (because withholding information is unethical or because of information spill-overs between treatment and control group).

In the UNECE report, we present several methods to address these issues and are therefore helpful in quantifying the return-on-investment of official school statistics. In one case study, I build on a unique policy experiment in England and Wales (reported in Burgess et al., 2013), where Wales stopped publishing school performance statistics in 2001 while England continued. The first method estimates the effect on a country’s economic growth. The second method quantifies the cost-savings that can be had from avoiding more costly investments in other areas, such as cutting class-sizes by hiring more teachers.

The report demonstrates that substantial gains in school accountability and student performance can be had from investing in and publishing school performance information. To our knowledge, this is the first return-on-investment estimate of official statistics that uses a counterfactual design, i.e. using a credible control group to compare a situation where statistics are available to one where they are not. We see this analysis as a step to build the case that investment in data has significant economic and social returns. In the case of Wales, the Welsh Ministry of Education that had decided to abolish school league tables in 2001 has now re-introduce league tables under the name of “school grading” as a measure to improve the performance of Welsh schools.

The full report was presented at the 65th plenary session of the Conference of European Statisticians (CES) and is available here.