The role of big data

With the Paris climate agreement and the UN’s sustainable development goals (SDGs), the international community has set itself some ambitious environmental targets. In many cases, however, there is no empirical basis for measuring progress towards meeting these targets. For example, there is no existing data on two thirds of the UN’s 232 SDG indicators. As a result, our best hope lies with data generated in the private sector. Forming partnerships between public bodies and private companies in order to access big data could play a decisive role in tackling global issues such as reducing CO2 emissions.

Political decision-makers are showing an increasing interest in big data analytics. Data on user activity and movement information from mobile networks, social media, sensors or geodata have enormous potential to complement and expand upon conventional statistics. A solid basis of data for making policy decisions can thus be generated in real time and in completely new fields of application.

At this year’s Climate Change Conference in Bonn, the UN presented its initiative “Data for Climate Action” (D4CA), a pioneering partnership in which private companies make their anonymised data available to selected groups of scientists to help fight climate change. Its great potential is illustrated by the various projects launched as part of the D4CA Challenge, whose participants include the ZEW Research Group “Market Design”. For instance, the winning project selected by the D4CA jury made use of movement data from mobile users to simulate the effects of various e-mobility concepts on air pollution in Mexico City. Based on detailed user data provided by the GPS navigation app Waze, researchers analysed movement patterns in order to estimate, for example, how the routes of new e-buses could be optimised to reduce greenhouse gas emissions.

Creating incentives for companies to provide their data

There are various incentives for private companies to share their data as part of projects like this. For one, exchanging data with public authorities allows companies to see a statistically representative analysis of their customer data. Secondly, collaborating with researchers helps companies to broaden their methods of analysis. Other significant incentives include regulatory conditions that require companies to share their data as well as corporate social responsibility. Many companies have also come to recognise the competitive advantages of sharing data.

On the whole, however, progress in the use of company data is still rather slow. One thing holding companies back are the legal restrictions they face regarding the confidentiality of customer data. More often than not, researchers are unable to access data due to the high costs associated with training employees for a task unrelated to the company’s core business as well as setting up the necessary IT infrastructure.

Conditions for the sustainable exchange of data still need to be created

In a joint study conducted by the Organisation for Economic Cooperation and Development (OECD) and PARIS21 – a global partnership for data and statistics – researchers documented and analysed business models for partnerships between the public and private sector which have already proven successful in practice. In many cases, data is either exchanged directly between companies and researchers, or access to data is provided via a trusted third party. However, there are also more promising IT approaches that are scalable and therefore more sustainable. One example is the “Open Algorithms” (OPAL) project, in which calculations using certified, open algorithms are carried out completely anonymously behind the company’s firewall.

The findings of the OECD/PARIS 21 study point to a number of recommendations for creating and maintaining a successful data partnership. For one, companies should appoint data stewards to act as a central point of contact for external data queries. Furthermore, governments can encourage data partnerships by announcing more competitions, documenting innovations and providing a secure IT infrastructure for the transfer of data.

The study can be downloaded at http://dx.doi.org/10.1787/18152031. This piece initially appeared in ZEW News on 23 January 2018.