Data gaps and development goals - counting the missing millions
2nd June 2016
Since the release, late last year, of the UN Secretary-General’s Expert Advisory Group report ‘A World That Counts: Mobilising The Data Revolution for Sustainable Development’ there has been a renewed flurry of activity and discussion in the development community about the potential for data to better inform development decision-making and practice.
At the end of April around 400 development professionals, government officials, technical innovators, statisticians, researchers and data scientists gathered for the Cartagena Data Festival to explore these opportunities, look at existing innovations in the collection and use of data and, also, examine the potential risks.
Eldis editor Alan Stanley was in Cartagena and caught up with ODI researcher Elizabeth Stuart to discuss what her research tells us about the opportunities and challenges that using data presents for development policy and practice.
Globally the development numbers that we think we know are actually built on quite broad assumptions, constructed models where the underlying assumptions are not necessarily correct - extrapolations, interpolations, some guess work basically. So for instance the figures for MDG 1 - the number of people living on less than $1.25 per day - could be out by up to 350 million people. On education we could be overstating the number of children attending school by over 10%. Maternal mortality could be 133,000 or it could be double – there is huge uncertainty. If we are about to agree a set of global sustainable development goals, it's questionable whether we would currently be able to measure whether they have been met or not.
The data gaps are there at the national level as well and, in fact, the issue here is much more problematic. Governments have large gaps in their understanding. For example looking at child labour in Bangladesh there are 3 household surveys that look at labour issues but none of these pick up on the child labour issue either because either they're not looking at the right age group, or they're not looking at the areas where child labour is most concentrated, or the sample size is not large enough. If you don't know what the situation looks like, and what people need, you can't adequately provide services like education and health care or targeted programmes like cash transfers. And what makes it worse is that the people missed out in the data tend to be the most marginalised the very people for whom it is most urgent to be able to provide these services effectively.
So there’s an opportunity, and this is situated within the wider “Data Revolution” - technological developments that were happening anyway, driven primarily by the private sector, which have resulted in a massive increase in technological capacity, the volume and availability of new different types of “Big Data” and the design of algorithms to analyse what that data can tell us.
The reason why we're all talking about it now is that the UN Secretary General called for a data revolution in the context of the post 2015 process and the design of the Sustainable Development Goals. This has kick-started a political process that could culminate in funding and mechanisms for improving data collection and use - a political process that can ensure this is delivered at the country level by bringing together all the different stakeholders to identify data gaps, who is going to fill them and how are they going to be filled.
I think it’s particularly important to make sure marginalised voices are being amplified and included in this process – they are absolutely not part of that debate at the moment. Part of it is about including the users of data when countries are figuring out what are the data gaps and data needs at the national level - including the users of data, including the citizens groups and citizens themselves. To do this we need to find a way to be able to communicate to people who may not traditionally have been involved. I give the example in our report of the New Zealand data futures forum which has been a nationwide consultation. From the beginning they've worked with the Maori community and said to them “what is it you need from government?” They’re putting the needs of the users front and centre and I think that's the kind of model that is needed.
This inclusive approach requires an intermediary to help translate their views and to be able to translate what is in the data. I think civil society can play that role but I think it's very useful to have a wide group of stakeholders involved and researchers need to be in that space as well - finding ways to act as intermediaries, to be data brokers if you like.
Another big part of this is how you design surveys so that groups aren't left out in practice. Much more work needs to be done on household surveys - looking at implementation issues, methodological issues and improving household survey design, as well as improving administrative data. Big data is going to fill some of the gaps we’ve talked about but you also need the ground truthing underneath and for that we need to improve household surveys to make sure they are including everyone - orphans, homeless people, migrant groups - that's a big area of new research that's needed.
The ODI report “The Data Revolution – Finding The Missing Millions” will be launched in the UK at an event in London on 21st May 2015.
Image: Orlando Cuéllar: One of the artworks exhibited at the Cartagena Data Festival as part of an exhibition organised by the Cartoon Movement.
At the end of April around 400 development professionals, government officials, technical innovators, statisticians, researchers and data scientists gathered for the Cartagena Data Festival to explore these opportunities, look at existing innovations in the collection and use of data and, also, examine the potential risks.
Eldis editor Alan Stanley was in Cartagena and caught up with ODI researcher Elizabeth Stuart to discuss what her research tells us about the opportunities and challenges that using data presents for development policy and practice.
The missing millions
Globally the development numbers that we think we know are actually built on quite broad assumptions, constructed models where the underlying assumptions are not necessarily correct - extrapolations, interpolations, some guess work basically. So for instance the figures for MDG 1 - the number of people living on less than $1.25 per day - could be out by up to 350 million people. On education we could be overstating the number of children attending school by over 10%. Maternal mortality could be 133,000 or it could be double – there is huge uncertainty. If we are about to agree a set of global sustainable development goals, it's questionable whether we would currently be able to measure whether they have been met or not. The data gaps are there at the national level as well and, in fact, the issue here is much more problematic. Governments have large gaps in their understanding. For example looking at child labour in Bangladesh there are 3 household surveys that look at labour issues but none of these pick up on the child labour issue either because either they're not looking at the right age group, or they're not looking at the areas where child labour is most concentrated, or the sample size is not large enough. If you don't know what the situation looks like, and what people need, you can't adequately provide services like education and health care or targeted programmes like cash transfers. And what makes it worse is that the people missed out in the data tend to be the most marginalised the very people for whom it is most urgent to be able to provide these services effectively.
The Data Revolution and the SDGs
So there’s an opportunity, and this is situated within the wider “Data Revolution” - technological developments that were happening anyway, driven primarily by the private sector, which have resulted in a massive increase in technological capacity, the volume and availability of new different types of “Big Data” and the design of algorithms to analyse what that data can tell us.
The reason why we're all talking about it now is that the UN Secretary General called for a data revolution in the context of the post 2015 process and the design of the Sustainable Development Goals. This has kick-started a political process that could culminate in funding and mechanisms for improving data collection and use - a political process that can ensure this is delivered at the country level by bringing together all the different stakeholders to identify data gaps, who is going to fill them and how are they going to be filled.
Making sure that the data revolution for development is inclusive and equitable
I think it’s particularly important to make sure marginalised voices are being amplified and included in this process – they are absolutely not part of that debate at the moment. Part of it is about including the users of data when countries are figuring out what are the data gaps and data needs at the national level - including the users of data, including the citizens groups and citizens themselves. To do this we need to find a way to be able to communicate to people who may not traditionally have been involved. I give the example in our report of the New Zealand data futures forum which has been a nationwide consultation. From the beginning they've worked with the Maori community and said to them “what is it you need from government?” They’re putting the needs of the users front and centre and I think that's the kind of model that is needed.
The role of research
This inclusive approach requires an intermediary to help translate their views and to be able to translate what is in the data. I think civil society can play that role but I think it's very useful to have a wide group of stakeholders involved and researchers need to be in that space as well - finding ways to act as intermediaries, to be data brokers if you like.
Another big part of this is how you design surveys so that groups aren't left out in practice. Much more work needs to be done on household surveys - looking at implementation issues, methodological issues and improving household survey design, as well as improving administrative data. Big data is going to fill some of the gaps we’ve talked about but you also need the ground truthing underneath and for that we need to improve household surveys to make sure they are including everyone - orphans, homeless people, migrant groups - that's a big area of new research that's needed.
The ODI report “The Data Revolution – Finding The Missing Millions” will be launched in the UK at an event in London on 21st May 2015.
Image: Orlando Cuéllar: One of the artworks exhibited at the Cartagena Data Festival as part of an exhibition organised by the Cartoon Movement.