Guidance to governance indicators
The tyranny of international index rankings
Are international index rankings reliable?
Authors:
B. Hoyland; K. Moene; F. Willamsen
Publisher:
Research Programme on Democracy (Democracy as Idea and Practice), 2009
This paper considers International index rankings and determines that they are perhaps too persuasive. The authors stress that these rankings emphasise country differences where similarity is the dominant feature. The document adds that the rankings can be misleading, not because of wrong indicators, but because the estimation of the scores ignores inherent uncertainty. Re-estimated with a method that captures this uncertainty, the authors assert that it is clear that the practice of ranking every adjacent country is a rather courageous activity.
Details of issues are given, including:
- The problem: measuring unobserved performance
- The approach: measuring latent variables
- Data and definitions: brief description of data
- Results: uncertain rankings.
To demonstrate the magnitudes of the inherent uncertainty in the rankings he singled out three indexes for scrutiny:
- The United Nations' Human Development Index: it ranks countries according to health, knowledge, and material resources
- The Freedom House's Freedom in the World: it creates an annual index of the political rights and civil liberties enjoyed in different countries
- The World Bank's Doing Business Index: it ranks countries on the basis of how business friendly the regulatory environment is.
Conclusions
- Human Development Index, Freedom House, and the Doing Business index all provide country rankings that are highly uncertain
- Country rankings are used as guides for economic, social, and legal reforms by governments, NGOs and international organisations but their popularity is not without impact
- The indexes may be poor guides for policies as each link between indicators and scores is noisy and uncertain, but presented as certain.



