A video and an essay all on the same subject: measurement in the social sciences. Summary: It’s really worth doing and doing better, even though it’s really hard.
First, health statistician and visualisation expert Hans Rosling, co-founder of Gapminder and justly famous for his presentations on the past 200 years of human development. In the video below, Rosling explains how the world’s poorest countries are developing:
Second, Bill Gates, now well into his second career as a development entrepreneur, makes an eloquent argument for more innovations in measurement, and explains why it matters (or at least his PR staff do):
Given how tight budgets are around the world, governments are rightfully demanding effectiveness in the programs they pay for. To address these demands, we need better measurement tools to determine which approaches work and which do not …
… I think a lot of efforts fail because they don’t focus on the right measure or they don’t invest enough in doing it accurately …
… As 2015 approaches, the world is taking a hard look at how it is doing on the [Millenium Development] goals. Although we won’t achieve them all, we’ve made amazing progress, and the goals have become a report card for how the world is performing against major problems affecting the poor. The MDG target of reducing extreme poverty by half has been reached ahead of the deadline, as has the goal of halving the proportion of people who lack access to safe drinking water. Living conditions for more than 200 million slum dwellers have also improved – double the target …
… It is the kind of good-news story that happens one life at a time and so it often doesn’t get the same visibility as a big setback like the outbreak of a new epidemic. From time to time we should step back and celebrate the achievements that come with having the right goals – combined with political will, generous aid, and innovation in tools and their delivery,
Both Rosling and Gates single out the example of Ethiopia, which they argue has made huge improvements in effective primary health care. Their message is that we can see various development initiatives succeeding in such countries.
I’m sceptical of their Ethiopian conclusions, because measurement is one thing and determining causation is another. Development experts have a history of claiming credit for improvements which turn out to have flowed simply from rising incomes (and which reverse when incomes stagnate or turn down). In many African countries, including Ethiopia, incomes have risen in recent years because of higher prices for resource exports. Ethiopia’s biggest export is coffee, and coffee prices have soared in the past decade. They’ve had a terms of trade boom since 2004, a phenomenon Australians should understand. (Update: As commenter Patrick notes below, they’ve also stopped fighting a war with Eritrea. Yep, that could help …)
Still, for whatever reasons, we’re seeing measurable improvement.
I’ve always been in awe of Tycho Brahe, who compiled detailed measurements of the movements of the planets, and Johannes Kepler, who analysed those measurements and made the mental leap necessary (from circular to elliptical orbits) necessary to create the laws of planetary motion. Brahe’s story breaks your heart: all that painstaking work, and he did not live to see where it led. But Brahe’s work mattered. We need great data sets in order to find truth, though creating them is a tough and often unrewarding pursuit.
It’s particularly tough to do decent measurement in the social sciences. I’d argue that far too much social science starts with poor data sets and plows vainly on through poor analysis to pointless conclusions. But good measurement in the social sciences is not impossible, and we should try to do more of it.
Footnote: It turns out Rosling’s latest video is embedded in Gates’ online essay, making it more likely that they are part of a calculated effort to create a powerful narrative around development. Make your own mental adjustments.