Reading Tim Harford’s excellent Adapt: Why success always starts with failure an idea occurred to me. He talks of the curse of the playpump – a photogenic aid strategy that appeals to celebrities and millionaires but which doesn’t work. It’s obvious that information about what works has been a huge obstacle to philanthropically motivated efforts to help the poor. Adam Smith said as much (when he said that people could be expected to do more good for the world investing their money to advantage themselves than they would investing it for the good of the world – because they know so little about the latter and so much about the former. Bill Easterly said as much.
I also mentioned this here. Anyway it got me thinking. What if the government set up an agency that evaluated the programs of any not-for-profit that asked to have its programs evaluated. This couldn’t really offend anyone, but if one could get some nibbles from the better charities. Then those of us who want to give their money to charities that generate the best impacts could actually find out which ones to give it to. The PC reported on the not-for-profit sector and seemed more concerned about information about the sector regarding its contribution to the economy. However it did have an important and worthwhile recommendation.
The Australian Government should provide funding for the establishment of a
Centre for Community Service Effectiveness to promote ‘best practice’
approaches to evaluation, with an initial focus on the evaluation of government
funded community services. Over time, funding should also be sought from
state/territory governments, business and from within the sector. Among its roles,
the Centre should provide:
- a publicly available portal for lodging and accessing evaluations and related information provided by not-for-profit organisations and government agencies
- guidance for undertaking impact evaluations
- support for ‘meta’ analyses of evaluation results to be undertaken and made publicly available.
At first glance this seems a very satisfactory response to my concern. But I think it misses something important. One could have lots of evaluation and that could be just fine, and one hopes that many not-for profits would take note of it. But there’s a problem. As I’ve argued with information on workplace conditions, what’s missing is a standard by which people can judge one offering against another. I think that needs to develop and proposals to improve information in the sector should also address that.
Interesting question, the metrics will be a challenge, but the result worth it. I hate giving when you don’t really know what is happening to the money/stuff.
I suppose you need to have a few types of standards because care for the sick, say, is quite a different charitable enterprise compared to relief of poverty or increase in educational attainment.
Yes, agreed.
May well already be being considered.
This sort of thing already exists in a few places in the US, for example: here.
Sorry, for the second post, but here is the other big one I know about.
Actually I think you’ve missed a point here. I think the biggest value of all in this scheme would be in remedying the current “lack of standard[s] by which people can judge one offering against another”.
As someone who’s done their fair share of program evaluation, I can tell you that the toughest part of the cycle is goal specification – without which, of course, an evaluation is not possible. Goal specification goes way beyond the simple KPIs because you have to smoke out and (crucially in this context) make transparent the program’s implicit goals, as well as develop the explicit ones. Let each nonprofit be clear exactly what they’re trying to achieve before they tell us whether they achieved it.