Club Troppo

Wikipedia defines ‘grey literature’ thus:

Materials and research produced by organizations outside of the traditional commercial or academic publishing and distribution channels. Common grey literature publication types include reports (annual, research, technical, project, etc.), working papers, government documents, white papers and evaluations. Organizations that produce grey literature include government departments and agencies, civil society or non-governmental organizations, academic centres and departments, and private companies and consultants.

Some of it is excruciating or just hard to take in. But there you go, no-one mistakes an annual report or many documents for anything less than bland contentless pabulum with some information that might be useful to someone if they know what to look for. And, on the upside, grey literature is often informative and unpretentious. Reports from ABS, BOM and other technical bodies are grey literature.

But there’s an active academic literature that seeks a similar audience. And here’s one formula for the worst of it. You wade into some topical area. You make some distinctions. You deal in ideas that might be the subject of learned investigations in specialist disciplines, but you take things pretty much as given. You might then do some research to get some ‘data’. You might count up the number of a certain type of organisations who have chosen to do Y and how many have chosen to do not Y. You then give talks and consult to said organisations or to those thinking and talking about them.

It’s all kept at a very general level, there are few if any examples and if they are, they’re cursory — illustrative, rather than to interrogate anything. Then you might call for more research. Here is a blog post whipped up from such an approach.

The rise of Knowledge Brokering Organisations (KBOs) has changed how decision-makers access evidence. Whereas in the past, decision-makers might have relied on internal research services, or favoured academics providing them with the latest research, governments across the world have invested in new organisations that can synthesise existing evidence of ‘what works’.

I’ve thought about What Works Centres. Do they work themselves? I have my doubts and set out what they were here. I’d go a little further here and say that they’re a good idea on a napkin. A good beginning to a discussion that might have led to some worthwhile institutional development. But the kinds of issues I’ve raised should have been a live part of their development. For instance, it’s telling that we speak about What Works rather than who made it work or where it worked and where it didn’t. Because it’s likely that, if something difficult is being done, it will be difficult to narrow it down to a stable, replicable ‘what’ and you might like to think more about promoting the agency of those who’ve done things that work. But then that would be more disruptive than putting out lists of decontextualised tips and tricks.

Of course people continue to say things like I said, but very much at the ‘ideas’ level. The What Works Centres themselves don’t seem to be wrestling with them, trying to transform themselves into things that might work better. (Or perhaps they are and I haven’t heard — that would be unsurprising.)

Here’s some more:

KBOs have emerged in countries with different political and policy systems. In all those countries, governments claim a commitment to ‘evidence-based policy-making’ and have invested funding to develop their capacity to use evidence, be that within government itself (for politicians and civil servants) and/or for practitioners such as teachers, doctors.

Explaining their emergence, our interviewees described various drivers, such as a charismatic individual inside or outside government pushing for the need for a new KBO, and the decreasing internal capacity of government and other decision-makers to fill this evidence function. Others spoke of KBOs being created to show that policy-makers cared about an issue and were aware of the lack of good quality evidence in that area.

And on it goes. Half-ideas lie strewn around. Another is social impact bonds. A good idea of sorts, but really only the beginning of something workable and obviously useful.

And on it goes. Reportage as analysis.