What works: getting to the land of ‘how’: Complete essay

Note, this essay was published in three parts in the Mandarin and is published in consolidated form (complete with its footnotes) here.

It is impossible to remember, until one gets in the country … that they care about their experiment more than about making things work.

John Maynard Keynes on Soviet Russia, to Lady Ottoline Morrell, May 2, 1928.[1]

Part One

I.         The land of ‘what’ and the land of ‘how’

From at least the years of the ‘third way’ in the 1990s under Blair and Clinton, we’ve been hearing what governments need to do to address our various social problems. Again and again, ‘thought leaders’ tell us what we must do – move beyond one-size-fits-all services to ‘joined up government’ to ensuring that programs do things ‘with’ people rather than ‘to’ them. Plausible as they are, these ideas have barely been tested. Because if they tell us what we have to do, we’ve scarcely learned how.

At the outset there seemed to be a seductive straightforwardness to getting to how. As Bill Clinton put it “nearly every problem has been solved by someone, somewhere”. The challenge was “to find out what works and scale it up”. For me these words stand as a creation myth of the problem I want to address. They even show us original sin, because if you pay close attention there it is! Clinton suggests we learn how to solve our problems by learning a ‘what’ – what works – and then scaling it.

Yet here we are nearly three decades on and despite numerous promising innovations in small scale programs, they all share the same fate. It’s hard to think of a single example of action to address social problems that’s started small and been ‘‘scaled’ as Clinton proposed. Yet despite endless inquiries into our failure to address social problems – for instance aboriginal or multi-generational disadvantage and endless restructurings and resettings of policy in response, we’ve never got far. Peter Shergold lamented the problem in 2005.[2] Yet despite a term as the nation’s chief public servant, he conceded in 2013 that the problem remains.[3]

This is the first installment of a three part essay which itself is part of a larger project. In this article I’ll set the stage showing the subtlety and depth of the problem. For, when it’s pointed out, we all understand that there’s a difference between ‘knowing what’ the rules of tennis or chess are and ‘knowing how’ to play. My claim is that in all kinds of ways we insensibly confuse the the two and so substitute ‘knowing what’ (or ‘knowing that’) and knowing-how. In this first part of this essay, I’ll show how this happens in our universities and the professions they teach. In the second, I’ll show how this occurs in government agencies and programs. The third part concludes with a look at recent initiatives like nudge units and What Works Centres that seek to foster greater ‘knowing how’ and innovation in government. The key to their success so far has been the way they bolt on to business-as-usual and, in so doing, improve it. This essay is written to try to articulate how they might envisage a more ambitious future.

II.         From the foundations to the commanding heights

Our confusion in understanding the distinction between ‘knowing that’ and ‘knowing how’ goes all the way back to our language itself. The lives we lead are built on a vast repertoire of tacit knowhow that we don’t and indeed can’t make fully explicit. Prosaic examples include anything taught by doing like riding a bike or kicking a football. But this stretches to skills you use all the time. Like those you’re using now – to help you interpret the strange squiggles your eyes are scanning while your brain converts them insensibly into meaning. These tacit skills are fundamental to making social and economic systems work, not least the judgements people make about what constitutes good work and what does not.

This issue is not discussed much in public administration or in economics, but one tradition has developed its ideas with close attention to it. As John Gray explains, Friedrich Hayek’s case for the indispensability of markets wasn’t just that they harnessed explicit local knowledge and expertise distributed throughout the economy. It is:

the far more fundamental problem of the practical knowledge on which economic life depends being embodied in skills and habits, which change as society changes and which are rarely expressible in theoretical or technical terms.[4]

Hayek further intimated the prospect that English speakers were disadvantaged in attending to the distinction between ‘knowing that’ and ‘knowing how’. The English language provides a more limited vocabulary for appreciating the distinction than many other European languages such as German. It was largely invisible to English speaking philosophy until Gilbert Ryle published on it in 1946.[5] There’s much to be unpacked here. In the academy the status of ‘knowing that’ is far higher than ‘knowing how’. I expect this is particularly pronounced in the Anglosphere. Knowhow is often gained via learning-by-doing within an apprenticeship rather than via the discursive methods of traditional teaching. This apprenticeship tradition remains much stronger in Germanic and Northern European cultures than in the Anglosphere.[6]

The great economist, cyberneticist and organisational theorist Herbert Simon is one of the few English speaking thinkers to foreground this distinction between ‘knowing what’ and ‘knowing how’.[7] For him, the distinction marks a faultline: On one side lie the sciences; on the other the professions. Sciences are for knowing what is given and necessary in nature. By contrast, the professions, such as engineering, medicine, business, architecture, and painting are for doing. They’re concerned “not with the necessary but with the contingent, not with how things are but with how they might be”.

In short, Simon concludes, the professions are all fundamentally about design – that is, with devising “courses of action aimed at changing existing situations into preferred ones”. This highlights the irony of a process that reached its apogee in the decades after WWII. As they sought to clothe themselves in the mantle of science the university faculties teaching the professions emptied their curricula of design content:

Engineering schools gradually became schools of physics and mathematics; medical schools became schools of biological science; business schools became schools of finite mathematics. The use of adjectives like “applied” concealed, but did not change, the fact. It simply meant that in the professional schools those topics were selected from mathematics and the natural sciences for emphasis which were thought to be most nearly relevant to professional practice. It did not mean that design continued to be taught, as distinguished from analysis.

Any economics graduate will appreciate Simon’s point. As he put it, “in terms of the prevailing norms, academic respectability calls for subject matter that is intellectually tough, analytic, formalizable, and teachable”. In Part Two I show these same confusions turning up in the world of work. There, the drivers are not only the greater prestige of ‘knowing that’ over ‘knowing how’ but also its greater ‘legibility’ to systems, particularly to those at the top.

Part Two

III.         From the commanding heights to everyday routines

The big public sector careers are built in the land of ‘what’ – that is in designing and administering policy in our social services systems in our capital cities. Meanwhile the land of ‘how’ is the land of ‘street-level bureaucrats’. It’s out in the sticks. And that’s inauspicious territory from which to build a public sector career. It’s a mark of how disconnected from the how those in the centre are that, as I’ve previously documented, they imagine that something like a market in ‘what works’ either already exists or will somehow construct itself around opportunities as they present themselves. And so, somehow, our complete failure to build even the rudiments of a system that might detect and then seek to expand what works wherever we find it goes unremarked. This is true of casual discussion, of senior bureaucratic and political leaders’ prepared speeches, in academic work and in the stream of independent reports every year or so including the recent comprehensive Thodey Review of the public service.

Of course people understand the difference in principle between ‘knowing what’ and ‘knowing how’ when their attention is drawn to the distinction. But they’re generally unaware of the way ‘knowing what’ has come to substitute for knowing how throughout their world. Because organisations and systems can’t easily scrutinise their agents’ know-how, it is made legible to them via the proxy of knowing what. Employees’ credentials come to stand for their know-how. As entrepreneur and essayist Paul Graham has written, this process begins at school and most of the confusion between the two occurs insensibly:

For me, as for most students, the measurement of what I was learning completely dominated actual learning in college. … Getting a good grade in a class on x is so different from learning a lot about x that you have to choose one or the other, and you can’t blame students if they choose grades. Everyone judges them by their grades — graduate programs, employers, scholarships, even their own parents.[8]

It took Graham the best part of his professional life to understand what was really going on, and how hard it was to get the high achievers he mentored in start-ups to focus on the real ‘how’ of their job – how to make great products – rather than looking round for tests to ‘hack’. (In that context this involved looking for tips and tricks to attract the money of investors and customers).

Where know-how pervades an organisation, the damage done by these expedients will often be contained. They may be a dominant consideration in recruitment, but then the actual contribution of those recruited can loom larger in the system’s apprehension of their abilities and its provision of training and further career progression. This will often be the case where a failure of know-how is made conspicuous by failure – as in the case of engineering – and/or where professional knowledge is given independent status within organisations – as with medicine. Elsewhere, the professions are both less well trained and enjoy less independent status. This is true in teaching, social work and public administration.

Here the various makeshifts by which organisations govern know-how – such as KPIs – can take on a life of their own just as tests come to dominate what is learned in school. This process is also a major factor in consulting. The biggest firms – now known as ‘brands’ – deliver professional services from people with some of the best resumes available in law, engineering, policy development, management, communications and PR. And whether they really have the appropriate know-how for the task at hand, or however well their incentives align with the longer term tasks to which they contribute, no-one ever got sacked for hiring them.

The point made in the previous installment of this essay regarding the disconnect between knowing that and knowing how in the professions might seem somewhat rarified. After all, economists and engineers tend not to deliver services on the ground. Yet the imaginative landscape implied by their discipline and other high status professions aspiring to scientific status casts a long shadow. The tendency for ‘knowing what’ – or various immitations of it – to take precedence over ‘knowing how’ characterises the way social policy is designed and delivered.[9]

Thus for instance, the New Zealand Treasury is proud of its wellbeing framework on which it’s worked for around a decade – and far more than Australia’s desultory exercise. On the arrival of a Prime Minister Ardern who was keen to bring wellbeing into the central objectives of government policy, it enabled the rapid production of a wellbeing ‘dashboard’ for policymakers. In principle it can answer questions like “what is the level of wellbeing amongst Māori in Christchurch?”

This information may prove to be valuable or otherwise. But it’s from the land of ‘what’ and tells us surprisingly little if we want to visit the land of ‘how’. It offers no direct insights into how one might improve the wellbeing of Māori in the Christchurch area. Yet this was surely the point of the exercise. As I’ve documented elsewhere, this is par for the course in most jurisdictions. We might equally invest in a dashboard showing us the prevalence of headaches without noting that people have used the salicylic acid occurring naturally in willow bark to alleviate headaches for three and a half millennia. It is now marketed as asprin.

When embarking on some difficult new venture, or the refurbishment of an old one, the habit is not to start from a stocktake of what knowhow and capability is and is not available to the system in the field. Nor is it to identify where performance has been best to ensure that those people and their insights help steer the process of lifting the system’s capability more broadly. This would be analogous to the way evolution and markets build from outbreaks of what works best wherever they occur in the field rather than designing and imposing a plan from the top-down.

Rather than a stocktake of capabilities of the existing knowhow of the organisation, the construction of frameworks becomes alluring. Frameworks are from the land of ‘what’. It might be the case that some frameworks embody very deep and rigorous thought. But no-one scrutinising them from the outside could ever tell. For, just as certain approaches to economics are proud of their ability to ignore whether their subject is silicon chips or potato chips, so, frameworks are written in a language which is largely independent of the work being done. They tie together inputs, outputs, outcomes and an overarching ‘vision’ all strung together with pleasing abstract nouns or adjectives. Words like ‘equitable’, ‘effective’, ‘efficient’, ‘accountable’ and ‘sustainable’ often figure as if asserting them vouchsafed their presence in the program.

All this makes the construction of frameworks an ideal displacement activity for the generalists running the show. It shows them to be busy leading and organising the troops while keeping them entirely insulated from having any skin in the game. If a bridge falls down or billions go missing, that word ‘accountable’ is in the framework for a reason. The malefactors can be rounded up from the land of ‘how’. As if betraying unquiet consciences at this vacuity, such framework documents frequently arrange their pleasing words into strategic diagrams which suggest specific relationships between them. Yet it’s surprising – or perhaps unsurprising – how gratuitous these diagrams are. Or on occasion worse. For on taking them seriously, it’s remarkable how often the relationships they imply actually make no sense or worse, betray the ultimate confusion and magical thinking at the heart of the project they illustrate.

At this point, it may have occurred to you that know-how is much harder to govern, and so a much harder thing to deliver than knowing what. It’s also often much less ‘scalable’. So not only have I no magic wand to wave that will fix it all, but if I did, I would probably be perpetrating the very thing that I’m critiquing – which is fads peddled by ‘thought leaders’ in TED talks and endless elite international events repackaging third way nostrums from the land of ‘what’ when we desperately want admission to the land of ‘how’.  But in Part Three, I’ll take a look at three new kinds of institutions designed to win new know-how and scale it in government service delivery: Government innovation ‘labs’, behavioural insights units and What Works Centres. The key to their success so far has been the way they bolt on to business-as-usual and, in so doing, improve it. But as I’ll explain, if these acorns are to grow into the oak trees that might herald a mature system in which know-how is won and ‘scaled’, they’ll have to move beyond their own tendencies to substitute know-ledge for the more difficult and distant destination of know-how.

Part Three

IV.         New ways of institutionalising know-how

Given governments’ evident failure in rising to the challenges of the third way, it’s not surprising that there are various promising initiatives intended to pursue better know-how in government. They are usually thought of as part of the ‘innovation in government’ agenda. Their contribution both began, and remains incremental. This is unsurprising given that these initiatives must germinate and grow within a much larger incumbent system to which they must make themselves useful. My critique below is not offered in any spirit of disapproval – rather the opposite. Instead, it pursues a fond hope that, with sufficient care to understand our situation and patient work to improve it, the initiatives we see before us are but acorns that might grow to a forest of oak trees over the next few decades.

Government innovation labs are a small, and so far relatively tokenistic nod towards the idea that innovation in government agencies cannot be specified in systematic ‘knowing what’ that might be imported from elsewhere, but must be won in the development of new ‘knowing how’. They house activities such as prototyping, human-centred design and small scale experiment where failure is regarded as a normal and necessary foundation for working towards success. They have certainly led to some worthwhile new initiatives and improvements on old ones.[10]

Behavioural insights or ‘nudge’ units have also proliferated, receiving far more recognition and status than labs, perhaps because of their alignment with a new development in the academy – the import of psychological research into economics known as ‘behavioural economics’. One of their stocks-in-trade is A/B testing which was pioneered in early 20th century media and subsequently adopted far more widely. This is used to optimise outcomes from government communication such as tax arrears letters and SMS reminders.

Once such know-how is won, it is typically easily scaled – not requiring any particular skill from those that use it. Behavioural insights units also capture the know-how residing in academic journals – for instance about the setting of defaults to optimise outcomes – to convert it into a ‘what’ that can be readily ‘scaled’ or applied system wide. Still the focus on testing the usefulness of interventions large and small before deploying them – via A/B testing or more elaborate randomised controlled trials – has produced large benefits compared with the small scale of the investment in such units.

Both nudge units and innovation labs have been helpful in introducing new ways of working. Thus for instance, as David Halpern reports, his own behavioural insights or BI unit embraced “a healthy dose of ethnography – a ‘method acting’ approach to policy – as an essential ingredient in translating BI-inspired ideas into the real world”. Likewise the Unit sought to maximise the use of feedback, not just to manipulate behaviour in presumptively beneficial ways, but also to ensure government systems themselves were continually improving their performance.[11] One might have hoped such methods were already well entrenched in social service policy and delivery. But the sad truth is that too often they’ve been conspicuously absent.

‘What Works Centres’ are another such institution. As Mulgan and Breckon put it the idea behind them is “very simple”:

It’s to help busy people who make decisions access the best available knowledge about what works. … They do this by orchestrating the best available evidence and making it usable for policymakers, for public servants, and for the wider public. Experience has shown again and again that it’s not enough to gather evidence and put it into repositories. Unless users are closely involved in how evidence is shaped and made accessible, behaviour is unlikely to change.

The proponents of What Works Centres are aware of the problems of context. Thus in one article Mulgan stresses the need to ask not just ‘what works’, but “for who, when, where and with who.” Just as with the other kinds of institutions mentioned above, What Works centres try to involve practitioners and to tailor their output to be useful to practitioners in the field. In this they help cultivate the public goods of professional activity, most particularly by nurturing the knowledge of communities of practice. They also stress the need for building “intelligent feedback loops” into service delivery.

Still, just as there’s much to the adage that what gets measured gets managed, so what gets codified in this process tends to be the core product of tips and tricks that ‘work’ with the caveats about context being downplayed. Thus for instance Mulgan celebrates the Crime Reduction Toolkit as a “good example” of the way What Works centres translate research into “useful products, distilling complex patterns into formats that can be used by busy professionals” in this case producing “a Which?-style guide at the College of Policing that weighs up the effectiveness of things like correctional bootcamps, CCTV or electronic tagging”. The tool is a table in which one can read off hundreds of potential interventions and see how they rank in five fields which rate the quality of the evidence available on the extent of their impact on crime, how they work, where they work, how to implement them and what they cost.[12]

Figure: The Crime Reduction Toolkit

V.         Conclusion

For as long as we continue to characterise what we’re trying to achieve with a name as amorphous as an ‘innovation agenda’, it’s hard to imagine it rising above anything more than being an add-on to business-as-usual – which of course guarantees that it will ultimately be constrained by the procedures and imperatives of business-as-usual. All the new initiatives I’ve identified operate as if the most prized kind of know-how they seek to generate is ‘tips and tricks’ – which is to say know-how that’s been codified so that it is a ‘what’ that can be applied more widely and ideally ‘scaled’ into whole systems. The A/B testing mentioned above provides the paradigm case of special-purpose know-how that, once won, can be scaled as a ‘what’. Given the straightforwardness of such techniques, it’s not too much to hope that they are increasingly built in to business-as-usual wherever they can be useful.

That such measures should be a priority makes perfect sense if one wishes to demonstrate ‘early wins’ on which one can gain some prestige within the system and parley it into building something more substantial. But to succeed in properly reorienting government service delivery towards knowing how to perform the difficult tasks society gives it – including most importantly of all, to identify and straightforwardly communicate where it has no such knowhow – would require a massive transformation of the system we have today. I intend to sketch possible elements of that transformation in subsequent essays. I have already sketched one such institution – an Evaluator General. It seeks to build accountability not on the accountability of those lower in a hierarchy to those above, but on those in the system – particularly the ‘street level bureaucrats’ out in the field – holding themselves to account for their practical achievements. Yet to effectively avoid wishful thinking such self-accountability needs to run the gauntlet of independent validation by others with domain expertise.

It is by way of such mechanisms that we might aspire to rebuild the know-how of the professions themselves around similar principles, rather than build them as they have been hitherto, on foundations of independence and prestige rendered accountable to a community of practice but much less to independent verification. But before doing that, in the next essay I want to sketch some additional kinds of cognition and action that systems need to take account of. Because social knowledge and cognition and so, improved social outcomes, require more than knowing how. They require both knowing-with and feeling-with others, and some faith in the likelihood of that taking place.

* Thanks to Gene Tunny and Paul Frijters for helpful comments on earlier drafts.

[1] Skidelsky, Robert, 1994. John Maynard Keynes, vol. 2: The Economist as Savior 1920–1937 (New York: Allen Lane, p. 236.

[2] As Shergold put it:

If there were a single cultural predilection in the Australian Public Service that I could change, it would be the unspoken belief of many that contributing to the development of government policy is a higher order function – more prestigious, more influential, more exciting – than delivering results. Perhaps it is because I have spent so much of my career in line agencies, learning to deliver indigenous, employment, small business, and education programs that I react so strongly against this tendency.

Mendham, Tim, 2005. “The State of Project Management”, CIO Magazine, 1 Nov, at https://www.cio.com/article/3494549/the-state-of-project-management.html

[3] As he put it in 2013:

Too much innovation remains at the margin of public administration. Opportunities are only half‐seized; new modes of service delivery begin and end their working lives as ‘demonstration projects’ or ‘pilots’, and creative solutions become progressively undermined by risk aversion and a plethora of bureaucratic guidelines.

Shergold, P. 2013 “My Hopes for a Public Service for the Future”, Australian Journal of Public Administration; March 2013, Vol. 72, Issue 1, p.7-13 at p. 7, also available at this link https://www.apsreview.gov.au/file/174/download?token=d5v23YKW. Note that it exists elsewhere. See for instance Savoie regarding Canada’s public service:

The ambitious know full well that the road to the top is through policy, generating ideas, managing the blame game, being visible in Ottawa circles, and central agencies, not through program management.

Donald J. Savoie “What Is Government Good At? A Canadian Answer” 2015.

[4] Thus Hayek often quoted the great Alfred North Whitehead:

It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilisation advances by extending the number of operations we can perform without thinking about them.

[5] Here’s Hayek citing that article:

The almost complete loss of the original connotation of ‘can’ in English, where it can scarcely any longer be used in the infinitive form, is … an obstacle to the easy discussion of these problems …. If a German says Ich weiß, wie man Tennis spielt this does not necessarily imply that he knows how to play tennis, which a German would express by saying Ich kann Tennis spielen. In German the former phrase states the explicit knowledge of the rules of the game and may – if the speaker had made special motion studies – refer to the rules by which the skill of a player can be described, a skill which the speaker who claims to know these rules need not possess. German, in fact, has three terms for the English ‘to know’: wissen, corresponding to ‘know that’, kennen, corresponding to ‘be acquainted with’, and können, corresponding to ‘know-how’.

Hayek, F.A., 1962. “Rules, perception and intelligibility”, in Hayek, 2014. The Market And Other Orders, The collected works of F. A. Hayek, Volume XV, Bruce Caldwell (ed), pp 232-53 at p. 233.

[6] In English speaking countries university-based know-ledge is the fulcrum of professional training. Apprenticeship based learning-by-doing typically supplements this only to the extent that the alternative would produce obvious dysfunction. Outside of lower status manual trades, in medicine, law and engineering, practice-based training is now merely part of the pathway from university into the workforce via internships.

[7] For readers interested in economists, it was one of the things that drove him to dismiss profit maximisation as a good way to think about what business managers did. It might be ‘what’ they thought they did, but the moment you focused on how they did it you had to imagine them doing something more prosaic and grounded in their practice – which Simon called ‘satisficing’.

[8] He continues:

Why did founders tie themselves in knots doing the wrong things when the answer was right in front of them? Because that was what they’d been trained to do. Their education had taught them that the way to win was to hack the test. And without even telling them they were being trained to do this. The younger ones, the recent graduates, had never faced a non-artificial test. They thought this was just how the world worked: that the first thing you did, when facing any kind of challenge, was to figure out what the trick was for hacking the test. …There are certainly big chunks of the world where the way to win is to hack the test.

[9] In economics, I’ve documented how this occurs with the crude distinction between ‘positive’ and ‘normative’ economics. The considerations issues raised by Simon above would be news to almost all economists – including professors of their field. Given this, economists imagine that normative conclusions about economic policy must follow relatively straightforwardly from positive models of the economy. (Yet the distinction between positive and normative should be the same distinction Simon references above between ‘applied’ analysis and ‘design’ or the knowhow to effect “courses of action aimed at changing existing situations into preferred ones”.)

[10] For an example of what looks like useful work labs can do see “Box 4.6: The case of New Zealand drivers’ licences” in Lateral Economics, 2017, “Through thick and thin: Cultivating self-organisation in Australia’s regions”, July at p. 40.

[11] See e.g.. Halpern, David, 1915 Inside the Nudge Unit, pp. 211–212.

[12] Thus for instance the What Works Centre for Crime Reduction promulgates an online tool that “allows users to weigh up evidence on the impact, cost and implementation of different interventions and use this to help shape their crime reduction efforts”. It lists numerous interventions like “Victim Offender Mediation”. The centre says that it has high quality evidence on this intervention, and that it produces a decrease in crime, but adds that “some studies suggest an increase”. Be that as it may, it does not take much imagination to believe that such interventions could be done well or badly depending on the know-how of those responsible for them. But it’s only the know-ledge that features prominently. Moreover if know-how was the heart of success, this could remain largely invisible to this methodology. It suggests an alternative methodology – in which more authority and resources are given to the individuals and teams who are seen to have demonstrated their knowhow with superior results. And this hints at a polycentric order of professionalism.

This entry was posted in Cultural Critique, Economics and public policy, Philosophy, Political theory, Science. Bookmark the permalink.

4 Responses to What works: getting to the land of ‘how’: Complete essay

  1. amanda keogh says:

    Great piece Nicholas. Couldn’t agree more that the problem of scaling what works is a persistent challenge to realising the benefits of social innovation. Have you come across David Snowden’s thinking on fractal engagement? He suggests that the best we can do is to find the parts of these complex adaptive systems that are producing good outcomes, and identify how we can encourage more of that in whatever capacity we have. It sounds tantalisingly simple, but as you’ve well noted, the art and science of finding the ‘how’ is far from that. Keen to hear your thoughts on Snowden’s work and looking forward to the next instalment.

  2. Nicholas Gruen says:

    Thanks Amanda,

    I think Dave Snowden’s stuff is very good. But it’s ultimately a consulting product and to that extent a ‘what’ if that makes sense. This is the logic of selling services – one creates a self-contained product which is then sold to an insider by an outsider.

    In that sense, it’s like the Nudge Units – a bunch of tools, but very much driven by the institutional imperatives of the system which prefer good results, but are mostly concerned with other things – maintaining stable routines, protecting the system from ‘reputational risk’ – that kind of thing. Put senior people in a situation where they have to chose between improved integrity and outcomes for a program and protecting their Minister from a bad front page, then the front page will win every time.

    Not that I’m being moralistic about that – that’s the institutional imperative and can’t be ignored. So the challenge is to produce a system that can honour it but via routing around it by finding ways to make its impact more transparent.

    The implicit argument in the piece is that, while productising the offering is (sadly) a necessary part of introducing it to the system, that ultimately constrains it to being the latest ‘what’ and keeps the purchaser following that recipe to get to the ‘how’.

    So what would a how look like? Those in an agency would have a degree of autonomy to do what seems to be called for in the circumstances. The closest we get to this is the high status professions like medicine, law and engineering. There, if you want work done, you take it to professionals whose job it is to
    1) remain on top of a domain of professional knowledge and
    2) discharge their duties which are not only to their boss but also as fiduciaries of the public and the client.

    Those professions are far from perfect however as they’re bound up in status which often takes the place of validated knowledge. My proposal for an Evaluator General is supposed to provide the framework around which such ‘evidence-based professions’ might grow.

  3. Nicholas Gruen says:

    I also received the comments below by email from philosopher John Burnheim. They appear together with my responses which are CAPITALISED.

    1. I would tend to add to your analysis of the relations between ‘what’ and ‘how’, and insistence on what anthropologists call ‘culture’. All cultures are the result of collective choices most of which operate by the simple educators of the process of growing up in relation to other members of the community, especially of those of whom one depends on approval. In general there tends to be a split between two sorts of approval, on the one hand that of those who claim superior status is defined by social practice and power particularly power that arises from the control of the things we need to survive and flourish in the circumstances. In most cultures there is also a strong tie to ones peers; which are often in tension with what claims authority and power. THIS REMINDS ME TO SAY THAT OFTEN IN THE DEBATE ON ‘WHAT WORKS’ PEOPLE THINK THAT THEY’RE AFTER A ‘THING’ THAT WORKS. BUT IN AREAS WHERE WE’VE HAD SOME SUCCESS – YOU WOUDN’T DREAM OF CONVEYING WHAT WORKS EXCEPT VIA A PROFESSION – WHICH IS A PROFESSIONAL CULTURE AND DOMAIN KNOWLEDGE AND KNOWHOW. THIS HAS BEEN LARGELY LOST TODAY BECAUSE WE FLATTER THE PEOPLE AT THE TOP OF THE SYSTEM. ECONOMICS EVEN HAS A SLOGAN FOR IT – AS EVER IT WAS INITIALLY PENNED IN HOSTILITY BUT IT’S OFTEN TAKEN ON BOARD PROUDLY BY ECONOMISTS. THEIR BOAST IS THAT THEIR ANALYSIS OF MARKETS ENABLES THEM TO ASSESS THEM WHETHER THEY ARE MAKING SILICON CHIPS OR POTATO CHIPS. THERE’S A SENSE IN WHICH THEY’RE RIGHT OF COURSE AND MORE WAYS IN WHICH THEY’RE WRONG.

    2. In traditional cultures this tension often results in the superiors exercising strong discipline over every aspect of the up and coming generations. The great revolution that has come about by modern education results from the accessibility to the coming generations of a colossal amount of knowledge which is at least in principle supported, not by power, so much as critical evaluation. This critical evaluation is protected from dismissal by the powerful, by the evident possibility of changes in most areas of life. Even though the critical knowledge is often useless or even dangerous in the realm of ‘how’ it has transformed social relationships. THIS IS TOO ABSTRACT FOR ME – LEAVING ME GROPING FOR YOUR MEANING.

    3. My view is that, roughly speaking, the surviving aspiration for a unified view of the world is the source of most of our dangers, partly because all the critical and experimental work produces is of a specific capacity to do certain things, particular things. All the bits of knowledge that we have as a result of enquiry and criticism have consequences that depend on how they’re used. We need to be reviewing those uses constantly and that is a matter, in most cases of changing our aspirations and desires not just as individuals but as communities, not just some one community but lots of particular communities to which various overlapping relationships and concerns give rise.

    So we’re back to my insistence that in thinking about community we drop the old model of the components making a single organism governed by the brain and instead cultivate the model of an ecosystem of many very different types of organism trying to do their own thing while adapting to the activities of other organisms. YES, I COMPLETELY AGREE WITH THIS – AND AM PLEASED YOU HAVE REMINDED ME OF IT. MY EVALUATOR GENERAL IS DESIGNED SIMPLY TO ALLOW THEORETICAL REFLECTION FROM THE BOTTOM-UP. BUT I CAN SEE THAT THE NAME MAKES IT SEEM LIKE I’M AFTER THE CONSTRUCTION OF A UNITARY SYSTEM. MY QUOTING OF SIMON IN THE PIECE IS SOMETHING I’LL KEEP MORE CLOSELY IN MIND. SCIENTIFIC MODEL BUILDING IS ABOUT A SINGULAR WORLD THAT NECESSARILY IS (EVEN IF IT CAN BE A BIG MISTAKE TO DWELL TOO MUCH ON THAT AND SPEND ONE’S TIME TRYING TO UNIFY THINGS THAT ONE DOESN’T KNOW HOW TO. DESIGN IS ABOUT TAKING WHAT WE KNOW ABOUT ASPECTS OF THE WORLD AND TRYING TO USE THEM TO BUILD A BETTER (ALTERNTATIVE) WORLD.

  4. Nicholas Gruen says:

    Note to self: thinking about the way knowhow is made legible to large systems, the comments of Rodney Brooks I quoted on my post on Descartes Ghost comments come to mind. In his paper “Intelligence without representation”. As he puts it.

    In this paper I … argue for a different approach to creating artificial intelligence:

    We must incrementally build up the capabilities of intelligent systems, having complete systems at each step of the way and thus automatically ensure that the pieces and their interfaces are valid.
    At each step we should build complete intelligent systems that we let loose in the real world with real sensing and real action. Anything less provides a candidate with which we can delude ourselves.

    We have been following this approach and have built a series of autonomous mobile robots. We have reached an unexpected conclusion (C) and have a rather radical hypothesis (H).

    (C) When we examine very simple level intelligence we find that explicit representations and models of the world simply get in the way. It turns out to be better to use the world as its own model.

    (H) Representation is the wrong unit of abstraction in building the bulkiest parts of intelligent systems.

    Representation has been the central issue in artificial intelligence work over the last 15 years only because it has provided an interface between otherwise isolated modules and conference papers.

    Likewise there are any number of activities that can seem useful relating to the ‘what’ or know-ledge. The classic one is rigging up a framework which makes everyone feel that things are ordered in their minds and in the organisation’s system before they’ve actually encountered the difficult material they’ll be working with and the choices they’ll have to be making. Likewise, Graham’s brilliant (and in retrospect obvious) way of highlighting the way ‘hacking the test’ leaches into pretty much everything, is a similar kind of story.

    The conference papers that Brooks mentions are the means by which the academic roboticists are trying to ‘hack the test’ – which is to say do well in their careers. But their work also feels serious. It also feels useful on the assumption that getting things clear will help. But of course, the only things that will help to get clear are things that it matters to get clear. Now one can never know that in advance, but if they’re on the way to building a prototype and making it work, chances are they’re things that it will matter – will be useful – to get clear.

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail. You can also subscribe without commenting.