Club Troppo

Here’s a paragraph I wrote about fifteen years ago.

The culture of economic expertise places inadequate weight on integrating insights from multiple perspectives, that it frequently places an unreasonably high ‘burden of proof’ on heterodox views, and that it has a penchant for spelling out the normative implications of its analysis by way of ‘peremptory rules’ which narrow the scope for dialogue.

This is just a sketch of a post, because I don’t have time for more, but for some time I’ve wanted to put down a marker about one of the ways in which results from Kaggle are instructive. The winners of Kaggle competitions often win not by trying to build the One True Model of the phenomena they’re trying to model, but rather by building a large number of models all of which have some explanatory power and which are independent of each other and then aggregating their insights. Random forests approaches often perform very well. One might call it a ‘wisdom of algorithmic crowds’ approach.

In any event, this is the antithesis of much economic modelling which involves reaching for the One True Model and then parameterising it.

I was reminded of this by David Colander’s latest piece urging humility on economists (pdf). His proposal? That instead of thinking of themselves as analogous to dentists (which Keynes suggested) economists should think of themselves as engineers:

An engineer’s approach to modeling such a complex system as the macro economy would likely focus much more on statistical models and methods of pulling patterns out of the data. It would explore a wide variety of formal models to gain analytic insight, and then would integrate the many variety of models with the statistical models to interpret the patterns. That would involve a fundamentally different way of doing macro and of thinking about macro problems. Similarly, with micro. Economists focus much of their applied micro policy discussion on Pareto optimal solutions even though we know that all actual policies will violate Pareto optimality. We can contort our micro policy models designed to provide Pareto optimal solutions to provide insight into non-Pareto optimal solutions, but, generally, that contortion comes at a cost. It means that we spend less time discussing other models that betting fit not Pareto optimal solutions, but “reasonable person solutions” that more closely reflect society’s value judgments. An engineering applied microeconomics would likely have an entire branch devoted to measuring society’s value judgments and integrating those judgments into applied policy instruments. Our scientific applied micro leaves the topic almost totally undiscussed.

Quite.

Other required reading on the point is this paper entitled “Identification Problems in the Social Sciences and Life”.

I made the latter point in a discussion of discursive collapse a while back.

In an earlier post, and one of a series by me and subsequently Ken as well, I suggested that an important part of any professional education should be a kind of counter-narrative in which those who learn a profession are also made familiar with that profession’s cognitive biases, with a view to lessening them in practice.

Nice to see this kind of thing is beginning to be taken seriously in management books. Actually it might have been taken seriously before now, I wouldn’t know because I don’t read management books. But I occasionally browse them and it hasn’t seemed to prominent in my browsing. In any event Daniel Kahneman et al have a long article in the HBR on the behavioural economics of business decision making.

And a useful check-list for when an organisation is making big decisions. Viz: (Continued)

Keynes famously said that the hardest part of coming up with the General Theory was not coming up with the new ideas so much as escaping from the old ones. I’ve just run into a great article on the implications of happiness research for making policy (and yes there are implications) at least according to John F. Helliwell. Read the whole article if you wish – it’s not very long, not technical and very interesting and it’s here (pdf). Anyway, here’s the first half of it’s conclusion which struck me, perhaps because I’ve been thinking about the Keynes quote a lot recently not in the intellectual but in the organisational context. Why is it that ideas that are clearly good ones, which have negligible risks and are clearly worth giving a go, are not given a go? Laziness? Inertia? The fact that people are busy and fully engaged in existing routines? Undoubtedly. But at least the chess example brought it home to me in a concrete way (and of course as a chess player it describes perfectly why I’m not very good – well one reason!)

There seems to be sufficient evidence already in hand to encourage policy field trials and policy experiments implementing what is already known from subjective well-being research. If this is so, why has so little changed? The relatively slow progress from accumulating evidence to even experimental changes in policies and procedures is partly due to the human predilection, evident in medicine and all sciences (Nickerson 1998), to adhere to old ways despite the arrival of contrary evidence. Even chess masters unconsciously stop looking effectively for better strategies once they have something plausible in hand, enough so to drag the quality of their play down by three standard deviations in the skill distribution (Bilalić et al 2008, 654). [this article is here btw, though there are better formatted, possibly later versions behind a paywall.]

Caution has its own rewards, however, as the inherent conservatism of science can at least reduce the likelihood of running off in all directions. But if taking subjective well-being more seriously has the potential for increasing the quality of lives while reducing pressures on available resources, should there not at least be a stronger commitment to broaden the range of policy alternatives to include those with a strong chance of improving subjective well-being?

Postscript: Here’s the problem in the experiment.

(a) 2-Solution problem; (b) 1-Solution problem (these positions are based on an idea of Pertti Saariluoma). White to move in both problems. In (a) the familiar smothered mate solution is possible: 1. Qe6+ Kh8 2. Nf7+ Kg8 3. Nh6++ Kh8 4. Qg8+ Rxg8 5. Nf7#. The shorter optimal solution is: 1. Qe6+ Kh8 2. Qh6 Rd7 3. Qxh7#, or 2.. . . Kg8 3. Qxg7#. In (b) the smothered mate is no longer possible because Black’s bishop now covers f7. The optimal solution is still possible. 1. Qe6+ Kh8 (If 1..... Kf8, 2 Nxh7#) 2. Qh6 Rd7 3. Qxh7#, or 2.. . . Kg8 3. Qxg7#, or 2 . . . Bg6 3. Qxg7#. The crucial squares for the familiar solution are marked by rectangles (f7, g8, & g5) and the optimal solution by circles (b2, h6, h7, & g7) in (a).

The experimental procedure then involved comparing the ease with which people presented with the first problem saw the most efficient solution – given that they would almost certainly have seen the less efficient (smothered mate) solution first.

And from the article:

Across a range of skill levels, the presence of a familiar solution that first came to mind [the smothered mate in the first problem] reduced the problem solving performance of the experts to that of players about three standard deviations lower in skill. Similar results were obtained using different problems and the more naturalistic instruction to find the best move (Bilalic´ et al., 2008). Three standard deviations is a gulf in skill level. The chance of a player being beaten by one 600 Elo points lower is close to zero. And yet the Einstellung effect temporarily reduced the problem solving ability of the experts to that of the less skilled players. It is a very powerful effect. . . .

We show, by measuring players’ eye movements, that the mechanism by which the first idea prevents a better idea coming to mind can be demonstrated. Crucially, we find that players believed that they were actively searching for better solutions when in fact they continued to look at aspects of the problem related to the first idea they consid- ered. This is why the Einstellung effect is pernicious – peo- ple do not realize that it is influencing their thoughts.

It was (I read somewhere) to avoid this (Einstellung) effect that Freud refused to read Nietzsche because he thought his ideas would pre-empt and so displace his own emerging ideas.

More generally while there are sub-disciplines variously entitled the ‘methodology’, the ‘sociology’, the ‘philosophy’, the ‘history’ and the ‘theory’ of one’s discipline, one of the most interesting and useful of these sub-disciplines might be called the ‘psycho-pathology’ of one’s discipline.  Such a sub-discipline, which I think there should be courses in which could be very valuable for the health and usefulness of the discipline would seek to generate greater self-awareness amongst practitioners of the psycho-pathology of their own discipline. Certainly there’s a dark psycho carnival going on in economics. Wouldn’t it be worth making some awareness of the psychological foibles of their discipline a routine part of the intellectual apparatus of economists?  Ditto, (mutatis mutandis) for lawyers, doctors and other professional practitioners.  Shouldn’t all people seeking to become ‘experts’ of one kind or another be familiar with all the foibles of expertise, as documented for instance by Philip Tetlock?