Herding: Part One

A few weeks ago I attended the latest F.H. Gruen lecture at ANU by the terrific English economist Andrew Oswald.* He’s one of those economists who, in addition to being formidable in his (many) fields within the profession, is also a great communicator. Though he has a lower international profile than say John Kay who is also an op ed writing academic, he’s also a master of the art of the clearly written op ed as a visit to his website will convince you. Those who choose the F.H. Gruen lecturers – I’m not sure but I presume Bob Gregory and/or Bruce Chapman have never missed in getting people who are talented economists with something to say at the same time as not being crazy – as quite a number of Nobel Prize winners in economics are.

In any event, Oswald’s topic was herding (and here are his presentation slides). As he suggested, you’d think that economics would have a good theory of herding, or at least that it would be a prominent subject within the discipline. Alas, if you thought that, you’d be mistaken. When Oswald looked at the biology of herding, the canonical article was Hamilton, W. D. (1971). “Geometry for the Selfish Herd”. Journal of Theoretical Biology 31 (2): 295–311.

This theory models herding as a ‘rational’ strategy to avoid predators. The ‘game’ that gets selected for is for each animal to try to avoid being on the outside of the herd so that the predator gets to eat the outsider. It’s a powerful theory which fits (ie ‘predicts’ a lot of of biological data).

IIRC Oswald said that this article had acquired thirty thousand references in subsequent academic journals, many in biology of course, but also in some social science disciplines such as psychology and sociology. You know how many times it’s been cited in the economics literature? Well it’s never been cited – at least when Oswald looked it up – it probably has now.  This simple fact is as good an introduction of the theme of this post as you’ll find. As I heard Oswald say this it struck me as itself a dramatic demonstration of herding. At the same time, it’s par for the course. This kind of thing happens all the time in economics. No doubt it happens in other disciplines, but it seems especially the case in economics.

By contrast, the prominent theory of herding in economics is herding as informational learning. Thus for instance people imitate others figuring ‘they must know something I don’t’.  In a cinema, someone yells “Fire!”.  People start running for the exit. Others up the back don’t hear what the yeller yelled, but they figure they could do worse than follow the herd.  The idea is illustrated at the end of this famous scene. This idea can help explain financial bubbles.

But the biological herding idea seems so much more powerful. Because it suggests that a dominant biological mode is one in which each ‘agent’ seeks the local optimum of their own survival, and that this gives the group some holistic coherence, but that no-one is thinking of the group, and the group’s survival and welfare – the global optimum – is therefore the (arbitrary) result of these individual optimisations. The biological theory of herding spells danger for the herd in many situations.

This idea is so powerful that once one hears it one sees it popping up all over the place. It doesn’t just help explain financial bubbles, it helps explain the entire culture of finance and of many other professions. No-one ever got sacked for buying IBM and no-one ever got sacked for hugging the financial index. Indeed it’s such a powerful force that, as Keynes quipped, it’s better to fail conventionally than to succeed unconventionally.

I was listening to this podcast (mp3) in which Robin Hanson makes a comment that astonished me. He said that academics don’t really care about being right or wrong, they just want to do their thing – which is to say deploy their methods. In other words academia is a cleverness competition and you don’t have to be right to show you’re clever.  Growing up as the son of an academic who had the converse values – he was someone who tried to exercise judgement in his work, rather than to prove how clever he was – this kind of thing incenses me.  It even incenses me that virtually everyone, including Hanson, treat it as if it were just a neutral fact. It’s a bloody scandal and represents one of the ways in which the whole of academia has gone backward since it’s full ‘professionalisation’ after WWII (note how closely this coincides with the formalisation of economics.) Anyway I’ll leave that to one side.

Of course in the sciences it really does help if you’re right – you end up getting Nobel Prizes and things. But there’s no such constraint in economics. Because virtually no proposition other than obvious ones end up being clearly demonstrated by the data. One can always reach for a protective belt of quibbles to shore up your own position compared with your opponents. In the meantime you can support all kinds of cockamamie theories which float your ideological boat.

Thus, as unemployment rises from around 5 towards 10 percent in the US following the financial crisis, a supposedly serious discussion breaks out amongst US commentators (aided and abetted by highly credentialed academics) about whether this is substantially driven by a new bout of work shyness – a proposition which is absurd on its face to anyone with a bit of commonsense. But of course you can do some exciting econometric jujitsu on some of the data and get an article out of your argument. And you can certainly get it in the Wall Street Journal.

And so here we have it. Pure herd behaviour in which everyone is seeking their own (local) optimum and the community optimum (actually finding our what the hell is going on) is left largely untendered.  And in the midst of this we get people saying that this is the contest of ideas which will ultimately favour the better ideas. Now, as indicated above, in the sciences there’s a plausible way for the fittest ideas to survive – because falsification works or comes close to working. In a market falsification also works – you can either go broke or prosper or somewhere in between.  But in the social sciences it’s not so good.

In this environment, something diabolical happens. Herd behaviour. Economics is a particularly clear case of it. Even where ideology isn’t the driver, a set of aesthetic standards apply to what’s good and not so good work, and, as Robin Hanson says, this aesthetic doesn’t relate to the rightness or the usefulness of your work but it’s technical merit – how much it impresses people as something which ‘gets with the program’ – however nutty the program might be. If the burden of your research program is devoted to propositions that involve as corollaries that the Great Depression was a spontaneous holiday – well that’s no problem. We just want quality work. Work that you can publish in the American Economic Review.

Paul Krugman has bemoaned the new dark age of macroeconomics and I agree with him. Yet it goes deeper than Paul’s critique and in my next post I’ll show how this kind of herding helps explain Krugman’s own choices, and his success, early in his career.

* Dad spent about eighteen months travelling from Canberra to Adelaide to do an arbitration between the Santos Cooper Basin JV partners and AGL for gas for Sydney in the 1980s and, being an ethical fellow, regarded this as income that the university should have (for paying his salary) rather than himself. This endowment funds various activities including the FH Gruen lectures. I’ve always been pleased that the trustees of this process (including Bob Gregory and perhaps Bruce Chapman, I am not sure) have invariably chosen lecturers who have been very capable, intelligent economists without being crazy. Such people are not that easy to find in our profession.

Postscript: here’s the podcast from the relevant ANU website. Slides [PDF,872 KB] and audio podcast: [.wma, 42.5MB]

This entry was posted in Economics and public policy, Philosophy, Political theory. Bookmark the permalink.
Notify of
Newest Most Voted
Inline Feedbacks
View all comments
Richard Tsukamasa Green

This reminds me of Noah Smith’s critque of the Riksbank prize as a prize for methodology alone.

The lack of methods of herding (or more generally complexity) is fairly strange when the ideas are implicit from 1st year economics. It’s there in the basic explanation of a bank run. It’s an extension of the Prisoner’s Dillema we get taught in micro. It’s certainly there in our understanding of recessions as a guessing game – even if the models never include it. It’s absolutely there in chapter 12 of the general theory.

But, as you describe, its a game in itself where you have to demonstrate understanding of a methodology with limited use to get any influence within economics, which crowds out the ability to learn something better. So you end up with many highly intelligent people trained in using CGE models which they’ll happily denigrate in front of you.

In my third year at uni my micro lecturer went on long digressions about deadends or possible paths in economics, and one of the readings was Axelrod’s paper on norms. I’ve been excited about the potential this kind of approach for a while, although I never expected to it applied in the next twenty years.

Thing is, the basic tools applied to animal herds will work in economics. The only comparative advantage economists would have is the ability to create realistic agents for Agent Based Modelling. Except of course so much effort was put into creating representational agents that are useless leaving it to psychologists like Kahneman and Tversky to do the real work. Probably economics will face some imperialists from biology doing superior work. I hope economics reacts like Japan rather than the Qing dynasty in the face of such imperialism backed by greater technology.

That said I have found people doing ABM in Australian economics – and on real policy questions rather than just wailing in the heterodox whinging circle. There’s hope yet.

[I meant to go to this lecture too, but I forgot dammit]

Tim van Gelder
10 years ago

What you say here about economics is, I think, even more true of philosophy, which is even less constrained by the possibility of empirical refutation.

Damien Hughes
10 years ago

For those that were interested, and perhaps you may have missed the opportunity to attend, the lecture was recorded with audio available from this link:

Research School of Economics | FH Gruen Lectures

10 years ago


“Thus, as employment rises from around 5 towards 10 percent in the US following …”

Shouldn’t that be “as UNempoyment rises”?

10 years ago

Wonderful article! Thanks.

I wonder if this explains “Moore’s Law”? i.e. the only reason the technology doubles every 18 months is because the herd of technologists expect it to, and falling behind is a failure. But every time someone comes up with a “breakthrough” technology, it never achieves any significance (until about the right time predicted by the “Law”, i.e. accepted by the Herd).

Chris Johnson
Chris Johnson
10 years ago

Nicholas,as a long term admirer of WD Hamilton, I like this post a lot.

Something else that you might have noted about that paper is that, even though it appeared in a pretty arcane academic journal, it is a superb piece of writing. It is s much a great piece of literature as it is a significant contribution to theoretical biology.

It’s interesting that some of the greatest biologists of the 20th century (and Hamilton was definitely one of those) were also among the best writers of the century. Have a look at anything written by Peter Medawar or JBS Haldane to find other examples.


[…] Just as an aside, since I mentioned how antithetical academic culture is to evidence-based policy, I note that think tanks have evolved to fill that space but sadly they are mostly funded to push ideological barrows. The British have introduced ‘what works’ centres to fill the void. They’re focused on discovering and communicating information to practitioners in a way that is useful – whereas academics are engaged in an activity that (scandalously) many argue is largely indifferent to being useful or even right. […]