Quantifying Institutions

How can we quantify culture?

This sounds ridiculous. It sounds like a quixotic intellectual conceit. But I think the idea is important to economics because of the way we are now using the concept of institutions to explain social and economic phenomena.

The fact that institutions matter has been known for yonks. The use of this knowledge has been limited however. It has proven too difficult to pin down just what we meant by the term. It was also hard to formalise when the larger part of the discipline was prizing formalism, so the use of the concept has been curtailed.

Institutions are culture. They don’t need to be any of the things we usually think of as culture. Whereas most of us think of food and philosophies and worldviews when we think of culture, things absorbed and made into identity, institutions are also culture. They are culture as they are not innate from a genetic level. They are the rules of the game in the societies we live and move between.

But can we pin these down, even quantify them? If we could it would be a great way to get more use out of something that has traditionally been mainly Applied Phebotinium.

Recently we’ve had Paul Romer promoting his Charter Cities concept for spreading good institutions, but with very little insight into what they are, save that places like Canada and Australia must have them. So my interest has been repiqued. I think the concept can be saved and I did my honours work dipping my toe into the area. I’m lucky enough to have this opportunity to shout out for feedback on a few questions (after all, who reads Honours theses). Does my critique of current approaches make sense? Is the concept of quantifying insitutions worth pursuing and what people think of my own tentative steps towards this aim.

This is a series of three posts. This one will talk about some recent work in empirical institutional literature and why I think it has serious flaws in the way it attempts the task. In the second I want to look at the alternatives that have been tried, the problems I have with them. In the last, I want to talk about my own modest efforts using language and hope to get some feedback.

In the last ten years there has been a surge in publishing on institutions in mainstream economic journals. Part of this was handwaving, excusing the failures of the Washington Consensus’ prescriptions. But much of it was a genuine attempt to give explanatory power.

There was a flowering of literature; Robert Hall and Charles Jones’ work on productivity in 1999; The prodigious output of the triumvirate Acemoglu, and Robinson kept things humming along and, by 2004, Dani Rodrik, Arvind Subramanian and France Trebbi declaring that Institutions rule. They were a solution to all our problems! Still, what are the damn things?

This literature was different from what had preceded it. It tried to be empirical. They were not just talking about the effect different institutions could have in different countries, they were using data to try and pin down this effect. Finally, something hard on an elusive concept.

Unfortunately their institutional measures were rubbish. Complete and utter rubbish.

Their favourite measure of institutions was the index. Constructed by an NGO or watchdog group, with names such as the Corruption Perception Index, the Protection from Expropriation Index, the Economic Freedom Index, these were good institutions. These were all apparently based on the thoughts of experts or surveys or some such judgement by a people. The problems involved with these are legion, but here are just a few I had:

Firstly, the methodology was vague and sometimes varied from year to year. I said “apparently” above for a reason. The Corruption Perception Index was illustratively shoddy. Survey question phrasing could change, or they might use a collection of various surveys amalgamated in a hopefully coherent mess. And what if there are different conceptions about what constitutes corruption/freedom/whatever across different economies? The findings could even be endogenous if a participant merely chose to agree with previous index findings rather than admit ignorance.

But there are deeper issues.

They are based on an unspoken assumption of convergence towards a single best model set of institutions. Good institutions are those that have drawn closest to this universal ideal. Perhaps this comes from an ingrained bias from unique equilibrium theories or maybe it’s a deeper enlightenment thing. In any case, it is a big assumption that there is a universal point to which institutional sets will converge as they improve, and it needs to be justified first. After all, we’ve seen many attempts at universalist historical laws before. They have us awaiting neo Roman decline and fall or the dawn, the Dictatorship of the Proletariat and a number of other inevitabilities.

There was a certain circularity as well. The people constructing the indices already had a good idea of what these ideal institutions looked like. You could see them in wealthy countries. That’s why they were wealthy, right? When applied to the indices you ended up with a scale of 1 to Switzerland. Eurocentric chauvinism to be sure, but it also meant you got a measure that told you that wealthy countries had institutions that looked like the institutions of wealthy countries, and that poor countries had institutions that looked like the institutions of poor countries. We can then prove by regression that A is A and B is B if A is A and B is B. A fairly meaningless correlation. Survey based indexes tended to reinforce this by way of the Halo Effect.

Finally (for now), the indices were far too focused on government and formal institutions. There seemed little or no effort to account for person to person trust in transactions, or expectations of return to effort, or any of the basic, diffused and universal institutions that actually matter to most of the economy and, indeed, were important to most of the early Institutionalists. It effectively treated government as exogenous and subject to human design rather than something built on deeper and less observable institutions. I think this was the most concerted effort to patch up the Washington Consensus. Rule of law was just another thing to be prescribed.

Even when they were testing hypotheses on the source of these institutions, the work relied on these indices. Acemoglu, Robinson and Johnson for instance posited an interesting hypothesis about the effects of disease on colonial settlers. Where there was less disease to which Europeans were susceptible, they were more willing to stick around and build better institutions. In the empirics however, they correlated the disease to indices, and then the indices to economic growth.

Even after all these flaws, I think the whole idea of quantifying this institutions caper has its merits. Is this a way we can stop the concept being only the Applied Phlebotinium, something used to mainly justify preconceptions and give us instead a little more explanatory power? After all, the virtue of Behavioural Economics isn’t that they discovered that humans aren’t rational, it is that they found ways to properly identify the relevant irrationalities and get a stronger sense of their implications. Perhaps we can do some similar things with institutions.

About Richard Tsukamasa Green

Richard Tsukamasa Green is an economist. Public employment means he can't post on policy much anymore. Also found at @RHTGreen on twitter.
This entry was posted in Economics and public policy and tagged , , , , . Bookmark the permalink.
Notify of
Newest Most Voted
Inline Feedbacks
View all comments
Don Arthur
Don Arthur
11 years ago

the indices were far too focused on government and formal institutions. There seemed little, or no effort to account for person to person trust in transactions, or expectations of return to effort, or any of the basic, diffused and universal institutions that actually matter to most of the economy and, indeed, were important to most of the early Institutionalists. It effectively treated government as exogenous and subject to human design rather than something built on deeper and less observable institutions.

I agree. It seems to me that the most powerful institutions are like a language’s grammar. When an institution like this is fully functional, most individuals behave in a predictable, orderly way without being conscious of rules or norms that regulate their behaviour.

While native speakers will have a strong sense of which kinds of utterances are right and which are wrong, they will struggle to list a set of rules. Most will stubbornly ignore the advice prescriptivist grammarians if the rules prove too hard to follow or produce utterances that sound abnormal.

As with language, many social institutions have norms that nobody can fully articulate. But at the same time, many people (particularly self appointed experts) will be willing to produce lists of rules that almost nobody actually follows.

It seems to me that there are three levels of institutionalisation:

1. Almost everyone behaves in a predictable, orderly way but nobody seems to know the norms or rules (or even imagines that there are norms or rules or that any sane, intelligent human being would behave differently).

2. Almost everyone is aware that that they are expected to behave in certain ways and not in others. But often they don’t behave the way they think they should – particularly when they think nobody is looking. Nevertheless, they feeling genuinely guilty when they fail to follow norms (and can’t imagine how any decent human being wouldn’t).

3. Almost everyone follows the norms but only because there is someone else punishing and rewarding them for their performance. Remove the incentives and the norm following vanishes.

Many economists talk as if all institutions are the third sort (which is by far the weakest). As you say, they seen to think that governments can just create the ‘rule of law’ by fiat.

But history is full of laws that failed because too many people regarded them as stupid, insane or unfair. For example 18th century attempts by Parliament to regulate the sale and consumption of gin failed to end the ‘gin epidemic’ sweeping London. As historian Jessica Warner and her co-authors argue:

The fact that consumption actually declined upon liberalization suggests that attempts to legislate morality can succeed only when they do not diverge significantly from the morality of the population whose behavior is deemed problematic.

Peter T
Peter T
11 years ago

What do you mean by “quantifying”? If you mean measuring some ways in which some sorts of institutions in particular settings contribute to the overall economic ecology, I would indeed see this as useful. If hard to do – most general institutions are promiscuous – they have more than one function, and the mix changes over time. Is a church an economic or political or social or moral or religious body – answer all of the above, in various proportions.

At a more general level, I think the factors you have identified come very strongly into play, with the addition that quantification as a process reduces the amount of information about anything that is reasonably complex.

Tony Harris
Tony Harris(@tony-harris)
11 years ago

Great post Richard! Welcome to Troppo!!

One of the things revealed by the surge of interest in institutions is the gulf between sociology (the home of the study of institutions) and economics.
One reason for this, apart from the usual complaint about narrowness and over-specialisation, lies in the history of ideas in sociology.

Despite the best efforts of Max Weber, his premature death left German sociology in the hands of “anti-economists”, the colleagues of Gustav Schmoller who fought the famous “methodenstreit” against Carl Menger, founder of the Austrian school of economics. Sociology continued in that mode, divorced from the scientific study of markets to such an extent that Ludwig von Mises gave up using the term “sociology” for the larger study of society (of which economics is a part).

The contemporary Austrians are very alert to the role of institutions as you can see from the work at the Mercatus Institute and places nearby at the George Mason University. http://mercatus.org/programs

Don Arthur
Don Arthur
11 years ago

Richard – I hadn’t thought enough about how context dependent norms are and how people are able to ‘code shift’ and they move between different environments.

You’re right — there must be institutions that don’t fit into the 1-3 taxonomy. Behaviour on buses might be one example. In some places people queue at bus stops, in other places they don’t. In some places people thank the driver when they get off, and in other places they don’t. Often it’s not obvious what’s going on. For example, there can be a ‘queue’ that exists only in people’s minds — nobody gets on the bus ahead of a person who was already at the stop when they arrived (unless they are old, disabled, have a baby etc).

If you violate norms like bus stop queuing people will get annoyed at you but may make allowances if they realise you’re not local.

But in some circumstances people will be reluctant to transact with you if you act weirdly (ie don’t follow the expected norms) or they think you might act weirdly (eg you’re a foreigner). The incentive to conform is that you can benefit from transactions. The penalty is to be excluded. So in this way it is like 3 (as you say).

So is this an exercise in economic anthropology you’ve been doing? Have you been reading people like Polanyi and Grannovetter?

Peter T
Peter T
11 years ago


Spot on. I recall having something of the same experiences.

On more narrowly economic behaviour, I found patterns of expected bargaining behavior differed really wildly across Asia in the 70s. In India, it was about setting a price about 10% above or below a signalled normal price, except for some things which were non-bargainable (you made a public fuss if anyone tried to bargain over these), but you had to signal that you knew the rules. In Afghanistan, by contrast, it seemed to be be set by estimates of who could cause more trouble, in Indonesia by social status (rich paid more). These were embedded in local culture, and I don’t imagine they are easy to change, yet would have profound effects on macro economic behaviour and outcomes.

Paul Frijters
Paul Frijters(@paul-frijters)
11 years ago


broad question, tough to answer!
Let’s answer this as if I were speaking to a recently minted honours student looking for a more in-depth PhD topic. Your best options to take this further, as I see it:

1. (empirical) Use the World Value Survey. It allows you to operationalise informal institutions by means of the average answers in regions and countries to questions on a variety of informal dimensions, such as trust, nationalism, work ethic, religiosity, etc.

2. (theory+lab). Operationalise the concept of informal institutions in terms of the modern utility-function approach that lumps altruistic, reciprocal, and expectational facets of culture into the cost-benefit calculus of utility. Then run experiments on whatever catches your fancy most accross a couple of cultures, preferably ‘in the field’. You can for instance take a van through Asia and let different villages play cooperation games. In stead of open experiments you can also try to ascertain informal institutions by having subject particpate in experiments without them knowing it, such as when you are leaving bikes unlocked or deliberately litter a public space. Whether people steal the bike or clean up the litter tells you about their informal institutions.

3. (pure theory). Figure out the bit of institutions you think matters most, formalise it in a choice framework and show how its predictions differ from non-institutional theories. There’s a whole wilderness of biologists and economists doing this already, but there’s plenty left undone, such as the role of violence in enforcing social norms and the difficulty of reconciling divergent expectations.

More generally, your critique of existing institutional measures is true and generally accepted, but it is ultimately uninteresting. We know the existing measures are circular and imperfect but for a long time they’ve been the only game in town. They have been used because they exist and are updated and easily available to honours students looking for a quick topic. What is needed is better data and, I think, particularly better theory that can form a guiding principle for how institutions matter.

Paul Frijters
Paul Frijters(@paul-frijters)
11 years ago

:-) ah, you want us to follow your example whilst you go out and do more interesting things yourself? Such altruism! I guess we’ll have to wait till post 3….


[…] the first post of this series I described recent work in empirical institutional economics and why I thought the […]


[…] something we may as well date to the introduction of Doi Moi (fn1) in the mid 80s.. Given my own interests, and continuing exposure to discussions about Adam Smith’s ideas on the marketplace and […]