Is the real genius of economics our ability to see things that are impossible to objectively measure? The examples I have in mind are incentives, market failures, groups, power, and corruption. Below, I will point out just how impossible these things are to objectively measure but how easy we as participating humans can spot them. I will argue that it is our ability to ‘see’ these things that is the real cause of the success of economics, not our superior connection to hard data.
- Incentives. Economists go on and on about incentives and how changes have to be ‘incentive compatible’. Yet incentives come in many shapes in sizes, both monetary and non-monetary. In the economists’ worldview, men and women have different incentives inside the home. Ministers and their constituents have different incentives. Firms and clients have different incentives. Yet, how on earth would you actually measure an incentive? It is damned hard to do. How would you for instance measure the incentives of a minister whose official duty is to do the right thing for Australia? How would you objectively say what the incentive is for a bank manager whose mission statement is one of ‘oneness with the world’? Neither their mission statement nor their list of official duties tells you much about their actual incentives for it is not those that determine whether they will get re-elected or promoted. In order to even start to measure incentives, a statistician would thus have to ignore most of what could be objectively measured as somehow not quite true. Yet, as a human being, incentives are almost childishly easy to observe. We ‘know’ that the baker and the butcher care for their own well-being. We ‘know’ the manager wants to solidify his power and get more sales. We ‘see’ the minister who wants re-election and does his cabinet team’s bidding. We ‘know’ young men by and large want sex. Etc. These incentives are sometimes uncomfortable to note, but very simple to observe. Why are they so simple to observe? Because we can use our introspection to guess the actual wishes of other people: other people are just like us and hence a little bit of honesty about ourselves allows us to immediately see what incentives others have in particular situations. We merely need to ask ourselves what we would find important in someone else’s position. Easy for us as individuals, virtually impossible for the statistician.
- Market failures. The basic policy recipe economists sell is to point to market failures and see whether the government bureaucracy is particularly suited for overcoming them. This needs economists to spot market failures, which is not easy to do objectively: merely statistically identifying a market, which is a fairly high level abstraction, is already quite difficult. But statistically identifying the objective importance of asymmetric information, missing markets, economies of scale, market power, and externalities? Nigh impossible. Market power in particular is notoriously difficult because every market has some concentration of ownership. Yet, again, as human beings it is not that hard to spot market imperfections and even to have a reasonable inkling of their importance. In the fuel market it is easy to spot market power by just driving around and noticing the same companies keep coming up whose prices look suspiciously similar. The same holds for banks and mines. Externalities are also easy to spot: we can spot noise pollution in a heartbeat, consumption externalities (eg. envy effects) and even production externalities (poaching of ideas) with contemptuous ease. Try measuring production externalities objectively as a statistician and you will find yourself in a nightmarish situation of having to define markets, shared labour force, product spaces, technologies, etc. You can basically forget it. Why are they simple to observe? For one, it is again the case that many market failures (asymmetric information, consumption and production externalities) are immediately apparent from introspection, such as by asking ourselves the question whether we would tell a competitor about the techniques of our current organisation if well-paid to do so. Also, our ability to abstract is far better than that of the statistician in that we have far more information observable than the statistician has: we can, for instance, spot product groups by looking at what the goods are used for and we can spot worker groups by the groups of workers that go to the pub together. As human beings, we have an enormous amount of soft information to help us identify things not available to the statistician.
- Peers and groups. Behavioural economists, often unknown to themselves, couch many of their arguments in terms of groups: ‘family’, ‘reference groups’, ‘peers’, ‘insiders’, ‘outsiders’, ‘management’, ‘workers’, ‘bureaucracy’, etc. In most cases, it is statistically very hard to get a handle on these concepts. ‘People we compare ourselves with’ can for instance include people in other countries who have been dead for a while. Try measuring them! Yet again, it is not that hard as individuals to observe these groups because people will themselves display all kinds of symbols of their group membership: the clothes they wear, the people they hang out with, the virtual communities they belong to, and of course you can just ask them. As humans we have a much more complete picture of our own peers and groups than is reasonably available to a statistician and we are in a better position than the statistician to observe other people’s peers and groups. This again goes to the value of fairly soft information.
- Power. When modelling relations, economists invariably presume they can spot who has power over what. This allows us to talk about principles (local deciders) and agents (local rule takers), social planners (all powerful) and atomistic individuals (no power). Statistical measures of power are notorious, whilst at the local level it is very easy: just see who does whose bidding. The person calling the shots is the one with power. Again, this relies on soft information about who barks which commands and which commands are followed up.
- Corruption and social norms. Suppose you are thinking of investing in an African country by setting up a chain store. If you would have to rely on available statistics surrounding the amount of hassle involved, you would almost certainly be misinformed: available statistics are invariably about the hassles that existing companies have to go through, i.e. the ones that survived all the corruption in a place (and that are more often than not owned by the people in power). Yet, by simply spending a week in a place you will learn fairly precisely what kind of hassles you can expect: whether the phones will be connected, whether the local police will come and demand a share, whether the local criminals will force you to employ their family, whether your transports will actually make it. Personal observation is far preferable in this case to statistics which suffer from the unavailability of data on things that didn’t happen (i.e. the reasons most people never even tried to set up a firm) and that are tainted by the wish to be politically correct (and hence believe official rules).
Much of the usefulness of economists surrounds the concepts above: when economists advocate a policy, it is usually because we can see a market failure and how to alleviate it, taking account of the expected effect of the changes in incentives of the affected groups, given the current power structures and social norms. The basic ingredients in our advise use almost no statistical information at all, but rely mainly on personal observations of soft information and introspection. One might call this a disadvantage of economics, but I think it is our great strength: our core stories rely on concepts not just fairly easily observable to us, but also to other human observers and yet not to our competitors (the statisticians). It is that connection to the observation space of others that makes economists eminently employable as top analysts of private companies and departments, and I think it is this ‘story telling’ what has allowed ’us’ to have a separate place at the table.
Actually, it is more Scott-ish; all that importance of unquantifiable and localised mètis rather than centrally quantifiable knowledge and control.
When I read this, I find it hard to find the difference between it and what crackpots like Freud would argue, or perhaps the more normal Heidegger. More seriously, the other problem with not measuring stuff is that I don’t see how you get around confirmation bias or how you begin to understand why some systems work in unexpected ways. As it happens, it may well be some things are very hard to measure, but I don’t think I’d want to use that as the basis of investigation in an area and if I did, I wouldn’t want it to end at the point.
SNAP Conrad – I was thinking Freud too. You can learn a lot from instropection, observation and extrapolation therefrom (and you can use what you learn for fun and profit!) but it ain’t scientific.
It’s also presisely the kind of thinking that creates market failures as often as it solves them, for the reasons you give.
conrad, I don’t the point isn’t about whether to measure things or not measure things. Measuring things is an important way get accurate and detailed information. But we need to keep in mind that the measured things are only a fragment of the whole picture. While we might have an accurate and detailed view of that fragment, that detail should not distract from the significance of the parts we can’t measure. You need a combination of measurement and human intuition to get the best perspective of the whole picture.
Conrad, FDB,
Interesting. What did you think the basis of economics was? That we had statistically and mathematically proven the true nature of the world?
Paul seems to me to be saying that people skills are more important for an economist, that understanding how people work is more valuable than the statistical knowledge that they tend to use as evidence that they know what they are talking about.
He’s perhaps suggesting that economists who do have a good understanding of how other people work will provide better advice than those who are more wonkish?
“That we had statistically and mathematically proven the true nature of the world?”
No, I was hoping that you’d come up with more than the wonders of introspection given this sort of stuff generally isn’t taken too seriously even in psychology and is well known to be very limited. Story telling will only get you so far in understanding many problems (sociologists really need to learn this too. Please don’t let economists start talking about this as their “lived experience”). Given this, I was hoping you’d come up with something more sophisticated in terms of how one understands complex domains than “Paul and other economists say”. It sounds all a bit too much like magic to me — Perhaps the people that do bother to come up with models and theories of the things you are talking about really do or will end up understanding what goes on better. Then what for economics?
I can think of examples in different domains here. For example, aspects of both your points (1) and (3) have been extensively investigated in different areas, and people have various theories of attitudes, different aspects of culture and so on. There are testable models of some of this stuff that do make predictions, and the fact that introspection once might have brought the general ideas to people’s attention doesn’t mean later work hasn’t been very informative. A good example here that is related to some of the things you talk about would be the work on collectivism and individualism across cultures, which people initially introspected (and probably most people that haven’t really thought about it still believe) something like “white people individualistic, yellow people collective”. Okay, that’s nice and perhaps that sucks up some variance, but it certainly isn’t the whole story.
For a less murky example, I always like looking at the end of year predictions for things like “how much GDP growth will we get this year and what were the predictions”. What you find out is that many economists are good at up coming with magic numbers, but not good at coming up with numbers that actually agree with each other since no-one actually seems to agree on what they should be introspecting. I assume the same is true for stock prices. However, it’s clearly the case that these things really are predictable as various companies make tons of money thinking of presumably ridiculously complicated maths to predict it (just think of all of those fast trading system). So it can be done better if people just put their minds to it.
Conrad,
You are too pessimistic here about economics. The type of information I call observable is also observable by others, so we are not talking about the fantasies of the individual economist here.
More importantly, of course we have theories and data galore on all of the above. But we have them in very stylised and limited ways, such as when power is suddenly nothing else than the position in a company or wages. Or when your peer group is defined to be everyone in your statistical cell, with the same age and education. The relations between those limited operationalisations of these broad concepts is studied a lot.
Yet, I do not know of a single person who has measured any of the concepts above in the way that we as participating humans assess these things.
I don’t think the reason for this is mysterious either: when observing groups for instance we can draw on hundreds of little mental models as to what groups do, how they distinguish themselves, and what roles people have within them. These little mental models allow us to categorise and interpret behaviour of an individual very quickly such that we can tell what groups a person is in. Now, in principle one might try to wade through all the mental models, try to measure all of those too, and then use those to reassess statistical information to get at groups. Yet, the data needs are fantastical to do this and you will undoubtedly end up with garbage as people will not respond to your questions or not even know themselves how their mental models operate. So good luck measuring them. Precisely this kind of problem is why fuzzy data, such as the indices on corruption which ultimately just aggregate the opinions of various people, are so useful and used.
The more important question is thus what you do if what you want to know cannot be measured statistically because the implicit constituting mental models are too numerous and ill understood?
“These little mental models allow us to categorise and interpret behaviour of an individual very quickly such that we can tell what groups a person is in”
They also allow us to be systematically wrong thanks to plain old well known cognitive biases (e.g., representativeness), various stereotypes, poor knowledge of the domain etc. .
“The more important question is thus what you do if what you want to know cannot be measured statistically because the implicit constituting mental models are too numerous and ill understood?”
I guess this depends on how nitty gritty you want your examples and hence what scale you want to look at. Many companies seem to do a pretty good job targeting particular groups with their marketing, and I assume the most successful of them have quite detailed models of their attitudes and (many certainly invest in quite complex science to find these things out). I imagine the most successful companies also have decent models of what makes an employee good and are not simply hiring “feels good to me” people to find them the best employees. If this is correct, then many relatively broad areas are well past introspection in terms of incentives they need for marketing and hiring. Even for your rather more specific example (a minister…), I’d be surprised if there wasn’t some data on the characteristics of people in those sorts of positions that allows them to be successful.
conrad,
yes, our stories can be plain wrong, but it is not quite true that one ends up ‘just having to believe the economist’ when stories are phrased in terms of these unmeasurable but observable concepts. Mainly, the stories have a logic to them that can be tested and compared to other stories. These tests are of the form ‘if it is true that a high level of asymmetric information in this particular emotional group leads to this and that problem, should i then not also see this in other similar settings’? By looking for analogies, both current and in the past, a bit of the ad hockery is taken out of the story telling. Though of course, saying of any historical analogy that they display particular levels of these unmeasurable concepts is itself a dodgy game.
As to the behaviour of successful companies, I fear you are too optimistic about their ability to do anything different from the rest of us. Don’t get me started on the companies who employ supposed psychometric tests to see whether they should hire someone….
Paul, good to see you making a sincere effort to provide a clear outline of where the problems are. All too often I see people skipping over the problems and jumping straight to the solutions. From my perspective (an empiricist) if it really is impossible to measure, then by definition it doesn’t exist. Having said that, I’ll hurry to add that “impossible” can mean “we just don’t know how to do it yet” or it can mean “this really is impossible, no one can do it, and no one ever will.”
I suspect the former case happens a lot more often, and every empiricist must accept that tomorrow the sun may not rise as we expected, and thus new evidence comes to light, err dark, well you know what I mean. Now, of course I must quibble over your subjective point of view (but perhaps that’s the whole idea of this posting).
You left out the word “Keynesian” and I’m sure you understand there are other types of economist.
Once again, you keep presuming that Keynesian economists are the only game in town. From my subjective point of view, I’d be tempted to say that someone employed by the state would be in the business of finding reasons for the state to justify it’s own existence. That’s my subjective observation. Difficult to prove in any concrete way.
On the question of power, people probably won’t agree with me on this, but Darwin offers you a little bit of help. You are unlikely to find power relationships that spiral either downwards into mutual self destruction, nor will you find systems that spiral upwards into universal takeover. With probability of nearly 100% you will find systems that pretty much sit on the level at the limit of what is self sustaining. The obvious theory here is that if the power relationship could quickly shift one way or the other, then it would have already done that before you got round to observing it. This doesn’t solve the “impossible” observations, but it does rule out quite a lot of silly interpretations.