John Quiggin’s objection to self-reported happiness data

In an earlier post I listed the main contentions of the happiness research program, and invited readers to contribute to the critique. The response was gratifying, and the student found them very helpful — in the comments here, in further posts by John Quiggin and Don Arthur, and in the comments on those posts (John even took the debate to Crooked Timber, which generated more discussion still). I didn’t follow the comments up because it was all too daunting, but John and Don have both returned to the topic, so they’ve spurred me to action.

Most of the respondents elaborated on the standard criticisms I listed rather than added to the list, so there weren’t many surprises, though I didn’t expect quite such hostility, from almost every point on the ideological spectrum. Ken Parish detected a plot to confiscate his car and computer and force him to attend line dancing classes, and most of the other comments also focused on the paternalist overtones of the project. I can understand this as a basis for healthy scepticism, but I was taken aback by the enthusiasm with which commenters clambered over each other (especially on Don’s thread) to dismiss the whole idea of enquiring into happiness and its causes as somehow shallow and nonsensical.

The one objection I hadn’t heard before was John Quiggin’s. Although I had put ‘Happiness can’t be measured’ first on the list of criticisms, and had also linked to an earlier post of his on the measurement issue, this particular challenge, as he reformulates it here, turned out to be one I hadn’t encountered:

Ive long argued that these questions cant really tell us anything, and an example given by Don Arthur gives me the chance to put it better than Ive done before, I hope.

Suppose you wanted to establish whether childrens height increased with age, but you couldnt measure height directly.

One way to respond to this problem would be to interview groups of children in different classes at school, and asked them the question Don suggests On a scale of 1 to 10, how tall are you?. My guess is that the data would look pretty much like reported data on the relationship between happiness and income.

That is, within the groups, youd find that kids who were old relative to their classmates tended to be report higher numbers than those who were young relative to their classmates (for the obvious reason that, on average, the older ones would in fact be taller than their classmates).

But, for all groups, I suspect youd find that the median response was something like 7. Even though average age is higher for higher classes, average reported height would not change (or not change much).

So youd reach the conclusion that height was a subjective construct depending on relative, rather than absolute, age. If you wanted, you could establish some sort of metaphorical link between being old relative to your classmates and being looked up to.

But in reality, height does increase with (absolute) age and the problem is with the scaling of the question. A question of this kind can only give relative answers.

I’m not convinced by this, and I don’t think John has thought it through.

Obviously subjective height data is inferior to objective height data, but that’s not the fundamental reason why the height-age hypothesis is rejected in John’s example. It’s actually because, in reporting their height, the children just happen to be adjusting for the very variable whose relationship with height we are investigating. Each child is using his class average as a yardstick, and of course children in a given class are all around the same age. One might just as plausibly suppose that each child will interpret the question to mean ‘are you tall for your age?’, because of course that’s what we usually mean when we comment that this or that child is tall. If they interpret the question that way, obviously no corellation between height and age will show up at all, not even in the class-level statistics. The only reason why there is some corellation at the class level in John’s example is that there is typically about eighteen months’ age variance in a given classroom.

But is there any danger that people will make an adjustment for income when you ask them how happy they are? For John’s analogy to work, it must be the case that if I’m asked to rate my happiness level on a ten-point scale, I say to myself: ‘Well, the question must be asking if I’m happy relative to other people on a similar income. Now, it’s true that my family is living on the street, that my children haven’t had a bite to eat for three days, and that I’m miserable compared to rich people, but compared to other destitute people I’m actually pretty happy, so I’ll report a score of 9.’

To avoid misunderstanding, this is not to say that people don’t compare themselves with a relevant reference group. Indeed, one of the findings of the happiness reseachers is that income and wealth are valued in large part because they confer status within one’s own peer group or socio-economic category (Don’s most recent post deals with this aspect). So, the person who lives in a smart three bedroom flat in Rooty Hill is happier than the person who has a modest five bedroom house in Dover Heights.

But to say ‘I’m happy because my income is high compared to others in my soci-economic category’ is not the same thing as saying ‘I’m happy compared to other people in my socioeconomic category.’ In the first case, you may indeed be happy relative to the population average. In the second, you are unhappy realtive to the society as a whole, but you are comparing yourself to a group of mostly unhappy people and finding that you are actually happier than they are.

If the ‘subjective’ nature of happiness data is a problem for happiness researchers, I think this applies more to its time-series claims than to its cross-sectional claims. The key finding, as John himself said (quoting me), is that, beyond a certain point, higher per capita income in a given country doesn’t seem to produce greater happiness. The standard explanation for this is that favourable or adverse changes raise or lower our happiness level only temporally: that is, we eventually become ‘habituated’ to an improvement or deterioration in our circumstances.

Habituation is normally taken to mean that, after some good or bad thing happens to a person, he reverts to the old level of happiness — the same point on the same scale. But what if it’s the whole scale that’s moving up or down — that what he previously called moderately happy is now very happy, and so on? This could happen unconsciously, in which case we might just as well speak of ‘resignation’ as of habituation; or it might be unconscious, that is, we might simply forget that what we now call happy is what we once regarded as only moderately happy. The unconscious case is a real problem for researchers because, even when the subject promises to be objective as possible, his reported increase or decrease in happiness may be illusory. A whole generation of respondents mistakenly report that they are no are happier than they were twenty years ago, and the reesracher walks away think he has evidence that happiness doesn’t rise permanently with income.

On reason why economists are wary in general about data on self-reported happiness is that our their teachers drummed into us that utility is incommensurable – that we’re not allowed to aggregate utility across individuals or make ‘interpersonal comparisons’.

But this is just an excuse. After all, we make interpersonal comparisons all the time, whether we’re calculating welfare loss triangles, conducting cost-benefit appraisals of public projects, or measuring GDP or aggregate consumption to gauge a nation’s performance. The real problem is that we are so accustomed to measuring welfare in money that substituting a less quantifiable variable like happiness seems like a retrograde step. How much money people spend, or would be Willing to spend, is not a bad guide if we know for sure that the spending translates into utility. But if we suspect that such spending is rash and myopic, or that it has external costs that can’t easily be quantified in money, then we must try harder to measure utility itself.

This is where the philosophical and political objections come in, and the accusations of paternalism. How dare you assume that people don’t know what’s best for them? That’s fine, but you can argue that measures of happiness are unnecessary or that they betray a authoritarian mentality, without necessarily arguing that they are meaningless.

Yes, self-reported data on psychological states are subjective and imprecise, but the same applies to data on pain. As far as I know, nobody is demanding that medical researchers quit using self-reported pain statistics on those grounds.