Is it the social scientists job (or anyone else’s) to make models of reality? (Hint: no).

Quantitative research for social scienceThere is still, I think, not enough recognition by teachers of the fact that the desire to think – which is fundamentally a moral problem – must be induced before the power is developed. Most people, whether men or women, wish above all else to be comfortable, and thought is a pre-eminently uncomfortable process.

Vera Brittain, Testament of Youth

As journalism is the first draft of history, some of my emails are the first draft of blog posts. As here where I was responding to someone with whom I’d already agreed that in economics there’s lots of emphasis on using models to understand reality but not much care given to the question of fitting them to the world.

To paraphrase in a way that’s unfair to him – but which is only intended to provide a point of departure here – he then suggested that social science was about model building and fitting the world to those models and, to shift the world towards better outcomes for social science (we were talking about economics), one needed different kinds of participants in this.

A model that comes to mind when thinking about all this is the model of how movements (e.g., Me too, civic rights, etc) happen. First we need a “0”, a person who would be willing to go against the social norm even if there were 0 people helping them. I would label Rosa Parks as a 0 in the scenario of civic right movement. But then you need a “1” to have observed the 0, and a 2 to have observed at least two people from type 0 and 1, and a 3 to have observed at least 3 people from 0, 1 or 2, and so on. Because they need to be observed to trigger additional behavior, in practice we need a bunch of 0s, a bunch of 1s, a bunch of 2s, and so on, otherwise the movement does not create its own momentum. In terms of making progress in the economic problems, I believe we need all kinds of people like that: the 0s, or the ones that have an original vision, the 1s, or those that make enough progress on those visions, and so on. As I said above, I do believe that in academic jobs in economics most people are in the 10,000s, so we need to incentivize more people to go down their numbers and create alternative paths for progress. For instance, I believe Glen Weyl has been doing important and timely work in economics. He is currently more like a 500 (where a 10 in this scenario might be Henry George).

My response – edited as I fancy going through it again was thus:

I don’t agree with much of that I’m afraid :)

I think it builds a metaphysics around one’s ordinary practice and convinces oneself that one’s own practice is paradigmatic – that reality is built in such a way that the practice not only makes sense, but contains within it the kernel of how all sense is actually made in the relation between us and the world. Here’s a nice passage on the point written in the 1920s you may have come across – I’ve edited and in doing so bowdlerised it a bit, but the original longer quote is here if you want to check it.

Now the history of mind reveals pretty clearly that the thinker who decries metaphysics will actually hold metaphysical notions. .… If he be a man engaged in any important inquiry, he must have a method, and he will be under a strong and constant temptation to make a metaphysics out of his method, that is, to suppose the universe ultimately of such a sort that his method must be appropriate and successful. .… but inasmuch as the positivist mind has failed to school itself in careful metaphysical thinking, its ventures at such points will be apt to appear pitiful, inadequate, or even fantastic.

E.A. Burtt
The Metaphysical Foundations of Modern Science.

I can think of two things I recoil from when I read what you’ve written.

1. Economic theory is very strong on resource allocation and has no strong theory of resource quality or what drives it up and down (other than reductive quantitative theories which might say that there’s a some co-relation between quality and market value). And your depiction of what we need to figure out the world has a given number of types of thinker and a division of labour. Discussing matters of thought in terms of resource allocation always reminds me of the Monty Python sketch where John Cleese explains acting in resource allocation terms – getting the right words, the right number, then getting them in the right order, inserting the pauses, getting the facial expressions right and so on. This also comes up in economics when people debate the role of formal analysis and it’s amazing – to me anyway – how often economists say that there should be a mix (which obviously there should be) and leave it at that.

As I agued in my debate with Krugman, I’m not arguing in any important sense for more or less of one or the other.1 It’s the quality of the thinking – of whatever kind that’s of the essence. And of course the way ideas from different perspectives using different techniques or skills are put together – echoing the original point on which we agreed that the fitting of models to the world is a critical matter which tends to be given short shrift.

2. The whole business of fitting models to the world has a very ‘positivist’ flavour. Reality is ‘out there’. We’re ‘in here’ (either in our own individual subjectivity, or in some discipline or department trying to understand the world as it is) and our job becomes building a ‘model’ or a ‘map’ of reality. That almost invariably begins with the now pat explanations that models aren’t reality, nor are maps of the territory – sigh. It seems to me that we need to be very careful in sketching a picture in our head of what’s ‘really’ going on.

To use a cool expression Wittgenstein used to describe philosophy, it represents the fly trying to get out of the fly bottle. It’s easy to turn one’s own intuition of the way the world is into a metaphysical map – as indeed was done when we thought that the sun revolved around the earth. That was ‘based’ if I can use that expression on the fact that it looked like the sun went round the earth. This was what I call metaphysics by default. As Wittgenstein put it, “and what would it look like if the earth went around the sun?”.

We need to be careful about ‘just so’ stories that reduce our making sense of the world to some simple reductive model. If we don’t pay close attention to what we’re doing, we end up simply reifying our own intuition. One thing we do know is that we don’t ‘build models of reality’ to navigate our daily lives. Yet how could that be if one needs to build models to understand reality? A tennis player seeking to run to strike a ball follows numerous heuristics to get the ball where they want. The knowledge they have – in their bodies and brains – is embodied.

They acquired their knowledge of the world from being in it and interacting purposively in it. In that regard their main cognitive job is not to build some God’s eye model of the world as it is, but to find ways of turning the physical and other features of the world into affordances for their purposes. It seems to me that engineering is the discipline that gets closest to this mindset (though I should make clear I have no extensive familiarity with the discipline(s) of engineering). Engineering, as I understand it, is a body of knowledge to assist in the articulation and pursuit of human purposes. If you want to build a bridge or a battleship or a chemical with certain qualities or a software program, you follow procedures that it is engineering’s task to explore, investigate and prove up to some point after which it’s up to your own critical nous to ‘apply’ in the field.

It’s only preoccupied with some God’s eye view of reality to the extent that that might become necessary or advantageous in the quest for the affordances it seeks. Thus there will be all manner of knowledge about the tensile strength of different materials, how they bear up under stress of different kinds, how they change through time and so on. It may be expedient to seek this knowledge via building God’s eye models of certain aspects of reality. It may be possible to acquire adequate knowledge from common knowledge or trial and error or some other method. The point is that one encounters the knowledge with purposes in mind.

In fact I can’t really see how one would encounter a God’s eye view of reality except as part of the lack of intellectual vigilance intimated by Burtt above which leads me to a corollary claim that seems attractive having come this far. I’d be highly suspicious of any abstract statement about what the physical or social world was like in any circumstances where the person making the assertion couldn’t explain how that conclusion arose from purposive activity – ‘where they were coming from’ as it were. If that couldn’t be done, I’d suspect their statement would be vague, possibly meaningless or poorly defined, or, to the extent that the claim that was being made is clear, wrong.

As an example I’d provide the Arrow Debreu ‘model’ of general equilibrium, though only a fool takes that to be a model of reality. Arrow and Debreu certainly didn’t accuse their model of being a model of reality. Perhaps a DSGE model is a better example. At least there, one may be able to explain the purposes with which it was built, which will then frame one’s investigations as to whether it’s of any use or not. As an aside however it will be difficult to explain why arbitrary (aesthetic) constraints were placed upon the assumptions that constituted the model.

  1. OK, I’ve allowed myself an intellectual shortcut here – which you can fill out for me as an exercise if you like. As things are so extreme it is literally true that the upshot of what I’m arguing would involve more discursive analysis and by implication relatively less formal analysis, but to focus on quantities of two different kinds of discourse is to pretty comprehensively miss my point. You can have any amount of discursive analysis and completely miss my point – as most ‘methodology’ or ‘philosophy of economics’ courses actually do.
This entry was posted in Economics and public policy, Methodology, Philosophy, Political theory. Bookmark the permalink.

5 Responses to Is it the social scientists job (or anyone else’s) to make models of reality? (Hint: no).

  1. paul frijters says:

    “we don’t ‘build models of reality’ to navigate our daily lives”

    some of us academics do and the results are mightily different from the models that those who do not take their own models seriously enough to apply to their own daily lives. The purpose of model building for most social scientists though is to escape the social world, not understand it. They hate the idea of truly meeting the gods they fantasise about.

    I guess I should say for the record to other readers that I am not the person Nick was corresponding with above :-)

  2. Nicholas Gruen says:

    Thanks Paul,

    When I wrote the headline, I knew it was a kind of exaggeration of my view, as I would readily concede that scientists – including social scientists – should build models to investigate the world.

    Then on rereading the headline, perhaps it was an accurate reflection of my views all along. Social scientists’ models are not models of reality and shouldn’t be confused with them – they’re clarified counterfactual scenarios investigated to shed light on certain things in the world. In that context they can be very useful. But they’re only a part of the toolkit. The bit of the tool kit that really gets short shrift is rigour, not in the mathematics but in the more discursive reasoning.

    I also ought to clarify that, though I expressed myself as formally in disagreement with my correspondent, that disagreement is one of tone, style, emphasis – even temperament. I’m saying that I’m coming from a different place – I think one way of looking at the issues is more useful than the style he’d adopted. I’m not really saying that he’s wrong in some technical sense.

    Then again, if my view of the world has some validity, someone might not be technically wrong for them to be spending their time relatively unproductively.

    • paul frijters says:

      sure, but what I am saying is that when one does demand from oneself that the models one builds are used in one’s own daily life, there is a huge disciplining force from that because one very quickly rejects things for oneself that one has no hesitation in recommending to others. There is an enriching effect too: one constructs with the whole of ones experience rather than a siphoned-off bit that is by construction outside of own experience and interests, as if social science is not about real humans at all. So the problem gets harder, but the hardware available to tackle it increases too.

      And yes, I agree with the ‘part of the toolkit’ bit. How we as individuals go through life also involves many different tools and tricks.

  3. desipis says:

    “It seems to me that engineering is the discipline that gets closest to this mindset”

    As an engineer, I’ve always noticed a distinct difference in a typical engineer’s approach to thinking about things compared to not just social scientists, but also philosophers as well. The necessity to include a certain amount of tangibly acquired knowledge, as distinct from socially acquired knowledge, in one’s epistemological framework helps to avoid a metaphysics that disappears up one’s own backside. It doesn’t matter how many people you convince if your work is going to fall apart in the real world. There’s an inherent falsifiability to the field.

  4. Nicholas Gruen says:

    Indeed

    There’s a stylised fact that experts are overconfident of their own knowledge. It’s certainly true of economists in general and Kahneman speaks of it as a pretty universal phenomenon in Thinking Fast and Slow. But as Tetlock points out weather forecasters don’t suffer from it. That’s because their forecasts are provided as forecasts of their own degree of knowledge. (“There’s 30 percent chance of rain” is a highly checkable claim to knowledge whereas the point forecasts of economists “growth will be 2.75% next year” isn’t a claim to a degree of knowledge – it is offered without confidence interval or claims regarding likelihood.)

    And engineers, I’m thinking, are highly trained to avoid thoughtless over-confidence. You don’t’ want your bridges or chemicals engineered by the over-confident. So as part of their disciplinary education and training engineers, learn techniques to ensure that the degree of knowledge claimed can be measured (as far as possible – for instance in measuring tolerances in real time) and delivered preferably by mechanisms of feedback and governance, but failing that, highly redundant systems that are as fail-safe as we know how to make them.

Comments are closed.