Have a look at the following picture that comes from a 2012 paper by Gotz and Hecq on forecasting growth. It tells you what the US growth rate at 3 different dates was estimated to be over time. This means that the start of the thick black line tells you what they thought in the 4th quarter of 1986 what economic growth in the 3rd quarter of 1986 was (basically 0.7% a quarter). Following the black line you get updates on what they thought growth in that particular quarter in 1986 was. In 1990 they thus thought that growth in the 3rd quarter of 1986 was just 0.2%. In 1993 they thought it was 0.6% and since about 2000 they think the growth was 1% in that same 1986 quarter. Hence depending on when you ask the question, you would either think that the US economy was close to recession or booming in the 3rd quarter of 1986.
The two other lines give you the estimated growth levels from particular other quarters, with equally astonishing changes over time.
Just think what our inability to measure growth accurately means for policy, media, and academia. For policy, it basically means you have to wait 20 years before you get to know whether you are in a recession today or not and hence whether or not you should stimulate the economy or depress it. For media it means almost every article written on current growth levels has a strong chance of being substantially wrong. For academia, it means the thousands of articles written in the 80s and 90s on growth dynamics in the 80s were using very dodgy data of which you cannot really trust the conclusions. It probably means all the ones written now on wobbles in growth the last 10 years face serious limitations.
Think of this the next time you sit in a seminar or a presentation and someone is trying to convince you that due to some miniscule blip in the data, or the coefficient on the AR(3) component of their Structural VAR (the tech equivalent of ‘some blip in the data’), they have found conclusive evidence of a coming recession, the advent of a new growth era, or the effect of admission to the WTO, to take but 3 ‘conclusions’ I have seen people make out of milking data like this.
What does this mean for the regular economist not looking to make a living out of data blips? Well, unless the economic movements are very strong and backed up by ancillary evidence (employment, deficits, trade), don’t put too much faith in the headline GDP figures.
Business schools were lauding Enron as a “new paradigm” until literally weeks before it imploded spectacularly. And not just Enron.
One wit noted that you could identify businesses that were about to fail, by whether they were the recent subject of a flattering case study at the business schools.
So much for all that concentrated expertise in forensic accounting, advanced financial analysis and business strategy! It all looks brilliantly obvious in hindsight, of course…
Jacques: How about the average prediction of the experts? I would expect the wisdom of crowds thing to work well here.
Paul: I wonder if it might not matter much if growth gets revised in the future. So long as you stick with the contemporaneous measure at all times then you are comparing like with like. The revisions are based on lagged data becoming available right? So it still might make sense to say we are in recession with 3 consecutive negative growths (as measured at the time) based on the same standard data available at that time. It appears as if growth estimates get revised upwards (at least in teh three cases displayed).
Maybe the problem is that experts suffer more from what Daniel Kahneman (Nobel Prize in Economics) calls WYSIATI (what you see is all there is)?
The data available to hand (or their preferred source of data) is assumed to fully represent the situation. This would allow confident expert opinion.
Assuming the experts have superior understanding, then they should be able to give a better prediction. If it is equal or worse than a group of lay people, then it may imply that superior understanding is being stymied by something, like for example greater faith in WYSIATI?
Soo, either the extra data wasn’t sufficient for an improved prediction, or their analysis was inadequate? Or both?
Having a bit more data to hand might still leave them vulnerable to WYSIATI.
e.g. The guys monitoring the Wivenhoe Dam at the time of the Brisbane floods might have some comments on WYSIATI. No shortage of analysis programs for dam and river levels, and more info to hand than the average householder, but still missing some vital upstream information on rainfall patterns in the catchment area.
Jacques,
yes I know of the experts-versus monkeys literature, though I often find the argumentation used to lambast experts flawed: saying no expert can predict the changes in the stock market tomorrow does not mean monkeys are just as good: it means the experts you ask cannot out-perform the other experts who have already priced the available information into the share price today. Similarly, in many areas expert’s actual role is not to forecast things but rather to say what should be done in order to prevent particular changes. Good luck asking your statistician the same question.
The thing about the above though is that the graph shows changes in the supposedly official statistics, so the uncertainty here is at an even more basic level; we wont know till 20 years from now what officially happened today!
The growth predictions seem to have the commonality of being short-term pessimistic and long-term optimistic. To borrow from Jethro Tull:
Obviously I’m not the only one who sees that these expert predictions are very much like a stopped clock; sometimes right, sometimes wrong, depends on when you check. By the way, surely if we discover evidence that expert opinions are not overly valuable, we should not be lambasting the experts themselves. The problem must reside in the process by which someone gets declared an expert.
The simplest design would be to only declare someone an expert after they have been able to deliver testable and provable predictions. Doing it that way we can guarantee a good correlation between expertise and quality judgement (albeit only in a retrospective way).
“Doing it that way we can guarantee a good correlation between expertise and quality judgement (albeit only in a retrospective way).”
Umm, so then we will have no experts in our own time?!!
JC, there are so many “industries” today – the economic modelling industry, the sex abuse industry, the Catholic Church industry, the Protestant Church industry, the foreign aid industry (thousands of 8 volume reports deeply analyzing poverty – generally caused through a lack of money), the latest uprising industry, the affirmative action industry (giving the modern empowered woman property rights ! – but funny that my darling grandmother bought land in her own name 100 years ago, without reference to her husband, and kept her bra on), the aboriginal industry (see “Why I burned my ‘Proof of Aboriginality'” on various fora), the education industry (a new new dynamic future for our children !! – what happened to the old just as dynamic future ??), the financial industry (we are here to “help” you !) and the biggest industry of all – the media industry – the list goes on.
All trying to make a living from doing something – don’t we all ? – the fact that a good proportion of “industries” output is bullshit does not enter the equation – hey ! I have a wife, 2.54 kids, a dog, cat and budgie, and a BMW to keep. Someone has to pay, and it will not come from the sweat of my brow – thank you very much !
I have seen hundreds of graphs on climate change temperatures from so many “renowned experts”. Microsoft Excel graph function has been hit fairly hard in the last 20 years.
The printer and paper industries love it all – think of the many thousands of A3 color prints handed out to conference delegates while they mull over a comparison to the 400 other climate change graphs they have on their computers.
They tell us the seas are rising – I would have thought that was obvious by the existence of submarine cities. Did they slide down the mountain to take up residence 20m submarine in an attempt to trick us ??
But the wheels must turn. We have to make a living somehow.