The double blind sector of the economy

Steve Randy Waldman is onto something in this post.

In the previous post, I identified government, health care, education, and finance as the “asymmetric information industry”. Arnold Kling makes an important point:

[I]nformation asymmetry is that the sellers know what they are selling much better than the buyers know what they are buying. However, I do not think this is what distinguishes those four industries. There are plenty of other situations of information asymmetry, including buying a house, buying a car, or buying a piece of electronic equipment.

I think what distinguishes these four industries is that the sellers themselves know less than what people expect. Educators do not know what, if anything, actually adds value. For all we know, test scores are determined by the backgrounds (mostly genetic) of the students, with remaining differences that are random and irreproducible.

Kling is right, of course.

This is quite topical for me for all sorts of reasons, not least that Lateral Economics has just finished a major study of Commonwealth purchasing of legal services (shortly to be posted to our outputs page – now available here – pdf)  in which we put at the centre of our analysis the fact that professional services like legal services are search goods (it’s costly to find the right one), experience goods (you’re best informed of the products quality after you’ve bought it) and credence goods (you often don’t know that much about the quality of the product even after you’ve bought it and experienced it.)

Anyway, I’d like to say quite a lot about this but don’t have the time.  But just as a marker, one of the points that Steve goes on to make is that the theatre of all these credence goods industries is a fine parody of the actual state of affairs.

More interesting is Kling’s point: “sellers themselves know less than what people expect”. That is, service providers in these industries are themselves uncertain of the value they are able to provide. Yet providers work hard to hide and downplay their uncertainty. Politicians pushing new programs offer authoritative projections of brilliant outcomes, although many initiatives fail once the lights of the bill-signing fade. Healthcare, finance, and education are built around credentials and prestige, despite questionable correlations between these tokens and value provided. Healthcare, finance, and educational institutions market themselves hard, portraying themselves as professional, competent, and above all, effective. These claims are not certain to be lies: High competence might sit within the wide confidence intervals that would surround a fair evaluation. But successful institutions do, and must, misrepresent those confidence intervals (to others, and sometimes to themselves). After all, would you go under the knife of a surgeon who told you that he thinks he might be competent? In all of these industries, there is an information asymmetry surrounding the degree of certainty that the services provided will in fact provide value. Providers have reason to be far less confident of their ability to deliver than they lead their customers to believe.

I think Steve could have thrown in ‘management’ as one of the asymmetric info industries and then you could see what a pretty pass we’re in. (The other way of accommodating the point would be to make ‘government’ a subset of ‘management’.  Then you have managers who are agents representing principles and they exist in large private firms and in government.)

One of the implications that comes from the picture that’s being painted of these double blind credence good industries is that the right ethos for these professions modesty. Alas, as Steve makes clear, the culture of these industries – I think with the exception of education – is the exact opposite. Arrogance, ‘trust me’ and hubris. John Kerin got sacked as Treasurer for telling the truth: When would the recession of the early 1990s end? “Your guess is as good as mine”.

One of the things it puts me in mind of is all the hubris of ‘strategic planning’ with all the nonsense that’s involved in it. So much of this is a kind of anti-thinking in which one pretends that strategic planning is all about everyone having a say, feeling good, ‘aligning’ with their boss and their team. It’s all part of the shtick of confidence in the elite – or pretending you know what you’re doing.

So much of what comes out of these things is self-evident nonsense – visions of the future in ten years time, plans to get there, when the best one could do in one’s ignorance would be to look around for a few things that you could be half-way confident might enable you to make a few things work a little better with some serious effort. Instead great visions are summoned up over a weekend (to be largely forgotten in the cold light of weekdays) all as a kind of cover for the profound ignorance of everyone concerned.

The other thing that it’s worth saying is that the predicament also raises the question of surfacing information about the performance of these sectors. That’s very hard to do in some areas – like government – but one can get some good information if not comprehensively, then at least about some important snippets of performance. For instance I know someone who’s seriously considering back surgery. Back surgery is notorious for bad results.  Yet it’s sometimes necessary. It’s a bloody scandal that no method exists other than word of mouth for determining questions like  “Which back surgeons are well regarded by those they’ve done back surgery on a year after the surgery”. One can track that information. But we don’t. It would also not be hard to track how gung ho surgeons are about operating – one could use Gruen tenders for instance.

But of course none of this is done. It’s true that policy makers are starting to get interested in this stuff, but they keep stuffing it up, for instance in the UK with crude targeting measures that resemble Gosplan rather than careful strategies to surface information whilst minimising the perverse incentives crude metrics can unleash. Oh and we seem to be coming up with our very own family of such disasters.

And you can use things like Gruen Tenders in ways that can actually influence the rewards to integrity. As we wrote in our report on the purchasing of legal services – discussing the merit of Gruen tenders in the purchasing of legal services:

Value for money is determined not just by price, but by quality which is itself a multi-dimensional concept embracing such things as thoroughness, relevance, accuracy, business relevance, timeliness, responsiveness, the client’s confidence that he has not been under or over-serviced and so on.

While various means have been developed to hold ‘multi-attribute auctions’ the problem is not merely the difficult technical problem of the architecture of the auction, but the more fundamental one which is that major aspects of job quality can never be perfectly reflected in the metrics into which bids in the auction would need to be converted. As a result, we expect that judgement will remain, necessarily, a major part of the process of choosing the best value for money provider. . . .

The Gruen Tender process can be valuable here also, as it can be used to develop prognostic foresight in any dimension. . . . Given the importance of determining the quality of particular people working on projects it would be possible to encode this information into the system and so generate prognostic information on the impact of such personnel on the various service quality metrics collected. One might also seek to use it to generate data on matters that are of great importance to the Commonwealth obtaining value for money such as the client’s perception that they have received the appropriate level of service, not having received too little or too junior attention to some matter and not having been charged for more hours of work than was appropriate to deal with a matter. . . .

Where the client depends so crucially on being able to trust the service provider’s judgement and integrity, a Commonwealth wide reputation for having the confidence of ones clients not to be over or under-servicing would be a highly valuable prize to win and keep.

This entry was posted in Economics and public policy. Bookmark the permalink.

10 Responses to The double blind sector of the economy

  1. Antonios says:

    I’ve experienced first hand a related phenomenon: the complicated and expensive being undertaken because it confers prestige on those undertaking the project.

    I’ve done some work in IT services firms, and it’s truly galling how often larger companies pay top dollar to use a particular software product for a “solution” even though there are better options available that are often free.

    Of course, managers in the larger companies want to run super-complicated multi-million dollar projects, not relatively simple and cost-free ones! Simple and cost-free just doesn’t garner the kudos that complicated and expensive does.

  2. conrad says:

    ”Which back surgeons are well regarded by those they’ve done back surgery on a year after the surgery”. One can track that information. But we don’t. It would also not be hard to track how gung ho surgeons are about operating – one could use Gruen tenders for instance”
    .
    This is at least somewhat in conflict with you agreeing to: “For all we know, test scores are determined by the backgrounds (mostly genetic) of the students, with remaining differences that are random and irreproducible.”
    .
    The reason for this is that if neither educators nor students (or doctors and patients etc.) know to what extent they are contributing to good outcomes, then you can measure whatever you want at the end, but it’s never going to have good validity. Now maybe it’s possible to actually get some measure for something pretty simple like a health outcome, but for other things, it’s just another confounded happiness measure or some measure with no validity (educators and governments are the masters at this sort of crap — even things where they arn’t using happiness measures like the My School website have poor validity).

  3. Marks says:

    This applies also to the PPP thread.

    About fifteen to twenty years ago public utilities were de-engineering and down-sizing. At that time, the Institution of Engineers, Australia warned that these utilities were in danger of becoming ‘uninformed buyers’ – exactly the situation described here as ‘asymmetric information’…and with the outcomes described here and in the PPP thread.

    Often, the sellers could and would sell a good product (bridge/water treatment plant, power station, freeway etc). However, if the buyer (the Government) did not know what they wanted (because they had lost the capacity to judge the various options), and the seller did not know what exactly it was that the client wanted other than some sort of generic ‘bridge’ or ‘freeway’ the outcome was not likely to be optimal – whatever the financial ‘engineering (LOL)’ involved.

    It is sort of sad that a situation that was forseen by a major professional body fifteen years or so ago is now only being recognised after the investment of huge amounts – and on several white elephants as well.

  4. Ken Parish says:

    I can’t help wondering why the Commonwealth needs to examine and create complex systems for outsourcing legal services at all. Why isn’t it instead looking at expanding AGS or making its lawyers more flexible and multi-tasking so they can do the vast majority of work in-house?

    The Commonwealth is a huge generator/consumer of legal services, and the work is mostly either:

    (a) of a nature which is so specialist and unique to government (e.g. legislative drafting; admin law judicial review, native title litigation etc) that it makes sense to retain the expertise in-house; or
    (b) mainstream commercial or litigious work which is generated in such large quantities that again it makes sense to have the expertise in-house.

    Undoubtedly there are some specialist matters that arise so infrequently for government that it DOESN’T make sense to retain the expertise in-house, but in many such cases government can access the expertise by briefing specialist independent counsel with all the logistic “grunt” still provided by AGS personnel. In a few cases the relevant expertise might only reside in a major corporate law firm, but there too government has the clout if it chooses to insist that the partner with the relevant expertise must contract his services to government in effect as independent counsel, again with all the logistic “grunt” provided by AGS (rather than by the corporate law firm’s junior lawyers and paralegals who are paid peanuts but charged out at premium rates, often for working very inefficiently due to their relative inexperience).

    Approached in this way, there is simply no need for complex systems. The pool of true specialists in a field sufficiently unusual as not to justify AGS employing its own in-house expertise will generally be so small as not to require complex systems to work out who is the best choice, and insisting that AGS does all the “dog work” in-house will preclude most of the overcharging, over-servicing etc. Of course, I wouldn’t blame you for not asking these questions. No doubt it wasn’t part of your brief. Then again, perhaps there are aspects of it that I’m simply not understanding. If so I’d be most interested to find out.

    BTW I’m not taking issue with your general points about informational assymmetry, Gruen tenders etc, just with the specific instance of government legal services.

  5. Peter T says:

    Just another observation that the world is riddled with uncertainty (not risk – which is calculable, but genuine uncertainty), plus that many outcomes are the product of more factors than we can calculate. Which are among the reasons we have complex social structures. Is this news?

  6. conrad says:

    “Why isn’t it instead looking at expanding AGS or making its lawyers more flexible and multi-tasking so they can do the vast majority of work in-house?”

    At a guess, here are two possibilities, neither of which are restricted to just legal work:

    (1) They don’t want more highly paid staff on their books — it’s something they get political grief for already, so, even if the overall cost is really higher, it looks better to give a lump sum to someone else who can then pay their employees however they feel like it.

    (2) The career structure and pay rates of the government are too rigid, and thus attracting and keeping the type of employees you need for this sector may be difficult.

  7. Nicholas Gruen says:

    Thanks Ken,

    I suspect there may be a fair bit in what you say. The basic decision to outsource was made on essentially ideological/managerialist grounds in the wake of the Logan Review which reported in the second year of the Howard Government.

    This has led to a lot of agency autonomy and a series of reports saying that a substantial number of Cth agencies are not ‘informed purchasers’ (and so, one imagines are gettting bad value for money.)

    It’s clear that the private sector are more expensive (sometimes a lot more expensive) than the public sector for lots of work, but not for all work. There are some very informed purchasers who do a lot of work in house (the other big competitor to a corporatised, but not privatised AGS) who regard the private sector as clearly better than AGS or in-house lawyers for some work (and better also means cheaper.) For instance the private sector can be good where work is commoditised (as it can be with large amounts of litigation) and where it’s commoditised the work can be quite satisfactorily kept under surveillance by the purchaser of the services.

    The other thing that our report argues is that the way firms are chosen (largely through open tenders for panels) operates as a major barrier to entry to smaller (and cheaper) firms.

    I’d guess that if you compared where we are compared with pre-Logan we may have a worse or at least more expensive system, but at least in principle, the architecture at least creates the scope to get the best of both worlds. Only high quality informed purchasing can do that, and getting it is not easy.

  8. Nicholas Gruen says:

    Conrad,

    AGS has autonomy to set its own internal pay scales commercially. For most purposes it’s effectively a private firm entirely owned by the Cth Government. One could argue that that is not appropriate – I’m not sure what I think – but that’s the way it is and has been for a decade.

  9. Nicholas Gruen says:

    Btw, the report is now available to be downloaded (pdf).

  10. Pingback: Club Troppo » The future of economic productivity inducing economic reform

Comments are closed.