As I hawked my father from one oncologist to another I invented the Gruen tender and published details of it in a much more general article here (pdf) in the Australian Journal of Public Administration in 2002. I have subsequently outlined it at greater length in this paper (pdf).
I presumed when I came up with the idea that it was sufficiently simple that someone else must have come up with it. It still surprises me that I can’t find anyone else who’s suggested it but there you go, I haven’t.
I’ve since expounded it in various contexts. Like auctions for goods, Gruen tenders provide a means by which someone responsible for allocating a job to a service provider can get the service provider to produce an unbiased estimate of the prognosis for the service provision. This offers a powerful tool for administrators who must allocate jobs to service providers and, also for consumers.
Step One: The service provider is required to predict in advance the prognosis in terms of a particular quantitative outcome and/or a statistical prediction of the likelihood of their achieving certain desired benchmarks.
Step Two: These prognoses are logged into a system and the service providers’ results are also logged when they become known. The system then produces an ‘optimism factor’ indicating the extent to which the service providers past predictions have tended to be optimistic or pessimistic.
Step Three: Once the system has sufficient data to give the ‘optimism factor’ some statistical robustness, ‘raw prognoses’ provided in Step One’ can be ‘moderated’ by reference to the ‘optimism factor’ applying to the service provider. The moderated raw prognoses then become unbiased predictions of actual results.
This is best explained with an example. This is easiest where the service provider’s prognosis can be measured as a predicted result such as the price a real estate agent indicates they will achieve upon selling a person’s house.
Each real estate agent must enter their predicted price (as a single point or an the average within a range) in a system and then return to that system to log the actual result each step of this process being subject to occasional audit.
After an appropriate number of observations have been made, an ‘optimism factor’ will be generated. The agent must then provide both their raw prognosis and their moderated prognosis to clients with the data being input.
Assume there is a client seeking to engage a an agent to sell their house. They receive a prognosis from three agents as indicated in the attached table. The first agent does not offer the most attractive raw prognosis, but when it is taken into account that it typically underestimates the prices it will achieve by 5% whilst the other two agents over-promise, its moderated prognosis is the most favourable.
The service providers might provide prognoses as follows with the indicated service provider being that with the best moderated prognosis.In the case of clinical service providers the prognoses would be in the form of some probabilistic standard of errors. Thus for instance on setting a broken bone the prognosis would be in the form of a probability that certain benchmarks would be met. Thus for instance the prognosis might be that there is a 92 per cent chance of the fracture being set without any adverse event as defined in some code such events may include infection, the need to reset the bone and so on.The merits of such an approach are several-fold.
- It produces simple numbers which generate important information about quality.
- Those numbers can be used by medical administrators, and by patients to select the medical service best meeting their needs either on its own or in conjunction with information about the price service providers will charge.
- It disciplines medical service providers to make predictions. In itself this process is likely to be beneficial in helping them to understand better their own competence and the factors influencing success.
- Publishing the raw performance of service providers can not only provide a highly misleading picture of service quality but can also create invidious incentives, particularly in the case of clinical service providers, an incentive to turn away the worst risks.
- Systems have tried to deal with this issue by ‘risk rating’ cases. But this is generally according to some mechanically followed ‘table of risks’ for different cases. Gruen tenders allow service providers to go by such a table of risks should they wish, but they can also ‘forward risk rate’ according to their own knowledge and experience.
- There is never any incentive for medical service providers to turn someone away because they fear they will harm their rating. They simply make a prognosis that reflects their assessment of the relative risk of their patients.
Postscript: It occurs to me to add that Gruen tenders could be established either by competition or by regulation. Excellent service providers have something to gain from submitting themselves to the rigours of Gruen tenders – they enable them to demonstrate their superiority to others (or at least have clients/customers asking ‘why won’t their competitors do the same). I would have thought that competing HMOs in America particularly, but insurers here have a huge amount to gain from using Gruen tenders. But change takes time. Likewise government funders of medical services would have an interest in using them to allocate work.
It’s an open question whether we should regulate to require Gruen tenders in some areas. Many professions are swamped in regulation which usually achieves little more than some basic consumer protection at considerable cost. Regulating to require real estate agents to use Gruen tenders would improve information in the market for real estate services considerably.
Further, Gruen tenders will be most effective where the event they measure is some discrete event that is well understood and where possible outcomes can be well specified before the event. Further where outcomes are probabilistic as the probabilities of failure fall to zero the number of observations needed for statistical significance rise asymptotically.
Thus setting a bone, or an obstetric delivery are good candidates because the incidence of unwanted outcomes is not extremely low (quite a fair percentage of settings of broken bones need resetting) and the result can be known within a short period of days in the case of a delivery and in a few weeks in the case of the setting of a bone. For my father’s cancer, it would have been trickier, because years can pass before one knows of one’s success or otherwise and often new things will have intervened in the meantime – new drugs, new treatments, new service providers.