As a summer exercise I’ve been thinking about places where more lotteries might be a good idea. By lotteries, I mean a decision maker selecting an option randomly, albeit perhaps from a selected pool, rather than using flawed criteria. After all, in a complex and uncertain world, even the best available criteria will be flawed and will insert some arbitrariness into the system. At best this just results random bad choices. At worst it will promote adverse behavior and selection and resentment amongst participants. Why not just make arbitrariness transparent and more purely random?
The Peter Principle is one (humourous but insightful) example, often put as “people are promoted to their level of incompetence”. Using the available criteria of people who are competent in their current jobs, selection of those to be promoted often fails to select the bets candidates for duties higher in the hierarchy. The bad selections oly become apparent when someone fails to perform in their new role. Plunchino and Rapisarda, after crunching the concept through game theory, found that random promotion may be the most efficient way of selecting those to promote.
Hierarchies, both in government and in the corporate sector, are rife with terrible selection procedures. The Peter Principle merely choses people on a criteria that doesn’t reflect what is really wanted. Many selection techniques actively choose bad traits. I’ve talked about the way narcissism may be partly responsible for a gender bias in executive positions. Those who are over confident are much more likely to put themselves forward, and to be able to sell themselves to decision makers. Unfortunately the most confident people are over confident. They’ll tend to be risk seeking, over estimate their own judgement and disregard advice, have an inflated sense of their own value (and subsequent entitlements) as well as an ability to deflect blame and subsequently fail to learn from mistakes. In short, you might be choosing the kind of person that takes a company down a risky road to the cost of stockholders, employees and eventually taxpayers – and then considers that they deserve a large bonus. Perhaps random promotion would have resulted in a broader array of neurological types, and more sober and responsible decision making from executives – or at least the kind that aren’t predisposed to exploit the principal-agent problem whenever it arises. It certainly would address the critique of quotas that Nicholas gives here, where a quota for women would merely promote narcissistic women.
Another problem with flawed criteria is that participants will quickly start altering their behavior to target it. If you start rewarding teachers based on test results, you risk them teaching to the test or “juking the stats”. When it comes to selections, this behavior may be very costly and, like persuasive advertising, is an aggregate economic loss. It’s only virtue is affecting the selection, often to the detriment of the quality of the decision, and at the expense of other behavior that might benefit the broader context, whether the organisation or society. In the hierarchical context, this can mean brown nosing and sycophancy at the expense of actual work (and which may be damaging to morale), or behavior that looks like hard work instead of productive work. The long working hours in Japanese firms are often filled with not very much work for the sole reason that selection favours the easily observable (working hours) over the less easily observable (productivity) – this has social costs of course, but it also means a tired and less productive workforce.
Another example includes our system of university admissions. The ranking (TER or UAI or ATAR or whatever the current acronym is) may be broadly a fairly good measure over all of intelligence, work ethic, studiousness etc. The things you want when selecting a university applicant. But at the margin small increments count for a lot. Towards the top rankings, where actual admissions are determined, particularly in popular courses, a few points make all the difference, and those increments become relatively less about the characteristics universities want (and what we might consider virtue) than they are about very costly behavior. Financially costly, like tutors and private schools (particularly those adept at gaming the system1); but also constant cramming at the expense of extra curricular activity, or of a social life (including every teenager’s right to do dumb things and make romantic mistakes), or of genuine learning or simply of sleep. Selection of subjects that the student is interested in comes second to their perceived contribution to the ranking – despite efforts to make selection neutral by ranking designers. All sacrificed things are valuable things in their own right might even contribute to the qualities universities might look for (and in other countries do look for by far more flawed application procedures than here), but all are sacrificed for a few more points, points that are mere noise when it comes to aptitude but are crucial to selection. Additionally of course, there is adverse selection, by choosing hothoused kids who need constant reinforcement in terms of marks and have narrow experience.
Why not just make the selection random above a certain threshold. A student with 90.00 is probably more adept than one with 80.00, but above that the gradations mean little in terms of what we want in university. Those above this threshold could simply be put in a lottery. That way we still select for quality without as much (we cannot eliminate it) incentive to engage in virtueless but costly activity at the margin. We might also avoid fostering a exam culture, such as those inspired by Confucian or Raj civil service exams.
A similar system may work for research funding. There are enough problems picking winners in research (particulary science), but we also have a great deal of wasted time when researchers write funding proposals and start trying to second guess what funding priorities might change, what “themes” the bureaucracy will pursue. Satisfactory proposals (and this still allows for a focus on certain areas) go into the hat, funded proposals come out and researchers can get on with research. Maybe research journals as well, which already use quite arbitrary techniques, and may lead academics to seek to satisfy the prejudices of editors. Likewise arts funding. Instead of selecting based on the tastes of the Australia Council fiefdom, or a desire to ape what has been done elsewhere by means of expensive cover bands, we fund random applicants who show a sufficient merit. We might even get a film watchable outside the self congratulatory arts stockade, hostile to the barbarians outside who like things like character and plots.
The main problem with all these is that transparent arbitrariness might cause resentment. But I am not convinced. Here’s Chris Blattman discussing random controlled experiments in aid.
Is it the trial that causes hard feelings, or the fact that we are giving aid to one person and not another? The latter happens as a rule. So long as aid is scarce, there’s always a ‘control group’ not getting the program.
There are two differences with an RCT: getting the program is random, and we’re collecting data on the unlucky ones. Does collecting the data create hard feelings? Not in my experience. What about the lottery aspect? The main reaction I’ve heard from communities about RCTs: “Finally, we know why some people get aid and others don’t, and we all have a fair shot.”
Aid allocation is often ad hoc, sometimes corrupt, and never transparent to the people who don’t get it. In some cases, lotteries among the deserving might actually be an improvement, or at least no worse. That’s when RCTs can be appropriate.
In an uncertain world with flwed criteria, there are always arbitrary decisions. A lottery makes this transparent. The losers may have missed out, but they know that they were always in the hat, and that their time might come again. They won’t be bitter that so and so was chosen because they were a suck up, or because of their colour or looks or height. They won’t resent that other schools gamed the system, or that they had one bad teacher, or that they got a bad question on the day of the exam. They won’t resent that a ministerial change meant that funding themes for research changed or that a rival had daddy’s friends on the funding council.
FIFA might learn that give world cups randomly to any proposal that is acceptable, lest they continue to be considered corrupt.
With flawed criteria, the losers suspect it was merely bad luck that resulted in their falure, but they also suspect that they were never in the hat to begin with. With a lottery, they know not to stop trying, because it was clear that they had as much chance as anyone else. It helps that losers know that they were in the hat.
1 Such as allowing a student a scribe in a arts exam because they had difficult writing, but a scribe who coincidentally was a masters student in fine arts.