Well, after a week and a half of our HIV progression comp we beat the best available model of HIV progression. Now the best entry in our Eurovision comp picked the winner. That’s not so amazing because it was a pretty one sided affair. What was worthy of note is that Kaggle‘s best entry did better than the betting markets!
From the Kaggle blog:
Interestingly, Kaggle’s statisticians outperformed the prediction markets, selecting seven of the countries that finished in the top ten, compared with five for the prediction markets.
One reason that statisticians may have outperformed the punters is that number crunchers can better assimilate voting patterns. Georgia and Greece, both countries that benefit from voting partnerships, appear in the Kaggle top ten but not in the betting markets top ten. Meanwhile Israel, a country that has no obvious voting partners appears lower down in the consensus ranking. Some have argued that the new voting system has put an end to political voting. However, it’s not difficult to pick out patterns in this year’s data. (Unless you think Georgians saw something that others missed in Belarus’s soporific performance when they allocated the maximum 12 votes to Belarus – the other 38 countries collectively awarded Belarus just six votes.)
??Another reason why the consensus among statisticians may have been more accurate is that prediction markets may be skewed by sympathies, while models tend to screen out emotions. Doubters might argue that individual punters may be emotional but that emotions average out in a large prediction market. This is likely to be true to an extent, however it’s almost certainly the case that emotion plays a greater roll in betting markets than in a consensus among statisticians.? (The existence of the favourite-longshot biassuggests an emotional aspect to betting markets.)
Finally, the betting-market data itself might have an impact on the outcome. This year, pre-contest favourites seemed unwilling to allocate votes to each other. Azerbaijan awarded Germany just one vote when other countries awarded Germany an average of 6.5. Germans returned the favour by not sparing a single vote for Azerbaijan.
We’ve got some more comps in the pipeline . . . watch this space.