Who is this man? And why should you care? He is a Portuguese physicist, Filipe Maia, a PhD student at Janos Hajdu Molecular Biophysics group at Uppsala University and he’s designed the best chess rating system the world has ever seen. Who knew he had it in him? Maybe him. Almost certainly no one else.
Which is as good a way as any of explaining the power of crowdsourcing. The power of Kaggle. Here’s an extract from Chessbase’s write up of the comp by Jeff Sonas whose competition it is.
The Kaggle website provides access to a training dataset of 65,000 recent chess games, that can be used to train their rating system, as well as a test dataset (a list of 7,800 recent chess games whose outcomes must be predicted and submitted). The website (which knows the results of those 7,800 games) automatically scores each submission and maintains a public leaderboard of each participant’s best-performing entry. A more complete score for each entry is kept secret so that participants cannot “decode” the leaderboard, and this secret list determines the final winners, to be announced at the end of the contest.
A wide range of approaches has been used, including almost every known chess rating system as well as other tries involving neural networks, machine learning, data mining, business intelligence tools, and artificial intelligence. In fact over 1,600 different tries have been submitted so far, and we anticipate far more submissions as the competition heats up over the final seven weeks.
The #1 spot is currently held by Portuguese physicist Filipe Maia, who confesses to little knowledge about statistics or chess ratings, but is nevertheless managing to lead the competition! He is also the author of El Turco, the first-ever Portuguese chess engine. Out of the current top ten teams on the leaderboard, seven use variants of the Chessmetrics rating system, two are modified Elo systems, and one is a “home-grown variant of ensemble recursive binary partitioning”. That last approach belongs to the #3 team on the public leaderboard, a team known as “Old Dogs With New Tricks”. This team is a collaborative effort between Dave Slate and Peter Frey, both prominent leaders in computer chess for many years.
Although the “Old Dogs With New Tricks” team clearly has a lot of chess expertise, and the #2 spot is held by Israeli mathematician and chess player Uri Blass (FIDE rating 2051), the top ten or twenty teams are primarily comprised of mathematicians, data miners, and other scientists having minimal direct experience with chess or chess ratings. This suggests that experts on chess rating theory might still have a lot to learn from experts in other fields, which of course is one of the desired outcomes of this contest! We have attracted interest from around the globe, with the top twenty comprised of participants from Portugal, Israel, USA, Germany, Australia, UK, Singapore, Denmark, and Ecuador.
Now wouldn’t it be good if all those people in the world, all those Treasuries of public and private sector agencies, marketing departments, strategy departments understood how incredibly cheaply and well crowdsourcing can solve some problems!