Club Troppo

Web 2.0 is proving very adept at finding needles in haystacks that we couldn’t have found before. Netflix is a company which rents videos and which relies on the ability of its algorithm to predict what movies you’re going to like from the ranking you’ve given past movies. Given its significance for their profitability they wanted to access the world’s best statisticians to improve their algorithm. But how do you do that. There are two problems – firstly the obvious one that in any imaginable world Netflix will be able to hire less than 0.1% of the world’s statisticians and the slightly less obvious one – that it won’t be easy for those in Netflix to work out who’s a good at their statistics and who is not.

So what better way to do it than host a competition on the internet?

And so a team has just won one million dollars for improving their suggestion algorithm by 10% – well in fact it was a team of teams as on their own, none of the teams could do it. The team that won contains some very seasoned statisticians, and some rank amateurs who nevertheless pulled their weight (I deduce from their membership of the team of teams).

Innocentive has established a Web 2.0 market for ideas in which ‘seekers’ post challenges (like how to get more toothpaste out of the tube), and promise prizes. Challenges that are on their website as I write this include the production of an Open and Re-closable Fastening System for clothes (other than the known methods like Velcro) and something that will help Reduce the Placebo Effect in Clinical Drug Trials.

Anthony Goldbloom, an econometrician working in the Treasury and then the Reserve Bank watched all this and decided to establish Kaggle.com which is a data-analysis and prediction marketplace where companies can run competitions like the Netflix prize. If you read about the Netflix prize you find that doing it well was quite an involved process – one that other firms might well want to contract out. Indeed, it turns out that their second offering of a major prize has had to be cancelled owing to privacy concerns. So expertise is needed to run these things well. And as is the case with e-Bay and Innocentive and plenty of other internet marketplaces, there are strong economies of scale and scope which means that there will be benefits in pooling resources in marketplaces. So I not only predict a bright future in Kaggle, I’ve become its Chairman (I think this qualifies as a disclosure of interest).

There are meanwhile lots of things to think about. Both Anthony and I have some hefty ambitions about what the site could become. We’re both keen on the way in which people can build reputations though Kaggle for knowing what they’re doing – reputations that are often not nearly as accurately formed within organisations for reasons discussed above.

There are also many different kinds of ways to run competitions and kinds of things one might want out of it. One might want to predict the future – by working out the odds of Collingwood winning next week and for such an exercise one would then require the elapse of some time before the prediction could come in. The Netflix prize is a little different in the sense that the prize can be given out when relationships are identified in the data that stand up to scrutiny given existing data. In such a circumstance one is really ‘predicting the past’.

We’ve just launched our first predicting-the-future competitition, which involves forecasting the voting for the 2010 Eurovision Song Contest. For those not familiar with the contest, it’s widely believed that voting outcomes are influenced by European politics. Contestants in Kaggle’s Forecast Eurovision Voting will attempt to exploit historical voting regularities, as well as other factors, to predict the voting for the 2010 Eurovision Song Contest. The winner of the Kaggle contest will collect a $USD1,000 cash prize. And just as the Eurovision Song Contest has launched the high-flying careers of ABBA, Celine Dion and Riverdance, forecasting accurately will earn competitors a top ranking on Kaggle’s league table.

A taste of the power of Kaggle is captured in its first, competition. Even without prize money, the demo footy tipping comp has attracted 158 legitimate entries from 7 teams, and the leader would have tipped 74 per cent of games correctly (76 per cent is required to win the rich Sportingbet tipping comp). What’s more this is based on a data set that was pretty much thrown together with the central criterion being that was easy to collect.

Here’s a quote I read today.

It’s how PR (Proportional Representation) systems are meant to operate, and is far preferable to a minority government. It’s a mature and sensible approach, and a step away from the pathologies of winner-takes all so common to Westminster systems with single member electorates. The result will be the representation of the will of a larger proportion of the electorate, and it’s hard to see how that’s anything other than a positive..

There’s nothing particular about the source or its context (Tasmania) I want to note, but its a sentiment I’ve heard many times and it happens to trouble me.

The positive in this instance seems to be that the government will be made up of politicians elected by a greater proportion. Likewise proportional representation is desirable because it increases a chance that an elector will have a politician in parliament to call their own.

The problem I have with this viewpoint is that it implicitly supposes the reason we have elections is to have politicians, and that I guess the point of politicians is giving viewpoints in parliament and arguing.

Are politicians really the end goal of democracy?

As far as I see it, politicians are only the means to an end: governance – which I’ll start calling policy. The identity of and colours flown by the people implementing the policy may be of great importance to political tragics (who should get football teams), but it’s what they do that counts more than whom they are. (Continued)

A friend of the family Tony Carson, an interesting fellow who was great at crosswords and so secured for himself a place at Bletchley Park during World War II, had a hand in designing the Smith Family’s program Learning for Life. It helps families pay for school books and also case manages things to try to address kids’ motivation etc.  In the bumph that the send me as a previous donor, the Smith Family shows me a picture of single father Andrew and his son Josh. It tells me that Josh had behaviour problems from years 5 to 7 but that attending the Learning Club made all the difference.  I have little reason to doubt this, but I wonder about other families. How successful is this program?

Of course the stuff I’m getting is marketing material. And the cartel of good intentions means that I’ll never hear of things that didn’t work out so well. For all I know the Smith Family is ruthless with our money making sure it’s working.  But I’m never going to be able to see this in action, because telling us of programs that didn’t measure up isn’t a very good way of marketing all the good that people’s money does, something that William Easterly points out about foreign aid. so it seemed to me that the some agency could do us all a favour by offering some kind of audit of effectiveness of such programs with public reporting of results.  It’s the kind of thing the government could help bring about, particularly if it tied deductible gift recipient status to the performance of such audits.  If it was done well, always a big assumption, it could make a big difference to the effectiveness of the charitable dollar.

I have long viewed sporadically gifted journalist Christopher Hitchens as a caricatured bullying buffoon, but until quite recently I admired Richard Dawkins.  Years ago I read The Selfish Gene and The Blind Watchmaker with fascination, along with the works of fellow biological sciences populariser Stephen Jay Gould.  They seemed to me to epitomise scientific rigor and rationalism.

However Dawkins seems to have gone completely off the rails over his atheism obsession.  His gratuitously offensive and silly reference to Benedict XVI as “Pope Nazi” at a recent atheists’ conference in Melbourne was bad enough.  But now he and Hitchens claim they want to arrest the Pope for ‘crimes against humanity’ for  ’the alleged cover-up of sex abuse in the Catholic Church’.

Leaving aside the fact that it’s highly unlikely that Benedict’s alleged actions (he wrote a letter as a Cardinal in 1985 indicating that moves to defrock a paedophile priest required “very careful review” and more time for investigation)  could be classified as a crime against humanity within the Rome Statute of the International Criminal Court even if he had ‘covered up’ child abuse,  the known facts don’t actually point to the commission of any crime at all by Benedict:

(Continued)

Before Freud was granted the exit visa he needed to escape from Vienna, he was made to sign a document: “I, Prof. Freud, hereby confirm that after the Anschluss of Austria to the German Reich I have been treated by the German authorities and particularly by the Gestapo with all the respect and consideration due to my scientific reputation, that I could live and work in full freedom, that I could continue to pursue my activities in every way I desired, that I found full support from all concerned in this respect, and that I have not the slightest reason for any complaint.”

Freud signed, but added in his own writing, “I can heartily recommend the Gestapo to anyone”.

From Jonathon Glover’s very interesting website: HT a Philosophy Bites Tweet.

I got this correspondence in my email today – last year we got a family membership to the Colliwobbles Football Club and enjoy going to most matches. I always email the words of our coach Mick Malthouse explaining the game on Saturday in hindsight on Mondays onto my son and sometimes if they are sufficiently cliched, to my daughter.  They are always words of great wisdom. Mick is really a performance artist with coaching a football side as his theme. In them he says things like “you’ve got to hand it to the boys”.  ”They fought it out to the end [as opposed, for instance to sitting down and having a sandwich]“.

Anyway, some of you may be vaguely aware that Mick Malthouse had an altercation with a player on the other side last Saturday. I haven’t read up on all the details, but someone called him a Poofter – and he called them a rapist. Which seems like a reasonable exchange.  Anyway, people objected to this and it has blown up. Some Troppodillians will probably know more about this than me. But I thought all Troppodillians would want to read Mick’s words this Monday showing just how he navigated the ethical issues that the situation produced.

Following my apology for inappropriate comments I made to Stephen Milne, I would like to take this opportunity to inform you of the reasoning behind the initial position I took in the post-match press conference on Friday night.

When asked if I was directing comments at St Kilda players or my own players, I chose the latter option basically under the old code of what happens on the field stays on the field. (Continued)

Sri Lankan asylum seekers in detention on Nauru in 2007

I was asked an interesting question this morning (well, interesting to me anyway) by a local media person about whether the seemingly imminent transfer of Christmas Island asylum seeker detainees to Darwin would mean an upsurge in refugee matters being litigated in local Darwin courts.

The simple answer is probably no.  In MZXOT v Commonwealth in 2008 the High Court ruled that amendments to the Migration Act 1958 (Cth) in 2005 meant that neither the Federal Court nor Federal Magistrates Court have jurisdiction to hear judicial review applications relating to ‘primary decisions’ under the Act, and the Court also held that State courts were not vested with jurisdiction to hear such matters.  Consequently only the High Court of Australia can hear judicial review matters concerning decisions dealing with “boat people” asylum seekers.

Conversely though, the High Court held in Plaintiff S157/2002 v Commonwealth that its own jurisdiction to judicially review such decisions was constitutionally entrenched by Constitution s 75(v) and therefore could not be removed by Parliament (or arguably even restricted to any significant extent – see Bodruddaza v Minister for Immigration and Multicultural Affairs (2007)).

On the other hand, in 2001 in Ruddock v Vadarlis the Full Federal Court held by majority that asylum seekers detained on the MV Tampa were not in law imprisoned in a sense that would allow the issue of the writ of habeas corpus, in that they were at all times free to leave and go anywhere they liked – except Australia!

(Continued)

I confess to not having read a proper biography of Leo Tolstoy. My conception of Tolstoy the man is based, unfortunately, on the relevant chapter of Paul Johnson’s notorious Intellectuals. If you haven’t come across this book, it’s a series of case studies (or hatchet jobs) advancing the thesis that modernity’s most cherished utopians were all monumental hypocrites and egotistical bullies in their personal lives. Johnson’s Tolstoy is one of the more extreme megalomaniacs of the bunch. A callous womaniser and uncontrollable gambler in his youth, he later develops a full blown messiah complex, abuses and betrays his most loyal friends, and reviles women as a class, seeing them as temptresses and hysterics. Unsurprisingly he does not not love his wife Sofya, whom he submits to unrelenting psychological cruelty.

I don’t know how accurate this version is, but Johnson himself asserts that these two are about as well documented as any nineteenth century lives, so presumably he stands by it.

Michael Hoffman’s film The Last Station offers a vastly more sympathetic picture of the writer. Based on a book (which I haven’t read) of the same name by Jay Panini, it treats the last months of the writer’s life, seen through the eyes of his adoring secretary Valentin (James McAvoy), an intense young chap whose endearing (though not to everyone) idiosyncracy is sneezing when he’s nervous, which is most of the time. Hoffman’s Tolstoy is a difficult character to be sure, mired in frustration and prone to tantrums; but under the surface there’s a kindliness and an ironic humour that can’t be reconciled with Johnson’s self-absorbed ogre. He is basically a victim of his own success, by all means impractical and gullible, but by no means a born tyrant. (Continued)

I have been sent the following guest post, by someone who wants to remain anonymous on account of his position in the public sector. (I know the author, but hey, here’s an offer to those hundreds of thousands of public servants out there – if you want to send me a post that’s worth posting, I’ll post it anonymously on Troppo for you.)

Recent moves by the RBA to move to a ‘leaning against the wind’ policy to head off incipient asset price bubbles is a welcome change for those of us who have advocated this approach for some time.

In the wake of the demise of the Greenspan doctrine that asset price bubbles are too hard to recognise, too hard to deal with and too benign to worry about, this is a logical response. It is an incremental approach and one that is not scaring the horses.

The other approach that may still be worth pursuing is one that improves the CPI measure (and underlying) used by the RBA to target inflation by incorporating house prices.

The CPI as it is currently is certainly not beyond improvement.

A variation of this approach was proposed late last year by Henry Thornton (the nom de plume of a prominent economist) on his blog and is worth real consideration. Provocatively dubbed “True Inflation” it is a work in progress and could do with a more theoretical detail, back testing and further debate but it is a great place to start. (Continued)

Recently there’s arisen a debate about having a debate on immigration and also an attempt to relive the glory days of asylum seeker politics. Whilst attempts to link the two have been cynical, I believe there might be a good reason to link them.

Why not draw almost all our new migrants from asylum seekers; more specifically, from boat people?

Lets work from the ground up.

I have no idea of the “carrying capacity” of the continent, and I don’t know how capable we are of actually calculating it, but I think it’s safe to say it’s in the range < infinity. Additionally, whilst we may not be anywhere near such a point at the moment, there will be a point where the weight of migration will completely overwhelm the existing culture. For reference note that my family has seven or so generations here but still has managed to avoid learning a single indigenous language. From this two points we can establish that there has to be some upper limit, no matter how high, on immigration intake. Some kind of quota.

Such a  quota necessitates some kind of selection mechanism. We could just do a first come first serve, or a lottery, but both would likely be unpopular and probably suboptimal from our perspective. A mechanism whereby we auction off immigration places would be likewise unpopular. (Continued)