Archiving Government websites: Should it really be this hard?

When I did the Government 2.0 Taskforce, one of the subjects that was earnestly discussed was archiving of government sites.  It’s a big problem in government. I could never see why it should be a big problem. After all you can look at anything written on ClubTroppo since it started.  We haven’t spent any huge amount of money to deliver that kind of functionality, haven’t burned any midnight oil. But IT people in government told that it’s very expensive to keep web pages live. I have no idea why but they swore black and blue that it was.

Anyway I recently sought to track down the results of Obama’s less than spectacularly successful community brainstorming on open government when he came into office. (The top two suggestions for promoting open government were legalising marijuana. The other big thing was releasing Obama’s birth certificate.) Anyway I emailed an American friend who’d been in the White House at the relevant time – now back in academia – asking for any write up of the program and she told me there was one in a 2009 annual review of operations.  But it’s gone from the website and no-one has been able to find it in a couple of weeks. This is 2009!

For another project I was also looking up the old Power of Information Taskforce in the UK.  Here’s Tom Steinberg’s blog entry announcing its release.

I’m delighted to announce that the review I’ve been working on with Ed Mayo and the Cabinet Office has launched today. You can get the official PDF version here or my friend Sam Smith’s annotatable version that he just threw together.

I clicked on the first link and it went through to here.

http://webarchive.nationalarchives.gov.uk/+/http://www.cabinetoffice.gov.uk/publications/reports/power_information/power_information.pdf

Which was promising. It said this.

This snapshot taken on 25/11/2010, shows web content selected for preservation by The National Archives. External links, forms and search boxes may not work in archived websites. Find out more about web archiving at The National Archives. See all dates available for this archived website 

Object moved to here.

Alas, it wasn’t there either and I was diverted to a Cabinet Office Page Not found signal – as you can see for yourself if you want to click on the link.

Meanwhile one of the things that the Power of Information Taskforce and Review did was to publish using commercial blogging platforms. And everything using that remains safe and sound. “Sam Smith’s annotatable version” that Steinberg says Sam “threw together” refers to on his blog is still there, safe and sound. Likewise the Government 2.0 Taskforce published to its own url using Wordpress software, and it’s still there too, it’s cost to government would be the same as the cost of Troppo to those of us who run it – the cost of the domain name registration, which is about $30 a year or something, though the cost to government of maintaining the UK’s Power of Information review, which is a sub-domain of wordpress.com is exactly zero.

So it still eludes me why, with all the resources to hand, governments make it quite so difficult for themselves.

Government 2.0: my first column of the Gittins Summer break

Ross Gittins asked me if I’d fill in for him during his summer break, which gives me a chance to get a few things off my chest. So here’s the first of four weekly columns.

In 2009, I chaired the federal government’s Government 2.0 Taskforce. We sketched out how government might be transformed by the open zeitgeist and tools of Web 2.0 – like Wikipedia, Facebook, Twitter and Google.

Web 2.0 massively scales up our capacity to communicate – with possibilities both trivial and earth-shaking. And it scales up simple improvisation. Whether you’re organising a party or a working bee, just hop on to Facebook or Twitter and Bob’s your uncle.

Two hours after the Christchurch earthquake, work commenced on a map on the net on which could be plotted emerging developments on the ground. The information, such as the address of pharmacies that still had insulin, was parsed from 300,000 tweets bearing hashtags like #eqnz.

If you think this was a job for official emergency services on the ground, think again. Tim McNamara wasn’t with the government, but spearheaded the initiative from the North Island capital Wellington. The people who parsed the tweets were further away still, a band of humanitarian ”Crisis Commons” volunteers spanning every continent. Continue reading

Designing better lives: An economist’s appreciation of design

Herewith an paper about my encounter with design, on taking up the Chairmanship of the Australian Centre for Social Innovation and encountering the Family by Family program.  The site where it’s been published doesn’t have any comments facility, so I’m opening up discussion here should anyone wish.

And I read today a quote that might have been a good complement to the quote appearing at the head of the article – immediately below.

Not only was he [Edward Land - inventor of 'instant cameras' and founder of Polaroid] one of the great inventors of our time but, more important, he saw the intersection of art and science and business and built an organization to reflect that.

Designing better lives: An economist’s appreciation of design

Design is often described as making things not only useable but useful and desirable/delightful. We’d agree this is important – but what is even more fundamental (and rare) is making things that prompt change. – Sarah Schulman and Chris Vanstone [2]

       I.

Design is on the march. Apple teeters on being the most highly valued company in the world – its core competitive strength lying in design and systems integration, not technology. ‘Design thinking’ is becoming increasingly prominent not only in the development of products and processes, but also in the delivery of services. So much so that Deloitte has recently begun investing heavily in its own ability to provide its clients with design knowhow as a crucial engine of its innovation and competitiveness. As I write this, a prominent article on Australia’s Deloitte Online’s homepage [3] is titled “Design thinking demystified”. So what is the core contribution of design and what is behind its rise?

Adam Smith’s invocation of the benefits of self-interest – or as he called it self-love – is famously encapsulated in this aphorism:

It is not from the benevolence of the butcher, the brewer, or the baker, that we expect our dinner, but from their regard to their own interest. We address ourselves, not to their humanity but to their self-love, and never talk to them of our own necessities but of their advantages.

Smith’s point is not that self-interest is good in itself, but that the self-interest of one person in a market brings them into relation with others’ self-interest. Note that Smith’s injunction (implicitly to both parties) is for each to seek their own interest by addressing themselves to the other’sinterest. Since Smith founded it, the discipline of economics has focused on the incentives facing each of the parties to a bargain and on their relative bargaining strength.

But there are more things in heaven and earth. For the butchers, bakers and brewers of Smith’s time there was no great mystery as to what constituted the customer’s wants or needs. Today’s world is much more complex. If you’re making computers or even cars, customers have specific wants that are not so easily divined by producers. Thus, part of Japan’s auto-producers’ recipe for competitive success was meticulous attention to consumer needs.[4]

This process has now gone much further. A great transformation occurred at the outset of the personal computer era when the Apple Macintosh showed that consumers didn’t just want more technical capability from their software and hardware – something that could be captured well enough in standard disclosures of those technical capabilities. They wanted user-friendliness – a very different thing and something inherently difficult to ‘disclose’ in specifications.

It turns out you can’t really make a car or a computer useable without a lot of work, almost invariably involving the users themselves. And indeed there is a discipline that has grown up under our noses which has been all but ignored by economists and policy makers but which nevertheless addresses itself to this issue. That discipline is design. Continue reading

Me: or recordings thereof

Here are two talks I’ve given in the last year. One was a couple of weeks ago at a Melbourne Conversation on Big Data. I talk about the serendipity of big data and the relevance for privacy regulation.  And tell a story about Kaggle. I recommend the talk before mine by David McCloskey of Deloitte. He’s a very thoughtful guy.

The other is a talk on Education 2.0 given to QUT senior staff. It’s only up for another week or so and it’s one gigabyte to download – which is ridiculous, but apparently reduced from four gigs.  It’s a zipped file.

Media Inquiry: Look forward, not back

[Cross-posted to Online Opinion]

I spend my working life running an online media firm – WorkDay Media, publisher of Banking Day – with its owner and editor-in-chief, Ian Rogers. Last month, Ian and I wrote a submission to the federal government’s Independent Media Inquiry. You can see the whole thing at the WorkDay Media site.

We’re trying to focus the inquiry a little more on what we might gain from the Internet’s transformation of communication, and a little less on what we might lose as newspapers inevitably dwindle.

It’s fairly obvious that Australians are relying less and less on information from “the mainstream media” – that is, existing newspapers, TV and radio stations. Instead they are getting and exchanging information from a far richer variety of Internet-based sources, from email newsletters to expert blogs to government and company records – plus, of course, Club Troppo.

This seems like good news. So why are we holding a media inquiry focused on mainstream media, and particularly on the newspaper industry?

The obvious answer is that the future for Australian newspapers looks pretty ugly. Once newspapers were the gatekeepers; now they are not. They are losing advertisers and readers to a fundamentally more attractive and efficient Internet. The media analyst Roger Colman calculates that “all metropolitan newspapers in print editions will be unprofitable, definitely, by 2020″.

Many of those who fear for the future of “the mainstream media” in Australia – like academic David McKnight, or publisher Eric Beecher – are concerned about how we will reproduce the activities of big newspaper newsrooms as newspapers gradually go out of business. They believe this is a very important question.

But this focus on the media past signals a failure of imagination. Big newspaper newsrooms will not be recreated in online form. Facts, news, analysis are all going to have to come out in different ways than they have in the past.

And they will. They already are. You have to be enormously enthusiastic about the old media environment not to believe this: the new media environment, for all its faults, is far better than what it is replacing.

Media thinkers worry that online sources would never have uncovered a Watergate scandal. They’re probably wrong, in every way. Now more than ever, the truth will out. Richard Nixon’s corruption was mostly uncovered by official investigators; Woodward and Bernstein, great journalists that they were, were merely conduits. In the age of the Internet, Watergate might have evolved over weeks, not years. Just in the past year we have seen yet another new information innovation – Wikileaks – whose model suggests secrets will be harder than ever to keep in the decades ahead.

There will probably be times in the future when Australia will look back at some event, some scandal, some development in the society, and say that newspapers might have done a better job than the new information sources. But we suspect those cases will be few and far between.

New online players would already be even more numerous in traditional media areas such as politics, public policy and business if not for the presence of mainstream media, particularly newspapers, whose large online presences are hugely subsidised by their traditional businesses. This is certainly the biggest bar to the expansion of many online information ventures, including WorkDay Media.

Australia has entered an age when media can be created, transformed and transmitted far more easily than ever before. Australians who believe in the importance of an informed society should treat the 2010s as an era of huge optimism and opportunity. For there is every reason to believe that the Australian society of the next 20 years will be better informed than ever before.

Facing such a future, it makes little sense to try to impose a more restrictive regime on the dwindling existing “mainstream media”, or to subsidise its continued existence. We can improve the Press Council. We can have governments make more information available to citizens. But there is no need to choose this moment to impose either a new regulatory regime or a new protection scheme.

This is a moment to embrace the information future, not to embalm the media past.

Kaggle closes its Series A round

I know you’re all on the edges of your seats about how Kaggle is going.

The answer is “very well”. We’ve just announced the closure of Series A funding.

And you can read all about it in the New York Times, the Independent or Gigaom.

Further information from our newsletter below the fold:  Continue reading

Games

Games seem frivolous. They can stand as metaphors for life, but typically, the outcome of games doesn’t really matter. I wanted Collingwood to win it’s last game this year, but it didn’t and that’s that. Doesn’t matter. Still as I gradually realised when working on the Government 2.0 work in 2009, the element of play is critically important and not just to high level ‘brainstorming’ activity, but to seizing the opportunities for innovation of all kinds from major disruptive innovation to the most minor improvised improvements in the way things are being done.  That’s why I thought things like mashups were so wonderful – they are low cost ways of breaking things up, and inviting others into play with one’s assets (or copies of them while the ‘real’ ones remain on the official website or otherwise in the system somewhere.

And with games becoming so much more prominent in young people’s use of their time, it’s not surprising that educators are arguing that games teach a new kind of literacy. Gamification is all the rage in Silicon Valley. As Wikipedia explains, gamification is

The use of game design techniques[1] and mechanics to solve problems and engage audiences. Typically gamification applies to non-game applications and processes (also known as “funware“)[2], in order to encourage people to adopt them. Gamification works by making technology more engaging[3], by encouraging users to engage in desired behaviors[4], by showing a path to mastery and autonomy, and by taking advantage of humans’ psychological predisposition to engage in gaming.[5] The technique can encourage people to perform chores that they ordinarily consider boring, such as completing surveys, shopping, filling out tax forms, or reading web sites.[3]

Michael Neilsen’s new book on Reinventing Discovery has this to say about Foldit, a very clever gamification of the arduous task of figuring out how proteins fold  which has generated new scientific insights about the formation of proteins.

I was skeptical when I first heard of Foldit. it sounded like the dull educational computer cames I saw in school when I was growing up in the 1980s. But I downloaded the game and spent hours playing it over several days. . . . . People play the game because it’s good.  It has the compelling, addictive quality all good computer games have: a task that’s challenging but not impossible, instant feedback on how well you’re doing, and the sense that you’re always just one step away from improvement.” (p. 146.)

But you can argue that games are more important than that. Continue reading

Information and Charities: an idea . . .

PlaypumpsReading Tim Harford’s excellent Adapt: Why success always starts with failure an idea occurred to me. He talks of the curse of the playpump – a photogenic aid strategy that appeals to celebrities and millionaires but which doesn’t work. It’s obvious that information about what works has been a huge obstacle to philanthropically motivated efforts to help the poor. Adam Smith said as much (when he said that people could be expected to do more good for the world investing their money to advantage themselves than they would investing it for the good of the world – because they know so little about the latter and so much about the former. Bill Easterly said as much.

I also mentioned this here. Anyway it got me thinking. What if the government set up an agency that evaluated the programs of any not-for-profit that asked to have its programs evaluated. This couldn’t really offend anyone, but if one could get some nibbles from the better charities. Then those of us who want to give their money to charities that generate the best impacts could actually find out which ones to give it to. The PC reported on the not-for-profit sector and seemed more concerned about information about the sector regarding its contribution to the economy. However it did have an important and worthwhile recommendation.

The Australian Government should provide funding for the establishment of a
Centre for Community Service Effectiveness to promote ‘best practice’
approaches to evaluation, with an initial focus on the evaluation of government
funded community services. Over time, funding should also be sought from
state/territory governments, business and from within the sector. Among its roles,
the Centre should provide:

  • a publicly available portal for lodging and accessing evaluations and related information provided by not-for-profit organisations and government agencies
  • guidance for undertaking impact evaluations
  • support for ‘meta’ analyses of evaluation results to be undertaken and made publicly available.

At first glance this seems a very satisfactory response to my concern.  But I think it misses something important. One could have lots of evaluation and that could be just fine, and one hopes that many not-for profits would take note of it. But there’s a problem. As I’ve argued with information on workplace conditions, what’s missing is a standard by which people can judge one offering against another. I think that needs to develop and proposals to improve information in the sector should also address that.