The importance of improvisation in innovation

In the conference I attended in Wellington NZ I saw a presentation by Tim McNamara a Wellington developer who spearheaded what seemed like a very successful volunteer web 2.0 effort that arose in the wake of the Christchurch earthquake. Using Ushahidi an open source package initially developed to generate and disseminate information on electoral violence in Kenya – Ushahidi is Swahili for “testimony” – Tim and others put together a map which he and scores of volunteers – from around the world no less – populated with information about what was working and what wasn’t and generally anything they thought would be of value to the stunned citizens of the shaken city.

They got much of this information from local organisations – firms, government agencies and others as well as from 300,000 tweets. The 300,000 tweets were parsed by volunteers to detect those that contained some useful fact about an identifiable place that could then be represented on the map. It’s an interesting and inspiring story. Anyway in Tim’s write up of these endeavours he says this:

Vendor neutrality

. . . As far as I could tell, we were the only website providing service to the community without corporate logos. 1 2eutrality meant that we a had high levels of trust and meant that it was safe for competitors to cooperate.

Our platform was neutral to non-code contributions also. This meant that we could provide non-essential community assets like bouncy castles in parks alongside critical information like water distribution points. The neutrality meant that while individual telcos had incentives to provide the locations of only their wifi hotspots, our incentive was to publish all.

In the end, most information in the system came from organisations reporting on their own facilities. They probably used Twitter as a medium to distribute that message. This message was picked up by our volunteers and plotted. Over time, businesses and the public sector began to input information directly. This distributes the burden for the information system to the organisations that have an incentive to keep it right, such as local business owners. Without neutral ground, the site would not have prospered as it had.

This echoes something that I came to understand was critical to the productivity that the internet delivers via net neutrality and open standards – something which is well illustrated in this article by Joi Ito as we quoted him in the Gov 2.0 Taskforce report.

3ry to imagine what it would have been like to create Google before the era of the Internet and open standards. You would probably have had to pay millions of dollars to create the necessary software on a proprietary operating system. The effort would have required a huge team of people taking many years. Since Google is a search engine, it most likely would have been given to the phone company to design and run. If you were using X.25, the international networking standard (the Internet equivalent of its time), you would have been charged for each packet of information that you sent or received, in a network in which each network operator had a bilateral agreement with every other network operator. This total project probably would have taken a decade, cost a billion dollars, and not have worked very well.

In fact, the actual cost of building and launching the first Google server was probably only thousands of dollars using standard PC components, mostly open-source software as the base, and connecting to the Stanford University network, which immediately made the service available, at no additional cost, to everyone else on the Internet.

The explosion of innovation unleashed by the Internet has been driven by an ecosystem of people who work in an open network defined by open standards. That openness has brought us technological marvels like Google, Amazon, and Wikipedia. Now, however, this era of innovation is bumping up against a new barrier.

But the passage in Tim’s article that immediately took my interest was the passage immediately before that quoted above.

Open source

One of the things that struck me was the breadth of skill that software developers who are familiar with open source development posses. Developers used to developing on a Microsoft platform were unable to install a local instance of a web server, get to grips with version control software or communicate effectively with tools used for synchronous remote communication, such as IRC.

In the end, several of the developers who came from a Microsoft/.Net platform went away disheartened.

The open source model was critical for the success of the project. There are technical as well as personal reasons for this. From a technical perspective, the open source community shares several practices and norms for effective remote communication and collaboration. This meant that it was simple to manage a software project that had developers working in multiple time zones in a very constrained timeframe.

However, a larger part of the success was due to the ability for multiple people and organisations. There are few projects in the world supported by Google’s Chief of Engineering as well as a high school students, simultaneously.

While Craig Neville-Manning may not have written any code for the project himself, he certainly approved the use of Google’s resources to move the project along. Without the open source mentality, I don’t think that it would have been possible for the Apache Foundation, Catalyst IT and the CTO of SilverStripe to work together without prior arrangement. As it stood, everyone could opt-in to the level that they were comfortable with. Everyone participating in the project understood that they could work on bug tickets that suited their skill level.

That spirit of collaboration has enabled the efforts of to support the people of Libya and Japan. Many of the improvements made by our team have already been incorporated in deployments to those areas. This includes everything from behind the scenes work, such as optimising clustering algorithms, to front-end usability changes made.

Typically I’ve seen the case for net neutrality and open standards being made in terms of maximising the number of potential connections and combinations and permutations of resources on the internet to find the best available configurations.  I guess it’s not a particularly new insight, more a sub-set of that original idea, but this framework does bring into clear view the importance of two things that I’ve been banging on about.  The first is gregariousness – the importance of picking up the phone, tapping out an email or a blog comment or post or tweet – as a means of connecting things and resources – including people. The second is the importance of improvisation.

One of the things that’s bemoaned about the division of labour is the way in which specialisation leads to fragmentation of knowledge. Likewise amidst this division of labour, organisations, as they grow standardise ways of doing things. This makes a lot of sense, as does the division of labour, but it comes with a mighty sting in the tail – the price of becoming more capable and smarter in some ways is that we become incapable and stupider – and the things we become incapable and stupid about are often some of the most important and commonsensical. And the routines, rules and regulations that organisations build into their modus operandi often not only displace but then actively foil commonsense and acts of improvisation.

To provide a simple example that comes to mind, though one runs into these things every day a couple of years ago I spent a good chunk of an afternoon talking with a CSIRO’s mathematical sciences unit about the qualities of an invention of mine.  They seemed genuinely interested indeed enthusiastic. But the meeting finished with them saying that they couldn’t spend any time on it – not even an hour or so a week snuck into the next month or so to see if they might develop it further. Why? Because they had to justify the time, and to do so they’d have to be convinced that there was money in it for their division (which might well emerge from further exploration).  And so nothing happened.

Of course they may just have been fobbing me off in as polite a way as possible, but others report similar experiences of CSIRO. And this is just not the way things happen in some of the most productive places on earth – like Silicon Valley where people are always improvising, trying things out. Of course that doesn’t mean that the inhabitants of Silicon Valley don’t think of their time as not having an opportunity cost and that they don’t find ways of treating it as the scarce resource it is. But the framework they use is looser, and much more open to improvisation.  Note also how Google’s 20 percent time enabling people to work on self directed projects one day per week (note this is still work for Google on which they are assessed for their contribution) explicitly builds the scope for improvisation into work routines, as well as opening up the scope for networks of self-directed people on their one day in five to link up with each other in collaborative networks that begin informally.

The power of the net and particularly web 2.0 has been the way in which weak ties, which we’ve known for some time are of great importance, have been brought into massively greater productive relation with each other. Here we have a new turbocharged market ecology.  And as Gruen’s law asserts, (well it does now) markets and indeed indeed social structures are always and everywhere an ecology of private and public goods. Here the public goods of net neutrality, open standards and a culture of tolerance, fun and awesomeness * and understanding of the power of openness to rev up the possibilities for serendipitous connections combine with individual energy, commonsense, gregariousness and improvisation to produce extraordinary things.

* Well folks, I don’t like the word ‘awesome’ in its current usage. I’ve been around long enough to know how embarrassed we’ll all be one day that we used it, like finding pictures of oneself with sideburns and flairs, but there you go.  ‘Awesome’ is the word used.

  1. Vendor[]
  2. n[]
  3. T[]
This entry was posted in Economics and public policy, Web and Government 2.0. Bookmark the permalink.
Notify of

Newest Most Voted
Inline Feedbacks
View all comments
13 years ago

“Of course they may just have been fobbing me off in as polite a way as possible, but others report similar experiences of CSIRO”

I’m not in the CSIRO, but I know people that are, and I can vouch for the second of these. Interestingly, with the French equivalent (the CNRS — it is a bit different, because it takes in a lot of research that universities in some places do), they have entirely the opposite strategy where, in some divisions that don’t require big teams, you can do almost anything you want. However, it’s not perfect either — it leads to a very bimodal distribution of people, where there are some outstanding people that have time to do truly outstanding work, but also people who appear to be really non-productive that they can’t get rid of. There are other problems with the CNRS, like pretty average pay, which means it’s not a great job in expensive cities like Paris, so it’s hard to know the extent that some of their problems are caused by this. In my books, it’s still a good deal overall, because it’s really hard to get truly outstanding people in some areas, and so they are really worth a lot.

“Note also how Google’s 20 percent time”

It would be interesting to know to how much of this effect is down to giving people jobs they enjoy, rather than just letting people do self-directed stuff. My bet is that it’s both, and there’s a big interaction (i.e., really smart people want to work there for that sort of reason, and really smart people help you in all aspects of development).


[…] recently posted about the Christchurch earthquake and the way in which Crisis Commons was able to help. Here’s an […]