Club Troppo

One of the core ideas in computer science is the power of indirection and abstraction. The single most powerful technique for understanding large systems is simply to ignore details: to build “abstraction barriers”, layer upon layer of them. At the top layer I may be thinking in terms of having a Post object talking to an Author object. Meanwhile, several dozen layers down, machine code is rearranging single digits billions of times a second.

I have often found that the concept of abstraction barriers explain why free markets work: the division of large systems into self-contained abstractions mimics the division of labour. I do not need to understand how my computer was manufactured in order to purchase it and put it to work. Someone else has taken care of those details. From the production of PCBs to the minutiae of money moving, I don’t need to know or care. My level of abstraction rests as a purchaser.

Related to free market ideas is Eric S. Raymond’s thesis of the Cathedral and the Bazaar. Raymond says that Bazaars may be unplanned, messy and ugly to the unsuspecting technocratic eye. Yet they prosper, they evolve, they embrace and expand. Linux is given as the canonical example of a bazaar – a spontaneously ordered product which turns out to be superior to technological cathedrals.

Recently this idea has been challenged by John Siracusa, a respected commentator at Ars Technica. He offers Sun’s new Zettabyte File System (ZFS) as an example of where Linux cannot penetrate certain layers of abstraction to reorder them, because it cannot enforce a vision from the top down.

ZFS is actually a brilliant piece of work. At a stroke it eliminates several layers of file system and volume management abstraction (and also a lot of complexity). Linux instead has followed a traditional route of divvying certain of these functions up, because that’s how it was done before. Instead, ZFS does not so much pierce several layers of abstraction as render them moot.

The core of the matter is that the cathedral has its place. From time to time brilliant technological advancements (and others which were merely overdue for the mainstream) need champions with clout to back them up. It is no accident that Java, in many ways a fairly uninteresting language, prospered largely through the backing of industry giants Sun and IBM. Similarly the x86 family has waxed through the support of IBM and then Microsoft; the early Internet ran largely on DEC Vaxen, Suns and IBM mainframes.

Sometimes it takes a bazaar to build an economy. And sometimes it takes a brilliant architect to inspire a new bazaar.