Club Troppo

Just as Asimov had his Three Laws (of Robotics), the other greatest science fiction author of all time, Arthur C Clarke, had his own Three Laws. Of interest today (and indeed always) is the widely-quoted Third Law:

Any sufficiently advanced technology is indistinguishable from magic.

Clarke’s Third Law has attracted many corollaries, such as Gehm’s Corollary to Clarke’s Third Law, which states that “any technology which is distinguishable from magic is insufficiently advanced”.

The central thesis of this essay will be that three relatively unknown corners of computer science represent some of the most important challenges facing humankind. Indeed, it is considered that the problem of software reliability, software security and artificial intelligence friendliness are far more central to the survival of the human race than the nearterm proliferation or any mainstream scenario of global climate change.

This is the first in a pair of essays I’ll write on this topic, inspired initially by a recent post from Paul Frijters.

From Little Things Big Trouble Grows

In this first half of the 21st Century, we are very likely to develop mature nanotechnology. This is a hotly contested statement and as a field nanotechnology has many skeptics; however the fact remains that it seems plausible given what we know of the laws of physics.

Before the advent of modern semiconductor manufacturing, computing was a largely theoretical exercise. However the theory suggested that artificial computing was possible. Turing’s Machine and Church’s Lambda Calculus both fed into actual hardware and software. They demonstrated that such things were possible (while also illuminating certain impossibilities which have yet to be solved).

I suggest that nanotechnology is at a similar state of development to computing in the 1940s. A lot of preliminary theory has been done, the field has not been shown to be in contravention of known laws of physics and designs are already appearing for mature nanotechnological devices (such as the Respirocyte design proposed by Robert A. Freitas Jr). It is being approached from several directions - the most important being biotechnological and semiconductor research, which are already starting to mess with nanoscale engineering.

Putting aside its feasibility as a future “magic”, nanotechnology has attracted powerful critics and naysayers on the basis of its potential impact. Probably best known is Bill Joy, co-founder of computer industry powerhouse Sun Microsystems. In 2000 Wired Magazine published his now-famous essay Why the Future Doesn’t Need Us. This famous essay argues that the future of the human race is threatened by rapid advances in genetic, nano and robotic technologies. For the common good, these technologies ought to be foregone.

I agree with Bill Joy that the “GNR” (Genetic, Nano, Robotic) technologies are existential risks for the human race. But I don’t agree that they can be foregone. Even if they were banned, research would continue somewhere anyhow; and that research would be more likely hostile than otherwise.

But his essay illuminates what futurists tend to overlook when enthusing about nanotechnology: when you make matter as malleable as software, you make it dangerous and unpredictable. To me the unconsidered conclusion is that research into software reliability and security are of paramount importance.

Security in particular: today millions of PCs are security-compromised “zombies” in “botnets”, arrayed into vast armies of distributed computing power and bandwidth. Botnets are able to take entire segments of the Internet offline by attacking not just web servers but key infrastructural elements with overwhelming volume. This problem is created not by black hat hackers having access to powerful computers. It is caused by the very wide incidence of software insecurity.

The security state-of-the-art for computers is well advanced over commerically-vended systems. Even between commercial offerings there is substantial differences. Windows 95 through to Windows XP are notorious for their thousands of exploitable flaws. Mac OS X is considered more reliable, though a recent “Month of Apple Bugs” shaked many people’s faith in that assertion. Then systems such as Trusted Solaris and SELinux go some steps further by implementing more complete security coverage (Role Based and Mandatory Access Control). Meanwhile, software security relying on sourcecode- and machine-bound changes (such as the No Execute bit, “parrot bits”, heap randomisation) has become prominent in the opensource world, particularly championed by the OpenBSD project.

But still, the research is far ahead of the practice. Experience with the history of programming language technologies suggests that the lag between research and practice can be as much as 30-40 years (the delay in the propagation of garbage collection ideas between Lisp and Java, for example). If this hold true for security practice in software, we should see Role Based and Capability security models becoming mainstream sometime circa 2030-2040, around the time we will see the early commodification of nanotechnology.

Is this really safe or acceptable? We need reliable software sooner to allow us to properly understand both theory and practice in time for widespread nanotechnology. We need to build into the first self-replicators the kind of security today only available in research operating systems and NSA computing centres.

Otherwise we risk subversion of our safety on an unprecedented scale. Should nanotechnology be as subvertable as current computer software, and on a comparably quasi-industrial scale, then it would be wisest to give it up.