Having begun an epic move from Darwin to Perth some weeks ago, I am at last established here, for the next 6 months at least. At the moment I am crawling along on dialup while Telstra — for the fifth consecutive time — play funny buggers with provisioning an ADSL connection at my local exchange.

What I would like to talk about briefly is what I will be doing with Club Troppo in the coming few weeks. Ken and I have been less than impressed with our move to a Virtual Private Server. It has, in actual fact, probably been slower than the already ordinary performance we got from Dreamhost, though somewhat more reliable. It turns out that while our hosting provider boasted good prices, excellent uptimes, generous memory, disk and bandwidth allowances, they very savagely cap CPU usage. Club Troppo is running on the equivalent of 250MHz of processing power. There isn’t enough grunt to complete MySQL queries before another one comes along, which produces an exciting pile-up of queries and page generation jobs strangling the available CPU time.

As some of you know, before leaving Darwin I spent about a week doing nothing but optimise the site. We’re running on lighter software, with aggressive tuning on MySQL, with fewer plugins, with 3 different layers of caching. Even so, the site chokes up with regularity: if you’re unlucky to ask something from outside the caches, you’ll essentially throttle the entire website. So, as I said, less than impressive.

Some reflection on the data I gathered while investigating optimisation options suggests that our delays are entirely server-bound. Bandwidth is not our problem. Ours is plenty fast and plenty generous. The home page, which is usually our heaviest page, still comes in under 100kb fully loaded in most cases.

This suggested to me a new approach to which Ken has agreed in-principle: to move Troppo again. This time we will trade off bandwidth for performance. I have ordered business-grade ADSL service with static IP and improved service level guarantees, and I will host Club Troppo on a server in my possession. This will give us about twice as much RAM and approximately 10-15x more CPU performance, which should more than meet our demands. Having a physical dedicated server also allows me to perform some more esoteric optimisations which are not available for a VPS, such as moving MySQL’s temporary table directory into a RAMdisk, or recompiling the entire software stack with the ICC compiler, or a number of other options.

There is some urgency in this move. My nightmare is the oncoming Federal Election. Club Troppo’s traffic surged dramatically in the 2003 election. I expect that 2007 will be no different, and I would like the site to be settled into a new server with surplus capacity well before John Howard drives up to the Governor-General’s digs for a friendly chat about election writs.

It also happens that Club Troppo is growing again. Traffic has been growing steadily month on month every month, even through the ongoing technical travails we’ve experienced. It has been (yet another) humbling experience for yours truly. I took up the role as site administrator confident that I could lick it without breaking a sweat. Instead I have learned far more about the Wordpress-to-Linux stack than I care to mention, and under quite stressful and trying conditions. I have been several times tempted to modify the “Managed by Jacques Chester” footer to read “Mismanaged by Jacques Chester”.

In any case, I thank Ken and Nick for putting up with my advice, which has typically been wrong; and for putting up with my services, which have been less than perfect. I’d also like to thank all of you, the readership and authorship of Club Troppo. This is a great site and it just keeps getting better in spite of me. So stick around — you ain’t seen nothin’ yet!