Shared Hosting is Doomed (and I have the graphs to prove it)

My abiding and irrational loathing for WordPress has at last yielded fruit.

WordPress thrives in the classic shared hosting market, where the LAMP stack — Linux, Apache, MySQL, PHP — is almost universally installed. It’s free, fairly user-friendly, well-marketed, widely used, has oodles of third party plugins and themes. My only objection to WordPress is that it’s rubbish11. Har Har: Yes, I realise I’m saying this on a WordPress website. Sometimes the poor workman doesn’t get to choose his tools. [].

But that’s not my point today. What WordPress has given me is the impetus to think, and think, and think some more about what blogging software is and what it should do. The dozen or so computer types who follow Club Troppo have seen my sketches in this direction before. Starting with a pre-history of blogging, I moved through a call for a “next generation”, through to consideration of the various interested participants in the world of blogging software. Later I returned to the topic of WordPress to complain about its architecture, then foreshadowed this post with remarks about a PHP performance benchmark.

My topic today is a discussion of how and why I think shared hosting is doomed. Let me start with a chart which I think will attract no argument22. About the charts: All charts in this post are illustrative, not data-driven. So YMMAPWV. []:


This chart looks at the long tend trends of wages and hardware. In the long run, administrator wages rise. How and why this is so is unimportant for our discussion. Likewise, in the long run, hardware prices fall. This trend is essentially informed by Moore’s Law.

It follows naturally that when a firm enters the hosting market, administrator wages will form a larger and larger fraction of their costs per customer, and hardware a smaller and smaller portion. So whatever can reduce administrative overhead will reduce costs, and in a competitive market, prices for hosting.

One way to do this is to set up your business as a host for Virtual Private Servers. This has the nifty property of pushing a lot of the administration cost back on to those customers who are able to sustain it:

So far, so good. Except that virtualisation has a large resource overhead. Each customer is running another copy of the operating system, another copy of the web server, another copy of WordPress or whathaveyou.

Furthermore virtualisation introduces facilities for guaranteeing a minimum portion of resources per customer, meaning that it is no longer possible more difficult to oversell each piece of hardware.

Thus, the current state of things is somewhat like this:

In the current market, this difference is so large that it dominates the cost of virtual private server hosting. Consequently, VPS hosting costs more, despite the fact that much of the admin overhead is pushed onto the client. A well-known top-tier shared host like Dreamhost charges $7.95 per month with very generous allocations of disk space and bandwidth. Meanwhile top-tier VPS host Slicehost charges $20 per month for their smallest option, with a tiny fraction of the bandwidth or disk space offered by Dreamhost. And you have to run it yourself.

A slam dunk for the shared host, it would seem. Except, as we recall from the chart at the top, hardware continues to grow cheaper and cheaper. So what does this do to the resource overhead problem? Assuming that the resource overheads remain essentially constant, then both shared hosting and VPS hosting overheads per customer will converge towards zero:

Eventually, while there is still a difference, it doesn’t matter any more. It becomes irrelevant.

The same thing happened to RISC chips when AMD and Intel introduced instruction decoders that took the CISCy x86 instruction set, chopped it up, and reissued RISCy micro-ops to the internal architecture. At first this cost a significant portion of the total transistor budget, but as time passed, the transistor budget grew while the cost of the decoder remained roughly constant. So here we are, a decade later, not caring about instruction set overhead.

So the hardware overhead is destroyed by Moore’s Law in time. What about administrative overhead?

I submit that the administrative overhead per customer rises faster for a shared host than for a VPS host.

In a shared host, every customer is running inside the same operating system instance. Several thousand customers share the same machine. Scheduling is handled by a single kernel, any program has access to the full resources of the machine, and so on.

But contrariwise, many more things can go wrong. Any customer could introduce badly-behaved software which gobbles RAM or hammers the CPU. Or software run by different customers may lead to an unexpected interaction. Or a customer might installed software with a security flaw, leading to the exploitation of the machine’s resources. And so on, and so forth.

Every customer adds a more than linear amount of risk and complexity to the host’s administration overhead. In a small shared host this isn’t a problem; in a large host it can significantly raise the cost per machine, largely negating the advantages of lower resource overhead.

Contrariwise, a VPS host has a fixed administrative cost per customer, because to them the customer’s virtual machine is administratively an opaque blob. They needn’t care what the customer is running, as the virtualisation software strictly isolates each customer from another. Without interactions, problems are isolated. And so the administrative overhead per customer is strictly linear.

Here’s what that looks like, in a dramatic-looking graph:

Once Moore’s Law has rendered resource overheads moot, this graph spells the doom of shared hosting.

Or does it? The canny reader has already noticed that I have forgotten that one of the tricks of the VPS host is to push a large portion of the administrative cost onto the customer. For a lot of customers, this just won’t fly. The average WordPress user is not a Linux administrator, MySQL corraller or PHP-hater. They are likely to be like the Club Troppo roster: lawyers, economists, thinkers, writers: in short, people who have neither the ability nor the interest to run their own server. That is, in large part, why I was brought into the Club Troppo tent.

This could represent a hard upper limit to the portion of the market which VPS hosts can capture.

The way around this impasse is the so-called “virtual appliance” — the VApp. These are essentially a prebuilt server designed to be deployed to a VPS host. They are configured with the desired software and carefully assembled by professionals. In the VApp model, administrative overhead is moved from the user and from the host to the upstream:

This allocation already exists, to some degree, in the world of Linux distributions. Each distribution assembles a large amount of software, and includes a lot of administrative know-how in the layout of files, default configurations, security policies and so on. The VApp, which adds some end-user software, merely expands on the concept of a distribution.

Already you can download VPS images for popular opensource software like WordPress, Drupal and the like — turnkey VApps. I speculate that eventually VPSes will become the target for software projects, rather than traditional shared hosts. This will allow a lot more flexibility in design, as it will allow the application to become decoupled from the LAMP stack at will, and to use operating system facilities in a way which are currently impossible. For example, application plugins could be run as separate operating system users, with all the security advantages obtained thereby.

Not everyone agrees with my thinking, mostly because of the current realities of the market. But this is an argument about the future, and the way that inexorable trends can reshape an industry without much consideration for who gets squished. To my mind, the twin trends of inflating wages and deflating hardware spell the end of shared hosting as the main way in which people publish websites. Before 10 years have passed, we’ll get to know if I was right.

31 thoughts on “Shared Hosting is Doomed (and I have the graphs to prove it)

  1. Wes;

    Good point, but I’ve sort of skipped over the differences as I don’t think they alter the substance of my argument.

    On the other hand, I’m getting slaughtered in comments at Reddit :)

  2. Google app engine doesn’t look anything like a VPS, though I think it’s more in line with the future. It’s closer to a shared hosting model, but leverages commodotized infrastructure and a small support staff.

  3. Phil;

    I used OmniGraffle and Numbers.

    Mike;

    I admit that I’ve not really looked too hard at GAE, but the economics are probably similar. Even more so for EC2.

  4. As an administrator at a hosting company with both shared and VPS accounts, I’m in agreement with your premise.

    By and large, shared customers are very needy. They want their hands held and tie up our front line support. VPS and dedicated customers on the other hand only ask for assistance when something has gone wrong and are usually more knowledgeable than their shared counterparts.

    I’ve used pre-built virtual machines like JumpBox before and they are probably the future. I can’t put 1000 virtualized customers on a box now (can’t even hit 50 Virtuozzo customers) but in the future if 2-3TB of memory starts to come standard on a machine, that could make it a reality.

  5. The bit missing from the equation would seem to be software. What if wordpress etc evolve and become much more flexible. They you can have a blog without having to know anything about the LAMP stack or any other OS. In fact the public wordpress site has already done this to a large extent relative to the hosted PHP bulletin boards that were all the rage about 5 years ago. Bloggers want to blog and look good. They generally don’t want to fiddle with MySQL and Linux or any other OS.

  6. An interesting prognostication, Jaques. I’ve had my share of shared host blues, but VPSs and VPMs are still beyond the budget of many small developers. I’m sure the situation will have evolved almost beyond recognition in another decade as we all move into the Cloud or whatever.

    WordPress may be bad, but nothing is as bad as BLOGGER:-)

  7. The comments at reddit pretty hilariously miss the point :-)

    Your graph assuming that the cost of hardware and hence the overhead of a VM will continue to plummet assumes that the software required in every VM doesn’t continue to grow. There’s a corollary to Moore’s law that says something like “software gets slower faster than hardware gets faster”; while that hasn’t entirely been true, I think it might be enough of an effect to slow down the exponential decline.

    Another issue you haven’t addressed is IP address space. IP addresses are not getting any cheaper and while every VM instance is typically assigned its own static IP address, any number of shared hosting customers can coexist easily on a single IP. This isn’t insurmountable – NAT exists – but does mean a significant change from the current VM hosting plans that are commonly offered today.

    The other point is that it’s always going to be far more efficient to have an application designed to host multiple users in a single operating system instance than it is to have. E-mail is now enough of a commodity that you can host arbitrarily many customers and domains on a single server without worrying about the security ramifications. Likewise for static web sites. I am envisioning that web applications can evolve to become something like flickr or livejournal with a single installation of the application hosting many users’ sites. If the source code was freely available, any web host can go and offer such a service, and provide the kinds of features that make WordPress more attractive than Blogger or Livejournal – more control over your own data, the capability to host your own domain, etc. Of course, the trade off for this is losing the flexibility to run arbitrary code on the hosted domain; for the “I don’t want to be a sysadmin” crowd, which is the majority of shared hosting customers, it may be worth it.

  8. Yeah, I did overlook that. But I think that the ratios of “Microsoft taketh away” will still remain fairly constant, barring miraculous breakthroughs in virtualisation technology.

    Hosted packages (aka SaaS) already exist and I think that’s the other end of the market that will squeeze out shared hosting. But a lot of people want more control over their stuff, which means shared hosts and VPSes.

    I never thought about IP space. My instinct is that in practice it will turn out to be orthogonal anyway — I’d be surprised to find a large shared host who isn’t doing some degree of NAT.

    The comments at Hacker News were better, including one guy with the same objection as yours.

  9. @Seth: You need to consider how the VPS-user demographics will change as the userbase grows. Sure, right now VPS users tend to be the “top 10%” in terms of being able to run their own systems and require little support. If/as VPSes displace shared hosting, though, as the grandparents start to go that way, the needy users would require support for the larger software base that they are now capable of screwing up. I think administering such systems would quickly become more of a pain than what we have now.

    @Jacques: You’re on to something, especially with the addition of comment 11. What might actually happen is a split between the Facebook/Myspace kind of environment and VPSes. In one, changing your background is the limit of customization; in the other, you can change everything down to the OS. Once someone leaves the first world, they’re likely to leap right over shared hosting to VPSes. Maybe the grandparents won’t start using VPSes after all. Maybe VPSes will remain a relatively small niche dominated by the technical elite, as shared hosting gets eaten up *from the other end*. I’ve already noticed a fair number of people who I know are capable of setting up their own host in their own domain instead choosing to use services like wordpress.com or Blogger just because they can’t be bothered. The question then becomes what will happen to either shared-hosting or VPS prices as overall volume for both combined decreases.

  10. Of course, Jeff, you’d like us all to move to single machines with 5,832 processors I’m guessing ;)

  11. I haven’t got any figures here but I’d suggest your first graph is wrong on Costs of Admin Staff. I’d provocatively suggest that the costs for the sort of people you are talking about over the last 10 years have been a pretty flat flat curve. But then people costs are recurrent – hardware somewhat less so.

  12. No, Jacques, that would mean more work for me. :p Seriously, though, I don’t consider myself to have a horse in this race. While an SC5832 might be able to replace quite a bit of hosting-service gear and save quite a bit of power in the process, it would be unappealing for other reasons (e.g. less favored ISA and Linux distro, initial unavailability of some packages popular in that market) and many of its most unique capabilities (e.g. strong floating point and very fast internode communication) would largely be wasted. I was just making my observations as a minor-league blogger and general technophile.

  13. Jeff;

    I know, just joshing. I’m jealous of where you work, it must be interesting. If I was Johnathan Schwartz I’d be out the front of your office banging loudly on the door, waving a blank cheque.

    I reckon the untapped “mainstream” market for sicortex gear will be in MMORPGs. You read it here first.

  14. I’d argue that most of your graphs are wrong.

    Server admin work can easily be tiered, so while it might take some additional expertise to setup a shared host on day one, the day-to-day operations can be farmed out so someone much more junior. If a particular day-to-day operation is difficult then the software gets improved to better service that operation resulting in lower admin costs over time. As the package (e.g. WordPress) gets more mature, both the rollout and day-to-day admin get smoother and costs are further reduced. In many ways, the VApp is just one example of what happens normally in any software lifecycle rather than anything magical about VMs. An additional factor is that computer administration is easy to learn (and getting easier) and the tools you need are getting cheaper, the barriers to entry are being removed (cheap PCs, cheap Internet, downloadable documentation). The only thing keeping wages high in this sector is that the market is growing faster than new skilled staff are entering the market — this won’t last.

    Further, the virtualisation technology also matures so the overhead of the virtual server gets smaller over time. Consider that hardware VM support is going to become standard. Look what happened with 3D graphics.

    As for the complexity explosion caused by more customers, that only happens if the customers are doing something unique and unusual. If all the WordPress customers want to basically run the same type of application and none of them do anything special, then complexity is pretty much flat (actually it gets cheaper to manage because you can afford to get an admin who specialises in that particular application, and spread the cost of their expertise over a larger group).

    One example of this is DNS hosting and management. Ultimately there’s only a limited number of things that you would want to do with a DNS database. Once suitable management software is written for those operations, why would anyone bother trying to do something special? Can you imagine what it would be like if everyone buying DNS hosting had to admin a virtual machine? Even a VApp is not going to compete in such a space.

    The place where VMs will compete is when customers want something a bit different from the other customers so they don’t want the standard app, or they want to start with a standard app and then heavily customize. This is the whole “long tail” market for people who get sick of working within the limitations of the mainstream. Notice that these people will START with a mainstream situation (on a shared host) and then outgrow it… so there will always be shared hosts for the newbies and there will always be hoards of newbies.

  15. A side note. I suggest you count the number of RISC and CISC CPUs in your house, the respective counts will probably make you reconsider your analogy.

  16. Interesting thinking.

    As a Solaris-weeny I have to mention Solaris Zones as a mid-way virtualised environment which don’t have the overhead of VMWare, and therefore could be useful in the near-term. That is, “VApps” in (Open)Solaris Zones.

    As a most learned colleague at RomanSandals once said to me though, “complexity is always preserved”. In your model the complexity is moved upwards towards the maintainers of the VApps, and downwards towards the customer.

  17. Craig;

    Sure, but total administrative load for everyone goes down. Compare a world without Debian to what we live in.

    Solaris has some nifty technologies, and Zones is something I’ve looked at a bit. But the hardware support is lagging too much :/

  18. I can certainly attest to your central point. Running a WordPress site is a huge problem for me, and I’d imagine I’m at least as comfortable with computers as the average blogger. I’m supposed to migrate from shared hosting to an accelerator this weekend (not precisely sure what this means, but hopefully better service) and even with a promise of some handholding, I’m viewing it with trepidation.

  19. Nice piece. Speaking from the perspective of specialist project hosting for Subversion and collaboration tools (falls halfway between VPS and shared web hosting), a provider is able to deliver the same service to thousands of people, which becomes highly scripted and efficient. This results in administrative costs really flattening. A little data- our typical 10-person customer (at CVSDude) spent 2,350 hours each year maintaining the open source tools that we offer (we learned this from 241 survey responses earlier this year), while often not doing the offsite backups and redundancies they should be. So outsourcing saves huge amounts of time and money (it’s cheap and provides instant access to security best standards). So I agree with your graph of hardware and administrative costs as they relate to shared web-hosting- it will be hard for shared web hosting providers to compete with VPS, ASSUMING that customers will have the time (and skills) to configure and manage their sites/apps as well as write the code. However I think that over the next 10 years, there will be a proliferation of hosting providers of all types of services in all industries, delivering SaaS to thin client users. This shift from a product to service oriented market has the benefit that industries will undergo a fairly serious ramp-up in productivity and profitability as maintenance costs decline. The (partial) losers are software companies who will no longer be able to sell their products at such high margins- but this is inevitable as the web brings transparency and openness to all.

  20. Working on mainframes I have a bit of a look into the future when it comes to the debate between VPS and shared hosting. That sounds a bit weird but makes sense when you realise that mainframes have always been shared environments running lots of applications in both production and development environments on the same hardware. For example sysplex’s (the mainframe equivalent of VPS) have been a standard part of all mainframe shops for decades.

    First point: It is possible to have a shared hosting environment where user applications cannot screw up the server. Most mainframes have at least a dozen different applications on them all running in the same shared environment. Despite the fact that a mainframe could have hundreds of programmers writing and testing thousands of different new and buggy application programs on the same shared host at the same time, nobody ever worries that one of these programs will crash the mainframe. It basically never happens. If mainframes can produce a shared hosting environment that is uncrashable by user apps then it is possible that Linux/Unix can do it as well.

    With that in mind I am not sure the below point about shared hosts will always be valid.

    But contrariwise, many more things [on a shared host] can go wrong. Any customer could introduce badly-behaved software which gobbles RAM or hammers the CPU. Or software run by different customers may lead to an unexpected interaction. Or a customer might installed software with a security flaw, leading to the exploitation of the machines resources. And so on, and so forth.

    I have heard it said that the reason that mainframes can do this and Unix doesn’t is that the mainframe security system (RACF) is better than Unix’s so maybe it really can’t be done in Unix. But if the amount of VPS’ starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

    A second observation is that the amount of server admin work seems to be largely proportional to the number of sysplex’s (one sysplex = one VPS). Each sysplex needs to have its OS and DBMS monitored, and upgraded seperately (upgrades are a huge amount of the workload). So on most mainframes you might have 10 or 30 applications, but usually only two or three sysplex’s. Its just too much work to keep lots of sysplex’s up to date with the latest patches and make sure they are all running smoothly. Tell me which is easier three MySQL upgrades for three VPS’s each with 10 shared users (in a bulletproof mainframelike environment) or 30 MySQL upgrades for each of 30 seperate VPS’ for 30 users? I know at the moment that a VPS can pass most of its admin work onto the users but that is not going to be the case forever.

  21. We’re just starting to use virtual machines at work, although I don’t have too much to with it directly. It’s really effective for maximising hardware use (and saving power), as a lot of servers have heaps of idle time.

    swio, I’m not sure you’d need to do software upgrades for each virtual machine, as you’d just point them at the same disk for software. (I may be mistaken here.) File systems on mainframes are somewhat different to those on PCs aren’t they? In any case, in a Windows environment at least, software upgrades to multiple machines are easily automatable with packages like ManuSoft.

  22. I have heard it said that the reason that mainframes can do this and Unix doesnt is that the mainframe security system (RACF) is better than Unixs so maybe it really cant be done in Unix. But if the amount of VPS starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

    You can do a lot, but Unix still has a simpler security and resource allocation model than a lot of other OSes. Different Unices often have extensions — such as jails, trusted extensions etc — but in practice software authors don’t use them because they aren’t present everywhere.

    Tell me which is easier three MySQL upgrades for three VPSs each with 10 shared users (in a bulletproof mainframelike environment) or 30 MySQL upgrades for each of 30 seperate VPS for 30 users? I know at the moment that a VPS can pass most of its admin work onto the users but that is not going to be the case forever.

    First, this is still a problem with shared hosts. When user A installs a buggy version of some PHP bulletin board software and doesn’t update it, you have a problem. If there’s a flaw in the underlying permissions, then everyone on that server now has a problem. Isolation is much weaker than the VPS scenario, which is why my graph talks about complexity and risk. Risk is about probability and consequences, both of which rise as more customers are added to the shared host.

    But you are right about the total load. However, my theory remains that virtual apps reduce total complexity for both host and client by moving it upstream to the VApp provider. They patch their image and it moves downstream to the users.
    Done properly this would be more secure, especially as a VApp can be designed to fully utilise operating system facilities rather than having to recreate them internally.

    Take WordPress, for instance. Here are some things it can’t do, because it has to be limited enough to run on a shared host:

    1. No cron. WordPress can’t run regular jobs through the cron system. Instead it has to bodgie up its own cron-alike system which relies on a PHP file being loaded and run every time a page is viewed, on the hope that a page view will occur around a scheduled time. Yuck.
    2. No isolation of plugins. Any plugin can see and mess up any data in any MySQL table because PHP has no way of isolating the database handler into different users. Every plugin has something like ‘root’ status over all of your data. Yuck.

    It goes on and on. Current web applications are coded to the LAMP stack, which is very common but omits key operating system tools — such as process management, security settings etc. A VApp can be architected to tightly target a particular OS and use features provided. On Linux it could use PAM for authorisation, on FreeBSD it would use Jails, on Solaris ZFS and Zones. And so on and so forth.

  23. If mainframes can produce a shared hosting environment that is uncrashable by user apps then it is possible that Linux/Unix can do it as well.

    Linux has a project where people execute chunks of random bytes just to see what happens, several Intel CPU hardware bugs have been found this way. It is possible that some extremely complex CPU bug requires a long setup sequence that would take too long to find by random search, but then probably no one will ever find such a bug.

    I have heard it said that the reason that mainframes can do this and Unix doesnt is that the mainframe security system (RACF) is better than Unixs so maybe it really cant be done in Unix. But if the amount of VPS starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

    There’s an explanation of “ulimit” here:

    http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm

    There are also extended security options (for those who want it) in the form of AppArmor and SELinux, which are part of a broader system of pluggable security modules (so you can write your own plug-in security policy it you want). You can even use the same security system to merely keep a log of what happens (without interference) so at least when something goes wrong, you can find out what caused it. In other words, there is really no limit to the fine-grained level of security that a BOFH can impose on other users.

    I’ll point out that it is almost certain that mainframes also have security loopholes somewhere in them, just less people are searching so less gets found (and things like System/370 are old and have been debugged over time, while the Linux/Intel platform is a fast moving target). It is a big mistake to believe that just because a system is proprietary, it is somehow more secure and/or more robust. It is also a big mistake to believe that complex security models give better protection than simple security models. In my experience, big complex machines tend to fail more often than simple machines.

    The main reason the extended security options in Linux don’t often get used is because of the admin overhead (and the plug-in security modules also necessarily increase CPU overhead). Complex security models require detailed understanding to use correctly, and are more likely to be misunderstood.

    Different Unices often have extensions such as jails, trusted extensions etc but in practice software authors dont use them because they arent present everywhere.

    The SELinux extension does not require any support from the app itself, it just sits and watches what the app does, maybe hits it on the head if it does the wrong thing. The restriction on cron jobs seems like it is designed for badly administered machines, there is no reason why individual users of a machine shouldn’t run cron jobs (and the paranoid sysadmin can use SELinux to nail down what those cronjobs may do, or just trust the normal user-level unix security).

    Apache suexec is kludgy and brain-dead. Newer versions of apache offer different unix users for different virtual hosts (which seems obvious but only recently has anyone bothered implementing it). PHP was never really designed for security from the outset but it is also getting better, if the unix user that PHP runs under is properly isolated and the MySQL user is also isolated, then the most anyone can do is bash their own data (that’s what you have backups for). I suspect there’s a fairly large number of app writers who don’t even really understand the basic unix security model, let alone other more complex models. Also, the app writers want to pander to every possible admin out there (which can never work).

  24. Tel;

    All good points. If I could summarise in a phrase why VApps can be better designed, it’s that traditional LAMP apps can only code to the lowest common denominator. So no SELinux, no jails, etc.

    In fact they generally wind up with a bad case of Inner Platform Syndrome.

  25. Pingback: andy.edmonds.be › links for 2008-07-23

  26. Pingback: A Strong SaaS Argument: Labor costs will continue to rise as hardware costs go down « Entrepreneur Musings - David Cummings

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.