All good points. If I could summarise in a phrase why VApps can be better designed, it’s that traditional LAMP apps can only code to the lowest common denominator. So no SELinux, no jails, etc.

In fact they generally wind up with a bad case of Inner Platform Syndrome.

If mainframes can produce a shared hosting environment that is uncrashable by user apps then it is possible that Linux/Unix can do it as well.

Linux has a project where people execute chunks of random bytes just to see what happens, several Intel CPU hardware bugs have been found this way. It is possible that some extremely complex CPU bug requires a long setup sequence that would take too long to find by random search, but then probably no one will ever find such a bug.

I have heard it said that the reason that mainframes can do this and Unix doesn’t is that the mainframe security system (RACF) is better than Unix’s so maybe it really can’t be done in Unix. But if the amount of VPS’ starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

There’s an explanation of “ulimit” here:

http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm

There are also extended security options (for those who want it) in the form of AppArmor and SELinux, which are part of a broader system of pluggable security modules (so you can write your own plug-in security policy it you want). You can even use the same security system to merely keep a log of what happens (without interference) so at least when something goes wrong, you can find out what caused it. In other words, there is really no limit to the fine-grained level of security that a BOFH can impose on other users.

I’ll point out that it is almost certain that mainframes also have security loopholes somewhere in them, just less people are searching so less gets found (and things like System/370 are old and have been debugged over time, while the Linux/Intel platform is a fast moving target). It is a big mistake to believe that just because a system is proprietary, it is somehow more secure and/or more robust. It is also a big mistake to believe that complex security models give better protection than simple security models. In my experience, big complex machines tend to fail more often than simple machines.

The main reason the extended security options in Linux don’t often get used is because of the admin overhead (and the plug-in security modules also necessarily increase CPU overhead). Complex security models require detailed understanding to use correctly, and are more likely to be misunderstood.

Different Unices often have extensions — such as jails, trusted extensions etc — but in practice software authors don’t use them because they aren’t present everywhere.

The SELinux extension does not require any support from the app itself, it just sits and watches what the app does, maybe hits it on the head if it does the wrong thing. The restriction on cron jobs seems like it is designed for badly administered machines, there is no reason why individual users of a machine shouldn’t run cron jobs (and the paranoid sysadmin can use SELinux to nail down what those cronjobs may do, or just trust the normal user-level unix security).

Apache suexec is kludgy and brain-dead. Newer versions of apache offer different unix users for different virtual hosts (which seems obvious but only recently has anyone bothered implementing it). PHP was never really designed for security from the outset but it is also getting better, if the unix user that PHP runs under is properly isolated and the MySQL user is also isolated, then the most anyone can do is bash their own data (that’s what you have backups for). I suspect there’s a fairly large number of app writers who don’t even really understand the basic unix security model, let alone other more complex models. Also, the app writers want to pander to every possible admin out there (which can never work).

I have heard it said that the reason that mainframes can do this and Unix doesn’t is that the mainframe security system (RACF) is better than Unix’s so maybe it really can’t be done in Unix. But if the amount of VPS’ starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

You can do a lot, but Unix still has a simpler security and resource allocation model than a lot of other OSes. Different Unices often have extensions — such as jails, trusted extensions etc — but in practice software authors don’t use them because they aren’t present everywhere.

Tell me which is easier three MySQL upgrades for three VPS’s each with 10 shared users (in a bulletproof mainframelike environment) or 30 MySQL upgrades for each of 30 seperate VPS’ for 30 users? I know at the moment that a VPS can pass most of its admin work onto the users but that is not going to be the case forever.

First, this is still a problem with shared hosts. When user A installs a buggy version of some PHP bulletin board software and doesn’t update it, you have a problem. If there’s a flaw in the underlying permissions, then everyone on that server now has a problem. Isolation is much weaker than the VPS scenario, which is why my graph talks about complexity and risk. Risk is about probability and consequences, both of which rise as more customers are added to the shared host.

But you are right about the total load. However, my theory remains that virtual apps reduce total complexity for both host and client by moving it upstream to the VApp provider. They patch their image and it moves downstream to the users.
Done properly this would be more secure, especially as a VApp can be designed to fully utilise operating system facilities rather than having to recreate them internally.

Take Wordpress, for instance. Here are some things it can’t do, because it has to be limited enough to run on a shared host:

1. No cron. Wordpress can’t run regular jobs through the cron system. Instead it has to bodgie up its own cron-alike system which relies on a PHP file being loaded and run every time a page is viewed, on the hope that a page view will occur around a scheduled time. Yuck.
2. No isolation of plugins. Any plugin can see and mess up any data in any MySQL table because PHP has no way of isolating the database handler into different users. Every plugin has something like ‘root’ status over all of your data. Yuck.

It goes on and on. Current web applications are coded to the LAMP stack, which is very common but omits key operating system tools — such as process management, security settings etc. A VApp can be architected to tightly target a particular OS and use features provided. On Linux it could use PAM for authorisation, on FreeBSD it would use Jails, on Solaris ZFS and Zones. And so on and so forth.

swio, I’m not sure you’d need to do software upgrades for each virtual machine, as you’d just point them at the same disk for software. (I may be mistaken here.) File systems on mainframes are somewhat different to those on PCs aren’t they? In any case, in a Windows environment at least, software upgrades to multiple machines are easily automatable with packages like ManuSoft.

First point: It is possible to have a shared hosting environment where user applications cannot screw up the server. Most mainframes have at least a dozen different applications on them all running in the same shared environment. Despite the fact that a mainframe could have hundreds of programmers writing and testing thousands of different new and buggy application programs on the same shared host at the same time, nobody ever worries that one of these programs will crash the mainframe. It basically never happens. If mainframes can produce a shared hosting environment that is uncrashable by user apps then it is possible that Linux/Unix can do it as well.

With that in mind I am not sure the below point about shared hosts will always be valid.

But contrariwise, many more things [on a shared host] can go wrong. Any customer could introduce badly-behaved software which gobbles RAM or hammers the CPU. Or software run by different customers may lead to an unexpected interaction. Or a customer might installed software with a security flaw, leading to the exploitation of the machine’s resources. And so on, and so forth.

I have heard it said that the reason that mainframes can do this and Unix doesn’t is that the mainframe security system (RACF) is better than Unix’s so maybe it really can’t be done in Unix. But if the amount of VPS’ starts increasing dramatically there is going to be increased incentive to get shared hosting working properly.

A second observation is that the amount of server admin work seems to be largely proportional to the number of sysplex’s (one sysplex = one VPS). Each sysplex needs to have its OS and DBMS monitored, and upgraded seperately (upgrades are a huge amount of the workload). So on most mainframes you might have 10 or 30 applications, but usually only two or three sysplex’s. Its just too much work to keep lots of sysplex’s up to date with the latest patches and make sure they are all running smoothly. Tell me which is easier three MySQL upgrades for three VPS’s each with 10 shared users (in a bulletproof mainframelike environment) or 30 MySQL upgrades for each of 30 seperate VPS’ for 30 users? I know at the moment that a VPS can pass most of its admin work onto the users but that is not going to be the case forever.

Sure, but total administrative load for everyone goes down. Compare a world without Debian to what we live in.

Solaris has some nifty technologies, and Zones is something I’ve looked at a bit. But the hardware support is lagging too much :/

As a Solaris-weeny I have to mention Solaris Zones as a mid-way virtualised environment which don’t have the overhead of VMWare, and therefore could be useful in the near-term. That is, “VApps” in (Open)Solaris Zones.

As a most learned colleague at RomanSandals once said to me though, “complexity is always preserved”. In your model the complexity is moved upwards towards the maintainers of the VApps, and downwards towards the customer.

Server admin work can easily be tiered, so while it might take some additional expertise to setup a shared host on day one, the day-to-day operations can be farmed out so someone much more junior. If a particular day-to-day operation is difficult then the software gets improved to better service that operation resulting in lower admin costs over time. As the package (e.g. Wordpress) gets more mature, both the rollout and day-to-day admin get smoother and costs are further reduced. In many ways, the VApp is just one example of what happens normally in any software lifecycle rather than anything magical about VMs. An additional factor is that computer administration is easy to learn (and getting easier) and the tools you need are getting cheaper, the barriers to entry are being removed (cheap PCs, cheap Internet, downloadable documentation). The only thing keeping wages high in this sector is that the market is growing faster than new skilled staff are entering the market — this won’t last.

Further, the virtualisation technology also matures so the overhead of the virtual server gets smaller over time. Consider that hardware VM support is going to become standard. Look what happened with 3D graphics.

As for the complexity explosion caused by more customers, that only happens if the customers are doing something unique and unusual. If all the Wordpress customers want to basically run the same type of application and none of them do anything special, then complexity is pretty much flat (actually it gets cheaper to manage because you can afford to get an admin who specialises in that particular application, and spread the cost of their expertise over a larger group).

One example of this is DNS hosting and management. Ultimately there’s only a limited number of things that you would want to do with a DNS database. Once suitable management software is written for those operations, why would anyone bother trying to do something special? Can you imagine what it would be like if everyone buying DNS hosting had to admin a virtual machine? Even a VApp is not going to compete in such a space.

The place where VMs will compete is when customers want something a bit different from the other customers so they don’t want the standard app, or they want to start with a standard app and then heavily customize. This is the whole “long tail” market for people who get sick of working within the limitations of the mainstream. Notice that these people will START with a mainstream situation (on a shared host) and then outgrow it… so there will always be shared hosts for the newbies and there will always be hoards of newbies.

I generalised from the property that wages rise in general. Inflation vs deflation, if you like.

I know, just joshing. I’m jealous of where you work, it must be interesting. If I was Johnathan Schwartz I’d be out the front of your office banging loudly on the door, waving a blank cheque.

I reckon the untapped “mainstream” market for sicortex gear will be in MMORPGs. You read it here first.

@Jacques: You’re on to something, especially with the addition of comment 11. What might actually happen is a split between the Facebook/Myspace kind of environment and VPSes. In one, changing your background is the limit of customization; in the other, you can change everything down to the OS. Once someone leaves the first world, they’re likely to leap right over shared hosting to VPSes. Maybe the grandparents won’t start using VPSes after all. Maybe VPSes will remain a relatively small niche dominated by the technical elite, as shared hosting gets eaten up *from the other end*. I’ve already noticed a fair number of people who I know are capable of setting up their own host in their own domain instead choosing to use services like wordpress.com or Blogger just because they can’t be bothered. The question then becomes what will happen to either shared-hosting or VPS prices as overall volume for both combined decreases.

Hosted packages (aka SaaS) already exist and I think that’s the other end of the market that will squeeze out shared hosting. But a lot of people want more control over their stuff, which means shared hosts and VPSes.

I never thought about IP space. My instinct is that in practice it will turn out to be orthogonal anyway — I’d be surprised to find a large shared host who isn’t doing some degree of NAT.

The comments at Hacker News were better, including one guy with the same objection as yours.