I was asked to co-present with an engineer from Sun at an upcoming conference in October. I asked him to do his slides and then shoot me over the presentation so I could fill in my half. I noticed that his view of virtualization and mine were very different. To put it into jargon speak, there is a difference between Redshift virtualization and Blueshift virtualization.

So what is this redshift/blueshift stuff about? Redshift is a theory created by Sun Microsystems CTO Greg Papadopoulos. Essentially it says that there are two different classes of business: “blueshift” companies that grow according to GDP and are essentially over-served by Moore’s Law that computing power doubles every two years, and “redshift” companies that grow off the charts, and which are grossly under- served by Moore’s Law.

At Joyent, we have both kinds of customers, but we are really born to serve the “redshift” market. When your site is in development, you want to save on cost. But when you open up and start to explode like a Twitter or FaceBook, you join the redshift elite – and suddenly cost falls away as a concern, replaced with a vicious need to scale to meet the demands. When users pour into your site, the ability to scale is the difference between life and death, between getting rich and updating your resume.

Virtualization is rarely properly distinguished between these redshift and blueshift environments, but the concepts are old ones: horizontal scaling (more systems) and vertical scaling (bigger systems). And here’s the rub, most vendors aim virtualization at the latter category (blueshifting), vertically scaling customers under the banner of “consolidation”; buy bigger systems that do more to fight the ills of under-utilization.

Of course, this matter of “under-utilization” is itself a tricky subject. Vendors show utilization graphs that demonstrate how most of your systems are sitting idle in racks, and that by consolidating into a larger virtualized system you can increase utilization dramatically. But is that what customers really want? Any sysadmin will tell you that if there is a load average above 1.0, users freak out. I’ve dubbed this phenomenon “pay for idle”; even in a consolidated environment, customers don’t want the system to do a lot of work.

And why? For excess capacity to meet the demands of the future. Customers want a 4-core Opteron system with 8GB of memory not because they need it, but because when Slashdot or Digg pays them a visit they might need it. To be frank, this is human nature and not restricted to computing… when you buy a truck you buy a truck that can haul the largest thing you can imagine ever needing to haul; otherwise sales of half-ton pickups wouldn’t be as high as they are, despite the fact that most are used to pick up groceries. People buy for what they may need at some point, not what they do need now.

Consolidation might increase utilization, but it doesn’t address human nature.

The answer is virtualization, but in two important forms: 1) horizontally consolidated environments, and 2) portability. That is, spread your application across several systems but in such a way that you can move any individual environment around to meet changing demands over time. Put into storage terms, RAID your application and get a vendor guarantee that you can put those disks into a bigger chassis when the need arises. These are the sorts of things we commonly attribute to disk solutions, but not to applications. Would a responsible company ever consolidate 100GB disks into a big 500GB disk? Of course not, we use a lot of small(er) disks in a RAID to enjoy the benefits of both increased protection and performance.

But consolidation is only part of the real promise of virtualization. Portability is key – the ability to duplicate (clone) environments quickly, to spread them over and increasingly large set of resources (physical servers), and to re-allocate resources seamlessly to meet the demands of a changing environment.

When you look at most VPS providers in the market today, they tend to only address convenient aspects of this formula. Solutions based on Xen, such as EC2, allow you to create additional clones of your application environment (horizontally), but not to re-size individual environments based on need (vertically). This isn’t because they can’t, but because it’s easier not to. Furthermore, many solutions out there leave management of the most critical resource – your load balancer – to you, rather than transparently managing a robust solution on your behalf.

True virtualization is about turning “servers” into “application environments” and treating those environments as building blocks that are easily duplicated, dynamically sized, and totally portable. That’s redshifting, and that’s what Joyent does best.