Comparing Triton containers to VMs and bare metal servers
Triton containers are new, but the technology, including the container hypervisor and container runtime, has been used in production by Fortune 500 companies for over ten years. Containers are at the heart of Joyent's cloud offerings, and they're the fundamental technology that allows our users to run their workloads securely on bare metal. But, how they work and how they differ from the alternatives deserves a full explanation.
Virtual machines have defined cloud infrastructure for so long that it's easy to forget there's an alternative. That's exactly what our containers are: an alternative to VMs that offer all the features you expect of VM infrastructure, without the performance limits you've struggled with for too long now.
Triton containers come in three types:
- Infrastructure containers running container-native Linux and SmartOS that work like a typical Unix host.
- Docker containers running your favorite Docker images.
- Containers hosting hardware virtual machines running Windows, FreeBSD, or others. These don't offer the same performance advantages as our other offerings, but they offer compatibility for workloads that can't be fully containerized.
Fully equipped
Every container comes fully equipped with everything you'd expect of a virtual machine, plus features that only containers can offer:
- Your choice of software, including your choice of popular Linux distros, popular and obscure applications, and Docker images.
- Bare metal performance that only containers can offer.
- Dedicated CPU shares, plus the opportunity to burst above that CPU share if capacity would otherwise go unused1.
- Dedicated memory.
- SSD-optimized, high-speed local RAID storage.
- One or more network interfaces.
- Free inbound and 1GB outbound bandwidth (additional bandwidth available).
- Resource isolation that ensures each container gets its share and prevents noisy neighbors from ruining the day for others.
- Resource pooling that allows containers to burst with CPU that would otherwise go unused and accelerates filesystem I/O with unused RAM.
- Tested and certified security isolation that matches that offered by virtual machines.
If you approach containers as alternatives to VMs, and that's what we did when we started using containers at Joyent almost ten years ago, this list starts out with what you'd expect. It's the CPU bursting and resource pooling that might look out of place, and the bare metal performance that you have to experience to believe. If, instead, you're comparing Triton containers to Docker, you'll see a number of differences that largely define our bare-metal, container-native advantages.
Something borrowed, something new
Containers have been at the heart of Joyent's offerings since 2006. What's new here is our support for container-native Linux, including Certified Ubuntu for Triton, and our native support for Docker containers on bare metal.
Joyent has refined the underlying technology behind Triton containers, adding features from our experience using them on multi-tenant bare metal. Some of those features include resource isolation that ensures every one of your containers gets the performance you're paying for, while also allowing you to enjoy the unused resources from others.
Compared to virtual machines and bare metal
If you have to trade performance for elasticity on VM infrastructure (and elasticity for performance on typical bare metal), you'd think you'd have to pay a premium for Triton. Well, this might be a surprise....
Here's the smallest instance type offered by the leading VM and bare metal server infrastructure providers:
| Instance type | EC2 t2.micro | Triton G4 standard 128M | Single CPU Bare Metal Hourly |
|---|---|---|---|
| Vendor | AWS | Joyent | Softlayer |
| Runs your choice of Linux | Yes | Yes | Yes |
| Memory | 1GB | 128MB | 8GB |
| CPU shares/cores (with hyperthreading) | 0.1 | 0.125 | 8 |
| Price/month | $9.36 | $2.30 | $334.80 |
| Price/GB | $9.36 | $18.43 | $41.85 |
| Price/CPU | $93.60 | $18.43 | $41.85 |
| Provisioning time | 30-60 seconds | 15-30 seconds | 20-30 minutes |
| Bare metal performance | No | Yes | Yes |
| CPU bursting | Limited | Yes | No |
| Live resize | No | Yes | No |
Can you believe AWS really charges that much for 1/10 of a CPU share?
Here's the comparison for a mid-sized host:
| Instance type | EC2 c4.4xlarge | Triton G4 standard 32GB | Single CPU Bare Metal Hourly |
|---|---|---|---|
| Vendor | AWS | Joyent | Softlayer |
| Runs your choice of Linux | Yes | Yes | Yes |
| Memory | 30GB | 32GB | 32GB |
| CPU shares/cores (with hyperthreading) | 16 | 16 | 8 |
| Price/month | $635.04 | $604.80 | $787.68 |
| Price/GB | $21.17 | $18.90 | $24.62 |
| Price/CPU | $39.69 | $37.80 | $98.46 |
| Provisioning time | 30-60 seconds | 15-30 seconds | 20-30 minutes |
| Bare metal performance | No | Yes | Yes |
| CPU bursting | No | Yes | No |
| Live resize | No | Yes | No |
The trend continues at 64GB too:
| Instance type | EC2 c4.8xlarge | Triton G4 standard 64GB | Dual CPU Bare Metal Hourly |
|---|---|---|---|
| Vendor | AWS | Joyent | Softlayer |
| Runs your choice of Linux | Yes | Yes | Yes |
| Memory | 60GB | 64GB | 64GB |
| CPU shares/cores (with hyperthreading) | 36 | 32 | 24 |
| Price/month | $1,269.36 | $1,209.60 | $902.16 |
| Price/GB | $21.16 | $18.90 | $14.10 |
| Price/CPU | $35.26 | $37.80 | $37.59 |
| Provisioning time | 30-60 seconds | 15-30 seconds | 20-30 minutes |
| Bare metal performance | No | Yes | Yes |
| CPU bursting | No | Yes | No |
| Live resize | No | Yes | No |
Consider this: you can start with the smallest container Joyent offers in Triton, just 128 megabytes, and live resize it up to 64 gigabytes or even larger without rebooting and with no downtime. The largest container we offer is 224 gigabytes, or 1,792 times the size of our smallest. That's a lot of elasticity, all with bare metal performance.
Get started with infrastructure containers
Create your account, then read on about how to containerize an application in infrastructure containers, with a practical example for deploying Couchbase in infrastructure containers.
Compared to Docker in a VM
| Question | Docker in VMs | Docker containers on Triton |
|---|---|---|
| Where do the containers run? | Docker containers run in Docker host VMs. | Docker containers run on bare metal. |
| How do I secure each container? | Weak isolation between containers on the same Docker host VM requires consideration of the security requirements and implications of each container sharing each VM. | Each Docker container is individually secured in a trusted execution environment. |
| How do I manage and scale container performance? | The performance of each Docker container is limited by the underlying Docker host VM. Provision a larger Docker host VM and re-provision containers to it to increase Docker container performance. | Resources are assigned and reserved for each Docker container using the -m and -c flags in the docker run command. Specify resources as needed for performance requirements. |
| I need a lot of containers and horizontal scale, how do I do that? | Start by provisioning VMs. If you're scaling horizontally, you'll probably need a lot of VMs to get the performance you need. | Just do a docker run.... Each new Docker container will be provisioned on a different host for the best scale. This works with docker-compose scale myapp=11 (or 1100) too! |
| How do I connect to containers on the network? | To connect to a container from another container on the same Docker host VM, use the IP shown in docker inspect for that container. To connect to a container on a different host VM, use the IP address for that VM and port mapping specified in docker run. | Use docker inspect to see the primary IP address for each Docker container, or sdc-listmachines to see all the IP addresses (including the public IP address if requested at docker run time). Public IP addresses are accessible from the public internet, private IP addresses are only accessible to other containers owned by the same customer. |
| How do I access the API? | Each Docker host VM presents as a separate API endpoint. | The entire data center, with multiple compute nodes, are all controlled via a single API endpoint. |
| How are fees assessed? | Fees accrue for each provisioned Docker host VM, even those without any running Docker containers. | Fees are charged by the minute for each Docker container. |
Get started with Docker containers
Create your account, then configure your Docker CLI to point to our cloud and try a docker run --rm -it ubuntu bash to see how it works, or go big and deploy a whole Couchbase cluster in moments.
Virtual machines in a container cannot burst CPU usage. ↩
Post written by Casey Bisson
