ParkMyCloud recently estimated that $17.6 billion will be wasted in 2020 due to idle and oversized resources in public cloud. But there is a related problem involving wasted spend on orphaned resources.
What is an orphaned resource? A resource becomes “orphaned” when it is detached from the infrastructure it was created to support. This can include a volume detached from an instance or a snapshot detached from any volumes. Whether an organization is aware these remain in their cloud environment or not, they can continue to incur costs, wasting money and driving up their cloud bill.
How Resources Become Detached
One form of orphaned resources comes from storage. Volumes or disks such as Amazon EBS, when created, are attached to an EC2 instance. Users can attach multiple volumes to a single instance to add storage space. If an instance is terminated but a volume attached to it have not, “orphaned volumes” have been created. Note that by default, the boot disk attached to every instance is designed to terminate when the instance is terminated (although it is possible to deselect this option), but any additional disks that have been attached do not necessarily follow this same behavior.
Snapshots can also become orphaned resources. A snapshot is a point-in-time image of a volume. In the case of Amazon EBS, AWS snapshots are stored in Amazon S3. EBS snapshots are incremental, meaning, only the blocks on the device that have changed after your most recent snapshot are saved. If the associated instance and volume is deleted, a snapshot could be considered orphaned.
Are All Detached Resources Unnecessary?
Just because a resource is detached it does not mean it should be deleted. For example, you may want to keep:
-
The most recent snapshots backing up a volume
-
Machine images used to create other machines
-
Snapshots used to inexpensively store the state of a machine you intend to use later, rather than keeping a volume around
However, like the brownish substance in the tupperware in the back of your freezer, anything you want to keep needs to be clearly labeled in order to be useful. By default, snapshots and volumes do not always get automatically tagged with enough information to know what they actually are.
I see exabytes of untagged storage in customer environments, with no way of knowing if it is safe to delete. In the AWS console, metadata is not cleanly propagated from the parent instance, and you have to go out of your way to tag snapshots before the parent instances are terminated. Once the parent instance is terminated, it can be impossible to identify the source of an orphaned volume or snapshot without actually re-attaching it to a running instance and looking at the data. Tag early and tag often!
The Size of Wasted Spend
To estimate the size of the problem of orphaned volumes and snapshots, I started with internal data showing customers spend approximately 15% of their bills on storage. I found that 35% of that storage spend was on unattached volumes and snapshots. As detailed above, while this does not mean that all of that is wasted, the lack of tagging and excess of snapshots of individual volumes indicates that much of it is.
Overall, an average of 5.25% of our customer bills is being spent on unattached volumes and snapshots. Then applied to the $50 billion estimated to be spent on Infrastructure as a Service (IaaS) this year gives us a maximum of up to $2.6 billion wasted this year on orphaned volumes and snapshots.
This is clearly a huge problem.
Katy Stalcup is Director of Marketing at ParkMyCloud