Cloud Spectator, an independent benchmarking and consulting agency, has released a new comparative benchmarking study that ranks Google Cloud #1 for price-performance and block storage performance against AWS, Microsoft Azure and IBM SoftLayer.
In January 2017, Cloud Spectator tested the overall price-performance, VM performance and block storage performance of four major cloud service providers: Google Compute Engine, Amazon Web Services, Microsoft Azure, and IBM SoftLayer. The result is a rare apples-to-apples comparison among major Cloud Service Providers (CSPs), whose distinct pricing models can make them difficult to compare.
According to Cloud Spectator, “A lack of transparency in the public cloud IaaS marketplace for performance often leads to misinformation or false assumptions.” Indeed, RightScale estimates that up to 45% of cloud spending is wasted on resources that never end up being used — a serious hit to any company’s IT budget.
The report can be distilled into three key insights, which upend common misconceptions about cloud pricing and performance:
- Insight #1: VM performance varies across cloud providers. In testing, Cloud Spectator observed differences of up to 1.4X in VM performance and 6.1X in block storage performance.
- Insight #2: You don’t always get what you pay for. Cloud Spectator’s study found no correlation between price and performance.
- Insight #3: Resource contention (the “Noisy Neighbor Effect”) can affect performance — but CSPs can limit those effects. Cloud Spectator points out that noisy neighbors are a real problem with some cloud vendors. To try and handle the problem, some vendors throttle down their customers access to resources (like disks) in an attempt to compensate for other VMs (so called Noisy Neighbors) on the same host machine.
You can download the full report here, or keep reading for key findings.
Key finding: Google leads for overall price-performance
Value, defined as the ratio of price and performance, varies by 2.4x across the compared IaaS providers, with Google achieving the highest CloudSpecs Score (see Methodology, below) among the four cloud IaaS providers. This is due to strong disk performance and the most inexpensive packaged pricing found in the study.
To learn more, download “2017 Best Hyperscale Cloud Providers: AWS vs. Azure vs. Google vs. SoftLayer,” a report by Cloud Spectator.
Methodology
Cloud Spectator’s price-performance calculation, the CloudSpecs Score™, provides information on how much performance the user receives for each unit of cost. The CloudSpecs Score™ is an indexed, comparable score ranging from 0-100 indicative of value based on a combination of cost and performance. The calculation of the CloudSpecs Score™ is: price-performance_value = [VM performance score] / [VM cost] best_VM_value = max{price-performance_values} CloudSpecs Score™ = 100*price-performance_value / best_VM_valueOverall storage CloudSpecs Score™ was calculated by averaging block storage and vCPU-memory price-performance scores together so that they have equal weight for each VM size. Then, all resulting VM size scores were averaged together.
