Bill shock is an incredibly common phenomenon for companies using AWS. You’ve probably heard stories of people getting bills in the tens of thousands of dollars with no warning, and companies which had massive cost blowouts that they weren’t able to resolve quickly. Most of these are caused by misconfigured workloads, but sometimes inefficiencies quietly build up in the background until your cloud spend is out of control.
The concept of financial operations, or FinOps for short, is gaining popularity as companies improve cost visibility in the cloud. FinOps isn’t just about minimising your cloud bill every month; the fundamental idea behind it is to get more value out of the money that you spend in the cloud. For example, you don’t want to arbitrarily downsize all of your EC2 instances, and then find that the reduction in processing power has caused a bottleneck in your system.
Knowing all of this, how can we get started with FinOps and cost control on AWS?
Introduction to Finops on AWS
Reading your AWS bill
The first step in this process is to read your AWS bill, and understand all of the services that you’re using. You can do this by searching for Billing in the service search, opening Billing Dashboard from the account dropdown in the top right, or clicking on this link to the AWS Billing Dashboard. Doing this from your org management account will give you an overview of bills and costs for the entire org.
The billing landing page gives you a summary of the money that you’re spending, and how it compares to your previous month’s spend. It also has a couple of graphs which show spending by service and by account. You can use this to get a high-level understanding of where your money is being spent, and whether you’ve had any major cost blowouts during the current billing period. You can view all of your previous invoices from the Bills section; these are broken down by service, so you’ll be able to see where you’re spending most of your money.
The other useful tool for visualising your AWS bill is AWS Cost Explorer. Cost Explorer displays your AWS bill in graph format. You can change the time interval that you want to view, and there are also a number of filters which you can use to find specific resources, including account, region, AWS service, and tag. If you have a sudden cost blowout, Cost Explorer is a great way to narrow down the culprit - particularly if your resources use appropriate tags.
You can also leverage the AWS Cost Explorer Reports tool, which is part of the AWS Samples open-source solution set, to generate Cost Explorer reports in Excel.
The importance of cost-benefit analysis
Now that you know how to read an AWS bill, we need to understand how to evaluate where the money is being spent. In small organisations, it’s pretty easy to trace resources back to the people and teams that created them. However, if you’re managing cloud accounts for a large company, you might need to work with dev teams to understand why they’re using particular services. People who haven’t worked in a cloud environment before may not know about specialised AWS services, which are often cheaper than running everything on EC2 instances.
Running performance testing can also help you to understand where you might be able to optimise the system to reduce costs. Sometimes, adding capacity in an area where you have a bottleneck will improve application speed, allowing you to reduce cost in other areas. In other situations, performance testing will help you to understand how much spare capacity you need to provision to handle spikes in traffic.
Finally, it’s important to find unused or under-used resources that you can turn off. This could be old services which are no longer needed, or resources with large amounts of redundant capacity. In addition, non-production environments can often be automatically shut down out-of-hours to reduce their cost.
What you can do to reduce cloud spend
Set billing alerts and alarms
Configuring billing alerts and alarms is the easiest way to prevent bill shock. In AWS, this is done through the AWS Budgets page. When creating a budget, you’ll need to set a maximum monthly spend on AWS. You can set billing alerts to send alarms when you hit a certain percentage of your budget, or when your forecasted spend for the month is projected to be higher than a certain percentage of your budget. You’ll need to provide one or more email addresses for the alerts to go to.
Budgets can be configured for specific services, or all of AWS. Alarms can be configured per-account, or for a whole AWS organisation.
Enforce tagging on resources
Tagging your resources will help you to break down your bill, and assign costs back to individual business units. This makes it easier to attribute large cost increases back to teams, and also allows you to readily identify orphaned and unused resources. You can filter resources by tag in both Cost Explorer and AWS Config, which will give you a more granular overview of who’s doing what and how costs have changed within individual business units.
If your organisation uses CI/CD, you can set policies in your pipeline which block deployment of any untagged resources. To audit the state of your environment you can use AWS Config Rules to identify resources which are missing particular tags, and automatically alert your operations team of non-compliant resources so that they can take appropriate action. Finally, to enforce standardisation and correct use of tags you should consider using AWS Organisations to manage Tag Policies.
Purchase reserved instances and savings plans
If you’re using EC2, and you expect your usage to remain constant for a year or more, you can purchase reserved instances from AWS to reduce your bill. When you buy a reserved instance, you commit to paying for a particular number and type of instances for a term of either one or three years. In return, AWS offers steep discounts over the on-demand price of the instances. The amount of money that you save depends on how long the reserved instance term is, and whether you pay for your RIs upfront. Reserved instances come in two different types:
Standard reserved instances: specific instances only, discounts up to 72%
Convertible reserved instances: can switch instance types during the term, discounts up to 66%
Savings plans are a more flexible version of reserved instances. Instead of committing to purchase particular instances, you commit to spending a certain amount of money per hour on the AWS platform. EC2-only savings plans offer similar discounts to reserved instances, but they can be applied to any instance within a particular instance family (e.g. m5, i3). Alternatively, you can purchase Compute Savings Plans, which can be applied to EC2, Fargate, and Lambda usage.
EC2 Savings Plans: you commit to $x/hour in one EC2 instance family, discounts up to 72%
Compute Savings Plans: you commit to $x/hour across EC2/Fargate/Lambda
EC2: up to 66% discount
Fargate: up to 52% discount
Lambda: up to 17% discount
In most cases, Savings Plans are a better option than Reserved Instances, as they offer similar discounts with more flexibility built in. To calculate the cost of reserved instances and savings plans, go to the Reserved Instances Pricing page or the Savings Plans Pricing page.
Set up, or improve, auto-scaling
Services with unpredictable workloads typically need to be overprovisioned, unless they have auto-scaling set up. Because auto-scaling allows you to adjust compute capacity on the fly to meet requirements, it can be a very effective way to keep cloud costs down.
The AWS Auto Scaling service supports EC2, EC2 spot instances, ECS, DynamoDB, and Aurora. It allows you to set up scaling plans based on several different methods and types of metrics, so even if you have auto-scaling built into your infrastructure, it can be worth checking your settings to make sure that you can scale up quickly whilst still keeping cost down.
Use AWS' cost control tools
Trusted Advisor, Cost and Usage Reports, and AWS Billing Conductor are AWS-native tools which can be used for FinOps and cost control.
Trusted Advisor provides rightsizing recommendations based on a scan of your AWS accounts in the form of regular emails. Trusted Advisor reports identify underutilised AWS resources, make recommendations on how to scale them down, and estimate how much money you can save by doing so.
Cost and Usage Reports provides detailed billing and cost tracking information. The CSVs that Cost and Usage reports produces can be used to power QuickSight dashboards for billing and cost control. AWS Billing Conductor allows you to assign bills directly to cost centres across your business, and produce reports and dashboards for each cost centre. Like Cost and Usage Reports, it integrates with tools such as QuickSight to provide fine-grained cost breakdowns and visualisation.
Use third party FinOps tools
Third-party cost dashboards generally provide more extensive visibility and observability into cloud costs than what you can get from AWS. Some third-party dashboards also provide cost-saving recommendations, and a few will automatically act on rightsizing recommendations for you. There are also a few cost management tools which are designed for specific use-cases, such as controlling Kubernetes cost and managing spot instance fleets. Most third-party cost management software supports all three major cloud providers, which allows multi-cloud organisations to see all of their cloud costs in a central location.
If you don’t have a budget for cost control, Cloud Custodian is a Linux Foundation-managed open-source tool for cloud governance and cost management which supports AWS, Azure, and GCP. Otherwise, the pricing models for third-party cost dashboards vary. Some paid tools have a flat price regardless of your cloud spend. Other tools charge per account, or charge a percentage of your monthly cloud bill. Depending on the size of your company, some tools may be more cost-effective than others. It’s important to do a cost-benefit analysis on these tools, as well; there’s no point in paying more for a third-party dashboard than you save by having it.
Try it yourself!
And now, you can try using these tools yourself! Hopefully they will help you to demystify your AWS costs, and put guardrails in place to avoid bill shock!
Innablr understand that architecture misalignment is a costly exercise that introduces unnecessary issues like bill shocks. Policy compliance is an intensive effort and an ongoing requirement, often with pesky deadlines. Extended periods of non-compliance introduce business impacts, both in terms of risk exposure and costs. Innablr are a leading edge platform consultancy and here to help drive more value out of your AWS architecture and help mitigate these risks. Please get in touch if you’d like to discuss more.