Author: David Matthews, Sr. Architect

Introduction

AWS Cloud costs have a tendency to grow and grow never looking back. There are new services to test, new instance types to migrate to, and more data to backup and store.  However, there is no reason to leave the cloud costs unchecked.  Let’s explore some areas to focus your attention on to optimize your cloud spend.

Right Sizing

When resources are first created estimations are made to pick instance type, size, and quantity. These estimations need to be revisited to determine if are continually accurate.  Consider the following when reviewing your environments sizing:

Is autoscaling possible?

  • EC2 and container based services off the ability to scale as needed. This functionality is critical to maintain the required compute resources without paying for idle systems.
  • Ensure that your application can handle scaling. There are times that legacy applications are built in a more monolithic manor and can’t handle this invariability.

Review instance metrics

  • What are the average, max, and min CPU usage in the past month?
  • Is Disk IO under utilized?

Review container level metrics

  • Can you change the Memory/CPU requests?
  • Do you have a lot of pods/containers that have to run on each instance? Could you change the instance size to limit this overhead? (Make sure to keep atleast 2 instances for high availability)

Update the instance type

  • Review the latest types
  • Could you switch to gravitons or AMDs?
  • Has your applications usage changed from CPU to memory intensive and should change from C class to R class instance types

Create a process to review the sizing on a semi-regular basis

  • Quarterly Internal Reviews
  • Monthly checks of AWS or third party recommendations

 Reserved Instances & Savings Plans

One of the most common ways to save on your AWS bill is Reserved Instances and Savings Plans. These tools offer a discount on your bill based on a commitment to use the resource for 1 or 3 years. There is no one right answer when it comes to what to buy. Use the following guide to think through your decision.

What does AWS recommend?

  • AWS cost management tool offers recommendations based on 7/30/60 days of historical usage.

 

Figure 1: Cloud spend example

 

What is your financial risk profile?

  • Cost saving is based commitment term and flexibility. If you want to save the most you could commit to 3 years for a specific instance type. Or you can commit to 1 year with flexible instance type
  • Consider a partial upfront or all upfront payment option to save additional money. Each transaction should be reviewed for its payoff, as in many cases the additional savings is not worth the capital outlay.

Should I buy Reserved Instances (RIs) or Savings Plans (SPs)?

  • Savings Plans were released in 2019 as a method to reduce the complexity of buying RIs. Compute Savings Plans offers the ability to purchase a set amount of compute spend ($) for a given hour, instead of declaring the exact regions and instance type.
  • Reserved Instances are still used for non-EC2 based resources (RDS, ElasticCache, Redshift, OpenSearch, MemoryDB). They can be used for EC2 resources, though the Savings Plans are often preferred due to simplicity.

Right Size the environment before buying RIs/SPs

  • Follow the advice in the Right Size section to ensure you are not committing to use resources for over a year that are not required for your environment.

Purchase a blend to maximize benefits but limiting risk

  • Utilize a 60% 1yr, 40% 3yr Savings plan purchase to increase your savings.
  • Utilize EC2 savings plans for the most prominent instance types in use.

Tagging

AWS cost management requires information about what is running in the environment. AWS Cost Allocation Tags are critical to ensuring you know how much things cost and where there might be cost overages. Follow these steps to ensure you are using best practices with tagging.

Create a Tagging Strategy (AWS Tagging Guide)

  • Identify Tag Requirements
  • Design Tag Structures
  • Determine how to responsibility and creation methods

Report on and correct untagged resources

  • Utilize automation to find untagged resources
    • AWS offers a Resource Groups & Tag Editor service (AWS Resource Group Console
    • Many third party cost management tools like CloudCheckr offer their own Tag management (CloudCheckr Tagging)
    • Create a process within your organization to correct the tagging in a timely manner

Create Reports and Budgets based on tags

S3 Optimization

Blob storage can cause your AWS bills to rise as time goes on, because users often use it as a set it and forget it approach. Data in S3 needs to be monitored, archived, and deleted as required by your organization. Consider the following when attacking your S3 spend.

Create/Review your company policy on data rentention

  • All companies should have a plan for how long they keep their data. Whether this if forever or 24 hours.
  • Ensure the plan is flexible to handle different file requirement (legal/compliance, short lived data, company standard, etc)

Use Intelligent-Tiering

  • This storage class automates the archiving and restoring of files as they are needed. It simplifies the S3 optimization process, though there is a fee to monitor files with this storage class.

Figure 2: Intelligent tiering

Use life cycle policies.

  • Set up life cycle policies that match your data requirements.
    • Ex. data is rarely accessed after 90 days, but legally we need to keep the files for 365 days. With this requirement we can create a lifecycle policy that moves to Infrequent Access after 90 days and delete the data after 365 days
  • Ensure that life cycle policies apply to file versions, if bucket versioning is enabled

Quick Wins

There are a few often overlooked items in AWS that cost money that get lost in the shuffle. These items should be reviewed at least quarterly to make sure you are not wasting money.

Unattached EBS Volumes
Sometimes when EC2 instances are deleted EBS volumes are not deleted. This results in unattached EBS volumes that are often forgotten.  Review your AWS account’s EC2 console for unattached EBS volumes. Delete those that are no longer needed.

Unattached Elastic IPs
AWS charges for all Elastic IPs. Review your AWS accounts to determine if you have IPs that are unused and could be released back to AWS.

DynamoDB Provisioned Throughput
DynamoDB is often setup with provisioned throughput, but many smaller or test tables don’t use this throughput. Review you metrics using cloudwatch to determine if you can switch autoscaling to save on your throughput costs

Cost Analysis Tooling

There are many options in the market to analyze you AWS spend. Using AWS or third-party tools you should look at you spend on a weekly/monthly basis to ensure that all spend is accounted for and optimized.  Consider the following options for analysis

AWS CUDOS Dashboard using QuickSIght (LINK)

AWS has a great blog post on setting this up.  It will require utilizing AWS QuickSight.  This tool offers additional visuals & dashboards over the traditional AWS Cost Explorer

AWS CUDOS Dashboard

AWS Cost Explorer (LINK)

Aws Cost Explorer offers 13 months of historical usage data to review. This can be easily searched based on a multitude of metrics. In addition you can create standard reports to view for consistency.

AWS Cost explorer

Third Party Services

There are many cost management tools on the market. One that hits on many of the cost savings items highlighted today is CloudCheckr CMX.  The tool offers both spend reporting and resource reporting.  This offers the ability for the tool to make right sizing recommendations, quick win analysis, and dashboarding all in 1 tool.

Uturn Data Solutions offers a managed FinOps offering that helps attack these areas and more. Reach out to us today to talk about how we can help to optimize your AWS Cloud costs.

About the Author:

David Matthews is a Senior AWS Architect. He has worked with many customers on redesigning their AWS environments to be Well-Architected as cost optimized. His technical background and work experience, plus a MBA in Finance, helps him work with both sides of the spend equation, Business & Technical team.