Architecture Available in v1.15.7+
Overview
ScaleOps integrates with AWS Cost and Usage Reports to provide accurate compute costs by reconciling current Kubernetes spend with your actual AWS bill.
This is done by creating an AWS Data Export that maintains an up-to-date cost and usage report in S3 (updated by AWS on a daily basis, typically several times a day). The report contains the actual cost of each EC2 instance, after applying the various discounts - such as Reserved Instances, Saving Plans and Spot instances.
ScaleOps periodically retrieves the report, and updates the current hourly prices of individual nodes accordingly. These prices propagate to the rest of the system, including ‘live’ cost, historical cost reports, etc.
Refer to AWS documentation for further information about CUR 2.0.
Diagram
Calculation of Discounted Prices Based on CUR
| Discount Type | Relevant Rows in CUR | Discounted Price Calculation |
|---|---|---|
| Spot Instance | pricing_term=Spot, line_item_line_item_type=Usage | line_item_net_unblended_rate OR (if “net” is not available): line_item_unblended_rate |
| Reserved Instance | pricing_term=Reserved, line_item_line_item_type=DiscountedUsage | reservation_net_effective_cost / line_item_usage_amount OR (if “net” is not available): reservation_effective_cost / line_item_usage_amount |
| Savings Plan | pricing_term=OnDemand, line_item_line_item_type=SavingsPlanCoveredUsage | savings_plan_net_savings_plan_effective_cost / line_item_usage_amount OR (if “net” is not available): savings_plan_savings_plan_effective_cost / line_item_usage_amount |
| None of the above | pricing_term=OnDemand, line_item_line_item_type=Usage | line_item_net_unblended_rate OR (if “net” is not available): line_item_unblended_rate |
Applying Discounted Prices from CUR to Cluster Nodes
Kubernetes cluster nodes are matched with respective EC2 instances in CUR by:
- AWS account ID where the cluster is hosted (
account_idcolumn in CUR), as well as - EC2 instance ID (
resource_idcolumn in CUR).
If the EC2 instance still does not appear in the report, ScaleOps attempts to infer the price from prices of other similar EC2 instances - e.g., those with same instance type, region/AZ and operating system (since the discounts typically apply to instance types rather than specific instances).
Known Limitations
- Currently, ScaleOps does not update discounted prices retroactively - meaning that historical prices prior to enablement of the CUR integration might not be accurate.
- Since CUR data is updated with delays of up to 48 hours (and sometimes even more), it is expected that the accuracy of the CUR based prices might be lower for new nodes (especially if there are no other nodes of the same instance type).