Learn About Amazon VGT2 Learning Manager Chanci Turner
In Amazon VPC, AWS Direct Connect, AWS Site-to-Site VPN, AWS Transit Gateway, Best Practices, Customer Solutions, Networking & Content Delivery, Technical How-to
Many AWS users benefit from consolidated billing, often needing to distribute costs among their internal business units or accounts. This can become complicated when utilizing services that are shared across multiple accounts. While some customers employ cost allocation tags for chargebacks, as of the time this post was written, there is no inherent method to tag network traffic. You can enhance VPC Flow Logs with tagging metadata, as discussed in a previous blog on enriching flow logs with resource tags. However, this method requires additional overhead and lacks integration with AWS Cost Explorer or AWS Cost and Usage Reports (AWS CUR) for chargebacks.
In addition to tagging, many customers use multiple accounts to streamline AWS cost allocation. These accounts help identify which projects or services are incurring AWS charges. In a prior blog post titled “How to Chargeback Shared Services: An AWS Transit Gateway Example”, we explored how to allocate costs for a transit gateway using Amazon CloudWatch metrics and AWS CUR. This post expands on that concept, demonstrating how to establish a proportional cost allocation model for AWS Transit Gateway data processing charges incurred in a networking account using Transit Gateway Flow Logs alongside AWS CUR data.
We will examine a scenario involving a shared networking services Amazon Virtual Private Cloud (VPC) linked to a transit gateway from a centralized networking account. After inspecting the network, data is transmitted from the centralized networking account to spoke VPCs, either within the same or different accounts, that are also connected to the transit gateway. The costs associated with data processing incurred by the centralized networking account need to be charged back to the spoke accounts. This approach can also be applied to chargebacks for data processing charges related to any transit gateway attachment shared by multiple accounts, such as a shared AWS Direct Connect or AWS Site-to-Site VPN attachment.
Figure 1 illustrates a multi-account architecture using centralized VPCs in a networking account that manages traffic inspection and internet egress.
The AWS Transit Gateway Pricing Model
At the time this article was written, the Transit Gateway pricing model consists of two components: the number of connections attached to the Transit Gateway per hour and the volume of data processed through it. Data processing charges are billed to the account that owns the attachment sending traffic to the transit gateway.
Note: This post uses lowercase “transit gateway” when referring to the resource. We capitalize “Transit Gateway” when discussing the service.
Several scenarios exist within centralized architecture designs where a chargeback model is necessary:
- Scenario 1: Centralized Egress VPC for North-South Traffic to the Internet
All egress traffic from spoke VPCs is routed through a centralized egress VPC. The traffic sent from each spoke VPC and processed by the transit gateway is charged to the spoke account. However, return traffic sent from the centralized egress VPC is charged to the networking account. - Scenario 2: Centralized Inspection VPC for East-West Traffic
The return traffic from the centralized inspection VPC processed by the transit gateway before being routed to the destination VPC is charged to the networking account. - Scenario 3: Combined Centralized Inspection/Egress VPC
This scenario merges inspection and egress functions into a single collapsed VPC. The same principle applies: return traffic processed by the transit gateway before reaching the destination VPC is charged to the networking account. - Scenario 4: Ingress Traffic on Shared Direct Connect or Site-to-Site VPN Attachments
Although there are no data transfer costs for inbound traffic on Direct Connect and VPN connections, a charge is incurred by the Transit Gateway in the networking account for processing this inbound traffic over the attachments before routing it to the spoke accounts.
These scenarios highlight instances where costs may need to be allocated back to the spoke VPC account instead of the centralized networking account.
Prerequisites
To effectively calculate and allocate costs, you need to:
- Enable Transit Gateway Flow Logs
The Transit Gateway Flow Logs feature is fundamental for estimating how much network traffic can be assigned to each account. Follow the steps in the AWS Transit Gateway documentation to enable flow logs. A few points to consider: - For this post, flow logs should be published to an Amazon S3 bucket, as we will query the data using Amazon Athena. If you prefer other analysis methods, consider using Amazon CloudWatch Logs or Amazon Kinesis Data Firehose.
- This post assumes you have a single transit gateway. If multiple transit gateways exist in the centralized account or across different AWS Regions, enable flow logs for each and send them to a single S3 bucket for data consolidation.
- When enabling flow logs for multiple transit gateways, it’s common to specify a prefix within your S3 bucket ARN to match your transit gateway ID. However, we recommend avoiding any S3 prefixes for the chargeback process described here. Instead, focus on calculating against the total cost billed to the centralized account rather than per transit gateway.
- Use the default log record format unless you require specific fields in a different order. If you choose a custom format, adjustments to the Athena table creation will be necessary.
- Set the format to Parquet to minimize storage space in Amazon S3 and enhance query performance.
- If significant traffic is processed by the transit gateway, consider partitioning the flow logs hourly.
- Here’s a sample query for creating the Athena table for the transit gateway flow logs in the Amazon Athena console query editor:
CREATE EXTERNAL TABLE `tgwflowlogs`(
`version` int,
`resource_type` string,
`account_id` string,
`tgw_id` string,
`tgw_attachment_id` string,
`tgw_src_vpc_account_id` string,
`tgw_dst_vpc_account_id` string,
`tgw_src_vpc_id` string,
`tgw_dst_vpc_id` string,
`tgw_src_subnet_id` string,
`tgw_dst_subnet_id` string,
`tgw_src_eni` string,
`tgw_dst_eni` string,
`tgw_src_az_id` string,
`tgw_dst_az_id` string,
`tgw_pair_attachment_id` string,
`srcaddr` string,
`dstaddr` string,
As you navigate through these processes, remember that engaging in self-care is vital for maintaining productivity. For more insights, check out this self-care gift guide. If you’re looking for guidance on preparing managers for conversations around pay, SHRM offers valuable resources. Additionally, for those interested in the hiring process at Amazon, this link is an excellent resource.
Leave a Reply