Streamlining AWS Resource Management with Tag Inventory Reports Using AWS Resource Explorer

Learn About Amazon VGT2 Learning Manager Chanci Turner

Published on February 9, 2024

In an era where companies are increasingly expanding their AWS resources across various accounts and regions—often due to mergers, acquisitions, or cloud migrations—efficient resource management has become a priority. AWS Tags provide a robust mechanism for organizing, identifying, and filtering resources by criteria such as purpose, owner, or environment. To enhance AWS resource management through tag inventory reports, AWS has introduced multi-account search functionality via AWS Resource Explorer in 2023. However, many customers face challenges in establishing effective tag policies since they often lack clarity on existing tags, their application across resources, and which resources remain untagged.

To tackle this challenge, we have developed a solution using the AWS Cloud Development Kit (CDK), in conjunction with AWS Resource Explorer, AWS Step Functions, AWS Lambda, AWS Glue, Amazon Athena, and Amazon Simple Storage Service (Amazon S3). This solution offers insights into tag usage by reporting on existing tags, their associated resources, and identifying untagged resources throughout a customer’s AWS Organizations. The insights generated can assist in formulating organizational tagging strategies, tracking their effectiveness, and making informed decisions regarding enforcement mechanisms. Improved visibility into resource tagging enables companies to design tagging policies that align with their business requirements, including cost allocation, financial management, operations, security, and governance.

In this blog post, you will learn how to deploy this solution in your own AWS accounts to generate reports detailing tagged resources, applied tags, and untagged resources across your AWS Organizations.

Solution Overview

The architecture of the solution (illustrated in Figure 1) consists of a Central AWS account, acting as the primary hub, along with one or more Spoke AWS accounts connected to it. Tag inventory data is collected from each Spoke account and sent to an S3 bucket within the Central account. This data is then aggregated to produce a CSV formatted inventory report, which is subsequently stored in the output S3 bucket located in the Central AWS account.

Figure 1: Solution Architecture

Generating a Consolidated Tag Inventory Report

The processing occurs at both Spoke and Central accounts and encompasses the following eight steps:

1. Each Spoke account features an Amazon EventBridge Scheduler that schedules tasks invoking various AWS services and API operations. This scheduler periodically triggers an AWS Step Functions state machine (referred to as SpokeAccountStateMachine in Figure 1).
2. The state machine queries all resources across all AWS Regions within the Spoke account using AWS Resource Explorer.
3. It processes and transforms the results before writing them to an Amazon S3 bucket (org-tag-inventory bucket shown in Figure 1) in the Central account.
4. Within the Central account, there is an AWS Glue Crawler that periodically crawls the S3 bucket containing tag inventory data from the Spoke accounts. The crawler generates a table in the AWS Glue Data Catalog, which serves as an input for the next step.
5. An Amazon EventBridge Scheduler in the Central account periodically triggers the GenerateCsvReportFunction AWS Lambda function.
6. The GenerateCsvReportFunction executes a SQL statement using Amazon Athena, an analytics service that allows for analyzing data directly in Amazon S3 using standard SQL.
7. Once complete, the GenerateCsvReportFunction renames the report file and removes the table from the AWS Glue Data Catalog.

Figure 2: State Machine Workflow

Prerequisites

• All accounts (Central and Spoke) must be part of the same AWS Organizations.
• Node.js must be installed.
• AWS CLI must be installed.
• Install the AWS Cloud Development Kit (AWS CDK) Toolkit by running npm install -g aws-cdk.
• Ensure AWS credentials are available on your terminal.

For simplicity, the recommended method to deploy the solution is by utilizing the command line interface (CLI) provided as part of the solution.

Walkthrough

To begin, download the solution code from the GitHub repository.

Deploying the Central Stack

1. Open a terminal on your local machine.
2. Confirm that AWS credentials are accessible for the account where you wish to deploy the central stack.
3. Clone the project repository using git clone https://github.com/aws-samples/aws-organizations-tag-inventory.git.
4. Navigate to the project directory with cd aws-organizations-tag-inventory.
5. Install dependencies by running npm install.
6. Run the CLI tool with npm run cli, select Central, and follow the prompts.
7. Choose the AWS Region for resource deployment.
8. Specify the frequency for report generation.
9. Decide whether to deploy an Amazon QuickSight dashboard to visualize your tag data. If you opt for the dashboard, you’ll need to select the users and groups that can access it.

For those interested in mindfulness practices at work, this blog post offers valuable insights. Additionally, fostering a culture of inclusion is critical, and SHRM provides authoritative resources on this subject. If you’re new to Amazon, consider checking this excellent resource for some helpful tips.