Four Methods to Access AWS Service Properties via AWS CloudFormation (Part 2 of 3)

More Info

This article is the second installment in a series focusing on customizing AWS CloudFormation. In the first part, we explored how to implement customizations using cfn-response and crhelper, detailing the scenarios in which they are most effective. In this segment, we will leverage AWS CloudFormation macros to fill in the gaps identified in our public roadmap.

Prerequisites

To complete the example outlined here, familiarity with YAML, AWS Lambda, and Python is necessary. Additionally, we will utilize the AWS Serverless Application Model (AWS SAM), a built-in transform that offers a shorthand syntax for defining functions, APIs, and other resources.

Option 3: Custom Resource Utilizing AWS CloudFormation Macros

Overview

• Estimated reading time: 15 minutes
• Time to complete: ~ 20 minutes
• Learning level: Advanced (300)
• AWS Services: AWS CloudFormation, AWS SAM, AWS Lambda, Amazon Elastic Compute Cloud (Amazon EC2)
• Software Tools: AWS CLI version 2, Linux, macOS, or Windows Subsystem for Linux

In part 1, we addressed the request from GitHub issue 157 regarding how to retrieve the Amazon EC2 security group name using the cfn-response module. In this post, we will utilize macros to obtain the same security group name. At the time of writing, there are approximately 70 similar issues on our public coverage roadmap, which have garnered over 168 reactions from the community.

Macros are AWS Lambda-backed functions registered in AWS CloudFormation, allowing for code transformations, such as string manipulation, find and replace, loops, or other substitutions. These macros are activated using the Transform keyword in your template. Once the template is uploaded to AWS CloudFormation and the parser encounters the Transform keyword, the AWS Lambda function executing your transformation logic is triggered. The macro processes occur post-template submission but before the CREATE, UPDATE, and DELETE stack operations commence.

Once deployed, template users remain focused on the core logic without being distracted by the macro code, offering the same flexibility in adding custom logic as before while abstracting the intricate details. Unlike other templating languages such as Jinja, macros are integrated into the AWS CloudFormation backend. You can utilize any programming language supported by Lambda functions, including those enabled by Lambda layers.

We create AWS CloudFormation templates to define the macro and its Python code. Since we are using AWS SAM, the code and templates must be organized in a specific manner. Here’s the directory structure we will establish:

$ tree option3 
option3 
├── cfn-ec2-custom-resource.yml 
├── cfn-macro.yml 
└── lambda_cr_function 
    ├── crhelper 
    ├── crhelper-2.0.6.dist-info 
    ├── tests 
    ├── macro.py 
    └── resource.py 

Let’s review how macros are incorporated into templates. Pay close attention to the sample template below, as it suggests how we will retrieve the missing attribute:

cfn-ec2-custom-resource.yml

AWSTemplateFormatVersion: '2010-09-09'

Transform: SGMacro

Parameters:
  vpcID:
    Type: AWS::EC2::VPC::Id
    Description: Enter VPC Id

Resources:
  CfnEC2SecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: CFN2 Security Group Description
      VpcId: !Ref vpcID

  CustomSecurityGroupNameMacro:
    Type: SGMacro
    Properties:
      RefId: !Ref 'CfnEC2SecurityGroup'

Outputs:
  SecurityGroupID:
    Description: Security Group ID
    Value: !Ref CfnEC2SecurityGroup
  SecurityGroupName:
    Description: Security Group Name from Macro
    Value: !GetAtt 'CustomSecurityGroupNameMacro.SecurityGroup-Name'

Template Explanation

We utilize the Transform keyword to indicate to AWS CloudFormation that we are using a macro named SGMacro. This keyword is positioned at the same global level as the Parameters, Resources, and Outputs keywords. The Parameters section captures the VPC ID, allowing for template reusability. Within the Resources section, we create an AWS::EC2::SecurityGroup in the specified VPC.

Subsequently, we call upon SGMacro to retrieve the EC2 security group name that was previously missing. Macros can be referenced in two ways in your template. In our instance, we employ the Transform keyword at the top of the file, allowing SGMacro to process the entire content of the template and potentially invoke it multiple times. For scoping down the content that a macro can modify, use the Fn:Transform intrinsic function within any resource definition. Both conventions can coexist in the same template for different macros. If you do so, refer to the evaluation order logic in the AWS CloudFormation User Guide.

Our SGMacro is defined to expect an EC2 security group ID as a parameter and return the corresponding EC2 security group name upon execution. This is one of the significant advantages of using macros: end users can invoke them much like they declare resources, without delving into the underlying function code.

Utilizing the !Ref and !GetAtt intrinsic functions, we showcase the EC2 security group ID and name in the Outputs section. Now that we understand the structure and syntax of our template, let’s proceed to construct the elements necessary for its simplified usage.

Creating Macros with AWS SAM

To create a macro, we first need to define and implement it using the open-source AWS SAM framework.

cfn-macro.yml

AWSTemplateFormatVersion: '2010-09-09'

Transform: AWS::Serverless-2016-10-31

Resources:
  ResourceFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.8
      CodeUri: lambda_cr_function
      Handler: resource.handler
      Timeout: 30
      Policies:
      - Statement:
        - Sid: CloudWatchLogPolicy
          Effect: Allow
          Action:
          - logs:CreateLogGroup
          - logs:CreateLogStream
          - logs:PutLogEvents
          Resource: '*'
        - Sid: EC2DescribePolicy
          Effect: Allow
          Action:
          - ec2:DescribeSecurityGroups
          Resource: '*'

  MacroFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.8
      CodeUri: lambda_cr_function
      Handler: macro.handler
      Environment:
        Variables:
          LAMBDA_ARN: !GetAtt ResourceFunction.Arn

  Macro:
    Type: AWS::CloudFormation::Macro
    Properties:
      Name: SGMacro
      FunctionName: !GetAtt MacroFunction.Arn

Template Explanation

Initially, we signal to AWS CloudFormation that our code requires the AWS::Serverless transform to create the necessary functions. This is an excellent resource for anyone looking to deepen their understanding of AWS CloudFormation macros, similar to what is discussed in this article.

For further insights, you can also check another blog post here. It provides additional context on the subject matter. As experts on this topic, they can be found here.

The location for our discussion is Amazon IXD – VGT2, situated at 6401 E Howdy Wells Ave, Las Vegas, NV 89115.