Amazon Onboarding with Learning Manager Chanci Turner

Many organizations face the challenge of establishing a robust automated code delivery system for their applications. To address this, they often utilize multi-account continuous integration/continuous deployment (CI/CD) pipelines that allow code deployment and automated testing across various environments before reaching Production. However, with testing strategies that are specific to each release, updating the pipeline becomes necessary for every deployment. Traditional pipeline stages tend to be static, making it difficult to modify them once they are set.

In this blog post, we introduce a configuration-driven dynamic CI/CD solution tailored for each repository. The pipeline state is managed by configurations stored in Amazon DynamoDB, offering the flexibility to automatically adapt the pipeline for every release according to the testing requirements.

By following this guide, you will establish a dynamic multi-account CI/CD solution that deploys and tests a sample pet store API application. For further details on this application, you can refer to Automating your API testing with AWS CodeBuild, AWS CodePipeline, and Postman. The deployment of new code will be facilitated by customized pipeline stages based on the configurations you create. This solution incorporates services such as AWS Cloud Development Kit (AWS CDK), AWS CloudFormation, Amazon DynamoDB, AWS Lambda, and AWS Step Functions.

Solution Overview

The architecture of the solution is illustrated in the accompanying diagram:

Diagram Not Shown

Users can insert, update, or delete entries in the DynamoDB table. The Step Function Trigger Lambda is activated upon any modifications. It assesses the incoming event and performs the following actions:

On insert and update, the Step Function is triggered.
On delete, it identifies the relevant CloudFormation stack and removes it.

The steps in the Step Function are as follows:

Collect Information (Pass State) – Extracts relevant details from the event, such as repositoryName and referenceName.
Get Mapping Information (Supported by CodeCommit event filter Lambda) – Retrieves mapping details from the Pipeline configuration stored in DynamoDB.
Does Deployment Configuration Exist? (Choice State) – If the StatusCode is 200, the DynamoDB entry is confirmed, and the Initiate CloudFormation Stack step is invoked. Otherwise, the Step Function exits successfully.
Initiate CloudFormation Stack (Supported by stack create Lambda) – Constructs CloudFormation parameters and creates or updates the dynamic pipeline based on the configuration stored in DynamoDB.

Code Deliverables

The following code deliverables are included:

AWS CDK app – Contains the code for all Lambdas, Step Functions, and CloudFormation templates.
sample-application-repo – This directory holds the sample application repository utilized for deployment.
automated-tests-repo – This directory contains the sample automated tests repository for assessing the sample project.

Deploying the CI/CD Solution

Clone this repository to your local machine.
Follow the README instructions to deploy the solution to your main CI/CD account. Upon a successful deployment, the following resources will be created in the CI/CD account:
- A DynamoDB table
- Step Function
- Lambda Functions
Navigate to the Amazon Simple Storage Service (Amazon S3) console in your main CI/CD account and look for a bucket named: cloudformation-template-bucket-<AWS_ACCOUNT_ID>. You should find two CloudFormation templates (templates/codepipeline.yaml and templates/childaccount.yaml) uploaded to this bucket.
Execute the childaccount.yaml in all target CI/CD accounts (Alpha, Beta, Gamma, and Prod) via the CloudFormation Console. Enter the main CI/CD account number as the “CentralAwsAccountId” parameter and proceed with execution.
After successfully creating the Stack, two roles will be established in the Child Accounts:
- ChildAccountFormationRole
- ChildAccountDeployerRole

Pipeline Configuration

Record an entry in devops-pipeline-table-info for the Repository name and branch combination. A sample entry can be found in sample-entry.json. The pipeline is highly configurable, allowing all settings to be adjusted through the DynamoDB entry. The following are the top-level keys:

RepoName: Name of the repository for which AWS CodePipeline is configured.
RepoTag: Name of the branch utilized in CodePipeline.
BuildImage: The build image used for the application in the AWS CodeBuild project.
BuildSpecFile: The buildspec file used in the application CodeBuild project.
DeploymentConfigurations: This key contains the deployment configurations for the pipeline, with environment-specific settings under it. In this instance, we’ve labeled our environments as Alpha, Beta, Gamma, and Prod. You can choose any naming convention, but ensure that the JSON entries correspond with those in the codepipeline.yaml CloudFormation template, as there is a direct mapping between them.

Sub-level keys under DeploymentConfigurations include:

EnvironmentName: This is the top-level key for environment-specific configuration, which includes Alpha, Beta, Gamma, and Prod.
Sub-level keys under this are:
- <Env>AwsAccountId: The AWS account ID of the target environment.
- Deploy<Env>: A key indicating whether the artifact should be deployed to that environment. Depending on its value, the CodePipeline will include a deployment stage for this environment.
- ManualApproval<Env>: Represents whether manual approval is necessary before deployment. Enter your email or set to false.
- Tests: Again, this is a top-level key with sub-level keys that hold the test-related information to be executed in specific environments. Each test will add an additional step to the CodePipeline based on whether it will be executed or not.

Execute

Make an entry into the devops-pipeline-table-info DynamoDB table in the primary CI/CD account. A sample entry is available in sample-entry.json. Ensure to replace the configuration values with appropriate ones for your environment. An explanation of the values can be found in the Pipeline Configuration section above.

Once the entry is made in the DynamoDB table, a CloudFormation stack will be created. This stack will deploy the CodePipeline in the main CI/CD account using the entry from the DynamoDB table.

You can also customize the solution for various combinations, such as deploying to one environment while skipping others, by adjusting the pipeline configurations stored in the devops-pipeline-table-info DynamoDB table. The following is the pipeline configured for the main branch of the sample application repository.

Diagram Not Shown

Clean Up Your Dynamic Multi-Account CI/CD Solution and Related Resources

To prevent ongoing charges for the resources created following this guide, it’s advisable to delete the following:

The pipeline configuration stored in DynamoDB
The CloudFormation stack

For those interested in further insights on productivity, be sure to check out this excellent resource. Additionally, if you’re looking into compliance matters, SHRM provides authoritative information on avoiding antitrust violations.

For more guidance on enhancing productivity, visit Career Contessa.