Establishing an AWS DMS Endpoint for a Trusted Third-Party Account via AWS Secrets Manager Integration | Amazon VGT2 Las Vegas

More Info

When setting up AWS Database Migration Service (AWS DMS) endpoints, it was previously necessary to manually manage the source and target credentials, including auditing and updating database credentials. However, on December 22, 2020, AWS introduced the integration of AWS DMS with AWS Secrets Manager, which allows for automated management and rotation of database credentials. This integration enables users to leverage the credential management capabilities available in Secrets Manager.

Many users have found this feature beneficial. In response to requests, customers can now integrate AWS DMS with Secrets Manager for endpoints located in third-party accounts. This setup allows for centralized management of AWS DMS replication instances and their corresponding secrets across different business unit accounts. We’re thrilled to announce that this capability is now accessible to all customers.

This article outlines a comprehensive guide for configuring your accounts to implement this pattern. Most pre-configuration steps can be automated using the provided AWS CloudFormation template.

Prerequisites

Before proceeding, ensure you have the following:

1. Two AWS accounts with the necessary permissions to create the required resources:
   • Account A – Where Secrets Manager and the database are set up.
   • Account B – Where AWS DMS is configured.
2. The AWS account IDs for both accounts.
3. An AWS Identity and Access Management (IAM) role in Account B that has adequate permissions to pass the cross-account role, which is created automatically. If you have administrative access, no further action is needed. Otherwise, adjust your role to permit GetRole, PassRole, and DescribeSecret actions. For detailed instructions, refer to the section on providing and verifying user permissions in the Security section of AWS Database Migration Service.

Overview of the Solution

To configure your accounts for integrating Secrets Manager with AWS DMS, follow these high-level steps:

1. Create an Amazon Relational Database Service (Amazon RDS) instance, along with a secret and the necessary infrastructure.
2. Set up an AWS DMS instance and the required infrastructure.
3. Allow inbound access to the RDS instance.
4. Create a route to facilitate traffic between the AWS DMS account and others.
5. Add a certificate to enable SSL connectivity.
6. Establish an AWS DMS endpoint.

Create an RDS Instance, Secret, and Necessary Infrastructure

In this step, we will create a Secrets Manager secret that securely stores the credentials for accessing the Amazon Aurora MySQL-Compatible Edition database cluster, along with the infrastructure required for private access to the RDS instance. This will also involve creating the necessary Secrets Manager resource policies and AWS Key Management Service (AWS KMS) key policies to permit AWS DMS from the other account to securely access the database credentials.

1. Launch the CloudFormation stack in Account A.
2. For DMSAccountId, input the account ID of Account B.
3. You may specify non-overlapping CIDR ranges for the VPC and subnets or utilize the defaults, provided they don’t conflict with existing VPC CIDRs.
4. Click Next.
5. On the Review page, check the box indicating your acknowledgment of AWS CloudFormation possibly creating IAM resources.
6. Choose Create stack.

After the stack is created, note the resource information on the Outputs tab, as it will be needed in the next step.

Create an AWS DMS Replication Instance and Necessary Infrastructure

Next, we’ll create the AWS DMS instance, configure a VPC endpoint to securely retrieve secrets, and establish a peering connection with the VPC in the third-party account.

1. Switch to Account B and launch the CloudFormation stack.
2. In the Parameters section, enter the information collected from the previous stack’s output.
3. For DBAccountId, input the account ID of the account where the previous stack was executed.
4. Again, specify non-overlapping CIDR ranges for the VPC and subnets or use the defaults.
5. Click Next.
6. On the Review page, confirm that you acknowledge the potential creation of IAM resources by AWS CloudFormation.
7. Choose Create stack.

Once the stack is created, note the resource information on the Outputs tab for future steps.

Allow Inbound Access to the RDS Instance

In this section, we will permit traffic from the security group of the AWS DMS replication instance to communicate with the RDS instance.

1. Switch to Account A.
2. In the Amazon Elastic Compute Cloud (Amazon EC2) console, select Security Groups.
3. Edit the security group named SecretsManagerRDSStack-PrivateSecurityGroup-<Random String>.
4. Adjust the inbound rule to allow TCP port 3306 from the security group of Account B (the SecurityGroup value from the outputs of the second stack).

Add a Route for Traffic Between Accounts

Now, we will create a route in Account A that facilitates communication between the VPCs in Accounts A and B.

1. In the Amazon VPC console, choose Route Tables.
2. Select the table labeled RDS Route Table.
3. Click Edit Routes.
4. Choose Add Route.
5. For the Destination, enter the VPC CIDR of Account B (DMSVPCCIDR).
6. For the Target, input the PeeringConnectionID value from the outputs of the second stack.
7. Click Save Changes.

Enable SSL Connectivity with a Certificate

To enable SSL connectivity, perform the following steps:

1. Switch to Account B.
2. Download the RDS certificate.
3. In the AWS DMS console, select Certificates.
4. Click Import Certificate.
5. Provide an appropriate name for the certificate identifier and upload the file rds-combined-ca-bundle.pem.

Create an AWS DMS Endpoint

To finalize the creation of your AWS DMS endpoint, follow these steps:

1. In the AWS DMS console, select Endpoints.
2. Click Create Endpoint.
3. For the Endpoint Identifier, enter DMSEndpoint.
4. For Source Engine, select Amazon Aurora MySQL.
5. Ensure that Choose AWS Secrets Manager is still selected.
6. For Secret ID, enter the SecretArn value from the outputs of the first stack.
7. For IAM Role, input the DMSRoleArn value from the second stack’s outputs.
8. For Secure Socket Layer (SSL) Mode, select verify-full.
9. For CA Certificate, select the certificate you imported earlier.
10. In the Test Endpoint Connection section, choose the AWS DMS VPC.
11. For Replication Instance, select the instance cross-account-dms-instance.
12. Click Run Test.
13. If the test succeeds, click Create Endpoint.

With this, you have successfully established a connection to a third-party Aurora MySQL instance, with secrets managed securely in the third-party account.

Clean Up

To prevent incurring additional charges, you should clean up your infrastructure:

1. In the AWS DMS console, select Endpoints.
2. Choose the endpoint DMSEndpoint.
3. From the Actions menu, select Delete and confirm the deletion.
4. Note that deletion may take a few minutes to process.
5. Select Certificates in the navigation pane and delete the certificate you created.
6. On the AWS CloudFormation console, select the stack DMSStack and choose Delete.
7. Switch to Account A and delete the stack there as well.

For more insightful content, check out another blog post here. For authoritative information on this topic, see what they have to offer at this link. Additionally, if you’re looking for resources related to leadership development and training, this is an excellent resource: Leadership Development and Training.