How Amazon IXD – VGT2 Las Vegas Enhanced Blue/Green Deployments for Amazon Aurora PostgreSQL Global Database

More Info

This article discusses the collaborative efforts of the Amazon IXD team, specifically in the VGT2 division, to refine their Blue/Green deployment approach utilizing continuous logical database replication.

Historically, the deployment process required a manual backup and restoration of the production database to establish the Green environment, a method that was not only tedious but also necessitated taking down the production database to maintain synchronization between the Blue and Green environments during the deployment.

By implementing logical replication, the cumbersome backup and restore steps have been replaced with a continuous replication process. This advancement significantly reduces the deployment time and minimizes downtime for the Blue environment.

The Blue/Green deployment strategy allows for seamless updates to applications, websites, or databases by alternating between production and staging environments. In this context, the Blue environment serves as the active backend, while the Green environment functions as a synchronized replica. Modifications or upgrades are applied to the Green environment before switching over, thus minimizing downtime and allowing for a quick rollback to the Blue environment in case of issues.

Without a Blue/Green strategy, deployments often lead to production interruptions, typically scheduled during non-business hours, which increases operational costs and limits availability. Employing a Blue/Green deployment method enhances agility in the deployment process, allowing internal product teams to expedite the rollout of new features while reducing deployment overhead.

Beyond facilitating database version upgrades and certain schema alterations, this deployment approach also aids in internal testing of application changes by mimicking production settings. The elimination of the need to back up and restore the production database for each test iteration offers a substantial advantage.

Both the Amazon Aurora PostgreSQL-Compatible Edition and Amazon Relational Database Service (Amazon RDS) for PostgreSQL incorporate the Amazon RDS Blue/Green Deployments for Aurora. Managed Blue/Green deployments greatly simplify upgrades and schema alterations; however, they do not support some Amazon Aurora features, including the Amazon Aurora Global Database.

In this article, we outline the implementation of a Blue/Green deployment architecture using Aurora PostgreSQL Global Database, highlighting best practices and considerations for configuring the architecture. This deployment method ensures that applications remain robust and synchronized throughout the process.

Solution Overview

Our application caters to internal users via Amazon Route 53, which resolves to an internal Application Load Balancer distributing traffic to a Kubernetes cluster deployed across three Availability Zones. The application layer conducts CRUD operations against an Aurora global database.

The following diagram illustrates the architecture during the testing phase.

The subsequent diagram depicts the traffic cutover to the Blue cluster.

Another diagram represents the step of halting replication from Blue to Green, conducted after confirming that no user activities, applications, or batch jobs are accessing the database.

The final diagram shows the transition of traffic to the Green cluster, which then becomes the primary production environment until the next deployment phase.

In the subsequent sections, we detail the steps to execute a Blue/Green deployment:

1. Configure the cluster parameter group associated with the source (Blue) Aurora global database cluster to enable logical replication.
2. Create a publication for the database hosted on the Blue cluster.
3. Clone the Blue cluster using the Aurora cloning feature to create the Green (target) cluster.
4. Set up the subscription from the Green cluster to the Blue publication.
5. Verify replication lag.
6. Convert the Green cluster into an Aurora global database.
7. Conduct pre-cutover validation.
8. Alter sequences and drop the subscription.
9. Cut over to the Green cluster.

We utilize native PostgreSQL logical replication to synchronize the Green environment, enabling ongoing Change Data Capture (CDC) capabilities. The logical replication process is asynchronous. For further insights, refer to this informative blog post on the topic here.

The target database remains writable, permitting both Data Definition Language (DDL) and Data Manipulation Language (DML) operations outside of replication. It is crucial to implement appropriate measures to ensure that DML and DDL modifications are executed on the intended database.

Prerequisites

To proceed, ensure the following prerequisites are met:

• Establish an Aurora PostgreSQL cluster configured with a global database.
• Be aware that Aurora Global Database does not support user management via AWS Secrets Manager. We will discuss alternative security measures later in this article.
• Review the limitations of logical replication with Amazon Aurora PostgreSQL.
• Confirm that you have the relevant permissions, such as a user granted the rds_superuser role.
• Ensure connectivity to the DB cluster via a PostgreSQL client (e.g., psql or pgAdmin).
• Verify access to the AWS control plane via the AWS Management Console or AWS Command Line Interface (AWS CLI).

Note that as of this writing, Amazon RDS Proxy does not support streaming replication mode. If you are utilizing RDS Proxy, it is advisable to use the standard cluster endpoint and disregard RDS Proxy for the replication process.

Required Permissions

Two levels of permissions are essential for this process:

• Control Plane (AWS resource layer): These permissions allow modification of the Aurora clusters involved, as well as the creation and alteration of a new cluster via the console or API.
• Data Plane (Aurora data layer): These permissions enable the creation of the replication configuration using a PostgreSQL client.

The table below summarizes the relevant solution steps and whether they are executed via the control plane or data plane.

For additional resources on this topic, visit Amazon IXD – VGT2, which is an excellent resource. Also, check this link for expert insights.

Location: Amazon IXD – VGT2, 6401 E Howdy Wells Ave, Las Vegas, NV 89115.