Amazon VGT2 Las Vegas: Establishing Scheduled Backups for Amazon DynamoDB Using AWS Backup

More Info

With the rapid expansion of AWS workloads across numerous resources, many users have highlighted the necessity for a centralized approach to manage and monitor backups. They desire a standardized method to efficiently handle backups at scale.

AWS Backup provides a solution to centralize and automate data protection across various AWS services. This fully managed, cost-effective service simplifies data protection by allowing users to schedule periodic backups. Users can define backup plans that include schedules and retention policies for their resources. AWS Backup automatically creates backups and removes older ones based on the specified retention schedule. Backups are essential for disaster recovery, and AWS Backup alleviates the burden of manually creating and deleting on-demand backups by automating these processes.

In this blog post, we will guide you through the steps to schedule regular backups of an Amazon DynamoDB table using AWS Backup. We will create a backup plan that includes daily backups retained for a month. Additionally, we will demonstrate how to restrict certain users within your organization from deleting backups while still permitting operations like backup creation and table management.

Overview of AWS Backup

AWS Backup is engineered to help you centralize and automate your data protection across AWS services. It enables the deployment of backup policies to configure, manage, and oversee backup activities across your organization’s AWS accounts and resources, which include Amazon EC2 instances, Amazon EBS volumes, Amazon RDS databases, DynamoDB tables, Amazon EFS, and more.

Here are some key AWS Backup terms to understand:

• Backup vault: A container for organizing your backups.
• Backup plan: A policy that outlines when and how to back up AWS resources, linked to a backup vault.
• Resource assignment: Specifies which resources to back up, selected by tags or ARNs.
• Recovery point: A snapshot of a resource backed up by AWS Backup, which can be restored using the service.

Solution Overview

Amazon DynamoDB offers two types of backups: point-in-time recovery (PITR) and on-demand backups. PITR provides continuous backups, allowing restoration to any time within the last 35 days. For longer retention, on-demand backups offer a consistent snapshot of your data indefinitely. While traditional backup solutions may require manual management, AWS Backup automates scheduling, expiration, monitoring, compliance verification, and restoration from a singular console.

You can schedule and manage backups via AWS Lambda functions or opt for AWS Backup, which simplifies the management process, streamlining backup handling akin to traditional solutions.

The following steps outline the process:

1. Launch an AWS CloudFormation template to create the necessary resources.
2. Load data into DynamoDB using a Lambda function.
3. Configure a scheduled backup for the DynamoDB table.
4. Restore the backup to a new DynamoDB table.

The architecture of this solution is illustrated in the accompanying diagram. Next, we will delve into scheduling a backup for a DynamoDB table and restoring it using AWS Backup.

Creating Resources with AWS CloudFormation

You can download data files and the CloudFormation stack file from GitHub. Follow these steps to create your AWS resources:

1. Sign in to the AWS Management Console.
2. Select “Launch stack.”
3. Enter a stack name and click “Next.”
4. Accept the default settings and click “Next.”
5. Acknowledge the IAM resource creation and select “Create stack.”

Loading Data into DynamoDB

To simulate a realistic scenario, we will mimic an online retail application. Upload raw data files to an Amazon S3 bucket and use a Lambda function to load the data into DynamoDB. Complete the following steps:

1. Upload the order_data.csv file to the S3 bucket created by CloudFormation.
2. In the Lambda console, select the stack <stack_name>-CsvToDDBLambdaFunction-*.
3. Click “Test.”

For the Event name, enter a name, remove the “template” event, and replace it with {}. Click “Create” and then “Test.” This will execute the Lambda function, loading the order_data.csv into the DynamoDB table. Wait for the success message to confirm completion. To view the data, navigate to the DynamoDB console and select the table order_transaction-*.

Configuring a Scheduled Backup

Before creating a backup plan or initiating a backup job, you must create at least one vault.

Creating a Backup Vault

In the AWS Backup console:

1. Select “Backup vaults” from the navigation pane.
2. Click “Create Backup vault.”
3. Enter a name for the Backup vault.

By default, AWS Backup generates an AWS KMS key with the alias aws/backup. You can use the default key or select another key from your account before clicking “Create Backup vault.”

The screenshot below confirms the successful creation of the backup vault.

Creating a Backup Plan

1. In the AWS Backup console, select “Backup plans.”
2. Click “Create Backup plan.”
3. Choose “Build a new plan.”
4. Enter a name for the Backup plan.
5. Specify a rule name for your backup rule.
6. Select “Daily” for the backup frequency.

The backup frequency determines how often backups are created. Options include every 12 hours, daily, weekly, or monthly. Choose “Customize backup window” to set the time and duration for backups to occur.

Next, configure the lifecycle for the backups, which defines when backups transition to cold storage and when they expire. In this example, the backup will transition to cold storage after 31 days and expire after 366 days. Note that currently, only Amazon EFS file system backups can transition to cold storage; the cold storage option is ignored for backups of Amazon EBS, Amazon RDS, Aurora, DynamoDB, and Storage Gateway.

For additional insights on AWS Backup, consider visiting this link. If you’re looking for more authoritative information on this topic, check out this resource. Moreover, for those interested in warehouse associate opportunities, this page is an excellent resource.