Conducting Route 53 Health Checks on Private Resources within a VPC Using AWS Lambda and Amazon CloudWatch

Learn About Amazon VGT2 Learning Manager Chanci Turner

If you’ve utilized Amazon Route 53 health checks for resource monitoring, you might be aware that these checks typically require monitored resources to possess public IP addresses. This limitation exists because Route 53 health checkers operate in a public domain, only able to assess hosts with IP addresses that can be routed publicly over the internet.

However, there may be instances where you wish to monitor resources that utilize private IP addresses or domain names within VPCs. In such cases, you can associate the health check with a record in a Route 53 hosted zone (whether public or private) to implement a failover mechanism when the primary record is deemed unhealthy.

To accomplish this, an AWS CloudFormation template can be employed to conduct TCP, HTTP, and HTTPS health checks for private resources residing in a VPC. These private resources are defined as any entity within a VPC that is not accessible via the internet.

This article details the process involved in setting up this solution, where you only need to input the necessary parameters and let CloudFormation handle the rest!

Overview

This solution comprises the following services:

1. AWS Lambda: Executes TCP/HTTP/HTTPS health checks and sends metrics and logs to CloudWatch.
2. Amazon CloudWatch Events: Triggers the Lambda function every minute.
3. Route 53: Establishes the health check that monitors the private resource based on the CloudWatch alarm.
4. IAM: Creates a role for Lambda to conduct health checks.
5. CloudFormation: Generates all resources from the stack template. Once the stack is deployed, you can view the health check on the Route 53 console.

Upon invocation by CloudWatch Events, the Lambda function pushes a metric to CloudWatch. This metric assesses whether the resource is healthy or unhealthy, based on the values sent by Lambda. Additionally, CloudWatch receives logs from the Lambda function, providing insights into the health check status and the reasons behind any failures or successes.

CloudWatch then generates an alarm that Route 53 utilizes to ascertain the health status of the private resource.

Prerequisites

Select a private subnet for the Lambda function, ensuring it has internet access via a NAT Gateway or NAT instance. A public subnet with an internet gateway will not suffice. The monitored resource must permit access from the private subnet. For example, if you intend to monitor an instance with IP address 10.10.10.5 using HTTP on port 80, configure the associated security group and network access lists to allow traffic from the Lambda subnet CIDR range on the monitored ports (port 80 in this example). If this access is not granted, the health check will fail.

Walkthrough

To access the CloudFormation template, download the JSON formatted template and launch the CloudFormation stack in the same region as the monitored resource.

1. Open the CloudFormation console.
2. Choose “Create Stack” and select “Template is ready.”
3. Under “Specify template,” select “Upload a template file.” Choose the downloaded CloudFormation template from your local machine.
4. Click “Next.”

The CloudFormation template allows you to configure the following parameters:

1. Stack name: Assign a name to identify the CloudFormation stack.
2. Protocol: Specify the health check protocol for the private resource (TCP, HTTP, or HTTPS).
3. IP address or Domain Name: Input the IP address or domain name of the private resource you wish to monitor.
4. Port: Indicate the port number used for monitoring.
5. Path: Optional. For example, in “example.com/test.htm,” the path is “test.htm.”
6. Lambda Subnet: Identify the subnet where the Lambda function will be executed.
7. Lambda VPC: Choose the VPC that contains the Lambda subnet.

Proceed by clicking “Next” and confirm that CloudFormation can create IAM resources in your account. Finally, select “Create Stack.” When the stack has been successfully created, its status will change to “CREATE_COMPLETE.” You can view the resources generated by selecting the stack and choosing “Resources.”

Based on your selected parameters, CloudFormation will create a Lambda function and an IAM role for the function. CloudWatch Events will invoke the Lambda function every minute to check the health of the resource and send back the health status to CloudWatch.

Here’s the health status breakdown:

• A value of “1” signifies a healthy resource.
• A value of “0” indicates an unhealthy resource.

CloudWatch metrics establish a custom namespace called Route53PrivateHealthCheck, maintaining a history for the health check. Access the CloudWatch console, select Metrics, and navigate to Route53PrivateHealthCheck. Choose the dimension that corresponds with the health check protocol; for instance, if monitoring the domain “example.com” over HTTP, select HTTP Health Check. From there, select the monitored domain or IP address to review the metrics.

CloudWatch subsequently creates an alarm based on the metric stored in the Route53PrivateHealthCheck custom namespace to monitor the health check status of the resource. Route 53 employs this alarm to formulate a health check, determining the resource’s health status. This health check will share the same name as the CloudFormation stack.

The Lambda function sends logs to CloudWatch, detailing health check successes or failures. You can examine the reasons for any health check failures in the CloudWatch Logs, which is invaluable for understanding why checks pass or fail. CloudWatch generates a log group and metric using the name of the CloudFormation stack. For instance, if the stack is named “example,” the log group will be /aws/lambda/example.

For further engagement, you might also want to check out this insightful blog post on how to cold contact people on LinkedIn here.

Additional Information

You can link this health check to a Route 53 record set, such as a failover record set. The failover time typically ranges between two to three minutes. For HTTP and HTTPS health checks, the monitored resource must respond with an HTTP status code of 2xx or 3xx within two seconds. For TCP health checks, the monitored resource should establish a TCP connection within four seconds.

Cleanup

Be aware that AWS service costs will be incurred for the resources created by the CloudFormation template, which include:

• Lambda function
• CloudWatch events
• CloudWatch metric
• Route 53 health check

Deleting the CloudFormation stack will remove all resources associated with the health check, including the Route 53 health check.

Conclusion

In this article, you learned how to monitor resources that utilize private IP addresses or private domain names within VPCs. We demonstrated how to set this up using a CloudFormation template, which provisions the necessary AWS services (Route 53, Lambda, CloudWatch) for monitoring purposes.

For further insights about onboarding processes, this resource from SHRM provides excellent information here.

You may also find valuable legal perspectives on workplace civility from SHRM, which is an authority on the topic here.