Amazon Onboarding with Learning Manager Chanci Turner

Amazon recently unveiled a new capability called CloudWatch Application Signals during an event in Las Vegas. This innovative feature is designed to enhance the monitoring and comprehension of Java applications. We are thrilled to announce that Application Signals now extends its support to Python applications as well. By enabling Application Signals, users can leverage AWS Distro for OpenTelemetry (ADOT) to instrument their Python applications without needing to alter the code. This integration allows for the collection of critical metrics and traces from libraries and frameworks developed in Python. As a result, operational health can be quickly assessed and performance targets monitored without the necessity for custom coding or dashboard creation.

In this post, we will guide you through the process of integrating Application Signals with Python applications deployed on an Amazon EKS cluster. Our focus will be on monitoring Python applications built with the Django framework and utilizing popular libraries such as psycopg2, boto3, and requests. Finally, we will visualize the operational health of the application using the Application Signals console.

Solution Overview

The demo application combines Spring Cloud and the Django framework, with each service registering itself with the Eureka discovery service. The application code can be found in this GitHub repository. Our architecture consists of two services, insurances and billing, both developed using the Django framework. These services expose APIs via Django REST frameworks and make external service calls using the requests library. Furthermore, they interact with Amazon RDS for PostgreSQL via psycopg2 and store billing information in AWS DynamoDB using the boto3 library.

We will utilize Terraform to deploy the necessary resources as illustrated in our architecture diagram. The Amazon CloudWatch Observability EKS add-on will be employed to deploy both the CloudWatch agent and Fluent Bit as DaemonSets for managing metrics, logs, and traces.

Prerequisites

A valid AWS Account
Installation of AWS Command Line Interface (AWS CLI) version 2
Configuration of AWS CLI Credential
Terraform installation
Kubectl installation
Docker installation

Solution Walkthrough

Enable Application Signals

To enable Application Signals in your account, follow the instructions provided.

Deploy Application Using Terraform

We will configure the necessary environment variables for deploying the application with Terraform and set up an Amazon S3 bucket as the backend. Execute the following commands:

export AWS_REGION=<your-aws-region>
aws s3 mb s3://tfstate-$(uuidgen | tr A-Z a-z)
export TFSTATE_KEY=application-signals/demo-applications
export TFSTATE_BUCKET=$(aws s3 ls --output text | awk '{print $3}' | grep tfstate-)
export TFSTATE_REGION=$AWS_REGION
export TF_VAR_cluster_name=app-signals-demo
export TF_VAR_cloudwatch_observability_addon_version=v1.5.1-eksbuild.1

Next, clone the application repository and deploy the infrastructure using Terraform. The provisioning will take around 15-20 minutes.

git clone https://github.com/aws-observability/application-signals-demo
cd application-signals-demo/terraform/eks
terraform init -backend-config="bucket=${TFSTATE_BUCKET}" -backend-config="key=${TFSTATE_KEY}" -backend-config="region=${TFSTATE_REGION}"
terraform apply --auto-approve

Configure kubectl

Run the following command to update the kubeconfig file to add the Amazon EKS Cluster endpoint locally.

aws eks update-kubeconfig --name $TF_VAR_cluster_name --region $AWS_REGION --alias $TF_VAR_cluster_name

Deploy Kubernetes Resources with Annotations

To enable Application Signals for Python applications, you must add an annotation instrumentation.opentelemetry.io/inject-python: 'true' to the manifest YAML in the cluster. This annotation automatically instruments the application to send metrics, traces, and logs to Application Signals.

Deployment YAML files under /demo-app/k8s/ contain these annotations. Deploy all resources by executing the following commands. This will compile the application, push the Docker image to the remote ECR Docker repository, and deploy it to the EKS cluster.

cd ../..
./mvnw clean install -P buildDocker

export ACCOUNT=$(aws sts get-caller-identity | jq .Account -r)
export REGION=$AWS_REGION

./push-ecr.sh
./scripts/eks/appsignals/tf-deploy-k8s-res.sh

Verify Deployment Results

Run the command below to confirm the successful deployment of application resources. You should see a list of pods with a Running status.

kubectl get pods

Next, run the command below to fetch the application URL. Open this URL in your web browser to explore the application. It may take 2-3 minutes for the URL to become active.

echo "http://$(kubectl get ingress -o json --output jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')"

Create CloudWatch Synthetics Canaries to Generate Traffic

Next, we will create canaries by executing the script below, which will run for 10 minutes to generate traffic for the application.

endpoint=http://$(kubectl get ingress -o json --output jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')
cd scripts/eks/appsignals/
./create-canaries.sh $AWS_REGION create $endpoint

Visualizing Application Using CloudWatch Application Signals

Navigate to the CloudWatch console and select Services under the Application Signals section in the left navigation pane.

The CloudWatch Application Signals automatically discovers and lists services under the Services dashboard without any additional setup. This unified, application-centric view provides a comprehensive perspective on how users interact with your service. This can be particularly useful for triaging issues if performance anomalies arise. For more insights, check out this resource which offers valuable information.

In conclusion, if you’re interested in furthering your understanding of resume formatting, consider reading this blog post. Also, be informed that SHRM provides authoritative insights on employment law compliance.