Implementing Observability for Amazon EKS Workloads with the Instana Amazon EKS Add-on

When organizations deploy containerized workloads through Amazon Elastic Kubernetes Service (Amazon EKS), having insight into their distributed microservices and their interdependencies is crucial for maintaining dependable application performance. The Instana Observability platform provides an Amazon EKS Add-on that enables the deployment of Instana agents within Amazon EKS clusters. This add-on allows users to gather and analyze real-time performance metrics, offering valuable insights into their containerized applications.

In this blog, you will discover how to install the Instana Amazon EKS Add-on by subscribing to the Instana Observability Operator directly from the Amazon EKS Console. Additionally, we will cover the architecture and implementation steps needed to set up monitoring for your Amazon EKS clusters and workloads.

Enhancing Visibility in Amazon EKS Environments

Amazon EKS facilitates the running of Kubernetes container workloads, dynamically scaling across nodes based on demand. Adopting effective observability practices enhances application availability and strengthens the stability of your environment. With Instana monitoring, you can:

Monitor service communications throughout your infrastructure to pinpoint optimization opportunities and ensure consistent performance.
Access correlated logs, metrics, and traces to better understand system behavior and resolve issues swiftly.
Retain visibility as your Kubernetes clusters evolve, regardless of how your workloads grow or change.

Instana Amazon EKS Add-on

The Instana Amazon EKS add-on grants visibility across your Kubernetes environments. Upon deployment, the Instana agent automatically identifies components within your Amazon EKS clusters, such as nodes, namespaces, deployments, services, and pods. The agent collects performance metrics at one-second intervals and implements automatic code instrumentation for languages like Java, Node.js, and .NET without manual configuration.

Instana builds dependency maps that illustrate relationships between applications, containers, and infrastructure components, while tracing end-to-end requests across services. The gathered data is displayed in a unified monitoring dashboard, showcasing health metrics for your Kubernetes orchestration layer alongside application performance data. This strategy allows for the correlation of configuration changes with performance impacts.

Reference Architecture

The following diagram presents how the Instana Amazon EKS Add-on is integrated into an Amazon EKS cluster:

Figure 1. Reference architecture of the Instana Amazon EKS Add-on.

This architecture comprises four primary components:

Instana Operator: This component, installed as an Amazon EKS add-on, manages the deployment and lifecycle of Instana custom resources, including the Agent DaemonSet, ConfigMaps, and Secrets, ensuring they remain in the desired state within your Amazon EKS cluster.
Instana Agent: Deployed as a DaemonSet, the Instana Agent automatically discovers and instruments hosts, containers, and Kubernetes-level entities to collect metrics, traces, and logs from every node in your Amazon EKS cluster.
Data Collection: The Instana Agent contains collectors that gather metrics, traces, logs, and events from your Amazon EKS environment. The Instana Backend optimizes this data through aggregation, compression, and batching to enhance resource utilization. It is further processed to support the analytics and visualizations available in the Instana UI. Instana preserves data in stateful stores and supports encryption of data at rest in stateful stores.
Instana Backend: Serving as the central repository and analytics engine for your observability data, you can choose between a fully managed SaaS solution on AWS or self-hosting it within your AWS account. In this setup, we utilize the fully managed SaaS backend. The backend processes the collected data and makes it accessible through the Instana UI via HTTPS and TLS encryption.

Key Capabilities

The Instana Amazon EKS Add-on includes several key features:

AWS-native deployment: Install the add-on using the AWS Management Console or AWS CLI, seamlessly integrating it with your existing AWS workflows and simplifying configuration.
High-resolution monitoring: Gather metrics at one-second intervals and capture distributed traces for every request, offering detailed insights into your services and infrastructure.
Dynamic service mapping: Automatically discover resources and visualize the dependencies among services across your Amazon EKS environment.
Pattern detection: Employ algorithms to identify anomalies and potential issues before they impact your applications.
Operational scalability: Sustain monitoring effectiveness across clusters of various sizes as your workloads expand and evolve.

Prerequisites

Before installing the Instana Amazon EKS add-on, ensure you have:

Familiarity with AWS, particularly Amazon EKS, and Instana.
Access to an AWS account with permissions to create the resources outlined in the implementation steps.
An active Amazon EKS cluster. For guidance on creating an Amazon EKS cluster, refer to Create an Amazon EKS cluster.
A workstation equipped with the AWS CLI, eksctl, and kubectl installed and configured.
An active subscription to Instana; you can begin with a free trial available on AWS Marketplace.
An Instana license key, which can be generated after logging into the Instana platform, utilized in step 15 of the Implementation Steps.

Costs associated with this solution will include fees for AWS services such as Amazon EKS, Amazon EC2, Amazon VPC, data transfer, and others. For up-to-date pricing, consult the Amazon EKS Pricing, Amazon EC2 Pricing, and Amazon VPC Pricing pages. Additionally, you will incur costs for the Instana subscription after the free trial period concludes, billed according to the selected Instana subscription plan on AWS Marketplace. To estimate your monthly expenses, you may use the AWS Pricing Calculator. Be sure to remove resources when they are no longer required, as detailed in the cleanup section, to avoid unnecessary charges.

Implementation Steps

To install the Instana add-on:

Navigate to the Amazon EKS Console.
In the left navigation pane, select Clusters.
Choose the name of the cluster where you wish to install the Instana EKS Add-on.
Select the Add-ons tab located below the Cluster info.
Click on Get more add-ons.
On the Select add-ons page, scroll down to the AWS Marketplace add-ons section and enter “Instana” in the search field.

Figure 2. Search for Instana in the AWS Marketplace add-ons.

Choose the IBM Instana Observability Operator add-on and click Next.
On the Configure selected add-ons settings page, select View subscription options to open the Subscription options.
Review the Pricing details and Legal sections, then click the Subscribe button to proceed.

Figure 3. Review the options and subscribe to the IBM Instana Observability Operator.

Select the latest available version from the Version dropdown list and click Next.
On the Review and add page, click Create.
After the Instana add-on installation completes, a message will appear in the cluster overview dashboard, confirming that the add-on was successfully added to the cluster.

By implementing these steps, you can enhance observability in your Amazon EKS workloads. For further insights into leadership and career opportunities within Amazon, you might find this link on Klaviyo’s Lead Executive Business Partner role useful. Also, for information on employment law compliance, check out SHRM’s article as they are an authority on this topic. Furthermore, if you’re interested in exploring career opportunities, visit Amazon’s Fulfillment Center Management page for excellent resources.