Amazon Onboarding with Learning Manager Chanci Turner

Learn About Amazon VGT2 Learning Manager Chanci Turner

In the realm of satellite communication networks, there exists a plethora of performance metrics, including signal-to-noise ratio (SNR) and bandwidth, which are crucial for remote terminals operating on land, sea, or air. These metrics empower customers to detect anomalies within networks and terminals, while also identifying trends that can significantly improve business outcomes, such as enhancing service level agreements (SLA). However, traditionally, accessing and analyzing this data has been hampered by factors like limited data science expertise, fragmented data repositories, and expensive business intelligence (BI) tools.

This article outlines a methodology utilizing serverless resources from Amazon Web Services (AWS) to establish satellite communication control plane analytics pipelines. With bursts of time-series metrics, the proposed architecture processes the data to extract key performance indicators (KPIs) of interest. These insights can then be visualized in BI tools, such as displaying data-rate trends on a geo-map, and machine learning (ML) can be implemented to identify unexpected SNR deviations.

While satellite operators are the primary audience for this architecture, the serverless data analytics pipelines discussed herein can extend to a wider range of customers. This post illustrates how to achieve business objectives through low/no-code analytics solutions, enabling fresh insights without the requirement of extensive data science teams. Additionally, you’ll learn about the advantages of serverless solutions, including automatic scaling, cost-effectiveness, and minimized operational burdens.

Prerequisites

To create these AWS pipelines, you will need an AWS account and a role with adequate permissions to create resources in these services:

• AWS Lambda
• Amazon Simple Storage Service (Amazon S3)
• Amazon Kinesis
• AWS Glue
• Amazon Athena
• Amazon QuickSight
• Amazon OpenSearch Service
• Amazon SageMaker

Use Cases for Satellite Communication Data Analytics Pipelines

Satellite operators provide solutions for diverse use cases ranging from Earth observation (EO) to satellite communications. In a typical network layout, communications flow from a remote terminal, like a ship or aircraft equipped with a modem and antenna, through a satellite to a central gateway earth station known as a teleport. At the teleport, the radio signal is converted to a lower intermediate frequency and subsequently demodulated and decoded to extract or encode the payload. A network management system (NMS) orchestrates this process, often located separately from the teleports, commanding various quality of service (QoS) profiles based on user traffic types.

Throughout this architecture, monitoring key metrics is essential to prevent SLA impacts, especially when thresholds are breached. Some critical metrics include:

• Carrier-to-noise ratio (C/N): This measures the strength of the received carrier relative to noise, which can be affected by weather conditions, often referred to as “rain fade.”
• Modulation and coding rates (MODCOD): This encompasses converting data into RF for satellite transmission and the error correction associated with it. Popular satcom waveforms, such as DVB-S2X, allow modems to dynamically adapt to higher MODCOD rates, optimizing throughput.
• Receive lock lost: If the remote terminal can no longer receive or transmit signals due to obstructions, the “lock” on the carrier signal may be lost, disrupting user traffic.

Customers of satellite communication services can utilize these KPIs to assess efficiency, raising alarms for unusual occurrences like high packet losses or frequent receive lock losses.

Setting Up AWS Serverless Data Analytics Pipelines

Now, let’s delve into how to establish a series of data analytics pipelines using AWS serverless best practices. This guide illustrates how to leverage these services to minimize costs and operational overhead, as they only operate during incoming data processing. The code and scripts for this tutorial can be found in the AWS samples GitHub repository under the satellite-comms-analytics-aws section. You can use AWS CloudFormation templates to provision these pipelines in your own AWS account, linking your data sources to the entry point.

This article explores the following use cases:

• Pipeline 1: Generating insights with a data lake, ETL transformation, and BI visualization tools
• Pipeline 2: Visualizing real-time KPIs with Amazon OpenSearch Service
• Pipeline 3: Detecting anomalies in SNR with machine learning

We begin with the Amazon Kinesis Data Generator (KDG), which simplifies the process of sending randomized test data to either Amazon Kinesis Data Firehose or Amazon Kinesis Data Streams. Follow the KDG documentation to create an Amazon Cognito user with permissions to access Amazon Kinesis. You can then enter one of the sample record templates found in the kdg/ folder, ensuring your setup aligns with the chosen AWS region, stream, and satellite beam details.

Pipeline 1: Generating Insights with a Data Lake, ETL Transformation, and BI Visualization Tools

The first analytics use case focuses on ingesting streaming data, performing extract, transform, and load (ETL) operations—such as filtering or joining—and deriving business insights using analytics visualization tools like Amazon QuickSight.

To deploy this solution in your own AWS account, select “Create stack (with new resources)” in the CloudFormation console. Download the streaming_s3_glue_pipeline.yaml template from the GitHub repository, select “Upload a template file,” and browse to the YAML file.

After naming your stack, provide parameters, including the creation of Amazon S3 buckets for the KDG input data, the outputs of the AWS Glue ETL job, and the necessary AWS Lambda and AWS Glue scripts. While this solution could be implemented with a single bucket and sub-folders, the structure is designed to emulate a larger-scale data lake where different functional groups may have varying access to assets.

Supplying Amazon S3 input and output bucket names without underscores or dashes enables automation of AWS Glue table creation without manual input. The Lambda function that post-processes the Kinesis Data Firehose stream is provided as Python source code in the GitHub repository. There are multiple ways to automate Lambda deployment, including embedding the code directly in the YAML file or referencing it as a zip file in an Amazon S3 bucket. We opt for the latter approach: zip the Python function and upload it to the SatComAssetsS3Bucket, similar to the KdfLambdaZipName parameter.

By leveraging these AWS resources effectively, organizations can streamline their operations and achieve significant cost savings while enhancing their analytics capabilities.

For further engagement, you might find it interesting to explore the future of collaborative work in another article from Career Contessa. Additionally, the National Labor Relations Board has intensified scrutiny on employer policies, which can offer insights into compliance matters. For those curious about the experience as an Amazon warehouse worker, there’s an informative resource available on Quora.