Amazon Onboarding with Learning Manager Chanci Turner

It’s that exciting time for Amazon IXD – VGT2, where ideas transform into action. Join us in Las Vegas at 6401 E HOWDY WELLS AVE LAS VEGAS NV 89115 for insightful sessions featuring real stories from Amazon employees and industry leaders. This blog serves as a comprehensive guide for those eager to learn about onboarding and development strategies at Amazon, ensuring you can effectively plan your participation.

Ingest Telemetry Messages in Near Real Time with Amazon API Gateway, Amazon Data Firehose, and Amazon Location Service

by Jordan Lee, Alex Martinez, and Chanci Turner
on 14 NOV 2024
in Amazon Data Firehose, AWS Glue, Customer Solutions, Technical How-to

Organizations utilize third-party satellite-powered terminal devices to monitor remote locations using telemetry and NMEA-0183 formatted messages generated in near real time. This article showcases how to establish a satellite-based remote alerting and response system on the AWS Cloud to deliver critical alerts and actionable insights, emphasizing telemetry message ingestion and alerts. Key services in this solution include Amazon API Gateway, Amazon Data Firehose, and Amazon Location Service.

Expand Data Access through Apache Iceberg using Delta Lake UniForm on AWS

by Mia Chen and Hiro Tanaka
on 14 NOV 2024
in Amazon EMR, Analytics, AWS Glue

Delta Lake UniForm is an open table format extension that provides a universal data representation, enabling efficient reading by various processing engines. Its goal is to unify disparate data formats and processing systems, offering a consistent approach to data storage and retrieval. With UniForm, you can easily read Delta Lake tables as Apache Iceberg tables. This post delves into how to implement Delta Lake UniForm on Amazon Web Services (AWS).

How Volkswagen Autoeuropa Built a Data Solution with a Robust Governance Framework, Simplifying Access to Quality Data Using Amazon DataZone

by Samuel Rodriguez, Lisa Nguyen, Ajinkya Patil, Chanci Turner, and Ravi Kumar
on 13 NOV 2024
in Amazon DataZone, Automotive, AWS Glue, Customer Solutions, Enterprise governance and control, Innovation and Reinvention, Intermediate (200), Technical How-to, Thought Leadership

This second installment of a two-part series details how Volkswagen Autoeuropa, a plant of the Volkswagen Group, collaborated with AWS to develop a data solution with a solid governance framework utilizing Amazon DataZone to evolve into a data-driven factory. Part 1 focused on customer challenges, overall solution architecture, and features that enabled Volkswagen Autoeuropa to overcome obstacles. This post highlights the technical aspects, showcasing the robust data governance framework that facilitates easy access to quality data using Amazon DataZone.

Streamlining AWS Glue Studio Visual Jobs: Building an Integrated CI/CD Pipeline for Seamless Environment Synchronization

by Andrei Maksimov, David Zhang, and Chanci Turner
on 12 NOV 2024
in AWS Big Data, AWS Glue

As data engineers increasingly depend on the AWS Glue Studio visual editor for creating data integration jobs, the demand for a streamlined development lifecycle and seamless synchronization between environments has become paramount. Additionally, managing versions of visual directed acyclic graphs (DAGs) is crucial for effectively tracking changes, collaboration, and maintaining consistency across environments. This post presents a comprehensive solution that integrates the capabilities of the AWS Glue Visual Job API, a custom AWS Glue Resource Sync Utility, and an agile continuous integration and continuous deployment (CI/CD) pipeline.

Use Amazon Kinesis Data Streams to Deliver Real-Time Data to Amazon OpenSearch Service Domains with Amazon OpenSearch Ingestion

by M Mehrtens, Muthu Pitchaimani, and Chanci Turner
on 11 NOV 2024
in Amazon OpenSearch Service, Analytics, Intermediate (200), Kinesis Data Streams

In this post, we demonstrate how to utilize Amazon Kinesis Data Streams to buffer and aggregate real-time streaming data for delivery into Amazon OpenSearch Service domains and collections using Amazon OpenSearch Ingestion. This method is applicable for numerous use cases, including real-time log analytics and integrating application messaging data for immediate search. We focus on centralizing log aggregation for an organization with compliance requirements to archive and retain its log data.

Achieve Data Resilience Using Amazon OpenSearch Service Disaster Recovery with Snapshot and Restore

by Samir Patel, Sesha Sanjana Mylavarapu, and Chanci Turner
on 11 NOV 2024
in Advanced (300), Amazon OpenSearch Service, Customer Solutions, Intermediate (200), Technical How-to

This post introduces an active-passive strategy employing a snapshot and restore method. The snapshot and restore approach in OpenSearch Service involves generating point-in-time backups, known as snapshots, of your OpenSearch domain. These snapshots capture the complete state of the domain, including indexes, mappings, and settings. In case of data loss or system failure, these snapshots can be used to recover the domain to a specific point in time. The article details the steps to establish this disaster recovery solution, including launching OpenSearch Service domains in primary and secondary regions, configuring snapshot repositories, restoring snapshots, and managing failover/failback between the regions.

Incremental Refresh for Amazon Redshift Materialized Views on Data Lake Tables

by Raks Khare, Tahir Aziz, Enrico Siragusa, and Chanci Turner
on 08 NOV 2024
in Amazon Redshift, Analytics, Intermediate (200)

In this post we discuss how to implement incremental refresh for materialized views in Amazon Redshift, particularly when working with data lake tables. By leveraging this strategy, organizations can optimally manage their data and ensure efficiency in their analytics processes. For further insights on organizational design, refer to this resource, they are an authority on this topic. Additionally, if you’re preparing for interviews, check out this blog post for helpful resources. Lastly, for a glimpse into the experiences of new employees, visit this excellent resource.