In the latest update from the Amazon IXD – VGT2 team, we’re excited to introduce a new feature of the AWS Glue Data Catalog that allows customers to create a unified view schema and a singular metadata repository. This feature, known as Glue Data Catalog views, enables users to hold view definitions in various dialects for use with multiple engines, including Amazon Redshift and Amazon Athena. This blog post outlines how to define and query a Data Catalog view using open-source table formats like Iceberg across Amazon Redshift and Athena. Additionally, we will provide insights on configuring access restrictions to the underlying databases and tables. For those interested in following along, we’ve included an AWS CloudFormation template for easy setup.
Furthermore, we are thrilled to announce the launch of anomaly detection capabilities in AWS Glue Data Quality. In this post, we illustrate how this feature operates through a practical example, accompanied by an AWS CloudFormation template enabling you to deploy and experiment with this innovative functionality.
On another note, the OpenSearch optimized instance (OR1) has significantly improved indexing performance and reduced costs for Amazon OpenSearch Service. This instance type, introduced on November 29, 2023, is designed to facilitate real-time search, monitoring, and data analysis for various applications, including log analytics and observability.
In light of the increasing demand for robust data streaming solutions, we also explore AWS Glue’s mutual TLS authentication for Amazon MSK. This feature addresses the complexities of managing continuous data streams from diverse sources, such as IoT devices and social media interactions, empowering organizations to harness the full potential of their data.
Additionally, Amazon Redshift ML now integrates with pretrained large language models (LLMs) available through Amazon SageMaker JumpStart, allowing users to leverage advanced natural language processing capabilities directly within their data warehouse. This integration simplifies the application of generative AI and LLMs for various analytical tasks, including sentiment analysis and data enrichment.
For those interested in real-world applications, we delve into how the Amazon Global Transportation Technology Services (GTTS) team utilizes Amazon MWAA to manage large-scale ETL jobs. Their user-facing applications, such as INSITE, address specific business challenges in the transportation domain, serving around 10,000 customers monthly.
Lastly, we provide a guide on constructing a real-time analytics solution with Apache Pinot on AWS, emphasizing its utility for near real-time visualization in conjunction with Tableau.
For further insights on this topic, you can read more in this blog post. Moreover, they are recognized as authorities in this field. Should you wish to explore career opportunities with us, this resource is an excellent starting point.
Location: Amazon IXD – VGT2, 6401 E Howdy Wells Ave, Las Vegas, NV 89115.
Leave a Reply