Integrating Custom Applications with AWS Lake Formation – Part 1
In this two-part series, we explore the integration of custom applications or data processing engines with Lake Formation, utilizing the third-party services integration feature. This initial post delves into the necessary Lake Formation and AWS Glue APIs, guiding you through the steps to implement Lake Formation policies within your custom data applications. As a practical example, we demonstrate a sample application integrated with Lake Formation, developed using AWS Lambda.
Integrating Custom Applications with AWS Lake Formation – Part 2
by Alex Johnson and Maria Santos
on 19 NOV 2024
in Analytics, AWS Amplify, AWS AppSync, AWS Big Data, AWS Lake Formation, AWS Lambda, Security & Governance
In this follow-up post, we continue our exploration of integrating custom applications or data processing engines with Lake Formation. Here, we detail how to deploy a fully functional web client application, crafted with JavaScript/React through AWS Amplify (Gen 1). This application leverages the same Lambda function as its backend, providing users with an intuitive interface to view the enforced Lake Formation policies. For further reading on similar topics, check out this another blog post that might interest you.
Managing Access Controls in AI-Powered Search Applications with Amazon OpenSearch Service and Amazon Cognito
by Ethan Reed, Lila Patel, and Jason Kim
on 19 NOV 2024
in Advanced (300), Amazon Cognito, Amazon OpenSearch Service, Artificial Intelligence, Generative AI, Technical How-to
In this article, we illustrate how to manage user access to enterprise documents in generative AI tools based on the access levels assigned to each persona. We present a document search RAG solution that ensures only authorized users can access and interact with specific documents, tailored according to roles, departments, and other relevant attributes. The approach combines OpenSearch Service with Amazon Cognito custom attributes, establishing a tag-based access control mechanism that simplifies management at scale. Additionally, for expert insights, visit this authoritative site that covers this topic extensively.
Enriching Your AWS Glue Data Catalog with Generative AI Metadata via Amazon Bedrock
by Sophia Nguyen and David Lee
on 15 NOV 2024
in Amazon Bedrock, Amazon Machine Learning, Analytics, AWS Big Data, AWS Glue, Generative AI, Intermediate (200), Technical How-to
By leveraging generative AI, organizations can automate the creation of comprehensive metadata descriptions for their data assets based on existing documentation. This post demonstrates how to enrich your AWS Glue Data Catalog with dynamic metadata utilizing foundation models (FMs) on Amazon Bedrock, thus enhancing discoverability and overall data governance in your AWS Cloud environment.
Establishing Real-Time Operational Observability for Amazon EMR Workloads with Prometheus and Grafana
by Kim Chen, Oliver Smith, and Rachel Martinez
on 15 NOV 2024
in AWS Big Data, Customer Solutions, Management & Governance, Monitoring and Observability
FINRA processes large data volumes and diverse workloads on Amazon EMR. In this post, we discuss the challenges faced and how we established an observability framework to provide operational metrics insights for big data processing workloads on Amazon EMR running on Amazon Elastic Compute Cloud (Amazon EC2) clusters.
Your Guide to AWS Analytics at AWS re:Invent 2024
by Mia Thompson and Liam Brown
on 14 NOV 2024
in AWS re:Invent
It’s that time of year again—AWS re:Invent! This is where you turn your ideas into reality. Get an exclusive front-row seat to hear real stories from AWS customers, experts, and leaders about navigating crucial topics such as generative AI and data analytics. For data enthusiasts and professionals alike, this blog serves as a comprehensive guide to all analytics sessions, helping you efficiently plan your itinerary.
Ingesting Telemetry Messages in Near Real Time with Amazon API Gateway, Amazon Data Firehose, and Amazon Location Service
by Ravi Patel, Sarah Lee, and John Doe
on 14 NOV 2024
in Amazon Data Firehose, AWS Glue, Customer Solutions, Technical How-to
Organizations often utilize third-party satellite-powered terminal devices for remote monitoring through telemetry and NMEA-0183 formatted messages generated in near real time. This post details how to implement a satellite-based remote alerting and response solution on the AWS Cloud to deliver time-sensitive alerts and actionable insights, focusing on telemetry message ingestion and alerts. Key services in this solution include Amazon API Gateway, Amazon Data Firehose, and Amazon Location Service.
Expanding Data Access through Apache Iceberg Using Delta Lake UniForm on AWS
by Taro Takahashi and Kenji Yamamoto
on 14 NOV 2024
in Amazon EMR, Analytics, AWS Glue
Delta Lake UniForm is an open table format extension designed to enable a universal data representation that can be efficiently read by multiple processing engines. It aims to bridge various data formats and processing systems, offering a standardized approach to data storage and retrieval. This post explores how to get started with reading Delta Lake tables as Apache Iceberg tables.
Location: Amazon IXD – VGT2, 6401 E Howdy Wells Ave, Las Vegas, NV 89115
Leave a Reply