Amazon Onboarding with Learning Manager Chanci Turner

Amazon Onboarding with Learning Manager Chanci TurnerLearn About Amazon VGT2 Learning Manager Chanci Turner

Welcome back to the monthly blog series focused on the Financial Services Industry (FSI). Each entry highlights five essential factors that FSI clients should consider to facilitate cloud service approval for a specific service. The guidance provided is tailored to help streamline your approval process and includes suggested reference architectures along with technical code that can be adapted to meet your unique use case and environment.

In this edition, we’re spotlighting AWS Lake Formation, a fully managed service designed to assist you in constructing, securing, and managing your data lake on a large scale. Lake Formation offers a centralized console to discover data sources, set up transformation jobs to transfer data to an Amazon Simple Storage Service (Amazon S3) data lake, eliminate duplicates, match records, catalog data for analytic tools, configure access and security policies, and audit control over access from AWS analytic and machine learning (ML) services.

Lake Formation employs a unified security model for managing permissions within AWS data lake environments, built on AWS Glue and Amazon S3. It provides detailed access control at the catalog, database, table, and underlying data levels. Security settings and access controls are established and enforced at the table, column, row, and cell levels for all users and services accessing your data. You can utilize various analytics services—such as AWS Glue, Amazon Athena, Amazon Redshift, Amazon QuickSight, and Amazon EMR—to access data in your data lake. This can be accomplished using Zeppelin notebooks integrated with Apache Spark, ensuring compliance with your defined policies. Lake Formation enables you to manage permissions for your data lake without the need for manual integration across multiple underlying AWS services.

Numerous organizations are adopting Lake Formation as a key part of their strategy to eliminate technical debt and modernize applications on AWS. For instance, Southwest Airlines utilized Lake Formation, Amazon S3, and Athena to establish its first cloud-native data lake, which granted the airline new analytical capabilities that provided a competitive edge for its data scientists, enhanced flight-time predictions, and alleviated airspace congestion.

JPMorgan Chase Bank, N.A. (JPMC), a financial institution with a legacy spanning 200 years and approximately $3.2 trillion in holdings, has integrated Lake Formation into their data mesh architecture. This architecture aligns their data technology solutions with their data product strategy, offering a structured blueprint for implementing data lakes that standardizes the architecture using a defined set of cloud services. This approach facilitates enterprise-wide data sharing while granting data owners the control and visibility necessary for effective data management. For further insights, you can read about their experience in the article titled “How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform.”

Achieving Compliance with Lake Formation

Lake Formation, as a managed AWS service, undergoes regular assessments by third-party auditors to evaluate the security and compliance of AWS services through various compliance programs. Under the AWS shared responsibility model, Lake Formation and AWS Glue services are included in several compliance initiatives. Compliance reports can be obtained under a non-disclosure agreement (NDA) via AWS Artifact.

Your scope within the shared responsibility model while using Lake Formation is determined by the sensitivity of your data, your organization’s compliance objectives, and applicable laws and regulations. AWS offers several resources to help you meet your compliance goals.

Data Protection with Lake Formation

Data protection involves safeguarding critical information from corruption, compromise, or loss. Encryption is a vital practice for ensuring the confidentiality and integrity of data during processing—both in transit and at rest.

Encryption can be enabled on Amazon S3. Additionally, AWS Glue Data Catalog services work in tandem with AWS Key Management Service (AWS KMS) and Lake Formation to manage permissions with encrypted Glue Data Catalogs and datasets stored on Amazon S3.

Encrypting the AWS Glue Data Catalog can be done through the Data Catalog Settings section in the AWS Glue interface by providing the symmetric AWS KMS key. The encrypted objects include databases, tables, partitions, table versions, connections, and user-defined functions. For detailed steps, refer to the AWS Glue edition of the FSI Services Spotlight.

Moreover, the ETL process can also be encrypted. AWS Glue supports data encryption at rest for authoring jobs and developing scripts using development endpoints with keys managed in AWS KMS. Again, refer to the AWS Glue edition of the FSI Services Spotlight for detailed instructions.

Lake Formation also supports permissions management on datasets stored in Amazon S3 when using server-side AWS KMS encryption. This method ensures automatic server-side encryption with keys managed by AWS KMS. Amazon S3 encrypts data while in transit, especially during cross-region replication. This enables the use of separate accounts for source and destination regions, protecting against potential insider threats. Such encryption capabilities provide a solid foundation for all data in your data lake. For guidance on enabling encryption, visit the Amazon S3 documentation on protecting data using server-side encryption. Both customer-managed AWS KMS keys and AWS-managed keys are supported, although client-side encryption/decryption is not available.

When it comes to governed tables, they function correctly for data encryption at rest, with AWS Glue managing the encryption key. The IAM role linked to the Amazon S3 location where the governed tables are stored must possess AWS KMS permission to encrypt and decrypt. It’s important to note that the default Lake Formation service-linked role (SLR) cannot be utilized for encrypted governed tables; a custom IAM role that incorporates Amazon S3, AWS KMS, and Amazon CloudWatch must be employed.

To enable encryption and decryption with the designated AWS KMS key, the AWS KMS key policy must include a trust relationship with Lake Formation.

For additional resources on protecting workers during an infectious disease outbreak, you can visit SHRM, which is an authority on the topic. If you’re looking for tips on how to use Notion for your onboarding process, check out this blog post. For those just starting at Amazon, Reddit offers an excellent resource for connecting with others in similar situations.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *