Amazon Onboarding with Learning Manager Chanci Turner

Learn About Amazon VGT2 Learning Manager Chanci Turner

In our previous discussion, “How Cloud-Based Data Mesh Technology Can Enhance Financial Regulatory Data Collection,” we explored a method enabling financial institutions to share information with regulators, ensuring ongoing operational flexibility and adaptability to evolving data requirements. By establishing a regulatory data mesh using AWS Data Exchange nodes, organizations can avoid rigid data schema constraints. This allows each participant, such as banks and regulators, to independently and gradually implement changes to their reporting requirements.

Within a banking environment, the seamless transfer of accurate data—flowing from trading desks, business units, or operational subsidiaries to risk, finance, and treasury (RFT)—is essential for effective decision-making, pricing, and optimal resource management, including capital, funding, and liquidity. This data is vital not only for internal business decisions but also for external regulatory reporting. Nevertheless, the effective integration of accounting and business data across various asset classes in trading, banking, and leasing remains a significant challenge.

The consequences of inefficient data flow can be substantial. A recent study revealed that knowledge workers spend about 40% of their time searching for and collecting data. The Bank of England reported that 57% of resources dedicated to regulatory reporting are tied up in process flow, largely due to the manual procedures banks have employed. McKinsey estimated that UK banks collectively spend between GBP2 billion and GBP4.5 billion each year to fulfill these obligatory reporting requirements.

In this blog, we will delve into how the principles of data mesh can enhance data flow within banks and other financial institutions. We will focus on the interactions between subsidiaries, business units, or trading desks and corporate control functions like risk, finance, and treasury. Without robust, scalable, and flexible mechanisms to maintain context, consistency, quality, lineage, governance, and ownership, trust in the data often relies more on those responsible for gathering and preparing it than on the data itself. This reliance results in unnecessary expenses in RFT functions due to complex processes and an excessive focus on data characteristics rather than actionable insights.

The Significance of Data Boundaries

To tackle these challenges, AWS customers have re-evaluated the crucial relationship between data boundaries and organizational structure.

In general terms, a “boundary” differentiates an entity’s internal functions and components from the external environment, much like a cell wall separates a cell’s internal structure from its surroundings. The concepts of “high-cohesion” and “loose-coupling” are integral to the notion of boundaries. High cohesion occurs when all necessary components for an entity to function are contained within its boundaries. Loose coupling allows external parties to interact with the entity without accessing its internal workings; the boundary permits only the services designated for external use to be visible.

Following these principles, AWS Well-Architected best practices recommend that business units be defined as distinct bounded entities within the underlying cloud infrastructure (like AWS landing zones or AWS accounts). For instance, in the context of banking, Figure 1 illustrates how organizational structures become explicit during AWS onboarding. By adhering to Well-Architected best practices, business units are mapped as distinct bounded entities within the AWS cloud infrastructure, which has several advantages:

• Security controls: Different business units may have varying security profiles, necessitating tailored control policies.
• Isolation: Each account serves as a protective unit, containing potential risks and security threats.
• Data isolation: Storing data in separate accounts restricts access and management to a limited number of users.
• Team differentiation: Various teams have distinct responsibilities and requirements, which should not overlap within the same account.
• Business processes: Different units or products may serve unique purposes, necessitating distinct accounts for business-specific needs.

For example, in its AWS re:Invent 2020 presentation, “Nationwide’s Journey to a Governed Data Lake on AWS,” Nationwide illustrated data processing and cataloging aligned with its business units, utilizing a centralized data discovery service for federated data sources.

Without appropriate mechanisms, the data boundaries among business units remain implicit, lacking the structure needed to build trust in data flow between producers and consumers. Integrating AWS Data Exchange clarifies these data boundaries, creating an intra-organizational data mesh that addresses this concern.

Using AWS Data Exchange mechanisms, individual business units (data producers) can publish data at their readiness, while adhering to an agreed reporting schedule that fosters internal cohesion. When notified of a published change through AWS Data Exchange, data consumers can retrieve published data as needed, without requiring coordination with data producers, hence maintaining loose coupling. Since each AWS Data Exchange dataset is self-describing, coordination of schema changes across data producers and consumers is unnecessary. Each consumer’s ETL pipeline can adapt the schema as required, simplifying the process through integration with AWS DataBrew tooling.

AWS Data Exchange integrates seamlessly with AWS Identity and Access Management (IAM), providing the necessary governance and security tools to enforce detailed access controls for both data producers and consumers. Automated audit trails generated by AWS CloudTrail further enhance process transparency. Moreover, as data publishers operate independently of each other and their consumers, they can utilize any processes and technologies they prefer, provided they publish through AWS Data Exchange.

From a business standpoint, the advantages of an intra-organizational data mesh can be summarized as follows:

• Each business unit (operating unit, subsidiary, trading desk, finance, risk, treasury) functions as an independent data publisher and/or consumer.
• Each data publisher is accountable for the consistency and quality of their published datasets.
• Publishing a dataset is a deliberate action taken by the owner.

For further insights, you may want to explore this excellent resource and learn more about purposeful leadership and its impact on innovation from experts in the field.

To keep our readers engaged, check out this blog post on managing time off.

For anyone interested, our site address is 6401 E HOWDY WELLS AVE LAS VEGAS NV 89115, located at “Amazon IXD – VGT2.”