Setting Up an AI/ML Center of Excellence at Amazon IXD – VGT2

In today’s fast-evolving landscape, artificial intelligence and machine learning (AI/ML) are reshaping industries at an unprecedented pace. Recent insights from a McKinsey report indicate that the financial services sector could see a boost of over $400 billion (5%) in revenue from productivity enhancements attributed to generative AI. Furthermore, Gartner forecasts that more than 80% of organizations will deploy AI technologies by 2026. At Amazon, we firmly believe that innovation—rethinking and reinventing processes—leads to enhanced customer experiences and improved operational efficiencies, ultimately driving greater productivity. To leverage the transformative potential of generative AI, financial services institutions must identify where its capabilities can most significantly benefit their customers.

Organizations across various sectors encounter numerous obstacles when implementing generative AI, including unclear business cases, challenges in scaling beyond pilot projects, insufficient governance, and a shortage of skilled talent. One effective strategy to address these challenges is to establish an AI/ML center of excellence (CoE). An AI/ML CoE serves as a dedicated unit—either centralized or federated—that coordinates and manages all AI/ML initiatives within an organization, seamlessly linking business strategy to value creation. A recent Harvard Business Review report highlights that 37% of large companies in the U.S. have already set up an AI/ML CoE. For organizations to succeed in their generative AI initiatives, fostering coordinated collaboration between business units and technical teams is becoming increasingly vital.

This article, along with the Cloud Adoption Framework for AI/ML and Well-Architected Machine Learning Lens, serves as a roadmap for effectively implementing an AI/ML CoE aimed at maximizing the potential of generative AI. This includes guiding practitioners to articulate the CoE’s mission, forming a leadership team, embedding ethical guidelines, prioritizing use cases, providing team upskilling opportunities, implementing governance structures, creating necessary infrastructure, ensuring security measures, and promoting operational excellence.

Understanding the AI/ML CoE

The AI/ML CoE plays a crucial role in collaborating with business lines and end users to identify AI/ML use cases that align with business objectives and product strategies. It recognizes reusable patterns across different business units, develops a cohesive AI/ML vision, and deploys an AI/ML platform using the most suitable combination of hardware and software. By fusing business acumen with deep technical know-how in AI/ML, the CoE team creates interoperable, scalable solutions throughout the organization. They also establish and enforce best practices encompassing design, development, processes, and governance operations to mitigate risks while ensuring robust frameworks for business, technical, and governance standards are maintained. The outputs of an AI/ML CoE typically fall into two categories: guidance—including published best practices and tutorials—and capabilities encompassing skills, tools, technical solutions, and templates.

Key Benefits of Establishing an AI/ML CoE:

Accelerated time to market with a clear pathway to production
Enhanced return on investments by realizing the promised outcomes of generative AI
Improved risk management practices
Structured team upskilling initiatives
Sustainability through standardized workflows and tools
Better support for and prioritization of innovation efforts

The following sections delve into the essential aspects for establishing a successful AI/ML CoE.

1. Sponsorship and Mission

The initial step in creating an AI/ML CoE is gaining support from senior leadership, defining its mission and objectives, and establishing a strong leadership framework.

Establish Sponsorship

Clearly defined leadership roles are essential to facilitate decision-making, accountability, and compliance with ethical and legal standards:

Executive Sponsorship: Secure commitment from senior leadership to advocate for AI/ML initiatives.
Steering Committee: Assemble a group of key stakeholders to oversee the CoE’s activities and strategic direction.
Ethics Board: Formulate a board to address ethical and responsible considerations in AI/ML development and deployment.

Define the Mission

Aligning the CoE’s mission with customer needs and organizational goals clarifies its purpose in achieving strategic objectives. This mission should include:

Mission Statement: Articulate the CoE’s role in enhancing customer outcomes through AI/ML technologies.
Strategic Objectives: Set measurable AI/ML goals that align with broader organizational aims.
Value Proposition: Quantify expected business value through Key Performance Indicators (KPIs) such as cost savings, revenue generation, user satisfaction, and faster time-to-market.

2. People

According to a Gartner report, 53% of business and technical teams rate their generative AI knowledge as “Intermediate,” while 64% of senior leaders identify as “Novice.” To cultivate a culture of continuous learning and a deep comprehension of AI/ML technologies, including the development of generative AI skills, tailored solutions must be created.

Training and Enablement

The AI/ML CoE should design training programs, workshops, certification offerings, and hackathons to educate employees on AI/ML concepts, tools, and methodologies. These initiatives can cater to varying expertise levels, helping staff learn how to utilize AI/ML for solving business challenges. Furthermore, the CoE could establish mentorship platforms for employees aspiring to enhance their AI/ML skills and introduce certification programs to recognize proficiency levels. Ongoing training is crucial for keeping teams current with the latest technologies and practices.

Dream Team

Cross-functional collaboration is vital for developing comprehensive AI/ML solutions. A multidisciplinary CoE that integrates industry, business, technical, compliance, and operational expertise can drive genuine innovation. This diverse team may include roles such as:

Product Strategists: Ensure product features align with the overall transformation strategy.
AI Researchers: Engage experts to foster innovation and explore advanced techniques like generative AI.
Data Scientists and ML Engineers: Build capabilities for data preprocessing, model training, and validation.
Domain Experts: Collaborate with business unit professionals to address specific applications and needs.
Operations: Develop KPIs, demonstrate value delivery, and manage machine learning operations (MLOps).

For more insights on career goals that can propel your professional journey, check out this resource. Additionally, understanding recognition and rewards can greatly enhance employee engagement—learn more from SHRM’s article. Lastly, for insights on common pitfalls to avoid, here’s an excellent resource to consider.