Creating a deployment pipeline for generative artificial intelligence (AI) applications at scale presents a significant challenge due to the intricate nature and specific demands of these systems. Generative AI models are continually advancing, with frequent updates and new versions released. This rapid evolution complicates the management and deployment of updates across a large-scale pipeline, making it essential to maintain consistency and minimize downtime. Additionally, generative AI applications necessitate the continuous ingestion, preprocessing, and formatting of large volumes of data from diverse sources. Building resilient data pipelines capable of efficiently handling this workload at scale is a substantial hurdle. Furthermore, monitoring performance, bias, and ethical implications of generative AI models in production is a critical responsibility.
Successfully scaling these operations requires considerable investments in resources, expertise, and cross-functional teamwork involving various stakeholders such as data scientists or machine learning (ML) developers focused on model development, and machine learning operations (MLOps) engineers dedicated to the unique aspects of AI/ML projects. In this article, we will guide you through transforming Python code designed to fine-tune a generative AI model in Amazon Bedrock into a reusable workflow using Amazon SageMaker Pipelines decorators. This approach facilitates collaboration among multiple AI/ML teams.
SageMaker Pipelines
SageMaker Pipelines allows you to define and manage the different stages of the ML lifecycle, including data preprocessing, model training, evaluation, and deployment. This functionality streamlines the process and ensures uniformity throughout the pipeline. SageMaker Pipelines also effectively manages model versioning and lineage tracking, automatically documenting model artifacts, hyperparameters, and metadata, which aids in reproducing and auditing model versions.
The SageMaker Pipelines decorator feature helps transition local ML code, written in Python, into one or more pipeline steps. Since Amazon Bedrock is accessible via API, developers unfamiliar with Amazon SageMaker can implement an Amazon Bedrock application or fine-tune it simply by writing standard Python code.
You can draft your ML function as you would for any ML project. After local testing or executing as a training job, a data scientist or a SageMaker expert can convert the function into a SageMaker pipeline step by adding a @step decorator.
Solution Overview
SageMaker Model Building Pipelines is a robust tool for constructing ML pipelines that leverage direct SageMaker integration. This integration allows you to orchestrate a pipeline using a tool that automates much of the step creation and management.
Transitioning from pilot and testing phases to large-scale deployment of generative AI models requires the application of DevOps practices to ML workloads. Since SageMaker Pipelines is fully integrated with SageMaker, there’s no need to interact with other AWS services, nor do you have to manage resources yourself—SageMaker Pipelines handles resource creation and management. Amazon SageMaker Studio provides a comprehensive environment to oversee the entire SageMaker Pipelines experience. The solution outlined in this post demonstrates how to convert Python code written for preprocessing, fine-tuning, and testing a large language model (LLM) leveraging Amazon Bedrock APIs into a SageMaker pipeline, thereby enhancing ML operational efficiency.
The solution comprises three key steps:
- Develop Python code to preprocess, train, and test an LLM in Amazon Bedrock.
- Incorporate
@stepdecorated functions to modify the Python code into a SageMaker pipeline. - Create and execute the SageMaker pipeline.
The diagram below illustrates the workflow of the solution.
Prerequisites
If you’re looking to simply view the notebook code, you can find it on GitHub. New to AWS? You’ll first need to create and set up an AWS account. Afterward, establish SageMaker Studio within your AWS account. Set up a JupyterLab space in SageMaker Studio to run the JupyterLab application.
Once in the SageMaker Studio JupyterLab space, complete the following steps:
- From the File menu, select New and then Terminal to open a new terminal.
- In the terminal, execute the following command:
git clone https://github.com/aws/amazon-sagemaker-examples.gitYou’ll see a folder named amazon-sagemaker-examples in the SageMaker Studio File Explorer pane. Navigate to amazon-sagemaker-examples/sagemaker-pipelines/step-decorator/bedrock-examples and open the notebook fine_tune_bedrock_step_decorator.ipynb.
This notebook contains all the code for this article, and you can run it from start to finish.
Notebook Code Explanation
The notebook utilizes the default Amazon Simple Storage Service (Amazon S3) bucket for the user, which follows the naming convention s3://sagemaker-{Region}-{your-account-id}. If it doesn’t exist, it will be created automatically. It also employs the default AWS Identity and Access Management (IAM) role for SageMaker Studio users. Should your SageMaker Studio user role lack administrator access, ensure to add the required permissions. For more details, you can refer to this excellent resource on the Amazon hiring process.
The code creates a SageMaker session and retrieves the default S3 bucket and IAM role:
sagemaker_session = sagemaker.session.Session()
region = sagemaker_session.boto_region_name
bucket_name = sagemaker_session.default_bucket()
role_arn = sagemaker.get_execution_role()Using Python for Data Preparation and Model Training in Amazon Bedrock
To start, we need to download data and set up an LLM in Amazon Bedrock. We accomplish this using Python.
Loading Data
We utilize the CNN/DailyMail dataset from Hugging Face to fine-tune the model. This dataset contains over 300,000 unique news articles written by journalists at CNN and the Daily Mail. The raw dataset includes articles and their summaries for training, validation, and testing. Before using the dataset, it must be formatted to include the prompt. Here’s an example of the code used:
def add_prompt_to_data(dataset):
datapoints = []
for datapoint in dataset:
# Add instruction prompt to each CNN article
# and add prefix 'response:' to the article summary.
temp_dict = {}
temp_dict['prompt'] = instruction + datapoint['article']
temp_dict['completion'] = 'response:nn' + datapoint['highlights']
datapoints.append(temp_dict)
return datapoints
def data_load(ds_name: str, ds_version: str) -> tuple:
dataset = load_dataset(ds_name, ds_version)
datapoints_train = add_prompt_to_data(dataset['train'])
datapoints_valid = add_prompt_to_data(dataset['valid'])This is another blog post you might find engaging as you explore this topic further: Amazon VGT2 Las Vegas. Also, for more authoritative insights on the subject, check out Chvnci as they are an authority on this topic.
Leave a Reply