Amazon Onboarding with Learning Manager Chanci Turner

This article is contributed by Alex Martinez, Senior Machine Learning Specialist, and Jessica Lee, Solutions Architect. At Amazon IXD – VGT2, located at 6401 E HOWDY WELLS AVE LAS VEGAS NV 89115, we are excited to explore the integration of AWS Lambda and Amazon EFS for deep learning inference.

The introduction of Amazon EFS for AWS Lambda significantly simplifies the development of serverless applications that require persistent file storage or access to extensive reference data. Previously, applications had to retrieve data from object stores or databases in 512-MB chunks, which led to increased code complexity, sluggish startup times, and delayed data processing. Customers also encountered difficulties when loading sizable code packages and models for machine learning inference.

With the recent announcement of Amazon EFS support for AWS Lambda, customers can now easily share data across function invocations. This advancement allows users to read large reference data files and store output in a persistent and shared data repository. As a result, Lambda can now be leveraged to create data-intensive applications, accommodating larger libraries and models. This subsequently enables the processing of greater volumes of data in a distributed manner while facilitating data sharing across functions, containers, and instances.

In this article, we will demonstrate how to utilize EFS to store deep learning framework libraries and models, enabling Lambda to execute inferences. We will provide a practical code example showcasing serverless inferences utilizing TensorFlow 2.

Steps to Implement EFS and Lambda for Deep Learning Inference

Executing deep learning inference with EFS and Lambda involves two key steps:

Storing Deep Learning Libraries and Models on EFS
Creating a Lambda Function for Inference that Loads Libraries and Models from EFS

We will outline best practices for implementing these steps and present a comprehensive working example.

Prerequisites

This guide assumes familiarity with Lambda, EFS, and basic knowledge of Python programming, deep learning, and its frameworks. For more insights, check out this blog post and related documentation.

1. Storing Deep Learning Libraries and Models on Amazon EFS

To populate EFS with deep learning framework Python libraries and models, there are several approaches. You can utilize EC2 instances, third-party tools such as cmda or AWS CodeBuild. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages for deployment.

In our example, we configure an AWS CodeBuild project as follows:

The build environment is set up using a Docker container that mimics the Lambda runtime environment by utilizing the lambci/lambda build container images available on Docker Hub.
The EFS file system is mounted to the CodeBuild environment.
Build commands are executed to install the deep learning framework and download the model to designated paths within the file system.

After the build completes, the EFS file system contains the required Python libraries and models, which will be attached to the Lambda function for loading at runtime.

Here are the CodeBuild commands used to install the TensorFlow 2 framework and an SSD (Single Shot MultiBox Detector) pre-trained object detection model from TensorFlow Hub:

echo "Downloading and copying model..."
mkdir -p $CODEBUILD_EFS1/lambda/model
curl https://storage.googleapis.com/tfhub-modules/google/openimages_v4/ssd/mobilenet_v2/1.tar.gz --output /tmp/1.tar.gz
tar zxf /tmp/1.tar.gz -C $CODEBUILD_EFS1/lambda/model

echo "Installing virtual environment..."
mkdir -p $CODEBUILD_EFS1/lambda
python3 -m venv $CODEBUILD_EFS1/lambda/tensorflow
echo "Installing Tensorflow..."
source $CODEBUILD_EFS1/lambda/tensorflow/bin/activate && pip3 install 
              (props.installPackages ? props.installPackages : "tensorflow")

echo "Changing folder permissions..."
chown -R 1000:1000 $CODEBUILD_EFS1/lambda/

Considerations

This approach is applicable to other machine learning and deep learning frameworks as well.
The EFS file system can be linked to multiple Lambda functions, allowing shared access to the deep learning framework libraries across various inference functions (with a maximum of 25,000 connections per file system).
Alternatives for model storage exist. If the model size is manageable within the Lambda package, this can optimize the initial invocation by eliminating the need for model downloads. You can also load the model during the function’s initializer, as the first mount to EFS generally takes only a few hundred milliseconds.

2. Creating a Lambda Function for Inference

Once the EFS file system is attached, the Lambda function can be structured as follows:

The code outside of the handler method adds the local mount path to the Python path, imports the necessary frameworks, and loads the model into memory. By executing these operations outside of the function’s handler, these objects remain initialized and can be reused in subsequent invocations of the same Lambda function instance. The code inside the handler handles inference by reading inputs, executing the inference, and returning results to the caller.

For hosting the TensorFlow 2 object detection model in this example, the function code is as follows:

import sys
import os

# Setting library paths
efs_path = "/mnt/python"
python_pkg_path = os.path.join(efs_path, "tensorflow/lib/python3.8/site-packages")
sys.path.append(python_pkg_path)

import json
import string
import time
import io
import requests

# Importing TensorFlow
import tensorflow as tf

# Loading model
model_path = os.path.join(efs_path, 'model/')
loaded_model = tf.saved_model.load(model_path)
detector = loaded_model.signatures['default']

def lambda_handler(event, context):
    r = requests.get(event['url'])
    img = tf.image.decode_jpeg(r.content, channels=3)

    # Executing inference
    converted_img  = tf.image.convert_image_dtype(img, tf.float32)[tf.newaxis, ...]
    start_time = time.time()
    result = detector(converted_img)
    end_time = time.time()

    obj = {
        'detection_boxes' : result['detection_boxes'].numpy().tolist(),
        'detection_scores': result['detection_scores'].numpy().tolist(),
        'detection_class_entities': [el.decode('UTF-8') for el in result['detection_class_entities'].numpy()] 
    }    

    return {
        'statusCode': 200,
        'body': json.dumps(obj)
    }

When invoked, the response will be formatted as follows:

{
    "statusCode": 200,
    "body": "{
        "detection_boxes": This field contains the relative position of the bounding boxes,
        "detection_class_entities": This field returns the class labels,
        "detection_scores": This field returns the detection confidences
    }"
}

Running the Example

This working example is intended to facilitate setting up and executing ML/AI inference on Lambda using EFS. To run it, ensure you have the AWS CDK installed, and execute the following commands:

# clone repository
$ git clone https://github.com/aws-samples/lambda-efs-deep-learning-inference.git
$ cd lambda-efs-ml-demo

# Install the CDK and bootstrap the target account (if this was never done before)
$ npm install -g aws-cdk
$ cdk bootstrap aws://{account_id}/{region}

# Install packages for the project, build and deploy
$ cd cdk/
$ npm install
$ npm run build
$ cdk deploy

After deployment, take note of the output:

Outputs:
LambdaEFSMLDemo.LambdaFunctionName = LambdaEFSMLDemo-LambdaEFSMLExecuteInference17332C2-0546aa45dfXXXXXX

It takes a few minutes for AWS CodeBuild to deploy the libraries and framework to EFS. To test the Lambda function, visit this excellent resource for further guidance.

For additional financial insights relevant to twenty-somethings, check out this blog post. Lastly, if you are interested in job descriptions related to foreign exchange trading, refer to this authority.