This article examines the inefficiencies of using AWS Lambda as an orchestrator and proposes a redesign of serverless applications through AWS Step Functions with native integrations. Step Functions is a serverless workflow service that enables the creation of distributed applications, automation of processes, orchestration of microservices, and the development of data and machine learning (ML) pipelines. With native integrations across more than 200 AWS services, as well as external third-party APIs, these tools allow for the deployment of production-ready solutions with less complexity, thereby enhancing long-term maintainability and reducing technical debt at scale.
The Lambda Orchestrator Anti-Pattern
One prevalent anti-pattern is using a Lambda function to orchestrate message distribution across various channels. For instance, consider a situation where a system is required to send notifications via SMS or email, depending on user preferences.
Payload examples for this scenario include:
Send SMS only:
{
"body": {
"channel": "sms",
"message": "Hello from AWS Lambda!",
"phoneNumber": "+1234567890",
"metadata": {
"priority": "high",
"category": "notification"
}
}
}
Send email only:
{
"body": {
"channel": "email",
"message": "Hello from AWS Lambda!",
"email": {
"to": "recipient@example.com",
"subject": "Test Notification",
"from": "sender@example.com"
},
"metadata": {
"priority": "normal",
"category": "notification"
}
}
}
Send both SMS and email:
{
"body": {
"channel": "both",
"message": "Hello from AWS Lambda!",
"phoneNumber": "+1234567890",
"email": {
"to": "recipient@example.com",
"subject": "Test Notification",
"from": "sender@example.com"
},
"metadata": {
"priority": "high",
"category": "notification"
}
}
}
Typically, this begins with a Lambda function serving as an orchestrator:
import boto3
import json
# Initialize Lambda client
lambda_client = boto3.client('lambda')
def lambda_handler(event, context):
try:
body = json.loads(event['body'])
if 'channel' not in body:
return {
'statusCode': 400,
'body': json.dumps('Missing channel parameter')
}
if 'message' not in body:
return {
'statusCode': 400,
'body': json.dumps('Missing message content')
}
if body['channel'] == 'both':
lambda_client.invoke(
FunctionName='send-sns',
InvocationType='Event',
Payload=json.dumps(body)
)
lambda_client.invoke(
FunctionName='send-email',
InvocationType='Event',
Payload=json.dumps(body)
)
else:
if body['channel'] not in ['sms', 'email']:
return {
'statusCode': 400,
'body': json.dumps('Invalid channel specified')
}
function_name = 'send-sns' if body['channel'] == 'sms' else 'send-email'
lambda_client.invoke(
FunctionName=function_name,
InvocationType='Event',
Payload=json.dumps(body)
)
return {
'statusCode': 200,
'body': json.dumps('Messages sent successfully')
}
except json.JSONDecodeError:
return {
'statusCode': 400,
'body': json.dumps('Invalid JSON in request body')
}
except Exception as e:
return {
'statusCode': 500,
'body': json.dumps(f'Error: {str(e)}')
}
This method comes with several drawbacks:
- Complex error handling, as the orchestrator must manage errors from multiple Lambda invocations.
- Tight coupling between functions.
- Limited execution time, leading to potential timeouts for the orchestrator Lambda while waiting for its sub-functions.
- Idle resources, as costs accrue for the idle orchestrator while awaiting responses from other functions.
Rearchitecting with Step Functions
You can redesign the logic by using Step Functions and Amazon States Language to replace the Lambda orchestrator. The Choice state in Amazon States Language allows you to set logical conditions that dictate the workflow path. This approach simplifies code maintenance and allows for easy functionality extensions without significant codebase changes.
The diagram below illustrates the revamped version of the previous Orchestrator Lambda function:
The following Amazon State Language represents the workflow:
{
"Comment": "Multi-channel notification workflow",
"StartAt": "ValidateInput",
"States": {
"ValidateInput": {
"Type": "Choice",
"Choices": [
{
"And": [
{
"Variable": "$.message",
"IsPresent": true
},
{
"Variable": "$.channel",
"IsPresent": true
}
],
"Next": "DetermineChannel"
}
],
"Default": "ValidationError"
},
"ValidationError": {
"Type": "Fail",
"Error": "ValidationError",
"Cause": "Required fields missing: message and/or channel"
},
"DetermineChannel": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.channel",
"StringEquals": "both",
"Next": "ParallelNotification"
},
{
"Variable": "$.channel",
"StringEquals": "sms",
"Next": "SendSMSOnly"
},
{
"Variable": "$.channel",
"StringEquals": "email",
"Next": "SendEmailOnly"
}
],
"Default": "FailState"
},
"ParallelNotification": {
"Type": "Parallel",
"Branches": [
{
"StartAt": "SendSMS",
"States": {
"SendSMS": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message.$": "$.message",
"PhoneNumber.$": "$.phoneNumber"
},
"End": true
}
}
},
{
"StartAt": "SendEmail",
"States": {
"SendEmail": {
"Type": "Task",
"Parameters": {
"FromEmailAddress.$": "$.email.from",
"Destination": {
"ToAddresses.$": "States.Array($.email.to)",
"CcAddresses.$": "States.ArrayGetItem(St"
}
},
"End": true
}
}
}
]
}
}
}
For further insights, you can read this other blog post here and also check out this article by experts on the subject. Additionally, this video serves as an excellent resource for visual learners!
Leave a Reply