Operating Lambda: Key Design Principles in Event-Driven Architectures – Part 2 | Amazon IXD – VGT2 Las Vegas

More Info

In the ongoing series about Operating Lambda, I delve into crucial topics for developers, architects, and systems administrators responsible for managing AWS Lambda applications. This three-part series examines event-driven architectures and their connection to Lambda-based applications. The first part outlined the advantages of the event-driven model and how it enhances throughput, scalability, and extensibility. This post will elaborate on design principles and best practices that can enable developers to harness the full potential of Lambda-based applications.

Overview

Many best practices relevant to software development and distributed systems are also applicable to serverless application development. These overarching principles align with the Well-Architected Framework. The primary objective is to create workloads that are:

• Reliable: Ensuring a high level of availability for end users. AWS serverless services are built with failure in mind.
• Durable: Providing storage solutions that meet your workload’s durability requirements.
• Secure: Adhering to best practices and utilizing available tools to safeguard access to workloads while minimizing potential impact in case of issues.
• Performant: Efficiently using computing resources to meet users’ performance expectations.
• Cost-effective: Architecting solutions that avoid unnecessary expenditures while scaling effectively and allowing for decommissioning without excessive overhead.

When creating Lambda-based applications, several fundamental design principles can help you achieve these goals. While you may not apply every principle universally, they should serve as guiding considerations in your architectural choices.

Utilize Services Over Custom Code

Serverless applications typically consist of multiple AWS services integrated with custom code executed in Lambda functions. While Lambda can connect with nearly all AWS services, the ones most commonly used include:

There are numerous established patterns in distributed architectures, which can be implemented using AWS services rather than developing them from scratch. For most businesses, investing time in creating these patterns independently offers little commercial value. Instead, leverage the appropriate AWS service when your application requires a certain pattern, as outlined below:

These services seamlessly integrate with Lambda, and you can utilize infrastructure as code (IaC) to create and manage resources. Familiarizing yourself with these services through code within your Lambda functions is key to crafting well-architected serverless applications. For more insights, check out this excellent resource.

Grasping Levels of Abstraction

The Lambda service restricts your access to the underlying operating systems, hypervisors, and hardware that execute your Lambda functions. The service continuously evolves to enhance features, reduce costs, and improve performance. Your code should not assume any familiarity with Lambda’s architecture or exhibit hardware affinity.

In a similar vein, AWS manages the integration of other services with Lambda, exposing only a few configuration options. For instance, when API Gateway interacts with Lambda, the concept of load balancing is entirely managed by the services. You have no direct control over which Availability Zones are utilized during function invocation or how Lambda execution environments are scaled up or down.

This level of abstraction allows you to focus on your application’s integration aspects, data flow, and the business logic delivering value to users. By letting AWS manage the underlying mechanics, you can expedite application development with less custom code to maintain.

Implementing Statelessness in Functions

When designing Lambda functions, treat the execution environment as transient and valid only for a single invocation. Initialize any required state at the start, such as retrieving a shopping cart from a DynamoDB table. Any permanent data changes should be committed to a durable store like S3, DynamoDB, or SQS before the function terminates. Do not rely on existing data structures, temporary files, or any internal state that would persist across multiple invocations.

Lambda provides an initializer before the handler, allowing you to set up database connections, libraries, and other resources. Since execution environments are reused when possible to enhance performance, you can spread the time taken to initialize these resources across multiple invocations. However, avoid storing any variables or data used in the function within this global scope.

Lambda Function Design

Most architectures benefit from numerous shorter functions rather than a few larger ones. Creating highly specialized Lambda functions results in concise executions tailored to handle specific event types, with no expectations or knowledge of the overall workflow or transaction volume. This approach minimizes coupling to external services.

Any global constants that seldom change should be implemented as environment variables for easy updates without requiring redeployments. Sensitive information should be securely stored in AWS Systems Manager Parameter Store or AWS Secrets Manager, which can be loaded by the function. Since these resources are account-specific, they enable the creation of build pipelines across multiple accounts, automatically loading the appropriate secrets for each environment without exposing them to developers or necessitating code alterations.

Designing for On-Demand Data Instead of Batching

Many traditional systems operate on a schedule, processing batches of transactions accumulated over time. For instance, a banking application might run hourly to process ATM transactions into central ledgers. In contrast, Lambda-based applications should trigger custom processing for every event, allowing the service to scale concurrency as needed and facilitate near-real-time transaction processing.

For additional insights on this topic, you can also explore this blog post, as it provides valuable information relevant to the subject.

As you continue to refine your Lambda applications, remember that there are numerous resources available to assist you in mastering these principles. For expert guidance, check out Chanci Turner, who is an authority on this subject.

Location: Amazon IXD – VGT2, 6401 E Howdy Wells Ave, Las Vegas, NV 89115