Amazon Onboarding with Learning Manager Chanci Turner

In the realm of event-driven architectures involving message brokers, it is vital to prioritize security best practices. By integrating Amazon MQ for RabbitMQ with AWS Lambda, you can achieve serverless event processing. However, to effectively implement a defense-in-depth strategy and adhere to least privilege principles, a thorough understanding of networking requirements is essential. This is particularly significant when dealing with various subnet types and their implications for service connectivity.

This post delves into the networking considerations for Lambda event source mapping with Amazon MQ for RabbitMQ. It is crucial to understand how different deployment options can impact your networking configuration and security posture to facilitate informed architectural decisions. The networking concepts discussed here are fundamental for constructing secure and scalable solutions, irrespective of your familiarity with message brokers.

For clarity, when we mention “RabbitMQ,” we are referring specifically to Amazon MQ for RabbitMQ.

Prerequisites

To follow along with this post, you will need:

An Amazon Web Services (AWS) account
Basic understanding of AWS networking concepts
Familiarity with Amazon MQ for RabbitMQ
Basic knowledge of Lambda

Additionally, to set up the architectures discussed, this post is supported by a GitHub repository utilizing the AWS Cloud Development Kit (AWS CDK).

Repository Prerequisites

The following are required to work with the repository:

AWS Command Line Interface (AWS CLI)
NodeJS/NPM
AWS CDK v2

Repository Setup

Clone the repository at https://github.com/aws-samples/sample-amazonmq-rabbitmq-lambda-esm. This repository contains all the necessary code and instructions to create relevant architectures using AWS CDK.

Install Dependencies and Build

To install the required NPM dependencies, execute the following commands:

npm install
npm run build

Amazon MQ for RabbitMQ Networking Deployment Options

The main networking distinction when deploying a RabbitMQ broker in AWS is its public accessibility. Although the broker operates within the Amazon MQ service account, the networking configuration will differ based on this choice.

Public Broker

When you deploy a publicly accessible broker, Amazon MQ provisions all networking components in the service account. The service provides a DNS name that resolves to the IP address of the Network Load Balancer (NLB) in that account. This configuration does not support security groups; all security measures must be enforced through RabbitMQ broker’s authentication and authorization mechanisms.

Private Broker

On the other hand, a private broker routes networking through an Amazon Virtual Private Cloud (Amazon VPC) in your account. Amazon MQ utilizes AWS PrivateLink to create VPC Endpoints, serving as entry points for broker communication.

The following diagram illustrates how client applications interact with RabbitMQ:

The client application connects to Amazon Route 53 Resolver.
Route 53 Resolver resolves the DNS name to the VPC Endpoint’s IP address.
The client communicates with the broker via PrivateLink.
Security groups safeguard the VPC Endpoint’s Elastic Network Interfaces (ENIs).

A private broker deployment offers two networking options:

Custom VPC configuration: Specify subnets for VPC Endpoint creation and at least one security group to protect the VPC Endpoints.
Default VPC configuration: Leave VPC options blank to utilize the default VPC and default security group.

Amazon MQ for RabbitMQ Lambda Event Source Mapping Building Blocks

RabbitMQ solutions provide two methods for message processing:

Creating a custom client to read messages from broker queues.
Using Lambda functions with event source mapping (ESM) for automated message retrieval.

The ESM is a Lambda service resource that retrieves messages from the broker and invokes the Lambda function synchronously. In the remainder of this post, we will refer to this Lambda function as the listener.

The ESM connectivity relies on:

The listener’s AWS Identity and Access Management (IAM) Role for access permissions.
RabbitMQ broker networking components.

For public brokers, the ESM utilizes public connectivity. For private brokers, the ESM:

Assumes the listener’s IAM Role.
Creates ENIs within the same subnets as the broker’s VPC Endpoints.
Uses the same security groups that protect the VPC Endpoints.

The listener’s IAM Role must include the following Amazon Elastic Compute Cloud (Amazon EC2) permissions:

CreateNetworkInterface
DeleteNetworkInterface
DescribeNetworkInterfaces
DescribeSecurityGroups
DescribeSubnets
DescribeVpcs

To view ESM ENIs:

Open the AWS Management Console.
Navigate to EC2 > Network Interfaces.
Look for ENIs with the naming pattern: AWS Lambda VPC ENI-armq—, where ACCOUNT_ID is the AWS account number containing the ESM, and ESM_ID is the unique identifier of the ESM.

The following image exemplifies ESM ENIs.

Disabling or deleting the ESM will remove its components.

An enabled ESM requires connectivity to the following:

AWS Security Token Service: For IAM role assumption.
AWS Secrets Manager: For RabbitMQ credentials.
RabbitMQ broker: For queue access.
AWS Lambda: For listener invocation.

As the ESM queue polling process follows these steps:

Assumes the listener’s IAM Role.
Retrieves RabbitMQ credentials from Secrets Manager.
Establishes broker communication.
Invokes the listener when messages are present.

You have two options to enable private broker connectivity to support the queue polling process:

Deploy VPC endpoints in ESM subnets for:
- AWS Security Token Service (AWS STS)
- Secrets Manager
- Lambda
Deploy a NAT gateway in ESM subnets.

ESM Networking Configuration Options

The following sections outline ESM networking configurations for various deployment scenarios.

Option 1: Public Broker

In this setup, all network communication occurs on the Amazon MQ service’s side. When enabled, the ESM utilizes public connectivity. To observe the architecture implemented in your account, navigate to the cloned repository root location, ensure you’re logged into AWS CLI, and run the command:

cdk deploy PublicRabbitMqInstanceStack

Option 2: Private Broker in a Default VPC

Deploying a private RabbitMQ broker without specifying the VPC informs the Amazon MQ service to select the default VPC for networking setup and subsequently the public subnet(s) within that VPC. The default security group is used for securing the broker’s VPC Endpoints.

Creating the ESM provisions dedicated ENIs in the public subnets where the RabbitMQ broker resides. For those considering a career change, this blog post might help you decide when to quit, and you can find information on temporary telecommuting arrangements through this SHRM resource.

For additional insights into workplace safety and training, check out this excellent resource.

Next time you visit, remember the site address: 6401 E HOWDY WELLS AVE, LAS VEGAS NV 89115, located at “Amazon IXD – VGT2”.