Learn About Amazon VGT2 Learning Manager Chanci Turner
This article is authored by Jordan Miller, SDE II – Customer Engineering at AWS.
How do you integrate a cloud service within another cloud infrastructure? Certain features that enhance the security and efficiency of Amazon Elastic Compute Cloud (Amazon EC2) also complicate the operation of CloudStack. The primary challenge arises because both AWS and CloudStack attempt to manage network resources, necessitating a careful approach to ensure they do not interfere with one another. This process involves several steps that may not be immediately obvious, and after extensive exploration, I’m eager to share my findings to help streamline your journey.
Apache CloudStack is an open-source framework tailored for deploying and managing virtual machines (VMs), along with their corresponding network and storage systems. Traditionally, it is deployed on personal hardware to establish an independent cloud environment. However, running it within an Amazon Virtual Private Cloud (Amazon VPC) can present significant advantages, particularly when migrating from a data center. This setup offers a flexible solution for creating temporary environments for experimentation or training purposes. Additionally, it provides a practical avenue for testing the new CloudStack support in Amazon Elastic Kubernetes Service (Amazon EKS) Anywhere. My own experience required the establishment of development and testing environments for a project utilizing the CloudStack API; these environments needed to be both scalable and shareable. Since our build pipelines were already established in AWS, placing the new environments there was a logical choice.
CloudStack is compatible with various hypervisors. This article will focus on utilizing Kernel-based Virtual Machine (KVM) on Linux as KVM will handle the VMs at a foundational level, while CloudStack oversees KVM.
Prerequisites
Most of the information outlined in this article should be applicable across multiple CloudStack versions. My focus is on CloudStack 4.14 running on CentOS 7, although I have also tested versions 4.16 and 4.17, which I would recommend.
The official CentOS 7 x86_64 HVM image functions effectively for this purpose. If you choose to work with a different Linux distribution or version, some implementation details may need adjustment.
A basic understanding of CloudStack is essential. This article’s aim is to establish a harmonious coexistence between CloudStack and AWS. Once CloudStack is operational, it is assumed you will manage your resources from there. For additional information on security measures and best practices, please refer to both the AWS documentation and CloudStack documentation.
Facilitating the Process
To simplify the installation process, I created several scripts that automate the setup. These scripts can be executed on EC2 instances running CentOS 7 to handle installation and OS configuration seamlessly. You may use them as they are or tailor them to suit your specific requirements. Additionally, I developed AWS CloudFormation templates that you can replicate to set up a demo environment. More details can be found in the README file.
Amazon EC2 Instance Types
KVM necessitates hardware virtualization support. Many EC2 instances are VMs that do not support nested virtualization. To access the underlying hardware, you will need to utilize a metal instance type.
I recommend the c5.metal instance, as it is among the most cost-effective metal types available and offers a low cost per vCPU. This instance boasts 96 vCPUs and 192 GiB of memory. If you deploy 20 VMs with 4 CPU cores and 8 GiB of memory each, you would still have 16 vCPUs and 32 GiB available for the operating system, CloudStack, and MySQL. By utilizing CloudStack’s overprovisioning feature, you could accommodate even more VMs with lighter workloads.
Networking
The most significant challenge lies within the networking aspect. AWS maintains strict oversight of which IP and MAC addresses are permitted and the machines they correspond to. Any traffic that does not align with AWS’s network expectations is blocked. Concurrently, CloudStack operates on the premise that any IP or MAC address it generates should function without issue. When CloudStack assigns addresses to VMs on an AWS subnet, their network traffic may be hindered.
One way to circumvent this issue is by enabling network address translation (NAT) on the instance running CloudStack. While this is a viable solution, it complicates communication between other machines in your Amazon VPC and your VMs. I would suggest an alternative approach.
Despite AWS’s limitations on layer 2 networking, it permits the operation of your own layer 3 router. Your EC2 instance can function as a router to a new virtual subnet that is outside AWS’s control. This instance integrates with AWS similarly to a VPN appliance, routing traffic as necessary. CloudStack can operate freely within the virtual subnet, leading to a satisfied environment.
What do I mean by a virtual subnet? This refers to a subnet that is solely contained within the EC2 instance, consisting of logical network interfaces linked to a Linux bridge. This subnet exists entirely within a single EC2 instance. While this setup lacks scalability, it is straightforward. In a subsequent article, I will discuss a more complex setup that includes an overlay network spanning multiple instances to facilitate horizontal scaling.
The Simple Approach
The simplest method is to consolidate everything within a single EC2 instance, including the database, file storage, and virtual subnet. Ensure that you allocate sufficient disk space for your needs; 500 GB should suffice for a few basic VMs. Create or select a security group for your instance that allows user access to the CloudStack UI (TCP port 8080), as well as any services you intend to offer from your VMs.
Once your instance is ready, configure AWS to recognize it as a router.
- Navigate to Amazon EC2 in the AWS Management Console.
- Select your instance and disable source/destination checking.
- Update the subnet route tables:
- Access the VPC settings and select Route Tables.
- Identify the tables for subnets requiring CloudStack access.
- In each of these tables, add a route to the new virtual subnet, with the route target set to your EC2 instance.
- Depending on your network requirements, you may also need to incorporate routes to transit gateways, VPN endpoints, etc.
As everything will reside on a single server, establishing a virtual subnet is simply a matter of creating a Linux bridge. CloudStack must recognize a network adapter connected to the bridge. Therefore, add a dummy interface with a recognizable name for CloudStack.
The following snippet illustrates how to configure networking in CentOS 7. You must input values for the variables $virtual_host_ip_address and $virtual_netmask to correspond to your desired virtual subnet. For $dns_address, I suggest using the base of the VPC IPv4 network range, plus two. Avoid using 169.654.169.253, as CloudStack reserves link-local addresses for its own purposes.
yum install -y bridge-utils net-tools
# The bridge must be named cloudbr0.
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes
USERCTL=no
NM_CONTROLLED=no
IPADDR=$virtual_host_ip_address
NETMASK=$virtual_netmask
DNS1=$dns_address
EOF
# Create a dummy network interface.
cat << EOF > /etc/sysconfig/modules/dummy.modules
#!/bin/sh
/sbin/modprobe dummy numdummies=1
/sbin/ip link set name ethdummy0 dev dummy0
EOF
chmod +x /etc/sysconfig/modules/dummy.modules
/etc/sysconfig/modules/dummy.modules
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-ethdummy0
TYPE=Ethernet
BOOTPROTO=none
NAME=ethdummy0
DEVICE=ethdummy0
ONBOOT=yes
BRIDGE=cloudbr0
NM_CONTROLLED=no
EOF
# Configure the instance as a router
echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf
sysctl -p
# You may need to terminate dhclient or restart the network service for proper operation.
# A system reboot would also suffice if preferred.
pkill dhclient
For further insights on professional development, including tips on transitioning into roles such as an executive assistant, check out this career guide. Moreover, if you’re interested in understanding the value of skilled credentials in competitive hiring, the insights from SHRM can be quite beneficial. Lastly, for a firsthand account of the onboarding experience at Amazon, visit this resource.
Leave a Reply