We are excited to announce the addition of a new region to our High Performance Computing (HPC) service on EC2, along with a new template that simplifies the process of quickly spinning up clusters. Let’s take a moment to review the evolution of High Performance Computing on EC2.
The Evolution of High Performance Computing on EC2
High Performance Computing (HPC) users have an unyielding demand for computational power, frequently hindered by the limitations of available resources. Scientists and engineers in this domain often find themselves constraining their research based on what they can access. Therefore, it was predictable that shortly after the launch of Amazon EC2 in 2006, customers began leveraging the utility computing model to eliminate these limitations by deploying large-scale computing clusters. This allowed them to expedite their projects and expand their research capabilities, enabling simulations with larger molecular counts, higher resolutions, or more extensive galaxies. From Monte Carlo simulations to life sciences R&D, these clusters have been put to the test.
In December 2009, the introduction of the Spot Market provided an additional advantage, offering customers the opportunity to run at a greater scale for the same cost or even select their compute run prices. This was particularly beneficial for batch processing and Hadoop users. However, we noted a desire from customers for the ability to enhance their large-scale systems with more tightly coupled and parallel jobs.
In July 2010, we responded to this need by launching a new instance type specifically designed for HPC clusters. The Cluster Compute instance was constructed on a high-performance network to facilitate tightly coupled parallel computations over a non-blocking, fully bisectional 10 gigabit Ethernet. It also introduced hardware virtualization and the concept of Placement Groups, allowing instances to be physically located closer for low-latency communication. The lineup was further enhanced with GPUs in November 2010 and the introduction of the second generation in November 2011 (cc2.8xlarge instances featuring the Intel Xeon E5 chip), which even made it to the Top 500 list of the world’s fastest supercomputers at position 42. Notable research organizations and businesses requiring high-bandwidth networking and extensive CPU cycles have adopted the Cluster Compute instances.
Expanded Availability
Today marks the expansion of CC2 instances into a new region, enabling users to benefit from high-performance computing capabilities on the West Coast. US West (Oregon) now joins US East (Virginia) and EU West (Ireland), allowing you to harness the power of cc2.8xlarge instances for your data and teams in the West.
Enhanced Usability
We continuously strive to improve the usability of our HPC services, helping scientists and engineers access their familiar tools swiftly. Today, we are introducing a new script that will facilitate the automatic provisioning, configuration, and installation of a fully operational HPC cluster in just five clicks.
Your first click is here:
This action will set up an AWS CloudFormation template in the console, where you can select the cluster size to launch and the name of the keypair for login. After reviewing your settings, click continue, and we will provision a fully functional HPC cluster, equipped with Open MPI, Open Grid Scheduler/Grid Engine, ATLAS libraries, NFS for data sharing, SciPy, NumPy, and iPython (thanks to the fantastic StarCluster). Additionally, example code and instructions for execution have been pre-loaded.
Once the stack is operational, you will find connection details in the ‘Outputs’ tab for SSH access or links to a webpage that provides further insights on running parallel tasks via MPI or the scheduler. You can also elastically scale your cluster up or down using StarCluster.
We encourage you to give it a try and share how you utilize all these FLOPS. For more insights, check out this related blog post. If you’re looking for authoritative resources on HPC, visit this site. Also, for details on training associates at Amazon fulfillment centers, explore this excellent resource here.
Amazon IXD – VGT2
6401 E Howdy Wells Ave, Las Vegas, NV 89115
SEO Metadata
Leave a Reply