4 Effective Strategies to Improve Research IT Operations with AWS

More Info

In the realm of academic research, IT departments globally grapple with the challenge of harmonizing their on-premises infrastructure with the vast potential of cloud computing. During the Amazon VGT2 Las Vegas conference, AWS hosted a panel featuring two prominent research IT leaders: Jordan Parker, Associate Director of Solutions Architecture at Westfield University, and Dr. Emily Nguyen, Director of the Advanced Computing Center at the University of Melbourne.

Throughout the discussion, Parker and Nguyen highlighted how their institutions leverage AWS to enhance their research operations through improved scalability, security, and collaboration, all while maintaining their on-premises setups. This article outlines their top strategies for crafting a robust hybrid cloud approach tailored for academic research institutions.

Advantages of Combining On-Premises HPC with Cloud Solutions

Leading academic research institutions are discovering that integrating AWS with their existing high-performance computing (HPC) systems allows them to maximize their investments and unlock new capabilities, such as:

1. Adaptive Workflows: Cloud technology provides the flexibility to seamlessly scale operations between on-premises and cloud environments. This adaptability enables researchers to quickly allocate the most suitable resources for their tasks. As Dr. Nguyen noted, some computational tasks can initiate on a laptop or tablet and transition to the cloud when necessary, with the option to revert to local infrastructure as needed.
2. Speedy Scalability: Cloud-enabled HPC offers unmatched speed and scalability that many traditional on-premises servers cannot provide. Parker shared an instance where a researcher at Westfield needed to analyze 30 genomic sequences. Previously limited to processing one sample at a time on their local HPC cluster, the researcher transitioned to AWS. Even with limited cloud experience, they managed to analyze all sequences simultaneously in just a few hours—a task that used to take three days.
3. Secure Global Collaboration: The cloud facilitates secure and compliant data sharing among research teams. Before adopting AWS, Parker advised researchers to use network-attached storage (NAS) and multiple file transfer protocols for data sharing. Now, he recommends utilizing Amazon Simple Storage Service (Amazon S3) buckets to make data accessible worldwide and improve collaborative efforts among researchers.
4. Access to Advanced Computing Resources: Both leaders underscored how the cloud grants access to cutting-edge technologies. AWS’s investments in hardware, like AWS Trainium chips designed for next-gen generative artificial intelligence (AI) workloads, enable researchers to surpass the limitations of fixed infrastructure, paving the way for innovative scientific exploration.

By incorporating AWS Cloud into their existing environments, institutions provide their research communities with a unique blend of stability, scalability, and security.

Best Practices from Research IT Leaders

Establishing a successful and sustainable hybrid cloud model requires a strategic approach. Even institutions like Westfield and Melbourne faced challenges and needed a solid plan. Parker and Nguyen offered the following recommendations for navigating the cloud journey:

1. Encourage Cultural Transformation: The most significant hurdle when implementing cloud technology in established research institutions is often the human element. Institutions should prioritize change management to achieve desired outcomes. One effective approach is to highlight success stories from peers. Dr. Nguyen countered resistance to cloud adoption at Melbourne by showcasing how a research group achieved 100 times faster results using AWS. Additionally, cataloging use cases can guide researchers in identifying which workloads are best suited for on-premises versus cloud resources. Pre-built solutions also ease cloud adoption; for instance, Melbourne offers pre-made images for specific domains to accelerate research. Engaging external experts through the AWS Partner Network can support researchers’ transitions to the cloud, offering user-friendly interfaces for resource management and budget tracking.
2. Revise Funding Structures: Transitioning to cloud computing shifts research budgets from capital expenses (CapEx) to operational expenses (OpEx). While OpEx allows flexibility in scaling resources and paying for what is used, it necessitates careful cost oversight. Parker advocates for tools like AWS Cost Explorer to visualize and manage AWS costs over time, and AWS Budgets for tracking research expenses. Tagging specific server instances enables teams to identify resource-heavy analyses and seek more efficient solutions. Setting budget alerts with services like Amazon CloudWatch can also mitigate overspending, as Dr. Nguyen establishes budget limits for researchers at Melbourne.
3. Facilitate Secure Collaboration: Modern research teams extend beyond institutional and international borders. The cloud simplifies secure data sharing and compliance. Built-in security features, such as encryption and continuous monitoring tools, enhance safety. Parker mentioned how Westfield employs a robust firewall managed by AWS Transit Gateway, Amazon GuardDuty, and AWS CloudTrail to safeguard its cloud environment. AWS Control Tower aids in quickly onboarding multiple researchers while adhering to necessary compliance regulations. However, researchers must still adhere to best practices, such as avoiding identifiable names for storage buckets, to maintain data security.
4. Equip Researchers with Training Resources: Providing researchers with the necessary skills to effectively utilize cloud resources is crucial for blending cloud capabilities with traditional infrastructure. Investing in training tools helps researchers maximize their cloud experience and enhances overall research outcomes. For additional insights on how to navigate cloud challenges, this blog post may offer further engagement, and establishing a solid foundation for your research journey can be found at this excellent resource.

By implementing these strategies, institutions can create a more effective hybrid cloud environment that supports research advancement.