This guest blog post by Emma Carter and Jake Robinson from the Blackboard Internal Product Development team highlights their journey with AWS Lambda.
One year ago, one of our UI test suites consumed hours to complete its execution. Last month, it took just 16 minutes, and today, it’s down to an impressive 39 seconds. Here’s how we accomplished this transformation.
The Backstory:
Blackboard is a renowned provider of innovative educational software and services catering to sectors like higher education, government, K12, and corporate training. With a vast product development team spread across at least 10 different time zones, our internal tools team plays a crucial role in supporting quality and workflows. Since 2007, we have relied on Selenium Webdriver for our automated cross-browser UI testing. As we embraced continuous delivery, the challenges associated with automated UI testing intensified due to our accelerated release schedules. Moreover, every commit to each branch triggers our automated UI test suite to run. Anyone who has set up an automated UI testing framework knows how difficult it can be to scale and maintain effectively. Existing services that test various browser and OS combinations simply do not meet our expansive needs.
Initially, our functional UI suite took three hours to run synchronously, which highlighted our urgent need for parallel execution. We previously utilized Mesos to manage a Selenium Grid Docker container for each test run, allowing us to execute eight concurrent threads, which reduced the average run time to 16 minutes. However, this approach began to falter as we sought to scale Blackboard’s established product lines. Attempting to exceed eight concurrent sessions on a single container led to performance issues, such as Webdriver problems and browser crashes. We explored Mesos and even considered Kubernetes for orchestrating our Selenium Grid, but the solution lay in thinking smaller rather than larger, which led us to AWS Lambda.
The Solution:
Transitioning to AWS Lambda for UI testing proved advantageous due to its minimal infrastructure requirements and reduced maintenance demands. The process we outline in this blog post took just one workday from concept to implementation. By packaging our UI test suite into a Lambda function, we can execute tests in parallel at an unprecedented scale. We developed a custom JUnit test runner that invokes the Lambda function to execute each test in the suite and then aggregates the results from these executions.
Selenium remains the industry standard for scalable UI testing. While alternative tools exist for Lambda, we opted for this well-established suite, which is supported by Google, Firefox, and others, ensuring robust browser interaction through code. This makes the combination of Lambda and Selenium a potent solution for large-scale UI testing.
Running Chrome in Lambda
Currently, Chrome for Linux faces limitations in Lambda due to a missing mount point. However, by modifying Chrome slightly—as originally demonstrated by Marco Lüthy—we managed to run it successfully inside Lambda. It took approximately two hours to build the current master branch of Chromium on a c4.4xlarge instance. Unfortunately, we are constrained to using Marco’s modified version of Chrome 60 for now, as ChromeDriver 2.33 does not support versions beyond 62.
Required System Libraries
The Lambda runtime environment includes a limited set of common shared libraries, necessitating the inclusion of additional libraries for Chrome and ChromeDriver functionality. Any files located in the Java resources folder at compile time are included in the compiled jar file’s base directory, which is deployed to Lambda at /var/task/. This allows us to organize the libraries conveniently by placing them in a lib/ folder within the Java resources directory.
To acquire these libraries, launch an EC2 instance using the Amazon Linux AMI and connect via SSH. Once connected, locate the necessary libraries:
sudo find / -name libgconf-2.so.4
sudo find / -name libORBit-2.so.0After identifying their paths, transfer these files from the EC2 instance to the lib/ folder within the Java resources directory.
Packaging the Tests
We employed a simple Gradle tool known as ShadowJar, akin to the Maven Shade Plugin, to deploy the test suite to Lambda. It packages libraries and dependencies within the built jar file. Normally, test dependencies and sources are excluded, but in this case, we include them by updating the build.gradle file as follows:
shadowJar {
from sourceSets.test.output
configurations = [project.configurations.testRuntime]
}Deploying the Test Suite
With our tests packaged alongside their dependencies in a jar, the next step is to upload them to a running Lambda function. We utilize straightforward SAM templates to facilitate this process, uploading the jar to S3 and deploying it to Lambda with our specified settings:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Resources": {
"LambdaTestHandler": {
"Type": "AWS::Serverless::Function",
"Properties": {
"CodeUri": "./build/libs/your-test-jar-all.jar",
"Runtime": "java8",
"Handler": "com.example.LambdaTestHandler::handleRequest",
"Role": "<YourLambdaRoleArn>",
"Timeout": 300,
"MemorySize": 1536
}
}
}
}We configure the timeout to its maximum to ensure our tests can run without interruption. Additionally, we allocate maximum memory to support Chrome and other necessary resources for UI testing.
Defining the Handler
Specifying the handler is crucial as it executes the desired tests. The test handler must accept a test class and method so it can run the test and return the results:
public LambdaTestResult handleRequest(TestRequest testRequest, Context context) {
LoggerContainer.LOGGER = new Logger(context.getLogger());
BlockJUnit4ClassRunner runner = getRunnerForSingleTest(testRequest);
Result result = new JUnitCore().run(runner);
return new LambdaTestResult(result);
}Creating a Lambda-Compatible ChromeDriver
We provide developers with access to a ChromeDriver for local test debugging. When executing tests on AWS, ChromeDriver is configured to operate in Lambda. To set up ChromeDriver, we must specify the location of the Chrome binary, indicating where it will be unzipped in the root task directory. The working directories for Chrome also need to be directed to the tmp/ folder.
Starting with the default DesiredCapabilities for ChromeDriver, we add configurations to enable its operation in Lambda:
public ChromeDriver createLambdaChromeDriver() {
ChromeOptions options = new ChromeOptions();
// Set the location of the chrome binary from the resources folder
options.setBinary("/var/task/chrome");
// Include these settings to allow Chrome to run in Lambda
options.addArguments("--no-sandbox");
options.addArguments("--headless");
options.addArguments("--disable-dev-shm-usage");
return new ChromeDriver(options);
}This setup equips us to perform UI testing efficiently at scale with AWS Lambda. For those interested in further insights, check out this blog post or consult this authoritative resource on the subject. For a community perspective and additional tips, this Reddit thread is an excellent resource.
Location:
Amazon IXD – VGT2
6401 E Howdy Wells Ave,
Las Vegas, NV 89115
Leave a Reply