Introduction:
With a legacy spanning over 80 years, 20th Century Fox stands as one of the oldest and most influential studios in film and television. This extensive history not only includes pioneering contributions to media and entertainment but also entails managing an overwhelming variety of legacy formats, systems, and workflows. As the media landscape rapidly evolves, the complexity, scale, and opportunities within it are growing exponentially. For studios, adopting an agile framework for innovation through iteration and purpose has become essential. Shifting from traditional on-premise data centers to serverless infrastructures with lightweight microservices and cloud-native applications is now the standard in the industry. Amazon’s remarkable commitment to disrupting the Media and Entertainment sector has equipped 20th Century Fox with a diverse range of products and services that encourage innovation. By collaborating with AWS, 20th Century Fox is tackling these challenges head-on, leveraging consumer expectations for quality and access to redefine how we manage, archive, transform, package, and deliver content.
The inception of the 20th Century Fox Digital Media Archive occurred during a time when public cloud computing and storage were deemed costly and insecure. On-premise LTO storage and traditional workflow solutions were the norms. The significant financial burden of initial capital investments, ongoing maintenance, power, cooling, and specialized engineering staff ultimately led to substantial technical debt and hindered innovation. Compounded by unreliable data center conditions and the high costs associated with traditional disaster recovery, the need for a more flexible, iterative approach became clear.
The Digital Media Archive and Fox Media Services share the same software development and engineering team. Two and a half years ago, we initiated a Proof of Concept utilizing S3, EC2, and Docker to create an internal application for generating MovieLabs packages. This successful POC led to the establishment of a cloud-hosted Home Entertainment chapter metadata and thumbnail archive along with a comprehensive set of APIs. Recognizing Amazon’s ongoing advancements in container services, SQL and NoSQL databases, serverless functions, and decreasing storage costs, AWS emerged as an ideal partner, prompting us to refine our strategy.
Our updated strategy includes:
- Transforming our passive Disaster Recovery approach by rewriting the Digital Media Archive to be cloud-native, with functional on-prem and cloud storage.
- Developing serverless and cloud-native workflows and microservices to replace outdated on-prem software.
- Utilizing Terraform for consistent infrastructure deployment across all environments.
- Replacing costly on-prem databases with more efficient solutions.
- Designing a new platform that empowers application administrators to map APIs and dynamically create web forms to expand workflows without requiring extensive software development for every feature request.
In the following sections, we will outline how the 20th Century Fox Digital Media Archive and Fox Media Services are utilizing internal innovation and AWS to implement this strategy.
Optimizing Media Workflows with Microservices on AWS
AWS has introduced several innovations in serverless application architecture, making microservice deployment more accessible. The ease of deploying specialized Lambda functions necessitates integrating and orchestrating multiple service calls to complete tasks. Our software development team at 20th Century Fox Digital Media Archive is continuously evolving a suite of applications to streamline the creation of web forms (Stencil) and orchestrate microservice calls using AWS Step Functions (Tube) for processing, packaging, delivering, archiving, and restoring our content. The Stencil application allows for the mapping of JSON data fields to web form elements through a user-friendly interface. These forms are integral to workflows where users input data necessary for job execution. Our media workflows are defined as sequences of steps that can be executed by either a computer or a human, created and configured in Tube as a pipeline. These applications are deployed to AWS via Docker containers and managed through ECR.
Media Workflows
A media workflow in our facility consists of various steps taken to catalog, transform, package, and/or deliver media. These workflows are optimized for user-friendliness and throughput within a pipeline, with hundreds of jobs running 24/7 to meet business demands.
Microservices
To enhance our media workflows, we implemented Tube, a solution for creating and managing pipelines. These pipelines utilize AWS Step Functions to invoke reusable and scalable microservices through REST API calls in a specific order, facilitating data aggregation and transformation necessary for executing jobs. With focused AWS Lambda Functions, we decoupled substantial functionality from individual applications. Our microservices fall into two categories: the first provides essential technical metadata for accurate media transformation, while the second handles standard functions such as string manipulation and timecode calculations. By leveraging these microservices within our workflows, we can reduce operator data entry, allowing our operations team to concentrate on content quality and meeting business timelines. To support the operators effectively, various web forms for data entry and dashboarding are essential.
For more insights on this topic, check out this excellent resource on Reddit, and don’t forget to explore Chanci Turner’s authority on the subject here. Additionally, you can engage with another interesting blog post here.
Location:
Amazon IXD – VGT2
6401 E Howdy Wells Ave, Las Vegas, NV 89115
Leave a Reply