In the realm of live streaming video workflows, AWS Elemental MediaStore stands out as a high-performance cloud origin. However, creating an Amazon CloudWatch dashboard to monitor live video streams can pose challenges for users. By default, AWS Elemental MediaStore doesn’t enable access logs, necessitating users to activate access logging for their containers. Doing so allows MediaStore to send detailed access logs to Amazon CloudWatch, providing essential information regarding object requests, including whether a request was to store (put) or retrieve (get) an object, the specific object in question, and the timestamp of each request.
In this post, we will explore how to leverage Amazon CloudWatch to establish a monitoring dashboard for live video streams in MediaStore. This dashboard will help track live stream ingress and egress status, offering insights into potential issues.
For additional information on MediaStore access logs, check out this blog post. You can also refer to this resource for guidance on sending live video to MediaStore.
Overview
This solution allows you to create a live streaming CloudWatch dashboard that monitors video stream ingest and egress metrics, including transactions per second, observed operation latencies, and HTTP operation statuses based on status codes like HTTP 200 OK, 400, and 500. All of this is achievable through MediaStore’s access logs.
Walkthrough
Begin by setting up a live channel using MediaStore as your origin, and start a player to stream directly from MediaStore to generate egress traffic. Following this, utilize MediaStore access logs and CloudWatch to construct a monitoring dashboard for both ingress and egress.
Prerequisites
Before proceeding, ensure you have the following:
- An AWS account
- An encoder, such as AWS Elemental Live, to push live streams to AWS Elemental MediaStore, or AWS Elemental MediaLive for streaming
- An HLS player capable of playing HLS streams
MediaStore and Live Stream Configuration
First, create a container for your live channel, naming it ‘mediastore_live’. Your live stream will follow a path structure resembling //main/index.m3u8; for example, /channel1/main/index.m3u8.
Once the MediaStore container is established, enable access logging by selecting the ‘mediastore_live’ name, then clicking “Enable access logging” in the Info section. Confirm the action in the pop-up window.
To facilitate direct streaming from MediaStore with HTTP/S access, navigate to the “Container policy” section, click “Edit policy,” and insert the following policy text, ensuring to adjust the “Resource” ARN to fit your container settings:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "MediaStoreFullAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxxxxx:root"
},
"Action": "mediastore:*",
"Resource": "arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "true"
}
}
},
{
"Sid": "PublicReadOverHttpOrHttps",
"Effect": "Allow",
"Principal": "*",
"Action": ["mediastore:GetObject", "mediastore:DescribeObject"],
"Resource": "arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*",
"Condition": {
"Bool": {
"aws:SecureTransport": ["true", "false"]
}
}
}
]
}In the “Container CORS policy” section, click “Edit CORS Policy” and paste the following text, then save:
[
{
"AllowedHeaders": ["*"],
"AllowedMethods": ["GET", "HEAD"],
"AllowedOrigins": ["*"],
"ExposeHeaders": ["*"],
"MaxAgeSeconds": 3000
}
]Next, configure a live stream using AWS Elemental MediaLive to connect to AWS Elemental MediaStore. For more details, refer to this authoritative article.
Stream Ingest Dashboard
Monitoring the health status of the stream ingest into the origin is crucial. MediaStore access logs provide valuable insights into live stream ingests. MediaStore has default soft limits on container operations, such as GetObject, PutObject, and DeleteObject. Keeping an eye on your live stream operations is vital to ensure that the stream isn’t throttled by these limits. Understanding your stream’s characteristics allows for proactive measures, such as requesting limit increases.
To create your dashboard, access the AWS management console, select CloudWatch, then Logs, and finally Insights. In the top dropdown, search for /aws/mediastore/mediastore_live, which corresponds to the container used. Paste the following text into the query box and click the Run query button. Switch to the Visualization tab to see the graph:
fields @message
| filter (Path like "/channel1/main") and (Operation="PutObject")
| stats count(*) as TPM by bin(1m)This query retrieves log messages containing ‘/channel1/main’ in the path with an operation type of “PutObject,” counting total Put operations per minute.
Next, click “Actions,” then select “Add to dashboard.” Create a new dashboard named “mediastore_live,” and confirm. Choose the Line widget type, label it “mediastore_live,” and add it to the dashboard.
From the CloudWatch dashboard, select the mediastore_live dashboard we created. Click “Actions” again and choose “View/edit source.” The dashboard source text box will appear; replace its contents with the following JSON structure. This will help you learn how to add individual widget blocks later. Once finished, click “Update.”
{
"widgets": [
{
"type": "log",
"x": 0,
"y": 0,
"width": 12,
"height": 6,
"properties": {
"query": "SOURCE '/aws/mediastore/mediastore_live' | fields @messagen| filter (Path like "/channel1/main") and (Operation="PutObject")n| stats count(*) as TPM by bin(1m)",
"region": "us-west-2",
"stacked": false,
"title": "mediastore_live channel1 TPM (Transaction Per Minute)",
"view": "timeSeries"
}
},
{
"type": "log",
"x": 0,
"y": 6,
"width": 12,
This guide provides a solid foundation for monitoring your live streaming applications. For additional insights, feel free to explore more on this blog post.
Leave a Reply