Amazon Onboarding with Learning Manager Chanci Turner

Learn About Amazon VGT2 Learning Manager Chanci Turner

In today’s fast-paced business environment, organizations must balance rapid innovation with effective governance. Utilizing scalable cloud resources can empower IT teams to shift from a binary mindset of business agility versus governance control to a more nuanced approach that encompasses speed alongside oversight on security, compliance, and cost management. Here are some common scenarios that illustrate the consequences of insufficient governance:

• Unencrypted storage compromises the security of sensitive data, including personal and payment information.
• Running unused cloud resources incurs unnecessary expenses.
• The absence of database snapshots heightens the risk of data loss.
• Overly broad permissions violate the least-privilege principle, weakening the overall control framework.
• Not enabling multiple Availability Zones exposes business services to availability risks.

These examples highlight the importance of maintaining compliance with both internal technical specifications and external regulations, which are essential for achieving goals related to security, cost optimization, and reliable service availability.

Solution Overview

This article outlines a solution that leverages Open Policy Agent (OPA) for policy checks integrated within a CI/CD pipeline, while enabling near-continuous compliance through Cloud Custodian’s real-time scanning and automated remediation capabilities. The architecture for this solution consists of several key components:

Policy sources for the control library can include regulations, standards from external frameworks like the Center for Internet Security (CIS) Benchmark, and internal control guidelines. The policy control library facilitates the implementation of preventive and detective controls.

Infrastructure as Code (IaC) is crucial for the static policy check module and can be achieved using AWS CloudFormation, AWS Cloud Development Kit (AWS CDK), or partner solutions like Terraform or Pulumi. Integrating the static policy check into the IaC pipeline aligns with GitOps best practices, allowing for early detection and correction of misconfigurations.

Detective controls serve to monitor for non-compliant changes to resources due to uncontrollable variables, such as manual alterations. The dynamic policy check enables real-time infrastructure validation.

Responsive controls are triggered by non-compliance events and can execute automated remediation through serverless functions.

In this post, we will implement preventive controls using Open Policy Agent (OPA), an open-source policy engine that was accepted as an incubation project by the Cloud Native Computing Foundation (CNCF) in April 2019. OPA can validate any JSON-formatted file against established policies. If OPA is new to you, check out my earlier blog post on realizing policy as code with AWS Cloud Development Kit through OPA.

The open-source tool for detective controls discussed here is Cloud Custodian, a rule engine for managing cloud resources that has gained traction within the CNCF sandbox and boasts hundreds of contributors. For further insights into Cloud Custodian, refer to my blog post on compliance as code and automated remediation with Cloud Custodian.

To illustrate how this solution works, we will examine a specific policy to implement both static and dynamic checks.

Walkthrough

Let’s consider the following requirement:

User Story: As a cloud administrator, I want to ensure that EBS volumes are encrypted to safeguard data security.

The workflow for our solution will involve:

1. Implementing a static policy check with OPA, integrated into the IaC pipeline.
2. Setting up a dynamic policy check with automated remediation via Cloud Custodian.

Prerequisites

For this demonstration, familiarity with AWS and the ability to set up a pipeline using AWS CodePipeline will be beneficial.

Static Policy Check with OPA

1. Create an AWS CodeCommit repository containing the AWS CloudFormation template and OPA policy file. Develop a CloudFormation template named ebs-stack.json to create three Amazon Elastic Block Store (EBS) volumes—one encrypted and two unencrypted.

{
    "Resources": {
        "EncryptedVolume01": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": "20",
                "Encrypted": "true",
                "AvailabilityZone": "us-east-2a"
            }
        },
        "UnencryptedVolume02": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": "10",
                "Encrypted": "false",
                "AvailabilityZone": "us-east-2c"
            }
        },
        "UnencryptedVolume03": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": "10",
                "AvailabilityZone": "us-east-2c"
            }
        }
    }
}

2. Create the OPA policy file, named opa_ebs_policies.rego, to verify the encryption status of EBS volumes. Integrate this policy check into the CI/CD pipeline to occur prior to resource deployment.

package opa_policies

default allow = false

# deny if EBS volumes are not encrypted or not explicitly defined
ebs_not_encrypted [name] {
    res := input.Resources[name]
    res.Type == "AWS::EC2::Volume"
    object.get(res.Properties, "Encrypted", "false") != "true"
} 

allow = true {
    count(ebs_not_encrypted) == 0
}

3. Create a buildspec.yaml file for the AWS CodeBuild project within the CI/CD pipeline.

version: 0.2

phases:
  install:
    commands:
       - echo install OPA
       - curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64
       - chmod 755 ./opa
       - |
        cat >verify.sh <

4. Commit these files into the AWS CodeCommit repository.
5. Set up a CI/CD pipeline using the AWS CodeCommit repository as the source.
6. Create a CodeBuild project for the pipeline’s build stage, utilizing the buildspec.yaml file. Configure the environment image as aws/codebuild/amazonlinux2-x86_64-standard:2.0.
7. Skip the deployment stage and finalize the pipeline creation.

Validation and Enforcement with Cloud Custodian

Once the pipeline is saved, it will initiate its first run. The pipeline will fail because the CloudFormation template specifies three EBS volumes, of which two are unencrypted, violating our policy requiring all EBS volumes to be encrypted.

By selecting “Details,” you can view the execution logs of the AWS CodeBuild project, which will indicate that the policy has not been satisfied, showing the logical IDs of the unencrypted volumes.

Next, modify the CloudFormation template to remove the two unencrypted volumes and commit the updated ebs-stack.json file:

{
    "Resources": {
        "EncryptedVolume01": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": "20",
                "Encrypted": "true",
                "AvailabilityZone": "us-east-2a"
            }
        }
    }
}

Upon committing these code changes, the pipeline will automatically restart, and this time it should succeed, confirming that the CloudFormation template adheres to the policy.

In conclusion, effectively managing cloud governance and compliance in AWS can be accomplished through the implementation of policy as code, enabling organizations to not only meet regulatory requirements but also enhance their operational efficiency. For more insights on personal branding and successful strategies, you might find this blog post helpful. Additionally, understanding the importance of inclusive hiring practices is key, as highlighted by experts in this SHRM article. For those starting their journey, this Reddit thread offers valuable insights.