Control Testing as Code

This is the third installment of our Test-Driven Compliance series. If you're new to the topic, I recommend starting with the previous posts ( 1 and 2).

Today, we'll take one of the controls defined in our last post and test it in a simulated AWS environment. This will help reinforce the concepts and demonstrate that compliance control testing is simply a form of testing, much like integration tests.

Check it out

The source code is fully available on GitHub. To run it, you'll need Git, Python, and uv.

Let's take a look:

 $ git clone https://github.com/GermanRED/encryption-at-rest-control-as-code.git
 $ cd encryption-at-rest-control-as-code

By default, the project simulates an AWS account with an S3 bucket and an RDS instance. You can view the contents using the following command:

 $ make export
 [
  {
    "arn": "arn:aws:s3:::encryption-at-rest-control-as-code-mock-bucket",
    "name": "encryption-at-rest-control-as-code-mock-bucket",
    "type": "S3",
    "encrypted": true,
    "encryption_algorithm": "SSE-S3"
  },
  {
    "arn": "arn:aws:rds:eu-central-1:123456789012:db:eatcac-mock-db",
    "name": "eatcac-mock-db",
    "type": "RDS",
    "encrypted": true,
    "encryption_algorithm": "KMS"
  }
 ]

We can see in the inventory that both resources are encrypted, so the tests will pass.

 $ make test
 uv run pytest -v
 ========================================================= test session starts ===========================================================
 platform linux -- Python 3.12.3, pytest-8.3.5, pluggy-1.5.0 -- ./encryption-at-rest-control-as-code/.venv/bin/python3
 cachedir: .pytest_cache
 rootdir: ./encryption-at-rest-control-as-code
 configfile: pyproject.toml
 collected 2 items

 tests/test_encryption_at_rest.py::test_storage_encryption[arn:aws:s3:::encryption-at-rest-control-as-code-mock-bucket] PASSED      [ 50%]
 tests/test_encryption_at_rest.py::test_storage_encryption[arn:aws:rds:eu-central-1:123456789012:db:eatcac-mock-db] PASSED          [100%]

 ==================================================== 2 passed, 4 warnings in 0.19s =====================================================

Just to clarify, these results are fully mocked, and the code does not connect to your infrastructure. Later, I’ll provide instructions on how to disable mocking and connect to a real AWS account.

How did this happen?

The key concept I want to highlight is the inventory. This inventory alone allows you to confidently engage with auditors, demonstrating that your compliance program is effective. We’ll use it to continuously verify that we don’t deviate from expected behaviors as our products evolve. Having sufficient coverage for compliance tests is liberating, you can aggressively push new features, knowing that your CI/CD pipeline will alert you if anything goes off track.

The code for extracting the inventory is located in src/inventory/assets.py. It connects to an AWS account, retrieves details of your S3 buckets and RDS instances, and presents them in a unified data model. The Storage class includes only the essential fields needed for this example, but in a full-fledged suite, you’d likely extend it to cover ownership, network configuration, backups, and more. Another potential improvement is adding caching, as repeated network queries will otherwise slow things down.

As mentioned earlier, by default, the tests run with mocked data. This is achieved using the excellent moto library, which seamlessly mocks AWS resources and integrates with the standard boto3 library. The mock setup can be found in src/inventory/setup.py. If you want to disable mocking and connect to a real AWS account, simply log in using the AWS CLI and run the tests with the environment variable AWS_MOCKED="false".

If you plan to run this code in your real infrastructure, point it to a development or test environment and have fun.

The tests are implemented using pytest and leverage parameterization to run a test for each resource. This approach ensures that all resources are checked while providing clear insights into any failures. The test code is located in tests/test_encryption_at_rest.py and can be executed as part of any CI/CD pipeline, just like any other pytest suite.

Conclusion

Test-driven compliance provides a structured, reliable approach to ensuring your cloud infrastructure meets security and regulatory requirements. By leveraging an inventory-driven strategy and automated tests, you can confidently verify compliance on every commit, enabling faster development cycles while maintaining security and audit readiness.

With the right tools and methodologies, compliance shifts from being a tedious checkbox exercise to an automated, scalable, and developer-friendly process.

Happy testing!