aws-ecs-cluster

AWS ECS (Elastic Container Service) is Amazon’s own fully managed container scheduling and orchestration service, making it easy to deploy, operate, and scale containerized applications and providing benefits such as serverless runtimes, autoscaling of worker nodes (in server-based implementations), high scalability, tight integration with other AWS services and cost efficiency.

Use Cases

Container orchestration

ECS is AWS’s own container orchestration solution, making it easy to deploy, scale, and manage containerized applications.

Microservices architecture

ECS allows you to deploy and manage individual microservices as separate containerized tasks, making it easy to scale, update, and maintain your application.

Batch processing

ECS can be used to run batch processing jobs on a schedule or in response to a specific event, such as the arrival of new data in an S3 bucket.

High-performance computing

ECS supports GPU-enabled container instances, making it a suitable option for running high-performance computing workloads such as machine learning and scientific simulations.

Configuration Presets

Development Cluster

This preset creates a cluster with a single node group of cost effective t3.medium instances.

Production Cluster

This preset creates a cluster with a single node group of compute optimized c5.2xlarge instances.

Design

ECS by itself is a simple control plane for scheduling containerized workloads. The Workloads themselves are composed of ECS tasks and services.

Tasks

Tasks are the containerized workloads that run in the ECS cluster. If you are familiar with Kubernetes, a similar concept would be a pod.

Services

An ECS service is an abstration on top of a set of tasks which will restart failed tasks, or autoscale them up or down to meet performance criteria. If you are familiar with Kubernetes, a similar concept would be a deployment.

Container Instances

The EC2 instances that ECS schedules the containers on are called container instances. If you are familiar with Kubernetes, a similar concept would be a node. ECS also has a serverless offering called Fargate, which requires no EC2 instances to schedule and run tasks.

Fargate

AWS Fargate is a serverless option to run containerized workloads. With this option, you don’t need to run EC2 instances for compute resources. Additionally, Fargate has the option to use Spot instances for running your workloads. This is a cheaper alternative to standard Fargate, but should only be used with interruption-tolerant workloads since Spot instances can be removed at any time (workloads are given a 2 minute window to shutdown gracefully). This bundle enables both Fargate and Fargate Spot as available runtimes by default.

As with most serverless options, it can be highly cost effective to run Fargate if your workloads have highly variable resource utilization. If your resource utilization is stable and highly optimized, EC2 instances can be up to 25% more cost effective.

Autoscaling Groups

If you choose to run EC2 instances for your workload scheduling, this bundle will automatically create AWS Launch Templates and Autoscaling Groups for the container instances. Additionally, the launch template is configured to always use the AWS recommended ECS optimized AMI, pulled via an SSM lookup.

Networking

By default, an ECS cluster itself isn’t associated with a VPC - it is simply a logical control plane for managing ECS tasks. However, since a VPC is required for both the EC2 instances in the workload pool as well as the ECS tasks that the cluster is scheduling, we include a VPC as a required connection on the ECS bundle itself. This greatly simplifies connecting and running applications within your ECS cluster.

Internet Ingress

If you plan to run workloads in your ECS cluster that should be accesible via the internet, you can enable internet ingress and associate Route53 domains to your ECS cluster. This will provision a layer 7 application load balancer (ALB), as well one or more ACM TLS certificates and associate the certificate(s) with the ALB to securely serve HTTPS traffic. Additionally, this bundle adds a default rule to the ALB on HTTP port 80 to automatically redirect to HTTPS on port 443.

Best Practices

ECR Integration

Worker nodes in the cluster have IAM roles permitting read access to ECR, allowing you to run with private images.

Fargate Support

Fargate and Fargate Spot are both enabled by default, making your cluster ready for serverless applications.

Secure Networking

Workloads are configured to run in private subnets, which allow internet egress through a NAT gateway. Internet ingress comes through an ALB running in a public subnet. This is AWS’s recommended setup for security.

Load Balancing

If you have web based applications you’d like to run in the cluster, an Application Load Balancer will be provisioned with proper TLS certificates and HTTP/HTTPS redirect enforced.

Security

Nodes Deployed into Private Subnets

Container instances are provisioned into private subnets for security.

Policy of Least Privilege

Container instances have minimal permissions only required for ECS integration. Your workloads will be configured with their own IAM roles and security group policies for maximum security

HTTPS Enforced

The ALB that is optionally configured with the cluster is pre-configured with a rule to redirect all HTTP traffic to HTTPS.