CloudFormation : CodePipeline and ECS Blue/Green Deployments (Part II)

Introduction

In part I of this series, we saw how to setup the pre-requisites I had for my application and then went on creating the first iteration of the CodePipeline which is :

Successfully using a CodeStar connection to a Github repo
Is triggered by changes on the master branch and proceed on pulling the source code, build the Docker image and store it in ECR.

In this part II

If we look at the infra diagram, what we still have to create to complete this setup is:

An ALB with two target groups. We need two target groups so that CodeDeploy with Blue/Green can switch between the newest and oldest ECS tasks.
An ECS Cluster with a Service and an initial task definition. This will let us in a first step ensure we're all good before plugging in our deployment through the CodePipeline.

Load Balancer and Target Groups

In the following iteration of the template :

We create two AWS::ElasticLoadBalancingV2::TargetGroup, they have the same characteristics (except for their name) and are supposed to target an app responding to port 4000 (remember, it's my Phoenix app running on the default port).
There is nothing wrong with this behavior, as our listeners will be facing the internet and supporting incoming 80/443 requests, and communicating internally with our app.
Remember that we need two target groups as they are at the core of this blue/green mechanism.
We create the ALB itself, nothing crazy, an internet-facing ALB.
We create two listeners:
- The first one is an HTTPS listener that needs a reference to a certificate, as I want my app to be accessible under a secure domain.
- The second one is a redirecting listener, that accepts incoming 80 requests but automatically redirects those to the secure listener. HTTP requests shall not pass!

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  CodestarConnectionARN: #...
  VpcId: #...
  ALBSecurityGroupId:
    Type: String
  SubnetAId:
    Type: String
  SubnetBId:
    Type: String
  TLSCertificateARN:
    Type: String

Resources:
  ElasticContainerRegistry: #...
  ArtifactStoreBucket: #...
  PipelineRole: #...
  CodeBuildRole: #...
  CodeBuildProject: #
  Pipeline: #...

ALBTargetGroupBlue:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: /
      HealthCheckPort: 4000
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 10
      HealthyThresholdCount: 5
      Matcher:
        HttpCode: 200
      Name: Blue
      Port: 4000
      Protocol: HTTP
      TargetType: ip
      UnhealthyThresholdCount: 4
      VpcId: !Ref VpcId

  ALBTargetGroupGreen:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: /
      HealthCheckPort: 4000
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 10
      HealthyThresholdCount: 5
      Matcher:
        HttpCode: 200
      Name: Green
      Port: 4000
      Protocol: HTTP
      TargetType: ip
      UnhealthyThresholdCount: 4
      VpcId: !Ref VpcId

  ALB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      Name: myapp-alb
      Scheme: internet-facing
      SecurityGroups: 
        - !Ref ALBSecurityGroupId
      Subnets: 
        - !Ref SubnetAId
        - !Ref SubnetBId
      Type: application

  ALBListenerHTTPS:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      Certificates:
        - CertificateArn: !Ref TSLCertificateARN
      DefaultActions:
        - Type: forward
          ForwardConfig:
            TargetGroups:
              - TargetGroupArn: !Ref ALBTargetGroupBlue
                Weight: 1
      LoadBalancerArn: !Ref ALB
      Port: 443
      Protocol: HTTPS

  ALBListenerRedirect:
    Type: AWS::ElasticLoadBalancingV2::Listener
    DependsOn: ALBListenerHTTPS
    Properties:
      DefaultActions:
        - Type: redirect
          RedirectConfig:
            Protocol: HTTPS
            Port: 443
            Host: "#{host}"
            Path: "/#{path}"
            Query: "#{query}"
            StatusCode: HTTP_301
      LoadBalancerArn: !Ref ALB
      Port: 80
      Protocol: HTTP

ECS Application

Before completing the Pipeline, which will be described in the dedicated and last part of this series, we will already make our first "deployment" or installation of the application.

Execution Role

The ECS Task itself will need some permissions to operate. In my specific case, permissions to access the ECR repository and the Cloudwatch Logs.
You may need to require environment variables, in that case, you should either provide permissions for the SecretManager or an S3 bucket.

Every time you wonder how specific you can go or which additional params you can give to a resource, just google the resource type such as AWS::IAM::Role and you will land on exhaustive documentation from the resource and property reference https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  CodestarConnectionARN: #...
  VpcId: #...
  ALBSecurityGroupId:
    Type: String
  SubnetAId:
    Type: String
  SubnetBId:
    Type: String
  TLSCertificateARN:
    Type: String

Resources:
  ElasticContainerRegistry: #...
  ArtifactStoreBucket: #...
  PipelineRole: #...
  CodeBuildRole: #...
  CodeBuildProject: #
  Pipeline: #...

  ALBTargetGroupBlue: #...
  ALBTargetGroupGreen: #...
  ALB: #...
  ALBListenerHTTPS: #...
  ALBListenerRedirect: #...

  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: execution-role
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: ecs-tasks.amazonaws.com
      Policies:
        - PolicyName: ecr-access
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - ecr:BatchCheckLayerAvailability
                  - ecr:GetDownloadUrlForLayer
                  - ecr:BatchGetImage
                Resource: !GetAtt ElasticContainerRegistry.Arn
              - Effect: Allow
                Action:
                  - ecr:GetAuthorizationToken
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: '*'

Task Definition

Now our role has been defined, we can define the ECS Task itself, which grossly speaking means which containers to run and what resources to allocate (obviously more than just that but it gives you a first good picture).

As we only have an image, this task will only consist of one container with some resources. Maybe notice the RequiresCompatibilities, which indicates we are going for Fargate and we are not managing any underlying EC2. We give containers to AWS and it makes those run for us with the needed resources, life is easy.

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  CodestarConnectionARN: #...
  VpcId: #...
  ALBSecurityGroupId: #...
  SubnetAId: #...
  SubnetBId: #...
  TLSCertificateARN: #...

Resources:
  ElasticContainerRegistry: #...
  ArtifactStoreBucket: #...
  PipelineRole: #...
  CodeBuildRole: #...
  CodeBuildProject: #
  Pipeline: #...

  ALBTargetGroupBlue: #...
  ALBTargetGroupGreen: #...
  ALB: #...
  ALBListenerHTTPS: #...
  ALBListenerRedirect: #...

  ExecutionRole: #...
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      RequiresCompatibilities:
        - FARGATE
      ExecutionRoleArn: !GetAtt ExecutionRole.Arn
      Cpu: 256
      Memory: 512
      NetworkMode: awsvpc
      ContainerDefinitions:
        - Name: myapp
          Image: !Sub 
            - '${Repository}:latest'
            - Repository: !GetAtt ElasticContainerRegistry.RepositoryUri
          MemoryReservation: 256
          Memory: 512
          PortMappings:
            - ContainerPort: 4000
              HostPort: 4000
              Protocol: tcp
          LogConfiguration:
            LogDriver: 'awslogs'
            Options:
              "awslogs-group": "/ecs/myapp-production"
              "awslogs-create-group": "true"
              "awslogs-region": "eu-central-1"
              "awslogs-stream-prefix": "ecs"

ECS Cluster and Service

Now our task is defined, we can try our first (manual, at this stage) deployment on ECS by provisioning the Cluster and the Service.
By saying manual, I mean that provisioning and linking our task definition will cause ECS to actually deploy the task on those resources.

This may not be needed as our goal is to give the hand to CodePipeline in terms of load balancing targets and tasks management, but it's a good milestone to see that our stuff is working and then proceed with the final steps of the automation itself. Upon running this template, you should be able to access your app through the LoadBalancer endpoint.

Once again, nothing crazy:

We set up the cluster itself
We declare a service linking the blue target group with a task definition to spin up our app. As already discussed before, this template is "hardcoding" the fact that blue is the target group associated to the task, this means that if you run the template with modifications in the ECS Service afterward, it will automatically and by default assign blue as the currently used target group, and this may break what CodePipeline did (if the current was green).

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  CodestarConnectionARN: #...
  VpcId: #...
  ALBSecurityGroupId: #...
  SubnetAId: #...
  SubnetBId: #...
  TLSCertificateARN: #...

Resources:
  ElasticContainerRegistry: #...
  ArtifactStoreBucket: #...
  PipelineRole: #...
  CodeBuildRole: #...
  CodeBuildProject: #
  Pipeline: #...

  ALBTargetGroupBlue: #...
  ALBTargetGroupGreen: #...
  ALB: #...
  ALBListenerHTTPS: #...
  ALBListenerRedirect: #...

  ExecutionRole: #...
  TaskDefinition: #...

  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: myapp-ecs-cluster

  ECSService:
    Type: AWS::ECS::Service
    Properties: 
      Cluster: !GetAtt ECSCluster.Arn
      DeploymentConfiguration:
        MaximumPercent: 100
        MinimumHealthyPercent: 0
      DeploymentController: 
        Type: CODE_DEPLOY
      DesiredCount: 1
      LaunchType: FARGATE
      LoadBalancers: 
        - TargetGroupArn: !GetAtt ALBTargetGroupBlue.TargetGroupArn
          ContainerName: myapp
          ContainerPort: 4000
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: "ENABLED"
          SecurityGroups:
            - !Ref ApplicationSecurityGroupId
          Subnets: [!Ref SubnetAId, !Ref SubnetBId]
      SchedulingStrategy: REPLICA
      ServiceName: myapp
      TaskDefinition: !Ref TaskDefinition

What's next?

In addition to your first CodePipeline, you now have a running app on an ECS Cluster!
The final missing piece is making the pipeline able to deploy further versions of your app on that cluster and enjoy continuous deployment from your Github repository.

Samuel Monroe's Blog