Read first:

PART 1: How I designed and deployed a scalable microservice architecture with limited resources

As mentioned in the last part, we have already setup and deployed our three services, but we don’t want to keep pulling the changes every time we make a small change in the codebase, that’s why we need to setup our CI/CD Pipeline.

Let’s get started…

Let’s first treat all our services as Monolithic, and deploy them to the instance.

Create IAM Roles for CodeDeploy and EC2 Instance.

Go to IAM → Roles in your AWS Console
Create an IAM Role with AmazonEC2RoleforAWSCodeDeploy and AutoScalingNotificationAccessRole policies.
Let’s name this IAM Role as CodeDeployInstanceRole
Create another IAM Role with AWSCodeDeployRole policy.
Let’s name this one as CodeDeployServiceRole

Configure EC2 Instance for the application

Make sure you already have an instance running.

You just gotta modify the tags to let the CodeDeploy Agent know which instances to deploy the code on.

Let’s put two tags: env: production and name: my-application

Create S3 Bucket for application revision

This bucket will be used to store our revised application before it is deployed to the instance.

Note: Creating this bucket is necessary if you want to add some files to the codebase that you couldn’t store in the Github repository, such as _.env_ files.

If you don’t have any such thing, then you can skip this step.

You may name the bucket whatever you like.
Make sure Block all public access option is checked.

Configure CodeDeploy

Navigate to CodeDeploy in AWS Management Console.
Create an application and give it a name.
Under Compute Platform choose EC2/On-premises
Create Application.

After creating the application, create a deployment group
Name it whatever you’d like, and under Service Role choose CodeDeployServiceRole.

Under Deployment Type choose In-place.
For Environment Configuration choose Amazon EC2 instances and specify the tags that we specified previously for our instances: env: production and name: my-application
For Deployment Settings choose CodeDeployDefault.OneAtATime.

Deselect Enable Load Balancing
Create Deployment Group.

Phew, we are done with setting up on the AWS side, now let’s get to the good stuff.

Setting up on the Code Repository Side

Create an appspec.yml file, and place it in the root of the directory.

      
        version: 0.0
      
        os: linux
      
        files:
      
          - source: /
      
            destination: /home/ubuntu/my-application
      
        hooks:
      
          BeforeInstall:
      
            - location: ./scripts/init.sh
      
              timeout: 300
      
              runas: root
      
          ApplicationStart:
      
            - location: ./scripts/start_app.sh
      
              timeout: 300
      
              runas: root
      
          ApplicationStop:
      
            - location: ./scripts/cleanup.sh
      
              timeout: 300
      
              runas: root

view raw appspec.yml hosted with ❤ by GitHub

Let’s setup our CI workflow now. I am using Github Actions for my CI/CD setup.

Create a .github/workflow/deploy.yml file

      
        name: CI/CD Deployment
      
        on: [push]
      
        jobs:
      
          buildAndTest:
      
            name: CI Pipeline
      
            runs-on: ubuntu-latest
      
            strategy:
      
              matrix:
      
                node-version: ['12.x']
      
            steps:
      
              - uses: actions/checkout@v2
      
              # Initialize Node.js
      
              - name: Install Node.js ${{ matrix.node-version }}
      
                uses: actions/setup-node@v1
      
                with:
      
                  node-version: ${{ matrix.node-version }}
      
              # Install project dependencies and test
      
              - name: Install dependencies
      
                run: npm install
      
              - name: Run tests
      
                run: npm run test
      
          deploy:
      
            name: Deploy
      
            runs-on: ubuntu-latest
      
            strategy:
      
              matrix:
      
                python-version: [3.8]
      
                node-version: ['12.x']
      
                appname: ['my-application-codedeploy']
      
                deploy-group: ['production']
      
                s3-bucket: ['my-application-codedeploys']
      
                s3-filename: ['prod-aws-codedeploy-${{ github.sha }}']
      
            needs: buildAndTest
      
            steps:
      
            - name: Checkout
      
              uses: actions/checkout@v2
      
            # Configure AWS Credentials
      
            - name: Configure AWS credentials
      
              uses: aws-actions/configure-aws-credentials@v1
      
              with:
      
                aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
      
                aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      
                aws-region: ${{ secrets.AWS_REGION }}
      
            # Fetch secrets from S3 Bucket
      
            - name: Configure Secrets
      
              run: |
      
                aws s3 cp s3://my-application-secrets/production/env/ ./env/ --recursive
      
            # Deploy push to AWS S3
      
            - name: AWS Deploy push
      
              run: |
      
                aws deploy push \
      
                --application-name ${{ matrix.appname }} \
      
                --description "Revision for the ${{ matrix.appname }}-${{ github.sha }}" \
      
                --no-ignore-hidden-files \
      
                --s3-location s3://${{ matrix.s3-bucket }}/${{ matrix.s3-filename }}.zip \
      
                --source .
      
            # Create deployment using AWS CodeDeploy
      
            - name: AWS Create Deployment
      
              run: |
      
                aws deploy create-deployment \
      
                --application-name ${{ matrix.appname }} \
      
                --deployment-config-name CodeDeployDefault.OneAtATime \
      
                --deployment-group-name ${{ matrix.deploy-group }} \
      
                --file-exists-behavior OVERWRITE \
      
                --s3-location bucket=${{ matrix.s3-bucket }},key=${{ matrix.s3-filename }}.zip,bundleType=zip \

view raw deploy.yml hosted with ❤ by GitHub

Note: In the _Configure Secrets_ step, we are fetching our secrets from a AWS S3 Bucket where we have stored to _.env_ files, as those can not be stored on the Github repository.

If you don’t have any secrets to configure, you can deploy directly from Github repository. Instead _s3-location_, you'd need to specify _github-location_: reference

Now, we just have to configure AWS Credentials that we used above.

Setting up CodeDeploy IAM User

Go to IAM -> Users in your AWS Console.
Create a new IAM User, let’s name it CodeDeployUser, and give it Programmatic Access

We need 2 sets of permissions: AmazonS3FullAccess and AWSCodeDeployDeployerAccess

Create the user, and save the user’s credentials ACCESS_KEY_ID and SECRET_ACCESS_KEY

Set those secrets in your Github Repository and you are all good to go!

Great! now, every push you do to your repository will be deployed to your EC2 instance.

But, wait. If you push to your repository after modifying just one of the services, all of them need to be restarted, that’s not how a Microservice Architecture works.

We need to decouple all our services from each other for all of them to operate separately.

Let’s get to that stuff in PART 3…

Read Next:

PART 3: How I designed a CI/CD setup for a Microservice Architecture at zero cost

ShauryaAg