Implementing a CI/CD Pipeline for Your Azure Application

This post is a continuation of our prior post Building Out an Application’s Azure Infrastructure. If you haven’t checked that one out already – give it a look.

At this point we have our Azure infrastructure in place and our engineering team has been putting the finishing touches on the enterprise application code base. Now it’s time to deploy and as part of a DevOps strategy, it is go-time for Continuous Integration and Continuous Delivery (CI/CD) pipeline implementation.

Continuous Integration refers to the practice of merging developer working code bases into mainline and/or a key branch several times a day in an automated fashion. Continuous Delivery refers to the ability to release code to a production or production-like environment with minimal manual effort/intervention. With CI/CD, you will be able to more efficiently and effectively manage your code releases and infrastructure. In addition to labor reduction, a CI/CD practice will help your team deliver quality software solutions faster AND empower your developers to focus on the task at hand, creating robust software in support of your company’s business objectives.

Our Azure Infrastructure Topology

In our previous Building Out an Application’s Azure Infrastructure post, we implemented our application’s cloud assets in Azure – this is the terrain we are deploying into. Here is the topology we will be working with:

Let’s Set a Baseline for Our CI/CD Pipeline Outcome Expectations

Before CI/CD automation became a stable practice, it wasn’t uncommon to hear the horrors of a deployment gone bad because of missed or botched manual steps. DevOps automation takes the stress of getting the code into the cloud and increases the predictably of enterprise software releases.

The most powerful tool we have as developers is automation.

Scott Hanselman

However, even with a well-constructed CI/CD pipeline, you have not spent your last effort on your software’s release – we don’t want to imply that what we are dispensing here is magic. Here are some expectations on what you should and should not expect from your CI/CD pipeline:

• Your team will no longer need to run down release action checklists
• All builds and releases will be tracked down to the individual build and release action
• More feasible (not quite magical, though) release rollback capability
• Automated testing prior to deployment
• Automated code health checks
• If your developers make big enough technological changes to the code base, you will likely need to modify or add a stage(s) to the software’s build steps
• If an automated test fails, an engineer is going to need to take action to clear the test failure before a deploy will succeed
• You can exceed build agent or release capacity subscription levels and need to tweak your CI/CD infrastructure subscription

What Will Our Workflow Look Like?

As we’ve discussed, the addition of a CI/CD pipeline will enable you to more effectively manage your code releases and infrastructure. We are about to get knee deep in pipeline construction so first, let’s take a look at the CI/CD pipeline practices we believe in:

Start Building the Pipeline

Our end goal is to create the artifact (i.e. code and data) that will be deployed to the cloud. This plan generally follows these steps to build the final deployable artifact:

1. Install build requirements
2. Restore dependencies (in this case, NuGet packages)
3. Build
4. Test
5. Publish (create application packages)
6. Create build artifact (to be used in future stages)

The integration/build portion of the pipeline will go into a single pipeline stage and a single job. We will start constructing the pipeline:

1. Build a New Pipeline. In Azure DevOps, select ‘Pipelines’ in the navigation and then ‘New pipeline’
2. Where is your code? We’re using Azure Repos Git here but if relevant you can set up a connection to a different repo and the pipeline setup will be the same
3. Configure your pipeline. There are options for some pre-made builds, which can be useful starting points and you can select Existing Azure Pipelines YAML file.

4. Run. Once you’ve selected the file, click the blue Run button
5. You’ll see a screen with the build information and a drill down into the currently running job – on this screen you should see the displayName property you set

Finishing the Build

Only one task has been added so far to our script. Let’s add the additional tasks.

stages:
- stage: 'Build_Stage' #Stage name cannot have spaces
  displayName: 'Build' #Name displayed when viewing in Azure DevOps
  jobs:

  - job: 'Build_Job' #Job name cannot have spaces
    displayName: 'Application Build' #Name displayed when viewing in Azure DevOps
    pool:
      vmImage: 'windows-latest' #See available agent images: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent
    steps:

    - task: DotNetCoreInstaller@1
      displayName: 'Use DotNet Core SDK'
      inputs:
        version: 3.x

    - task: DotNetCoreCLI@2
      displayName: Publish App
      inputs:
        command: publish  #The publish command does a restore, build and creates a zip file so no need to add extra steps
        projects: '**/*.sln'
        arguments: '--configuration Release --output $(build.artifactstagingdirectory)/application'

    - task: DotNetCoreCLI@2
      displayName: Test
      inputs:
        command: test #Don't forget to test!
        projects: '**/*.Tests.csproj'
        arguments: '--configuration Release'

    - task: PublishBuildArtifacts@1
      displayName: 'Publish Artifact'
      inputs:
        pathtoPublish: '$(build.artifactstagingdirectory)\app\application.zip' 

#Publish (different from the command used above) allows the artifact to be used in later jobs
        artifactName: 'app'

Check in the code and what the update pipeline run in Azure DevOps which should look a little bit like the following:

Implement the Deployment Portion of the Pipeline

It is now time to deploy the code that we’ve now packaged in our build step to two different app services—we’ll call them “staging” and “production”—with the appropriate dependencies between the stages. Remember that a pipeline is a collection of stages that can run sequentially or in parallel. There are nuances but generally speaking, jobs in a stage all run in parallel and tasks within a job run sequentially.

To skip the step-by-step procedures, take a look at the following shortcut to get a look at the results of adding deployment to your pipeline at fullpipelinept2.yml (github.com). Some things to point out:

• Our new deployment work, instead of displaying as a job it is instead named deployment. This is a specially-named job that allows for additional options than a standard job type, including history
• There is a property named environment that is set to ‘Staging’; in the deployment stage to the production assets it will be named ‘Production’
• The strategy section has a variety of lifecycle hooks (also specially-named jobs) that can be used in different deployment strategies; this example shows the simplest strategy of RunOnce
• Each lifecycle hook has their own set of steps to execute; first we extract files from the zip that was created in the build, then deploy those files to an Azure App Service

This is what the pipeline would look like in Azure DevOps if the production stage only had a dependency on the build stage:

Notice that at this point, the dependency lines show that both staging and production will run at the same time after the build stage – not something we would want in the real world. Instead, let’s make a change so that the production stage has all the proper dependencies and commit the code.

Push Button Release

While the build and release to the application’s development environment is automatic and frequent (each approved PR into code mainline kicks off the automation), promotion to Staging and Prod environments requires a human “push button” approval.  Azure DevOps enforces a limited set of users who are approved to release to these higher environments and we are careful to set this group for each application.

Blue/Green Deployments FTW

When deploying to a PaaS infrastructure (like the Azure assets we are using in this example), we use a separate App Service for each environment to enable implementation of the blue/green deployment pattern.

It is common within each App Service to create a slot and direct the deployment pipeline to this slot.  This use of slots isn’t always the best strategy but if implemented, following a successful deployment the pipeline performs a “slot swap” to both warm up the slot and then sets this slot as the default for the App Service (and again, PaaS for the win also).  Applications often brings a “startup cost” and this slot pre-warming reduces this potential impact to first user of a slow or unresponsive app.

The Wrapup – and Some Resources

There is a whole lot to know about CI/CD pipeline implementation and best practices but hopefully this post has whet your appetite. We covered several things in this series but there is surely a lot more to know in the details of creating a CI/CD pipeline with Azure DevOps. Here are some additional resources to dive deeper if you find yourself ready to get going on a pipeline for your organization:

• Creating a Multi-Stage Pipeline in Azure DevOps
• CI/CD with Azure Pipelines and Bicep files
• Build your first Bicep deployment pipeline by using Azure Pipelines

---

Interested In Moving Your Application to the Cloud?