CI/CD - Continuous Integration & Deployment

// Lesson 1: Introduction to CI/CD

What is CI/CD?

CI/CD stands for Continuous Integration and Continuous Delivery (or Deployment). It's a methodology and set of practices that automate the building, testing, and deployment of software.

The Three Pillars

Continuous Integration (CI)

Developers merge code changes frequently, typically several times per day. Each merge triggers an automated build and test process:

Code is automatically compiled
Automated tests run
Code quality is analyzed
Build artifacts are created

Continuous Delivery (CD)

Code changes are automatically prepared for release to production. The deployment to production is triggered manually but is automated:

Artifacts are stored
Staging environments are updated
Release is ready at any time
Human approval may be required

Continuous Deployment

Every change that passes tests is automatically deployed to production. No manual intervention needed:

Full automation from commit to production
Fast feedback on code changes
Requires excellent test coverage
Feature flags enable rapid iteration

Benefits of CI/CD

Faster releases: Deploy multiple times per day
Fewer bugs: Automated testing catches issues early
Better collaboration: Teams can work on the same codebase with confidence
Rollback capability: Quick recovery from issues
Reduced manual work: Automation frees developers

The CI/CD Pipeline

A typical CI/CD pipeline includes:

Commit: Developer pushes code to version control
Build: Code is compiled and dependencies are resolved
Test: Automated tests run (unit, integration, e2e)
Analyze: Code quality and security scanning
Package: Artifacts are created and stored
Deploy: Code is deployed to environments
Verify: Post-deployment testing

Next: Version Control →

← Back to Lessons

// Lesson 2: Version Control & Git Workflows

Effective CI/CD starts with good version control practices. Your branching strategy directly impacts your CI/CD pipeline design.

Branching Strategies

Trunk-Based Development

Developers work in short-lived branches (less than 2 days) or directly on main:

main (or trunk)
├── feature/auth-login (2 days)
├── feature/api-v2 (1 day)
└── hotfix/security-patch (hours)

Benefits: Less merge conflicts, faster integration, easier CI/CD

Git Flow

More structured approach with dedicated branches:

main
├── develop
│   ├── feature/new-feature
│   ├── release/1.0.0
│   └── bugfix/fix-login
└── hotfix/urgent-fix

Best for: Projects with scheduled releases

GitHub Flow

Simple, branch-based workflow:

main
├── feature/add-login
├── feature/update-api
└── fix/typo-in-readme

Branch from main
Work and commit
Open pull request
Review and discuss
Merge to main
Deploy immediately

Commit Message Best Practices

# Good commit messages
feat: add user authentication
fix: resolve memory leak in cache
docs: update API documentation
refactor: simplify user service
test: add unit tests for auth module
chore: update dependencies

Conventional Commits

<type>(<scope>): <description>

[optional body]

[optional footer]

Next: Automated Testing →

← Back to Lessons

// Lesson 3: Automated Testing

Automated testing is the foundation of CI/CD. Without reliable tests, you can't have confidence in your automated pipeline.

Testing Pyramid

Unit Tests

Test individual functions and methods in isolation:

// Example: Python unit test
def test_calculate_discount():
    assert calculate_discount(100, 10) == 10
    assert calculate_discount(100, 0) == 0
    assert calculate_discount(100, 50) == 50

Characteristics: Fast, isolated, many per feature

Integration Tests

Test how components work together:

// Example: API integration test
def test_create_user():
    response = client.post('/api/users', {
        'name': 'John',
        'email': 'john@example.com'
    })
    assert response.status_code == 201
    assert 'id' in response.json()

Characteristics: Moderate speed, test interactions

End-to-End Tests

Test complete user workflows:

// Example: Cypress e2e test
it('user can login', () => {
  cy.visit('/login')
  cy.get('[data-testid=email]').type('user@example.com')
  cy.get('[data-testid=password]').type('password')
  cy.get('[data-testid=login-btn]').click()
  cy.url().should('include', '/dashboard')
})

Characteristics: Slow, test complete flows

Test Coverage

Aim for meaningful coverage—not necessarily 100%:

# Using pytest with coverage
pytest --cov=src --cov-report=html

# Example output
Name                  Stmts   Miss  Cover
--------------------------------------------
src/models/user.py       50      5    90%
src/services/auth.py    100     20    80%
src/api/routes.py       80      0   100%
--------------------------------------------
TOTAL                 230     25    89%

Next: GitHub Actions →

← Back to Lessons

// Lesson 4: GitHub Actions

GitHub Actions is GitHub's built-in CI/CD platform. It integrates directly with your GitHub repository and allows you to automate workflows.

Basic Workflow

Create a workflow file in .github/workflows/ci.yml:

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      
      - name: Run tests
        run: |
          pytest --cov=src
      
      - name: Upload coverage
        uses: codecov/codecov-action@v4

Workflow Syntax

Triggers

on:
  push:
    branches: [main]
    tags: ['v*']
  pull_request:
    paths:
      - 'src/**'
      - 'tests/**'
  workflow_dispatch:  # Manual trigger
  schedule:
    - cron: '0 0 * * *'  # Daily at midnight

Jobs

jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    outputs:
      version: ${{ steps.version.outputs.version }}
    steps:
      - id: version
        run: echo "version=1.0.0" >> $GITHUB_OUTPUT
  
  test:
    needs: build
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
    steps:
      - uses: actions/checkout@v4
      - run: echo "Testing with ${{ matrix.python-version }}"

Actions Marketplace

Reusable actions from the community:

steps:
  - uses: actions/checkout@v4
  - uses: actions/setup-node@v4
    with:
      node-version: '20'
  - uses: aws-actions/configure-aws-credentials@v4
    with:
      aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
      aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      aws-region: us-east-1

Next: GitLab CI →

← Back to Lessons

// Lesson 5: GitLab CI

GitLab CI is GitLab's built-in CI/CD platform. It's known for its powerful pipeline features and easy setup.

Basic Pipeline

Create .gitlab-ci.yml in your repository:

stages:
  - build
  - test
  - deploy

build:
  stage: build
  image: python:3.11
  script:
    - pip install -r requirements.txt
    - python -m compileall src
  artifacts:
    paths:
      - build/

test:
  stage: test
  image: python:3.11
  script:
    - pip install pytest pytest-cov
    - pytest --cov=src
  coverage: '/TOTAL.*\s+(\d+%)$/'

deploy:
  stage: deploy
  script:
    - echo "Deploying to production"
  only:
    - main

GitLab Runner

Runners execute pipeline jobs. You can use shared runners or register your own:

# Register a runner
gitlab-runner register \
  --url https://gitlab.com \
  --token REGISTRATION_TOKEN \
  --executor docker \
  --docker-image ubuntu:latest

Advanced Features

Rules

deploy:
  script: echo "Deploying"
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Dependencies

build:
  stage: build
  script: echo "Building"
  artifacts:
    paths:
      - build/

test:
  stage: test
  script: echo "Testing"
  needs:
    - build

Next: Jenkins →

← Back to Lessons

// Lesson 6: Jenkins

Jenkins is the oldest and most widely used CI/CD tool. It's highly customizable with thousands of plugins.

Installing Jenkins

# Using Docker
docker run -d -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

# Using apt (Ubuntu)
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins

Jenkinsfile (Declarative Pipeline)

pipeline {
    agent any
    
    environment {
        DOCKER_IMAGE = 'myapp'
        REGISTRY = 'docker.io'
    }
    
    stages {
        stage('Build') {
            steps {
                sh 'pip install -r requirements.txt'
            }
        }
        
        stage('Test') {
            steps {
                sh 'pytest --cov=src'
            }
            post {
                always {
                    junit 'reports/*.xml'
                    coverageParser 'coverage.xml'
                }
            }
        }
        
        stage('Build Docker') {
            steps {
                sh '''
                    docker build -t $DOCKER_IMAGE:$BUILD_NUMBER .
                    docker tag $DOCKER_IMAGE:$BUILD_NUMBER $DOCKER_IMAGE:latest
                '''
            }
        }
        
        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh 'kubectl apply -f k8s/'
            }
        }
    }
    
    post {
        success {
            slackSend color: 'good', message: "Build succeeded!"
        }
        failure {
            slackSend color: 'danger', message: "Build failed!"
        }
    }
}

Jenkins Pipeline Syntax

Scripted Pipeline

node {
    stage('Checkout') {
        checkout scm
    }
    
    stage('Build') {
        sh 'make build'
    }
    
    stage('Test') {
        sh 'make test'
    }
}

Next: Build Automation →

← Back to Lessons

// Lesson 7: Build Automation

Build automation tools compile your code, manage dependencies, and create deployable artifacts.

Makefiles

.PHONY: build test clean install

build:
	go build -o bin/app

test:
	go test -v -coverprofile=coverage.out ./...

clean:
	rm -rf bin/ coverage.out

install:
	go install

run:
	go run main.go

docker:
	docker build -t myapp:latest .

Maven (Java)

# pom.xml
<project>
    <groupId>com.example</groupId>
    <artifactId>myapp</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
    </dependencies>
</project>

# Commands
mvn clean package    # Build
mvn test            # Test
mvn spring-boot:run # Run

Gradle (Kotlin/Groovy)

// build.gradle.kts
plugins {
    kotlin("jvm") version "1.9.0"
    application
}

application {
    mainClass.set("com.example.App")
}

dependencies {
    implementation("org.jetbrains.kotlin:kotlin-stdlib")
}

tasks.test {
    useJUnit()
}

// Commands
gradle build       # Build
gradle test        # Test
gradle run         # Run
gradle bootJar     # Spring Boot JAR

npm Scripts

// package.json
{
  "scripts": {
    "start": "node server.js",
    "dev": "nodemon server.js",
    "test": "jest --coverage",
    "build": "webpack --mode production",
    "lint": "eslint src/"
  }
}

Next: Artifact Management →

← Back to Lessons

// Lesson 8: Artifact Management

Build artifacts need to be stored, versioned, and managed. Artifact repositories provide centralized storage.

Artifact Types

JAR/WAR: Java archives
Docker Images: Container images
NPM Packages: JavaScript/TypeScript
Python Wheels: Python packages
Binary Executables: Compiled programs

GitHub Packages

# .npmrc for NPM
@myorg:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${NPM_TOKEN}

# In workflow
- name: Publish to GitHub Packages
  run: npm publish
  env:
    NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Container Registries

# GitHub Container Registry
docker build -t ghcr.io/user/image:latest .
echo $CR_PAT | docker login ghcr.io -u USER --password-stdin
docker push ghcr.io/user/image:latest

# AWS ECR
aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_REGISTRY
docker build -t $ECR_REGISTRY/myapp:latest .
docker push $ECR_REGISTRY/myapp:latest

JFrog Artifactory

# Upload artifact
curl -u admin:password -X PUT \
  "https://artifactory.example.com/artifactory/libs-release/myapp.jar" \
  -T myapp.jar

# Download artifact
curl -u admin:password -o myapp.jar \
  "https://artifactory.example.com/artifactory/libs-release/myapp.jar"

Next: Deployment Strategies →

← Back to Lessons

// Lesson 9: Deployment Strategies

How you deploy to production matters. Different strategies trade off between risk, speed, and complexity.

Rolling Deployment

Gradually replace old instances with new ones:

# Kubernetes rolling update
kubectl rollout restart deployment/myapp

# Kubernetes deployment config
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

Pros: Simple, no downtime

Cons: Can't easily roll back instantly

Blue-Green Deployment

Two identical environments—swap traffic at once:

# Deploy to green
kubectl apply -f green-deployment.yaml

# Test green
curl -H "Host: app.green.example.com" http://lb

# Switch traffic
kubectl patch service myapp -p '{"spec":{"selector":{"version":"green"}}}'

Pros: Instant rollback, no partial states

Cons: Requires double resources

Canary Deployment

Gradually shift traffic to new version:

# Istio canary
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myapp
spec:
  hosts:
    - myapp
  http:
    - route:
        - destination:
            host: myapp-v1
          weight: 90
        - destination:
            host: myapp-v2
          weight: 10

Pros: Test with real traffic, gradual rollout

Cons: More complex setup

Feature Flags

// In code
if (featureFlags.isEnabled('new-feature')) {
    return renderNewFeature();
} else {
    return renderOldFeature();
}

// Toggle without deployment
featureFlags.enable('new-feature')
featureFlags.disable('new-feature')

Next: Infrastructure as Code →

← Back to Lessons

// Lesson 10: Infrastructure as Code

CI/CD works best when your infrastructure is also defined in code. This ensures consistency and repeatability.

Terraform in CI/CD

name: Terraform

on:
  push:
    paths:
      - 'infra/**'
  pull_request:
    paths:
      - 'infra/**'

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.5.0
          
      - name: Terraform Format
        run: terraform fmt -check
        working-directory: infra/
        
      - name: Terraform Init
        run: terraform init
        working-directory: infra/
        
      - name: Terraform Plan
        run: terraform plan
        working-directory: infra/
        
      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve
        working-directory: infra/

Ansible in CI/CD

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          
      - name: Install Ansible
        run: pip install ansible
        
      - name: Run Ansible Playbook
        run: ansible-playbook -i inventory/prod deploy.yml
        env:
          ANSIBLE_HOST_KEY_CHECKING: false

Next: Container CI/CD →

← Back to Lessons

// Lesson 11: Container CI/CD

Containers have become the standard deployment format. Let's look at building and deploying containers in CI/CD.

Building Containers

name: Build and Push

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        
      - name: Login to Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
          
      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest

Multi-Platform Builds

- name: Build multi-platform
  uses: docker/build-push-action@v5
  with:
    context: .
    platforms: linux/amd64,linux/arm64
    push: true
    tags: ghcr.io/${{ github.repository }}:latest

Security Scanning

- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    scan-type: 'fs'
    scan-ref: .
    format: 'sarif'
    output: 'trivy-results.sarif'
    
- name: Upload Trivy results to GitHub Security
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: 'trivy-results.sarif'

Next: Production Patterns →

← Back to Lessons

// Lesson 12: Production Patterns

Enterprise CI/CD requires additional considerations for security, compliance, and observability.

GitOps

GitOps uses Git as the single source of truth for infrastructure and applications:

# GitOps workflow
# 1. Developer commits code
# 2. CI builds and tests
# 3. CD updates Git with new image tag
# 4. GitOps operator (ArgoCD/Flux) syncs to cluster

# Example: Update deployment in Git
- name: Update deployment
  run: |
    cd infrastructure
    yq -i '.spec.template.spec.containers[0].image = "${{ github.sha }}"' deployment.yaml
    git config user.email "ci@example.com"
    git config user.name "CI"
    git add deployment.yaml
    git commit -m "Deploy ${{ github.sha }}"
    git push

ArgoCD Example

# Application manifest
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/user/repo.git
    targetRevision: HEAD
    path: k8s
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Observability

Monitor your CI/CD pipelines:

Pipeline metrics: Build times, success rates
Deployment frequency: How often you ship
Change failure rate: What percentage of deploys fail
Mean time to recovery: How fast you fix issues

The Road Ahead

Congratulations on completing this guide! You've learned:

CI/CD fundamentals and benefits
Git workflows and branching strategies
Automated testing practices
GitHub Actions and GitLab CI
Jenkins pipelines
Build automation tools
Artifact management
Deployment strategies
Infrastructure as Code integration
Container CI/CD
GitOps and production patterns

Continue learning with:

ArgoCD or Flux for GitOps
Spinnaker for advanced deployments
Tekton for Kubernetes-native CI/CD
Service meshes for traffic management

← Back to Lessons

AutomateYour Pipeline

// The Path to CI/CD Mastery

Introduction to CI/CD

Version Control

Automated Testing

GitHub Actions

GitLab CI

Jenkins

Build Automation

Artifact Management

Deployment Strategies

Infrastructure as Code

Container CI/CD

Production Patterns

// Lesson 1: Introduction to CI/CD

What is CI/CD?

The Three Pillars

Continuous Integration (CI)

Continuous Delivery (CD)

Continuous Deployment

Benefits of CI/CD

The CI/CD Pipeline

// Lesson 2: Version Control & Git Workflows

Branching Strategies

Trunk-Based Development

Git Flow

GitHub Flow

Commit Message Best Practices

Conventional Commits

// Lesson 3: Automated Testing

Testing Pyramid

Unit Tests

Integration Tests

End-to-End Tests

Test Coverage

// Lesson 4: GitHub Actions

Basic Workflow

Workflow Syntax

Triggers

Jobs

Actions Marketplace

// Lesson 5: GitLab CI

Basic Pipeline

GitLab Runner

Advanced Features

Rules

Dependencies

// Lesson 6: Jenkins

Installing Jenkins

Jenkinsfile (Declarative Pipeline)

Jenkins Pipeline Syntax

Scripted Pipeline

// Lesson 7: Build Automation

Makefiles

Maven (Java)

Gradle (Kotlin/Groovy)

npm Scripts

// Lesson 8: Artifact Management

Artifact Types

GitHub Packages

Container Registries

JFrog Artifactory

// Lesson 9: Deployment Strategies

Rolling Deployment

Blue-Green Deployment

Canary Deployment

Feature Flags

// Lesson 10: Infrastructure as Code

Terraform in CI/CD

Ansible in CI/CD

// Lesson 11: Container CI/CD

Building Containers

Multi-Platform Builds

Security Scanning

// Lesson 12: Production Patterns

GitOps

ArgoCD Example

Observability

The Road Ahead

Automate
Your Pipeline