Home Guides Tests Games Pathway Foss Blog

BUILD A CI/CD PIPELINE

// Automate everything from code to production.

// WHAT WE'RE BUILDING

In this tutorial, you'll build a complete CI/CD pipeline using GitHub Actions. Every time you push code, your pipeline will automatically run tests, build Docker images, and deploy to your server. No more manual deployments - just push and watch your code go live.

// WHY THIS MATTERS

Manual deployments are error-prone and time-consuming. With CI/CD, you get: faster releases, fewer bugs in production, automatic testing, and the confidence to deploy frequently. Every serious development team uses some form of CI/CD.

// What You'll Learn

This project teaches you about:

┌──────────┐     ┌──────────┐     ┌──────────┐     ┌──────────┐
│   Git    │────▶│  Build   │────▶│  Test    │────▶│ Deploy   │
│   Push   │     │  Image   │     │  Suite   │     │  to VPS  │
└──────────┘     └──────────┘     └──────────┘     └──────────┘
     │                │                 │               │
     │            Dockerfile         Run Tests      SSH Pull
     │            & Build            & Lint         & Restart

Why GitHub Actions?

GitHub Actions is free for public repositories and has generous free tiers for private repos. It's deeply integrated with GitHub, meaning you don't need a separate CI server. Everything lives in your repository.

Prerequisites

Before starting this tutorial, make sure you've completed:

If you haven't completed these, check out our other tutorials first!

// Understanding CI/CD Concepts

What is CI/CD?

CI (Continuous Integration) means every time you push code, it's automatically tested and validated. Developers might push dozens of times a day, and each push triggers a build that runs tests.

CD (Continuous Deployment) takes it further - after tests pass, the code is automatically deployed to an environment (staging or production).

The Pipeline Flow

  1. Trigger - Push to branch (or PR, tag, schedule)
  2. Checkout - Clone the repository
  3. Build - Compile code, build Docker image
  4. Test - Run unit tests, linting, security scans
  5. Deploy - Push to server, restart services

GitHub Actions Terminology

// Part 1: Create a Sample Application

First, let's create a simple web application to deploy. We'll use a basic Node.js app with Docker.

1.1 Initialize the Project

$ mkdir -p ~/projects/cicd-demo
$ cd ~/projects/cicd-demo
$ git init
$ git config user.name "Your Name"
$ git config user.email "you@example.com"

1.2 Create the Application

$ cat > package.json << 'EOF'
{
  "name": "cicd-demo",
  "version": "1.0.0",
  "description": "CI/CD Demo Application",
  "main": "server.js",
  "scripts": {
    "start": "node server.js",
    "test": "jest --coverage"
  },
  "dependencies": {
    "express": "^4.18.2"
  },
  "devDependencies": {
    "jest": "^29.7.0"
  }
}
EOF

1.3 Create the Server

$ cat > server.js << 'EOF'
const express = require('express');
const app = express();
const PORT = process.env.PORT || 3000;

app.get('/', (req, res) => {
  res.json({
    message: 'Hello from CI/CD Pipeline!',
    timestamp: new Date().toISOString(),
    version: '1.0.0'
  });
});

app.get('/health', (req, res) => {
  res.json({ status: 'healthy' });
});

app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});
EOF

1.4 Create a Test

$ mkdir -p tests
$ cat > tests/app.test.js << 'EOF'
const app = require('../server');

describe('App tests', () => {
  test('health endpoint returns healthy', async () => {
    const response = await request(app).get('/health');
    expect(response.status).toBe(200);
    expect(response.body.status).toBe('healthy');
  });
  
  test('root endpoint returns greeting', async () => {
    const response = await request(app).get('/');
    expect(response.status).toBe(200);
    expect(response.body.message).toBeDefined();
  });
});
EOF

1.5 Create Dockerfile

$ cat > Dockerfile << 'EOF'
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY server.js ./

EXPOSE 3000

CMD ["npm", "start"]
EOF

1.6 Create .dockerignore

$ cat > .dockerignore << 'EOF'
node_modules
npm-debug.log
.git
.gitignore
README.md
tests
*.test.js
jest.config.js
EOF

1.7 Create .gitignore

$ cat > .gitignore << 'EOF'
node_modules/
.env
*.log
.DS_Store
coverage/
EOF

// Part 2: Create GitHub Actions Workflow

Now let's create the workflow that will run our CI/CD pipeline.

2.1 Create Workflow Directory

$ mkdir -p .github/workflows

2.2 Create the CI Workflow

$ cat > .github/workflows/ci.yml << 'EOF'
name: CI Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    name: Test Application
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '18'
        cache: 'npm'
    
    - name: Install dependencies
      run: npm ci
    
    - name: Run tests
      run: npm test -- --passive
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage/lcov.info

  build:
    name: Build Docker Image
    runs-on: ubuntu-latest
    needs: test
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3
    
    - name: Log in to Docker Hub
      uses: docker/login-action@v3
      with:
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_TOKEN }}
    
    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v5
      with:
        images: yourusername/cicd-demo
        tags: |
          type=ref,event=branch
          type=sha,prefix=
          type=raw,value=latest,enable
    
    - name: Build and push
      uses: docker/build-push-action@v5
      with:
        context: .
        push: ${{ github.ref == 'refs/heads/main' }}
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
EOF

Understanding the Workflow

// Part 3: Create Deployment Workflow

Now let's add a deployment workflow that deploys to our VPS when code is pushed to main.

3.1 Create Deployment Workflow

$ cat > .github/workflows/deploy.yml << 'EOF'
name: Deploy to Production

on:
  push:
    branches: [ main ]
  workflow_dispatch:

jobs:
  deploy:
    name: Deploy to VPS
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Setup SSH
      uses: webfactory/ssh-agent@v0.8.0
      with:
        ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
    
    - name: Deploy to server
      env:
        HOST: ${{ secrets.SERVER_HOST }}
        USER: ${{ secrets.SERVER_USER }}
      run: |
        echo "Connecting to $USER@$HOST"
        ssh -o StrictHostKeyChecking=no $USER@$HOST << 'REMOTE'
        
        # Pull latest image
        docker pull yourusername/cicd-demo:latest
        
        # Stop old container
        docker stop cicd-demo || true
        docker rm cicd-demo || true
        
        # Start new container
        docker run -d \
          --name cicd-demo \
          -p 3000:3000 \
          --restart unless-stopped \
          yourusername/cicd-demo:latest
        
        # Show logs
        docker logs cicd-demo --tail 20
        
        echo "Deployment complete!"
        REMOTE
    
    - name: Verify deployment
      run: |
        sleep 5
        curl -f https://yourdomain.com/health || exit 1
        echo "Deployment verified!"
EOF

3.2 Alternative: Use a Deploy Action

You can also use community actions for cleaner deployment:

$ cat > .github/workflows/deploy-action.yml << 'EOF'
name: Deploy to Production

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Deploy to Server
      uses: appleboy/ssh-action@v0.1.7
      with:
        host: ${{ secrets.SERVER_HOST }}
        username: ${{ secrets.SERVER_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          cd /opt/cicd-demo
          docker pull yourusername/cicd-demo:latest
          docker stop demo || true
          docker rm demo || true
          docker run -d \
            --name demo \
            -p 3000:3000 \
            --restart always \
            yourusername/cicd-demo:latest
          docker image prune -f
EOF

// Part 4: Setup Server for Deployment

Prepare your VPS to receive deployments.

4.1 Install Docker on VPS

your-username@vps:~$ sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
your-username@vps:~$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
your-username@vps:~$ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
your-username@vps:~$ sudo apt update && sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

4.2 Create Deployment Directory

your-username@vps:~$ sudo mkdir -p /opt/cicd-demo
your-username@vps:~$ sudo chown $USER:$USER /opt/cicd-demo

4.3 Configure Firewall

your-username@vps:~$ sudo ufw allow 22/tcp
your-username@vps:~$ sudo ufw allow 3000/tcp
your-username@vps:~$ sudo ufw enable

4.4 Add nginx Reverse Proxy (Optional)

your-username@vps:~$ sudo apt install -y nginx
your-username@vps:~$ sudo cat > /etc/nginx/sites-available/cicd-demo << 'EOF'
server {
    listen 80;
    server_name yourdomain.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_cache_bypass $http_upgrade;
    }
}
EOF
your-username@vps:~$ sudo ln -s /etc/nginx/sites-available/cicd-demo /etc/nginx/sites-enabled/
your-username@vps:~$ sudo nginx -t
your-username@vps:~$ sudo systemctl reload nginx

// Part 5: Configure GitHub Secrets

Secrets store sensitive data securely in GitHub.

5.1 Generate SSH Key Pair

$ ssh-keygen -t ed25519 -C "github-actions"
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519): /home/user/.ssh/github_actions
Enter passphrase (empty for no passphrase): 
Enter same passphrase again:

5.2 Add Public Key to VPS

$ cat ~/.ssh/github_actions.pub | ssh your-username@your-vps-ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

5.3 Add Secrets to GitHub

Go to your repository on GitHub:

  1. Navigate to Settings → Secrets and variables → Actions
  2. Click New repository secret
  3. Add these secrets:
# SSH_PRIVATE_KEY
# Copy the PRIVATE key content (not .pub)
$ cat ~/.ssh/github_actions
# Copy entire output including -----BEGIN OPENSSH PRIVATE KEY-----

# SERVER_HOST
# Your VPS IP address or domain
# Example: 123.45.67.89 or yourdomain.com

# SERVER_USER
# Your SSH username
# Example: your-username

# DOCKER_USERNAME
# Your Docker Hub username

# DOCKER_TOKEN
# Docker Hub access token (not password)

Security Note

Never commit secrets to your repository. Use GitHub Secrets for everything sensitive. SSH keys should have a passphrase for additional security. The private key you add to GitHub should be the one without a passphrase for automation, or use ssh-agent with a passphrase.

// Part 6: Push and Watch the Pipeline

Let's push our code and watch the magic happen.

6.1 Push to GitHub

$ git add .
$ git commit -m "Initial commit with CI/CD pipeline"
$ git remote add origin git@github.com:yourusername/cicd-demo.git
$ git push -u origin main

6.2 Watch the Workflow

  1. Go to your repository on GitHub
  2. Click on the Actions tab
  3. You should see your workflow running
  4. Click on the workflow run to see details
# You'll see output like this in the Actions tab:
CI Pipeline
✓ Test Application - 1m 23s
✓ Build Docker Image - 2m 45s

6.3 Check Deployment

Once deployed, verify it's working:

$ curl https://yourdomain.com/health
{"status":"healthy"}
$ curl https://yourdomain.com/
{"message":"Hello from CI/CD Pipeline!","timestamp":"2024-01-15T10:30:00.000Z","version":"1.0.0"}

// Part 7: Advanced Features

7.1 Environment Variables

Pass environment variables to your container:

$ cat > .github/workflows/deploy.yml << 'EOF'
# ... inside the deploy job:
    - name: Deploy to Server
      uses: appleboy/ssh-action@v0.1.7
      with:
        host: ${{ secrets.SERVER_HOST }}
        username: ${{ secrets.SERVER_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        envs: NODE_ENV,LOG_LEVEL
        script: |
          docker stop demo || true
          docker rm demo || true
          docker run -d \
            --name demo \
            -p 3000:3000 \
            -e NODE_ENV=production \
            -e LOG_LEVEL=info \
            --restart always \
            yourusername/cicd-demo:latest
EOF

7.2 Multiple Environments

Deploy to staging first, then production:

$ cat > .github/workflows/multi-env.yml << 'EOF'
name: Deploy to Multiple Environments

on:
  push:
    branches: [ develop ]

jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    environment: staging
    
    steps:
    - uses: actions/checkout@v4
    - name: Deploy to Staging
      uses: appleboy/ssh-action@v0.1.7
      with:
        host: ${{ secrets.STAGING_HOST }}
        username: ${{ secrets.STAGING_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          cd /opt/cicd-demo
          docker pull yourusername/cicd-demo:staging
          docker stop demo || true
          docker run -d --name demo -p 3000:3000 yourusername/cicd-demo:staging

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: production
    
    steps:
    - uses: actions/checkout@v4
    - name: Deploy to Production
      uses: appleboy/ssh-action@v0.1.7
      with:
        host: ${{ secrets.PROD_HOST }}
        username: ${{ secrets.PROD_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          docker pull yourusername/cicd-demo:latest
          docker stop demo || true
          docker run -d --name demo -p 3000:3000 yourusername/cicd-demo:latest
EOF

7.3 Scheduled Runs

Run jobs on a schedule:

cat > .github/workflows/scheduled.yml << 'EOF'
name: Nightly Maintenance

on:
  schedule:
    - cron: '0 2 * * *'  # 2 AM UTC every day

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
    - name: Clean up old images
      uses: appleboy/ssh-action@v0.1.7
      with:
        host: ${{ secrets.SERVER_HOST }}
        username: ${{ secrets.SERVER_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          docker image prune -af --filter "until=72h"
          docker system prune -af
EOF

7.4 Docker Compose Deployment

For more complex applications, use docker-compose:

$ cat > docker-compose.yml << 'EOF'
version: '3.8'

services:
  app:
    image: yourusername/cicd-demo:latest
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - PORT=3000
    
  nginx:
    image: nginx:alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - app
EOF
cat > deploy.sh << 'EOF'
#!/bin/bash
docker-compose pull
docker-compose up -d
docker image prune -f
EOF
chmod +x deploy.sh

// Part 8: Troubleshooting

Workflow Not Running?

SSH Connection Failed?

Docker Build Failed?

Deployment Works but App Crashes?

Debug Tip

Add continue-on-error: true to steps to see more output. You can also add run: echo "${{ secrets.SECRET_NAME }}" to verify secrets are being passed correctly.

// Summary

You've built a complete CI/CD pipeline!

What You Can Do Now

Every professional development team uses CI/CD. You now have a skill that's in high demand.

The revolution will not be proprietary.