DevOps CA3 - CI/CD Pipeline Implmentation

---

Student Information

• Student Name: Cathal Flood

• Student Number: C22312211

• Course: DevOps - Continuous Integration and Deployment

• Assessment: CA3 Complete CI/CD Pipeline Implementation

• Submission Date 14/12/2025

Repository information

• GitHub Repository: 'https://github.com/floodcathal/devops-cicd-pipeline'
• Azure DevOps Project: 'https://dev.azure.com/C22312211/C22312211_CA3'
• Azure DevOps Pipeline: 'https://dev.azure.com/C22312211/C22312211_CA3/_build?definitionId=1'
• Azure Test Environment URL: 'https://c22312211-ca3-test-heead9ecemdydtdh.francecentral-01.azurewebsites.net/'
• Azure Production Environment URL: 'https://c22312211-ca3-prod-cte7gfdcgvhwa4gm.francecentral-01.azurewebsites.net/'

---

Table of Contents

1. Project Overview
2. Repository Information
3. Application Architecture
4. CI/CD Pipeline
5. Branch Protection Rules
6. Testing Strategy
7. Deployment Environments
8. Setup and Installation
9. Pipeline Execution
10. Security and Performance
11. Learning Outcomes
12. Troubleshooting Guide
13. References

---

Project Overview

This project implements a complete CI/CD pipeline for a Flask based calculator web application The pipeline automates the entire software delivery process from code commit to production deployment, incorporating comprehensive testing, security scanning, and approval gates.

---

Key Features

• Multi-stage CI/CD pipeline with 7 distinct stages
• Automated testing including unit tests, security scans, performance tests, and UAT
• Multi-environment deployment (Test and Production)
• Approval gates for controlled production releases
• 100% code coverage with comprehensive test suite
• Security scanning using Bandit and Safety
• Browser automation testing using Selenium
• Zero-downtime deployments using Azure ZipDeploy API

Technologies Used

• Application: Python 3.11, Flask 3.0.0, Gunicorn
• Testing: Pytest, Pylint, Bandit, Safety, Selenium
• CI/CD: Azure DevOps Pipelines
• Deployment: Azure App Services (Linux)
• Version Control: Git, GitHub
• Containerization: Docker (via Azure App Services)

---

Repository Information

GitHub Repository

• URL: https://github.com/floodcathal/devops-cicd-pipeline
• Branch: main

Azure DevOps Project

• URL: https://dev.azure.com/C22312211/C22312211_CA3
• Pipeline: C22312211_CA3

Live Environments

• Test: c22312211-ca3-test-heead9ecemdydtdh.francecentral-01.azurewebsites.net
• Production: c22312211-ca3-prod-cte7gfdcgvhwa4gm.francecentral-01.azurewebsites.net

---

Application Architecture

Repository Structure

C22312211_CA3/
├── app.py                      # Flask web application
├── calculator.py               # Calculator business 
logic
├── requirements.txt            # Python dependencies
├── azure-pipelines.yml         # CI/CD pipeline 
definition
├── test_calculator.py          # Unit tests
├── test_security.py            # Security tests
├── test_performance.py         # Performance tests
├── test_selenium_uat.py        # User acceptance tests
├── .coveragerc                 # Coverage configuration
├── .gitignore                  # Git ignore rules
├── README.md                   # This file
└── screenshots/                # Documentation screenshots

---

CI/CD Pipeline

Pipeline Overview

The pipeline consists of 7 sequential stages that ensure code quality, security, and reliable deployment:

1. Pipeline Overview

Stage 1: Build and Unit Test

Purpose: Compile code, run unit tests, perform static analysis, and create deployment artifact.

Steps:

1. Set up Python 3.11 environment
2. Install project dependencies
3. Run Pylint static code analysis
4. Execute unit tests with pytest
5. Measure code coverage (target: 100%)
6. Publish test results and coverage reports
7. Create deployment artifact (ZIP file)

Success Criteria:

• All 8 unit tests pass
• Code coverage ≥ 100%
• Pylint score acceptable
• Artifact created successfully

Duration: ~3 minutes

Stage 2: Security Testing

Purpose: Scan code for security vulnerabilities and dependency issues.

Tools Used:

• Bandit: Scans Python code for security issues
• Safety: Checks dependencies for known vulnerabilities

Checks Performed:

• SQL injection vulnerabilities
• Use of insecure functions
• Hard-coded credentials
• Weak cryptography
• Known CVEs in dependencies

Success Criteria:

• No high-severity security issues
• Security reports published

Duration: ~2 minutes

---

Stage 3: Performance Testing

Tests:

• Calculator operation speed (< 1ms per operation)
• Web application response time (< 2 seconds)
• Load testing with multiple iterations

Success Criteria:

• All performance benchmarks met
• No degradation from previous builds

Duration: ~1 minute

---

Stage 4: Deployment to Test Environment

Purpose: Deploy application to Test environment for validation.

Process:

1. Download build artifact from Stage 1
2. Extract deployment files
3. Deploy to Azure Web App (Test) using ZipDeploy API
4. Wait for application startup (30 seconds)
5. Perform health check verification

Technology:

• Azure App Services (Linux)
• Python 3.11 runtime
• Gunicorn WSGI server

Success Criteria:

• Deployment completes without errors
• Health endpoint returns 200 OK
• Application is accessible

Duration: ~3 minutes

---

Stage 5: User Acceptance Testing (UAT)

Purpose: Automated browser testing on deployed Test environment.

Technology:

• Selenium WebDriver
• Chrome browser (headless mode)

Test Scenarios:

1. Page loads successfully
2. Environment badge displays "TEST"
3. Calculator performs addition (10 + 5 = 15)
4. User interface is responsive
5. Error handling works correctly

Success Criteria:

• All Selenium tests pass
• Calculator functions correctly in browser
• No JavaScript errors

Duration: ~4 minutes

---

Stage 6: Manual Approval Gate

Purpose: Manual approval required before production deployment.

Process:

1. Pipeline pauses after UAT completion
2. Designated approver reviews:
   • Test environment functionality
   • Test results
   • Security scan reports
   • UAT test outcomes
3. Approver grants/denies production deployment

Timeout: 3 days (auto-reject if not approved)

Success Criteria:

• Manual approval granted

Duration: Variable (depends on approver)

---

Stage 7: Deployment to Production

Purpose: Deploy approved application to Production environment.

Process:

1. Download same build artifact used in Test
2. Extract deployment files
3. Deploy to Azure Web App (Production)
4. Wait for application startup
5. Perform health check
6. Mark deployment as complete

Success Criteria:

• Deployment completes successfully
• Health endpoint returns 200 OK
• Production environment accessible
• Calculator functions correctly

Duration: ~3 minutes

---

Branch Protection Rules

GitHub Branch Protection Rules

To ensure code quality and prevent accidental changes to the main branch, the following branch protection rules are enforced:

Protected Branch: main

Required Settings:

1. Require pull request before merging

   • At least 1 approval required
   • Dismiss stale pull request approvals when new commits are pushed

2. Require status checks to pass before merging

   • Azure Pipelines CI build must succeed
   • All tests must pass

3. Require conversation resolution before merging

   • All review comments must be resolved

4. Include administrators

   • Rules apply to repository administrators as well

2. Branch Protection

Azure DevOps Branch Policies

Protected Branch: main

Policies Enabled:

1. Require a minimum number of reviewers: 1
2. Check for linked work items: Optional
3. Check for comment resolution: Required
4. Build validation: Azure Pipelines CI must succeed

Configuration:

1. Azure DevOps → Repos → Branches
2. Click "..." on main branch → Branch policies
3. Configure policies as above
4. Save changes

---

Testing Strategy

Framework: Pytest with pytest-cov

Coverage:

• 8 test cases covering all calculator operations
• 100% code coverage of calculator.py
• Edge cases and error handling included

Test Cases:

1. Addition of positive numbers
2. Subtraction with negative results
3. Multiplication including zero
4. Division with decimal results
5. Division by zero (error handling)
6. Power operation
7. Square root of positive number
8. Square root of negative number (error handling)

Execution:

pytest test_calculator.py --cov=calculator --cov-report=html -v

---

Security Testing

Tools:

1. Bandit - Python Security Scanner

   • Scans for common security issues
   • Identifies insecure code patterns
   • Checks for hardcoded secrets

2. Safety - Dependency Vulnerability Scanner

   • Checks dependencies against CVE database
   • Identifies outdated packages with known vulnerabilities
   • Recommends security updates

Execution:

bandit -r calculator.py app.py -f txt
safety check

---

Performance Testing

Metrics Measured:

• Operation Speed: < 1 millisecond per calculation
• Response Time: < 2 seconds for web requests
• Throughput: 1000+ operations per second

Execution:

python test_performance.py

---

User Acceptance Testing (UAT)

Technology: Selenium WebDriver with Chrome

Test Flow:

1. Launch headless Chrome browser
2. Navigate to deployed application
3. Locate input elements by ID
4. Enter test data (10, 5)
5. Click "Add" button
6. Verify result displays "15"
7. Check environment badge

Execution:

export TEST_URL=https://[test-url]
python test_selenium_uat.py

---

Deployment Enviroments

Test Environment

Purpose: Pre-production validation and testing

Configuration:

• URL: c22312211-ca3-test-heead9ecemdydtdh.francecentral-01.azurewebsites.net
• Runtime: Python 3.11 on Linux
• Server: Gunicorn WSGI
• Environment Variables:
  • ENVIRONMENT=TEST
  • DEBUG=True
  • SCM_DO_BUILD_DURING_DEPLOYMENT=true

Features:

• Orange environment badge
• Verbose logging enabled
• Automated deployments from pipeline

---

Production Environment

Purpose: Live production system

Configuration:

• URL: c22312211-ca3-prod-cte7gfdcgvhwa4gm.francecentral-01.azurewebsites.net
• Runtime: Python 3.11 on Linux
• Server: Gunicorn WSGI
• Environment Variables:
  • ENVIRONMENT=PRODUCTION
  • DEBUG=False
  • SCM_DO_BUILD_DURING_DEPLOYMENT=true

Features:

• Blue environment badge
• Minimal logging (errors only)
• Manual approval required for deployment

---

Setup and Installation

Prerequisites

• Git
• Python 3.11+
• Azure DevOps account
• Azure subscription
• GitHub account

Local Development Setup

1. Clone the repository:

   git clone [https://github.com/floodcathal/devops-ci-calculator]
   cd C22312211_CA3

2. Create virtual environment:

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Run tests:

   pytest test_calculator.py --cov=calculator -v

5. Run application locally:

   python app.py

   Open browser to http://localhost:5000

---

Azure DevOps Setup

1. Create Azure DevOps project:

   • Name: C22312211_CA3
   • Visibility: Private

2. Create Variable Group:

   • Name: CA3-Deployment
   • Variables:
     • AZURE_WEBAPP_NAME_TEST
     • AZURE_WEBAPP_NAME_PROD
     • AZURE_WEBAPP_PUBLISH_PROFILE_TEST (secret)
     • AZURE_WEBAPP_PUBLISH_PROFILE_PROD (secret) 3. Variable Groups

3. Create Environments:

   • Test (no approvals)
   • Production (optional additional approvals)

4. Create Pipeline:

   • Connect to GitHub repository
   • Use existing azure-pipelines.yml

---

Azure Web Apps Setup

1. Create Resource Group:

   az group create --name CA3-Resources --location francecentral

2. Create Test App Service:

   az webapp create \
     --resource-group CA3-Resources \
     --name c22312211-ca3-test \
     --runtime "PYTHON:3.11" \
     --os-type Linux

3. Create Production App Service:

   az webapp create \
     --resource-group CA3-Resources \
     --name c22312211-ca3-prod \
     --runtime "PYTHON:3.11" \
     --os-type Linux

4. Configure both apps:

   • Set startup command: gunicorn --bind=0.0.0.0 --timeout 600 app:app
   • Enable SCM Basic Auth
   • Download publish profiles
   • Add to variable group

---

Pipeline Execution

Triggering the Pipeline

Automatic Triggers:

• Push to main branch
• Pull request to main branch

Manual Trigger:

1. Navigate to Azure DevOps → Pipelines
2. Select pipeline
3. Click "Run pipeline"
4. Select branch (main)
5. Click "Run"

---

Monitoring Pipeline Progress

1. View real-time logs:

   • Click on running pipeline
   • Select stage to view
   • Click on job to see detailed logs

2. Review test results:

   • Navigate to "Tests" tab
   • View pass/fail status
   • Examine failed test details

3. Check code coverage:

   • Navigate to "Code Coverage" tab
   • View coverage percentage
   • Drill down to file level

---

Approving Production Deployment

1. Receive notification
2. Navigate to pipeline run
3. Click "Review" button on Approval Gate stage
4. Review checklist:
   • All tests passed
   • UAT completed successfully
   • security reports reviewed
   • test environment verified
5. Add comment
6. Production deployment begins automatically

Security and Performance

Security Measures Implemented

1. Code Scanning:

   • Bandit for Python security issues
   • No high-severity vulnerabilities allowed

2. Dependency Management:

   • Safety checks for CVEs in dependencies
   • Regular updates of packages

3. Secure Configuration:

   • Debug mode disabled in production
   • Environment variables for sensitive data
   • Secrets stored in Azure DevOps Library

4. Deployment Security:

   • Publish profiles as secrets
   • HTTPS enforcement
   • Basic auth for deployment endpoints

---

Performance Optimizations

1. Application Level:

   • Efficient Python code (< 1ms per operation)
   • Gunicorn for production WSGI server
   • Proper timeout configurations

2. Deployment Process:

   • Artifact caching
   • ZipDeploy for fast deployments
   • Parallel test execution where possible

3. Infrastructure:

   • Azure App Services scaling
   • CDN for static assets (if needed)
   • Health check monitoring

---

Pipeline Execution

1. Pipeline Overview - All 7 stages completed

Complete pipeline with all green checkmarks

---

2. Build Stage - Unit tests and coverage

Test results showing 8/8 passed and 100% code coverage

---

3. Security Testing - Bandit and Safety results

No high-severity security issues found

---

4. Performance Testing - Benchmark results

All performance metrics met

---

5. Test Deployment - Successful deployment

Deployment logs and health check verification

---

6. UAT Testing - Selenium test results

Browser automation tests successful

---

7. Approval Gate - Manual approval step

Manual approval interface and checklist

---

8. Production Deployment - Final deployment

Production deployment completed successfully

---

Application Screenshots

9. Test Environment - Calculator interface

Calculator with TEST environment badge

---

10. Test Calculation - Working calculator

Calculation result: 10 + 5 = 15

---

11. Production Environment - Live application

Calculator with PRODUCTION environment badge

---

12. Production Calculation - Verified functionality

Calculator working correctly in production

---

Azure Portal Screenshots

13. Resource Group - Azure resources

All Azure resources for the project

---

14. App Services - Web apps configuration

Test and Production app services configuration

---

15. Variable Group - Secure configuration

Pipeline variables with secrets hidden

---

GitHub Repository

16. GitHub Overview - Repository structure

Complete repository file structure

---

Learning Outcomes

What Went Well

1. Pipeline Design:

   • Multi-stage pipeline successfully implements all required components
   • Clear separation of concerns between stages
   • Efficient use of artifacts to avoid rebuilding

2. Testing Implementation:

   • Achieved 100% code coverage
   • Comprehensive test suite covering all scenarios
   • Successful integration of Selenium for UAT

3. Deployment Strategy:

   • ZipDeploy API provides fast, reliable deployments
   • Health checks ensure application availability
   • Environment variables enable proper configuration

4. DevOps Practices:

   • Automated entire delivery pipeline
   • Manual approval gates provide control
   • Clear visibility into all stages

---

Challenges Faced

1. Service Connection Authentication:

   • Initial attempts to use service principal failed due to insufficient permissions
   • Solution: Used publish profile authentication with ZipDeploy API
   • Learning: Multiple authentication methods available; choose based on permissions

2. JavaScript Syntax Error:

   • Multi-line comment syntax broke JavaScript execution
   • Solution: Used single-line comments (//) instead of multi-line (/* */)
   • Learning: Careful attention to embedded code syntax in templates

3. DNS Resolution for Health Checks:

   • Generic Azure URLs didn't resolve immediately
   • Solution: Made health checks non-blocking with fallback
   • Learning: Azure DNS can take time to propagate

4. YAML Syntax Issues:

   • YAML anchors not supported in Azure Pipelines
   • Solution: Removed anchors, used explicit step definitions
   • Learning: Azure Pipelines YAML has specific limitations

---

Troubleshooting Guide

Common Pipeline Issues

Issue 1: Pipeline Fails at Build Stage

Symptoms:

• Build stage shows red X
• Error: "ModuleNotFoundError: No module named 'flask'"

Solution:

# Check requirements.txt includes all dependencies
pip freeze > requirements.txt
git add requirements.txt
git commit -m "fix: Update requirements.txt"
git push

---

Issue 2: Unit Tests Fail

Symptoms:

• Tests pass locally but fail in pipeline
• Error: "AssertionError" or test failures

Solution:

1. Run tests locally with same Python version:

   python --version  # Should be 3.11.x
   pytest test_calculator.py -v

2. Check test file encoding (should be UTF-8)

3. Verify calculator.py logic matches test expectations

---

Issue 3: Deployment Fails - PublishProfile Error

Symptoms:

• Deploy stage fails
• Error: "Input required: PublishProfilePassword"

Solution:

1. Verify variable group has correct publish profiles:

   Azure DevOps → Pipelines → Library → CA3-Deployment
   

2. Re-download publish profiles from Azure Portal:

   • App Services → Your app → Get publish profile
   • Copy ENTIRE XML content
   • Update variable in variable group
   • Lock the variable (make it secret)

3. Verify variable names match YAML:

   • AZURE_WEBAPP_PUBLISH_PROFILE_TEST
   • AZURE_WEBAPP_PUBLISH_PROFILE_PROD

---

Issue 4: Health Check Fails

Symptoms:

• Deployment succeeds but health check fails
• Error: "curl: (6) Could not resolve host"

Solution:

1. Check app is actually running:

   # Open in browser
   https://your-app-name.azurewebsites.net

2. Verify app name in variable group matches Azure app name

3. If DNS issue persists, health checks are non-blocking (won't fail pipeline)

---

Issue 5: UAT Tests Fail - ChromeDriver Error

Symptoms:

• UAT stage fails
• Error: "ChromeDriver version mismatch" or "WebDriver not found"

Solution: Pipeline installs Chrome automatically. If fails locally:

# Install Chrome
sudo apt-get install google-chrome-stable

# Install ChromeDriver
wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/120.0.6099.109/linux64/chromedriver-linux64.zip
unzip chromedriver-linux64.zip
sudo mv chromedriver-linux64/chromedriver /usr/local/bin/
sudo chmod +x /usr/local/bin/chromedriver

---

Issue 6: JavaScript Not Working in Browser

Symptoms:

• Page loads but buttons don't work
• Console error: "calc is not defined"

Solution:

1. Check browser console for errors (F12)

2. Verify JavaScript is properly closed:

   <script>
   // JavaScript code here
   </script>

3. Check for syntax errors:

   • Use // for single-line comments
   • NOT /* */ without closing

4. Hard refresh browser: Ctrl + Shift + R

---

Issue 7: Security Scan Shows High Severity

Symptoms:

• Bandit reports high-severity issues
• Pipeline continues (doesn't fail)

Solution:

1. Review Bandit report in pipeline logs

2. Common issues:

   • debug=True → Use environment variable
   • Hardcoded secrets → Use environment variables
   • Insecure functions → Replace or suppress with # nosec

3. Suppress false positives:

   app.run(host='0.0.0.0')  # nosec B104

---

Issue 8: Pipeline YAML Syntax Error

Symptoms:

• Pipeline shows "Informational run"
• Error: "YAML error preventing Azure Pipelines from determining if pipeline should run"

Solution:

1. Validate YAML online: https://www.yamllint.com/

2. Common issues:

   • Incorrect indentation (use 2 spaces, not tabs)
   • Missing quotes around strings with special characters
   • Duplicate keys

3. Use provided YAML file from repository (tested and working)

---

References

Documentation and Resources

1. Python Testing: -pytest documentation. pytest: helps you write better programs. Available at: https://docs.pytest.org/en/stable/ [Accessed: November 2025]

   • pytest-cov Documentation. pytest-cov documentation. Available at: https://pytest-cov.readthedocs.io/en/latest/readme.html [Accessed: November 2025]

2. Azure DevOps:

   • Microsoft. Azure Pipelines Documentation. Available at: https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops [Accessed: November 2025]
   • Microsoft. YAML schema reference Available at: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines [Accessed: November 2025]

3. Code Quality:

   • Pylint Documentation: Pylint user manual. Available at: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines [Accessed: November 2025]
   • Python Software Foundation: PEP 8 - style guide for python code. Available at: https://peps.python.org/pep-0008/ [Accessed: November 2025]

Documentation

4. Flask (n.d.). Welcome to Flask — Flask Documentation (1.1.x). [online] flask.palletsprojects.com. Available at: https://flask.palletsprojects.com/. ‌
5. docs.pytest.org. (n.d.). pytest: helps you write better programs — pytest documentation. [online] Available at: https://docs.pytest.org/. ‌
6. Selenium. (n.d.). The Selenium Browser Automation Project. [online] Available at: https://www.selenium.dev/documentation/. ‌
7. SyntaxC4 (2025). Azure App Service documentation - Azure App Service. [online] Microsoft.com. Available at: https://docs.microsoft.com/azure/app-service/.

Tools and Technologies

8. Python 3.11: PYTHON (2025). Python. [online] Python.org. Available at: https://www.python.org/. ‌
9. Flask 3.0.0: Flask (n.d.). Welcome to Flask — Flask Documentation (1.1.x). [online] flask.palletsprojects.com. Available at: https://flask.palletsprojects.com/. ‌
10. Gunicorn: gunicorn.org. (n.d.). Gunicorn - Python WSGI HTTP Server for UNIX. [online] Available at: https://gunicorn.org/. ‌
11. Bandit: Readthedocs.io. (2025). Welcome to Bandit — Bandit documentation. [online] Available at: https://bandit.readthedocs.io/ . ‌
12. Safety: pyupio. (2025, November 6). GitHub - pyupio/safety: Safety checks Python dependencies for known security vulnerabilities and suggests the proper remediations for vulnerabilities detected. GitHub. https://github.com/pyupio/safety ‌
13. Selenium: Selenium (2023). SeleniumHQ Browser Automation. [online] www.selenium.dev. Available at: https://www.selenium.dev/.

---

Last Updated: 14th of December 2025

---

End of README