Skip to content

Chore/robustness and enhancements #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
letsconfuse wants to merge 6 commits into from
Closed

Chore/robustness and enhancements #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6ef5daf
chore: add comprehensive enhancement documentation
letsconfuse Jul 13, 2026
a1449c2
chore: add RBAC configuration for least privilege access
letsconfuse Jul 13, 2026
c9533a4
chore: enhance terraform security and add VPC configuration
letsconfuse Jul 13, 2026
7f3e868
chore: enhance kubernetes deployments with resource limits and health…
letsconfuse Jul 13, 2026
296b704
chore: add network policies for zero-trust networking
letsconfuse Jul 13, 2026
8199278
chore: update README with enhanced deployment and security details
letsconfuse Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 201 additions & 27 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,18 @@

> GitHub: [github.com/letsconfuse](https://github.com/letsconfuse)

A production-grade, end-to-end DevOps deployment of the **Weaveworks Sock Shop** microservices architecture. This project serves as a comprehensive portfolio piece demonstrating modern Cloud-Native practices: from containerization and local orchestration to automated CI/CD, GitOps-style Kubernetes deployments, Infrastructure as Code (AWS), and a full Observability stack.
A production-grade, end-to-end DevOps deployment of the **Weaveworks Sock Shop** microservices architecture. This project demonstrates modern cloud-native practices with enhanced security, robustness, and automated CI/CD workflows.

---

## System Architecture

```mermaid
graph TD
User([User]) -->|HTTP 80| Ingress[K8s Ingress]
User([User]) -->|HTTP 80/443| Ingress[K8s Ingress]
Ingress --> FrontEnd[Front-End Service]

subgraph K8s Cluster [Kubernetes Cluster / Docker Compose]
subgraph K8s Cluster [Kubernetes Cluster]
FrontEnd --> Catalogue[Catalogue API]
FrontEnd --> Carts[Carts API]
FrontEnd --> Orders[Orders API]
Expand Down Expand Up @@ -51,58 +51,232 @@ graph TD
| **Containerization** | Docker | Multi-stage builds, non-root users (`appuser`) for security. |
| **Local Environment** | Docker Compose | Custom `docker-compose.yml` networking 13+ microservices. |
| **CI/CD Pipelines** | GitHub Actions | Automated Linting (`hadolint`, `yamllint`), Testing, and Docker pushes. |
| **Orchestration** | Kubernetes | Deployments (`RollingUpdate`), Services, Ingress, Secrets, ConfigMaps. |
| **Infra as Code (IaC)** | Terraform | AWS EC2 provisioning with S3 Remote State backend. |
| **Automated Testing** | Bruno / Playwright | API Smoke tests run automatically as a pre-deployment gate. |
| **Observability** | Prometheus & Grafana | Custom SLI alerts (`FrontEndDown`) and auto-provisioned dashboards. |
| **Orchestration** | Kubernetes | Deployments with RollingUpdate, health checks, Pod Disruption Budget, Network Policies. |
| **Infra as Code (IaC)** | Terraform | AWS VPC, EC2 provisioning, security hardening, S3 Remote State backend with DynamoDB locking. |
| **Automated Testing** | Bruno / Curl | API smoke tests and health verification in CI/CD pipeline. |
| **Observability** | Prometheus & Grafana | Metrics scraping, custom dashboards, and alert rules for critical services. |

---

## Security Features

### AWS Infrastructure
- Dedicated VPC with private subnets for network isolation
- Restricted security groups: SSH limited to specific IP ranges, Kubernetes API internal-only
- IMDSv2 enforcement to prevent metadata exploitation
- EBS volumes encrypted with AWS KMS
- Elastic IP for stable public access

### Kubernetes Security
- Network policies implementing zero-trust networking (deny-all by default)
- RBAC with minimal privilege service accounts and role bindings
- Security contexts enforcing non-root users, dropped Linux capabilities, read-only filesystems
- Pod Disruption Budgets maintaining availability during cluster maintenance

---

## Production Robustness

- Resource limits and requests prevent resource exhaustion
- Liveness and readiness probes for container health monitoring
- Pod anti-affinity rules spread replicas across nodes
- Rolling update strategy with zero downtime (maxUnavailable: 0)
- Health checks integrated into CI/CD pipeline

---

## How to Run Locally

You can spin up the entire microservice ecosystem and the observability stack on your local machine using Docker Compose.

1. **Clone the repository**:
1. Clone the repository:
```bash
git clone https://github.com/letsconfuse/sock-shop-devops.git
cd sock-shop-devops
```
2. **Start the application**:

2. Start the application:
```bash
docker-compose -f docker/docker-compose.yml up -d
```
3. **Access the Application & Tools**:
- **Storefront**: `http://localhost:8079`
- **Grafana Dashboard**: `http://localhost:3000` (Pre-configured)
- **Prometheus**: `http://localhost:9090`

*To tear down the environment:* `docker-compose -f docker/docker-compose.yml down`
3. Access the applications:
- Storefront: `http://localhost:8079`
- Grafana Dashboard: `http://localhost:3000` (credentials: admin/admin)
- Prometheus: `http://localhost:9090`

To tear down the environment:
```bash
docker-compose -f docker/docker-compose.yml down
```

---

## CI/CD & Delivery Flow

The project utilizes two distinct GitHub Actions workflows to ensure code quality and seamless delivery:
The project utilizes three GitHub Actions workflows to ensure code quality and safe delivery:

### Continuous Integration (Pull Requests to main)
File: `.github/workflows/ci.yml`

Triggered on pull requests, this workflow:
- Lints YAML files with `yamllint`
- Lints Dockerfiles with `hadolint`
- Builds the custom front-end Docker image
- Spins up ephemeral Docker Compose containers
- Performs health checks on key services
- Runs API smoke tests

### Continuous Deployment (Merges to main)
File: `.github/workflows/cd.yml`

1. **Continuous Integration (PRs to `main`)**
- Triggers `hadolint` for Dockerfiles and `yamllint` for manifests.
- Builds the custom `front-end` Docker image.
- Spins up ephemeral `docker-compose` containers and runs **Bruno API Smoke Tests**.
Triggered on commits to main, this workflow:
- Runs all CI checks to guarantee code integrity
- Builds and pushes the image to Docker Hub with Git SHA tag
- Authenticates with the Kubernetes cluster
- Applies all Kubernetes manifests
- Updates front-end deployment with the new image
- Monitors rollout status

2. **Continuous Deployment (Merges to `main`)**
- Repeats the CI checks to guarantee integrity.
- Securely pushes the image to Docker Hub tagged with the Git SHA.
- Injects the AWS `KUBE_CONFIG` via GitHub Secrets to trigger a GitOps-style `kubectl apply` and a zero-downtime Rolling Update to the cluster.
### Terraform Validation
File: `.github/workflows/terraform-validate.yml`

Triggered on pull requests modifying Terraform files:
- Validates Terraform format with `terraform fmt`
- Validates syntax with `terraform validate`
- Checks best practices with `tflint`

---

## Infrastructure & Kubernetes

- **Terraform (`terraform/`)**: Fully automates the provisioning of the underlying AWS infrastructure. Uses an S3 bucket with DynamoDB state locking to simulate a team environment safely.
- **Kubernetes (`kubernetes/`)**: Core application components are mapped to dedicated manifests. Sensitive data like database credentials are decoupled using Kubernetes `Secret` resources, while environment variables are passed via `ConfigMap`.
### Terraform (terraform/)
Fully automates AWS infrastructure provisioning:
- VPC with public subnets and Internet Gateway
- EC2 instance with Ubuntu 22.04 for Kubernetes runtime
- Security groups with restricted inbound rules
- Encrypted EBS volumes for data protection
- S3 backend with DynamoDB for state locking (supports team environments)

Key variables:
- `aws_region` (default: us-east-1)
- `instance_type` (default: t3.medium)
- `key_name` - SSH key pair for EC2 access
- `allowed_ssh_cidrs` - Restrict SSH to specific IP ranges

### Kubernetes (kubernetes/)

Core application components organized into dedicated manifests:
- **Deployments**: Enhanced with resource limits, health checks, and security contexts
- **Services**: Expose microservices within the cluster
- **ConfigMaps**: Manage application configuration
- **Secrets**: Handle sensitive data (database credentials)
- **Network Policies**: Implement zero-trust networking
- **RBAC**: Define minimal privilege access controls

### Monitoring (monitoring/)
- **Prometheus**: Scrapes metrics from services with 15-second intervals
- **Grafana**: Pre-configured dashboards for visualization
- **Alert Rules**: Custom alerts for critical service failures

---

## Deployment to AWS

### Prerequisites
- AWS account with credentials configured
- Terraform 1.0 or later
- kubectl 1.27 or later
- Docker Hub account for image pushes

### Setup Steps

1. Configure Terraform variables:
```bash
cd terraform/
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values
```

2. Deploy infrastructure:
```bash
terraform init
terraform plan
terraform apply
```

3. Connect to the instance:
```bash
ssh -i your-key.pem ubuntu@<public_ip>
```

4. Initialize Kubernetes:
```bash
minikube start --driver=docker
```

5. Deploy applications:
```bash
kubectl apply -f kubernetes/deployments/
kubectl apply -f kubernetes/services/
kubectl apply -f kubernetes/configmaps/
kubectl apply -f kubernetes/network-policies/
kubectl apply -f kubernetes/rbac/
```

6. Configure GitHub Secrets for CD:
- `DOCKER_USERNAME` - Docker Hub username
- `DOCKER_PASSWORD` - Docker Hub access token
- `KUBE_CONFIG` - Base64-encoded kubeconfig

---

## Architectural Decisions & Learnings
## Documentation

For detailed information about security enhancements and production best practices, see [ENHANCEMENTS.md](docs/ENHANCEMENTS.md).

This document covers:
- Security hardening details
- Resource limits and health checks configuration
- Network policy implementation
- RBAC setup
- Performance impact analysis
- Production checklist

---

## Additional Resources

- [Kubernetes Documentation](https://kubernetes.io/docs/)
- [Terraform AWS Provider](https://registry.terraform.io/providers/hashicorp/aws/)
- [Docker Security Best Practices](https://docs.docker.com/engine/security/)
- [Prometheus Documentation](https://prometheus.io/docs/)
- [Grafana Documentation](https://grafana.com/docs/)

---

## Troubleshooting

### Check pod status
```bash
kubectl get pods -n sock-shop
kubectl describe pod <pod-name> -n sock-shop
kubectl logs <pod-name> -n sock-shop
```

### Verify network connectivity
```bash
kubectl exec -it <pod-name> -n sock-shop -- sh
wget http://target-service:port/
```

### Terraform validation
```bash
cd terraform/
terraform validate
terraform fmt -check .
```

---

To see a detailed log of *why* certain technical choices were made (e.g., why CI and CD are separated, or why the front-end Dockerfile was rewritten from scratch), please read the **[Decision Log (docs/decisions.md)](docs/decisions.md)**.
**Last Updated:** July 2026
Loading
Loading