Aegis-Ops: Automated GitOps for Multi-Environment K8s

Production-grade GitOps platform using ArgoCD on Kubernetes, implementing the App-of-Apps pattern with automated drift detection, multi-environment Kustomize overlays, custom Lua health checks, and Bitnami Sealed Secrets.

Architecture

                    +---------------------+
                    |    GitHub Repo       |
                    |  (Single Source of   |
                    |       Truth)         |
                    +----------+----------+
                               |
                    +----------v----------+
                    |   ArgoCD (root-app) |
                    |   App-of-Apps       |
                    +----------+----------+
                               |
               +---------------+---------------+
               v               v               v
         +-----------+  +-----------+  +--------------+
         | Frontend  |  | Backend   |  | Sealed       |
         | (Nginx)   |  | (Echo API)|  | Secrets Ctrl |
         | ns:       |  | ns:       |  | ns:          |
         | frontend  |  | backend   |  | kube-system  |
         +-----------+  +-----------+  +--------------+

Each application supports environment-specific configuration through Kustomize overlays:

         base/ (shared manifests)
           |
    +------+------+
    |      |      |
   dev  staging  prod
  1 rep  2 rep  3 rep
  low $  med $  high $

Directory Structure

.
+-- bootstrap/
|   +-- root-app.yaml              # App-of-Apps root (auto-sync + self-heal)
|   +-- argocd-cm-patch.yaml       # Custom Lua health checks (SealedSecret, Ingress)
+-- apps/
|   +-- argocd-apps/               # Child ArgoCD Application manifests
|   |   +-- frontend.yaml          # Points to frontend overlay
|   |   +-- backend.yaml           # Points to backend overlay
|   |   +-- sealed-secrets.yaml    # Helm-based Bitnami chart (v2.16.2)
|   +-- frontend/
|   |   +-- base/                  # Kustomize base (Nginx + ConfigMap)
|   |   +-- overlays/
|   |       +-- dev/               # 1 replica, minimal resources
|   |       +-- staging/           # 2 replicas, moderate resources
|   |       +-- prod/              # 3 replicas, production resources
|   +-- backend/
|       +-- base/                  # Kustomize base (Echo API + SealedSecret)
|       +-- overlays/
|           +-- dev/               # 1 replica, minimal resources
|           +-- staging/           # 2 replicas, moderate resources
|           +-- prod/              # 3 replicas, production resources
+-- .github/workflows/
|   +-- lint.yaml                  # yamllint + kubeconform + kustomize build (all overlays)
+-- scripts/
|   +-- setup.sh                   # One-command bootstrap
+-- .yamllint.yaml                 # YAML linting rules
+-- LICENSE

Key Features

1. App-of-Apps Pattern

A single root ArgoCD Application (bootstrap/root-app.yaml) manages all child applications. Adding a new service is as simple as dropping a YAML file into apps/argocd-apps/. ArgoCD discovers and deploys it automatically.

2. Automated Drift Detection (Self-Heal)

Every ArgoCD Application is configured with:

syncPolicy:
  automated:
    selfHeal: true   # Revert manual kubectl changes
    prune: true      # Delete resources removed from Git

If anyone runs kubectl scale or kubectl edit directly against the cluster, ArgoCD detects the drift and reverts to the state defined in Git within seconds.

3. Multi-Environment Deployments with Kustomize

Each application uses a Kustomize base/ with environment-specific overlays/:

Environment	Replicas (frontend / backend)	Resource Profile
dev	1 / 1	Minimal (25-50m CPU)
staging	2 / 2	Moderate (75-150m CPU)
prod	3 / 3	Production (100-200m CPU)

To promote a change from dev to prod, update the ArgoCD Application's spec.source.path from apps/frontend/overlays/dev to apps/frontend/overlays/prod. The overlay structure ensures environment-specific resource tuning while sharing the same base manifests.

4. Custom ArgoCD Lua Health Checks

Defined in bootstrap/argocd-cm-patch.yaml:

• SealedSecret: Checks whether the sealed-secrets-controller successfully decrypted the secret by inspecting status.conditions for Synced: True.
• Ingress: Marks Ingress as healthy only after a load balancer IP/hostname is allocated.
• Deployments: Standard probes (readinessProbe, livenessProbe, startupProbe) on all pods feed into ArgoCD's health assessment.

5. Sealed Secrets

Passwords and credentials are encrypted with kubeseal and stored as SealedSecret resources. Only the in-cluster controller can decrypt them. Plaintext secrets never enter Git.

kubectl create secret generic db-credentials \
  --namespace backend \
  --from-literal=DB_HOST=postgres.backend.svc.cluster.local \
  --from-literal=DB_PORT=5432 \
  --from-literal=DB_USER=appuser \
  --from-literal=DB_PASSWORD=changeme-use-strong-password \
  --dry-run=client -o yaml \
  | kubeseal --format yaml \
    --controller-name sealed-secrets-controller \
    --controller-namespace kube-system \
  > apps/backend/base/sealed-secret.yaml

6. CI Pipeline (GitHub Actions)

On every push to main or develop, and on pull requests, GitHub Actions runs three validation jobs:

Job	Tool	Purpose
yamllint	yamllint	YAML syntax and style enforcement
kubeconform	kubeconform	Kubernetes schema validation (including ArgoCD CRDs)
kustomize-build	kustomize	Verify base and all overlays (dev/staging/prod) render cleanly

Environment Promotion Workflow

  Git commit to main
        |
        v
  CI validates all manifests (yamllint + kubeconform + kustomize build)
        |
        v
  ArgoCD detects change (auto-sync)
        |
        v
  Dev overlay deploys (default)
        |
        v
  Manual verification in dev
        |
        v
  Update ArgoCD app path: overlays/dev -> overlays/staging
        |
        v
  Staging deploys with moderate resources
        |
        v
  Update ArgoCD app path: overlays/staging -> overlays/prod
        |
        v
  Production deploys with full resources

Prerequisites

• Minikube (or any Kubernetes cluster)
• kubectl
• Helm
• kubeseal
• kustomize (bundled with recent kubectl versions)

Quick Start

# 1. Clone the repo
git clone https://github.com/jasonjacinth/project-aegis-gitops.git
cd project-aegis-gitops

# 2. Run the bootstrap script (starts Minikube, installs ArgoCD, deploys root app)
chmod +x scripts/setup.sh
./scripts/setup.sh

# 3. Access the ArgoCD UI
kubectl port-forward svc/argocd-server -n argocd 8080:443
# Open https://localhost:8080
# Username: admin
# Password: printed by setup.sh

Verifying Drift Detection

# Scale the frontend manually
kubectl scale deployment frontend -n frontend --replicas=5

# Watch ArgoCD auto-revert it back to the overlay-defined replica count
kubectl get deployment frontend -n frontend -w

Switching Environments

To deploy the staging profile instead of dev, update the ArgoCD Application path:

kubectl patch application frontend -n argocd --type merge \
  -p '{"spec":{"source":{"path":"apps/frontend/overlays/staging"}}}'

Or edit apps/argocd-apps/frontend.yaml and change path: apps/frontend/overlays/dev to path: apps/frontend/overlays/staging, then push to Git. ArgoCD picks up the change automatically.

License

This project is licensed under the MIT License. See the LICENSE file for details.