Assignment 1 — K3s on AWS

Full Name: Musa Bolo

Student Number: 221394486

Repository: MusaBolo4/-K3S-AWS

Date: 23 March 2026

---

Table of Contents

1. System Requirements
2. Architecture Explanation
3. Environment Setup
4. Installation Steps
5. Evidence of Deployment
6. NGINX Ingress Controller
7. Storage Configuration
8. Uninstalling K3s
9. Troubleshooting
10. Reflection

---

System Requirements

Component	Specification
Cloud Provider	AWS (us-east-1 / N. Virginia)
Instance Type	t3.large
vCPUs	2 per node
RAM	8 GB per node
OS	Ubuntu 24.04 LTS
Nodes	3 x EC2 instances
K3s Version	v1.34.5+k3s1

---

Architecture Explanation

What is K3s?

K3s is a lightweight, CNCF-certified Kubernetes distribution developed by Rancher Labs (now SUSE). It packages the entire Kubernetes control plane into a single binary, making it ideal for edge computing, IoT, CI/CD pipelines, and resource-constrained environments. Unlike standard Kubernetes, K3s removes legacy features, uses embedded etcd for HA, and bundles everything needed — container runtime, CNI, ingress controller, and storage provisioner — out of the box.

Cluster Architecture

┌─────────────────────────────────────────────────────────────┐
│                   AWS VPC (172.31.0.0/16)                   │
│                                                             │
│  ┌──────────────────┐  ┌──────────────────┐                │
│  │   k3s-master-1   │  │   k3s-master-2   │                │
│  │  172.31.85.171   │  │   172.31.86.1    │                │
│  │  18.212.55.65    │  │  3.82.105.121    │                │
│  │                  │  │                  │                │
│  │  Control Plane   │  │  Control Plane   │                │
│  │  etcd (embedded) │  │  etcd (embedded) │                │
│  │  API Server      │  │  API Server      │                │
│  └────────┬─────────┘  └────────┬─────────┘                │
│           │                     │                           │
│           └──────────┬──────────┘                           │
│                      │ K3s Cluster                          │
│           ┌──────────┴──────────┐                           │
│           │    k3s-master-3     │                           │
│           │   172.31.92.252     │                           │
│           │   18.207.218.28     │                           │
│           │                     │                           │
│           │   Control Plane     │                           │
│           │  etcd (embedded)    │                           │
│           └─────────────────────┘                           │
│                                                             │
│  Security Group: k3s-ha-sg                                  │
│  Ports: 22, 6443, 2379-2380, 8472/UDP, 10250, 30000-32767  │
└─────────────────────────────────────────────────────────────┘

Node IP Reference

Hostname	Private IP	Public IP
k3s-master-1	172.31.85.171	18.212.55.65
k3s-master-2	172.31.86.1	3.82.105.121
k3s-master-3	172.31.92.252	18.207.218.28

Key Components

Component	Implementation	Purpose
Control Plane	All 3 master nodes	Runs Kubernetes API server, scheduler, and controller manager
etcd	Embedded (per master)	Stores cluster state and provides high availability
Container Runtime	containerd 2.1.5-k3s1	Responsible for running containers on each node
CNI	Flannel (VXLAN)	Provides pod networking across nodes (UDP port 8472)
Ingress	NGINX Ingress Controller	Handles HTTP/HTTPS routing into the cluster
Storage	local-path-provisioner	Provides dynamic hostPath-based volume provisioning

---

Environment Setup

Step 1 — Create Security Group

A security group named k3s-ha-sg was created with the following inbound rules:

Type	Protocol	Port Range	Source	Purpose
SSH	TCP	22	0.0.0.0/0	Remote access
Custom TCP	TCP	6443	0.0.0.0/0	Kubernetes API server
Custom TCP	TCP	2379-2380	k3s-ha-sg (self)	etcd cluster communication
Custom UDP	UDP	8472	k3s-ha-sg (self)	Flannel VXLAN overlay network
Custom TCP	TCP	10250	k3s-ha-sg (self)	Kubelet API
Custom TCP	TCP	30000-32767	0.0.0.0/0	NodePort services

Step 2 — Launch 3 EC2 Instances

Three EC2 instances were launched with the following configuration:

Names:           k3s-master-1, k3s-master-2, k3s-master-3
AMI:             Ubuntu 24.04 LTS (64-bit x86)
Instance type:   t3.large
VPC:             Default
Security group:  k3s-ha-sg

---

Installation Steps

Step 3 — Prepare All Nodes

Run on each node (adjusting hostname per node):

# Set hostname
sudo hostnamectl set-hostname k3s-master-1  # change per node

# Update packages
sudo apt-get update && sudo apt-get upgrade -y

# Set timezone
sudo timedatectl set-timezone UTC

# Update /etc/hosts on ALL nodes
sudo tee -a /etc/hosts <<EOF
172.31.85.171  k3s-master-1
172.31.86.1    k3s-master-2
172.31.92.252  k3s-master-3
EOF

# Disable swap
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

Step 4 — Install K3s on Master-1 (Cluster Init)

sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/config.yaml <<EOF
cluster-init: true
node-ip: 172.31.85.171
advertise-address: 172.31.85.171
tls-san:
  - 172.31.85.171
  - 18.212.55.65
  - k3s-master-1
disable: [servicelb, traefik]
EOF

curl -sfL https://get.k3s.io | sh -

Retrieve the join token:

sudo cat /var/lib/rancher/k3s/server/token

Step 5 — Join Master-2

sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/config.yaml <<EOF
server: https://172.31.85.171:6443
token: <token-from-master-1>
node-ip: 172.31.86.1
advertise-address: 172.31.86.1
tls-san:
  - 172.31.86.1
  - 3.82.105.121
  - k3s-master-2
disable: [servicelb, traefik]
EOF

curl -sfL https://get.k3s.io | sh -s - server

Step 6 — Join Master-3

sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/config.yaml <<EOF
server: https://172.31.85.171:6443
token: <token-from-master-1>
node-ip: 172.31.92.252
advertise-address: 172.31.92.252
tls-san:
  - 172.31.92.252
  - 18.207.218.28
  - k3s-master-3
disable: [servicelb, traefik]
EOF

curl -sfL https://get.k3s.io | sh -s - server

Step 7 — Verify All Nodes

sudo kubectl get nodes -o wide

Expected output:

NAME           STATUS   ROLES                       AGE   VERSION
k3s-master-1   Ready    control-plane,etcd,master   51m   v1.34.5+k3s1
k3s-master-2   Ready    control-plane,etcd,master   8m    v1.34.5+k3s1
k3s-master-3   Ready    control-plane,etcd,master   5s    v1.34.5+k3s1

---

Evidence of Deployment

All 3 Nodes Ready

###Welcome to my Web App

Pods Running

###AWS Console

###K3s Uninstall output on Master 1

App accessible at: http://18.212.55.65:30080

---

NGINX Ingress Controller

K3s deploys Helm charts automatically by placing a manifest in /var/lib/rancher/k3s/server/manifests/.

# Download and deploy the NGINX ingress manifest
curl -O https://raw.githubusercontent.com/MusaBolo4/-K3S-AWS/refs/heads/patch-1/nginx-ingress.yml
sudo cp nginx-ingress.yml /var/lib/rancher/k3s/server/manifests/nginx-ingress.yaml

# Verify
sudo kubectl -n ingress-nginx get pods
sudo kubectl -n ingress-nginx get svc

Output:

The nginx-ingress.yml manifest includes the following AWS NLB annotations:

service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"

---

Storage Configuration

Local Path Provisioner (Default)

K3s ships with local-path-provisioner built-in. It was tested as follows:

curl -O https://raw.githubusercontent.com/MusaBolo4/-K3S-AWS/refs/heads/patch-1/pvc.yaml
curl -O https://raw.githubusercontent.com/MusaBolo4/-K3S-AWS/refs/heads/patch-1/pod.yaml

sudo kubectl create -f pvc.yaml
sudo kubectl create -f pod.yaml

sudo kubectl get pvc
sudo kubectl get pv
sudo kubectl get pod volume-test

Output:

---

Uninstalling K3s

# Run on all master nodes
/usr/local/bin/k3s-uninstall.sh

# Verify removal
sudo kubectl get nodes
# Expected: command not found

---

Troubleshooting

Nodes stay in NotReady

• Check security group rules — ensure ports 8472/UDP (Flannel VXLAN) and 10250/TCP (Kubelet) are open between nodes
• Verify hostname resolution: ping k3s-master-2 from k3s-master-1

API server unreachable

• Ensure port 6443/TCP is open in the security group
• Verify the server: URL in config points to the correct private IP

Check K3s logs

journalctl -u k3s -f

Check etcd health

sudo k3s etcd-snapshot list

Reflection

What Did I Learn?

Deploying a K3s high-availability cluster on AWS provided practical insight into how Kubernetes works at the infrastructure level. Before this assignment, I understood terms like control plane, pods, and services conceptually, but had never provisioned a real multi-node cluster from scratch. Working through each step showed me how tightly coupled infrastructure and software configuration are — a small mistake such as a wrong port number in a firewall rule can completely block access to a running application. I also learned how embedded etcd replicates cluster state across all master nodes, meaning the cluster continues to function even if one node fails.

Challenges Faced and How I Resolved Them

The first challenge was accidentally running the cluster-init configuration on all 3 nodes instead of only master-1, causing all three to start as independent cluster founders. The fix was to run the K3s uninstall script on masters 2 and 3 and rejoin them correctly using the server: configuration pointing to master-1's private IP.

A second issue was a firewall misconfiguration — port 30800 was opened in the AWS security group instead of 30080. The application was confirmed running via curl on the server, but browser access failed until the correct port was opened and saved.

The AWS Learner Lab environment also presented a challenge with SSH key formats — the lab provides .ppk keys (PuTTY format) while standard tools expect .pem (OpenSSH format). This was worked around by using EC2 Instance Connect directly in the browser.

K3s in 5G Cloud-Native Architectures

K3s is highly relevant in 5G environments where Multi-access Edge Computing (MEC) requires Kubernetes to run at the network edge with limited resources and minimal latency. K3s runs as a single binary under 512MB RAM, making it ideal for edge deployments. Cloud-native 5G network functions such as the AMF and UPF are increasingly containerised and orchestrated using Kubernetes, and K3s's full API compatibility means the same tooling used in central data centres can be used at the edge.

Virtualisation and Containerisation for Scalable Services

The EC2 instances acted as virtual machines on AWS physical infrastructure, while containerisation through containerd added another layer — applications ran in isolated containers sharing the host OS kernel. This layered architecture enables horizontal scaling where additional nodes can be added without modifying deployed applications, and Kubernetes automatically handles scheduling, load balancing, and self-healing.