Skip to content
This repository was archived by the owner on Aug 23, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2aeea3f
Add moduletemplates
tsaishawn Mar 17, 2026
c9b1570
Build moduletemplate together
tsaishawn Mar 17, 2026
ba5b069
Change gitrepository ref to tag
tsaishawn Mar 17, 2026
3508676
Refactor: change folder structure
tsaishawn Mar 17, 2026
d7bd3e5
Remove redundant files
tsaishawn Mar 17, 2026
3ca5b76
Merge branch 'main' into feat/add_addons_templates
tsaishawn Mar 17, 2026
447813b
Rename gitrepository
tsaishawn Mar 17, 2026
165094b
Revise kustomization.yaml
tsaishawn Mar 17, 2026
8167e66
Remove chinese
tsaishawn Mar 17, 2026
9ace9cd
Sync gitrepo tag version
tsaishawn Mar 17, 2026
32e9b36
Update AGENTS.md
tsaishawn Mar 17, 2026
dadd03b
feat: remove Samba operator resources and templates from kustomization
iasthc Mar 18, 2026
0c5bf2e
feat: add dependency on CDI in KubeVirt module template
iasthc Mar 18, 2026
2b434fa
feat: add dependency on gpu-operator in HAMI module template
iasthc Mar 18, 2026
2fc6831
feat: add dependencies for gpu-operator, istio-gateway, and gateway-a…
iasthc Mar 18, 2026
2ba86ca
feat: add dependency on rook-ceph for kube-prometheus-stack module te…
iasthc Mar 18, 2026
c2a54b8
feat: remove module.yaml references and add new module samples for va…
iasthc Mar 18, 2026
6eb5020
feat: remove unused kustomization.yaml files for various addons
iasthc Mar 18, 2026
a006324
feat: add module templates and Helm repositories for various componen…
iasthc Mar 18, 2026
c13ff9c
feat: update namespaces from flux-system to otterscale-system in modu…
iasthc Mar 18, 2026
bc5c821
feat: update GitRepository references from otterscale-repository to o…
iasthc Mar 18, 2026
88db872
feat: update helm release intervals to 30m for various module templat…
iasthc Mar 18, 2026
025301f
feat: add Kustomization and ModuleTemplate files for CDI, Gateway API…
iasthc Mar 18, 2026
9a653eb
feat: remove NodePort service type from Prometheus module template
iasthc Mar 18, 2026
9c4e088
feat: remove gateway-api dependency from llm-d-infra and rook-ceph de…
iasthc Mar 18, 2026
384da96
feat: add version placeholders and update module templates for CDI, G…
iasthc Mar 18, 2026
65c8ece
feat: add Gateway API Inference Extension module with versioning and …
iasthc Mar 18, 2026
7b303d3
feat: update GitRepository version placeholder to use double undersco…
iasthc Mar 18, 2026
c38d27e
feat: update version numbers for CDI, Gateway API, and KubeVirt modules
iasthc Mar 18, 2026
13c34e9
feat: reorganize kustomization.yaml for clarity and consistency in re…
iasthc Mar 18, 2026
1c702b0
feat: update namespaces in module templates for consistency
iasthc Mar 18, 2026
4493585
feat: update namespace in prometheus module template for consistency
iasthc Mar 18, 2026
c6a36c2
feat: update namespace in llm-d-infra module template for consistency
iasthc Mar 18, 2026
5246f68
feat: remove istio-gateway module template and update apply order in …
iasthc Mar 18, 2026
9553365
feat: change CreateOrUpdate to CreateOrPatch in helmrelease and kusto…
iasthc Mar 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,10 +273,16 @@ make build-installer IMG=<registry>/<project>:tag

**Key points:**

- The `dist/install.yaml` is generated from Kustomize manifests (CRDs, RBAC, Deployment)
- The `dist/install.yaml` is generated from Kustomize manifests (CRDs, RBAC, Deployment, HelmRepository, GitRepository, addon ModuleTemplates)
- Commit this file to your repository for easy distribution
- Users only need `kubectl` to install (no additional tools required)

**GitRepository version substitution:**

- `config/modules/gitrepository.yaml` uses the placeholder `__MODULE_OPERATOR_VERSION__` for the Git ref tag
- The `build-installer` target replaces it with the current version (`git describe --tags --always`) via `sed` before writing `dist/install.yaml`
- Modules (CDI, KubeVirt, etc.) are thus sourced from the same version as the operator being built, enabling feature-branch testing without hardcoded tags

**Example:** Users install with a single command:

```bash
Expand Down
16 changes: 14 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -158,10 +158,22 @@ docker-buildx: ## Build and push docker image for the manager for cross-platform
rm Dockerfile.cross

.PHONY: build-installer
build-installer: manifests generate kustomize ## Generate a consolidated YAML with CRDs and deployment.
build-installer: manifests generate kustomize ## Generate dist/install.yaml (CRDs+deploy+HelmRepository+ModuleTemplate+GitRepository+addon ModuleTemplates).
mkdir -p dist
cd config/manager && "$(KUSTOMIZE)" edit set image controller=${IMG}
"$(KUSTOMIZE)" build config/default > dist/install.yaml
{ "$(KUSTOMIZE)" build config/default; echo "---"; "$(KUSTOMIZE)" build config/modules; } | \
sed 's|__MODULE_OPERATOR_VERSION__|$(VERSION)|g' | \
sed "s|__CDI_VERSION__|$$(cat config/modules/cdi/VERSION | tr -d '\n')|g" | \
sed "s|__KUBEVIRT_VERSION__|$$(cat config/modules/kubevirt/VERSION | tr -d '\n')|g" | \
sed "s|__ROOK_CEPH_VERSION__|$$(cat config/modules/rook-ceph/VERSION | tr -d '\n')|g" | \
sed "s|__GATEWAY_API_VERSION__|$$(cat config/modules/gateway-api/VERSION | tr -d '\n')|g" | \
sed "s|__GPU_OPERATOR_VERSION__|$$(cat config/modules/gpu-operator/VERSION | tr -d '\n')|g" | \
sed "s|__HAMI_VERSION__|$$(cat config/modules/hami/VERSION | tr -d '\n')|g" | \
sed "s|__ISTIO_VERSION__|$$(cat config/modules/istio/VERSION | tr -d '\n')|g" | \
sed "s|__PROMETHEUS_VERSION__|$$(cat config/modules/prometheus/VERSION | tr -d '\n')|g" | \
sed "s|__LLM_D_INFRA_VERSION__|$$(cat config/modules/llm-d-infra/VERSION | tr -d '\n')|g" | \
sed "s|__GAIE_VERSION__|$$(cat config/modules/gateway-api-inference-extension/VERSION | tr -d '\n')|g" \
> dist/install.yaml

##@ Deployment

Expand Down
6 changes: 6 additions & 0 deletions config/manager/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,8 @@
resources:
- manager.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
images:
- name: controller
newName: controller
newTag: latest
1 change: 1 addition & 0 deletions config/modules/cdi/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.65.0
8 changes: 8 additions & 0 deletions config/modules/cdi/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# CDI deployment — remote resources for FluxCD Kustomization
# Source: https://github.com/kubevirt/containerized-data-importer
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/kubevirt/containerized-data-importer/releases/download/v__CDI_VERSION__/cdi-operator.yaml
- https://github.com/kubevirt/containerized-data-importer/releases/download/v__CDI_VERSION__/cdi-cr.yaml
24 changes: 24 additions & 0 deletions config/modules/cdi/moduletemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ModuleTemplate — CDI (Containerized Data Importer) v__CDI_VERSION__
# Uses FluxCD Kustomization (not Helm). Deploys from remote release YAMLs.
#
# Prerequisite: GitRepository (otterscale-module-operator) must exist in otterscale-system.
# Apply: kubectl apply -k config/modules/cdi/
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: cdi
spec:
namespace: cdi

description: >-
Containerized Data Importer v__CDI_VERSION__ — PVC data import for KubeVirt VM disk images.
Deploys via FluxCD Kustomization from upstream release YAMLs.

kustomization:
interval: 30m
prune: true
sourceRef:
kind: GitRepository
name: otterscale-module-operator
namespace: otterscale-system
path: config/modules/cdi
1 change: 1 addition & 0 deletions config/modules/gateway-api-inference-extension/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.3.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Gateway API Inference Extension v__GAIE_VERSION__
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/v__GAIE_VERSION__/manifests.yaml
24 changes: 24 additions & 0 deletions config/modules/gateway-api-inference-extension/moduletemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ModuleTemplate — Gateway API Inference Extension v__GAIE_VERSION__
# Uses FluxCD Kustomization. Requires Gateway API CRDs.
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: gateway-api-inference-extension
spec:
namespace: default

description: >-
Gateway API Inference Extension v__GAIE_VERSION__ — CRDs and controllers
for AI/ML inference routing via Gateway API.

kustomization:
interval: 30m
prune: true
dependsOn:
- name: gateway-api
namespace: default
sourceRef:
kind: GitRepository
name: otterscale-module-operator
namespace: otterscale-system
path: config/modules/gateway-api-inference-extension
1 change: 1 addition & 0 deletions config/modules/gateway-api/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.5.1
6 changes: 6 additions & 0 deletions config/modules/gateway-api/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Gateway API v__GATEWAY_API_VERSION__ — Standard channel CRDs
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/kubernetes-sigs/gateway-api/releases/download/v__GATEWAY_API_VERSION__/standard-install.yaml
20 changes: 20 additions & 0 deletions config/modules/gateway-api/moduletemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# ModuleTemplate — Kubernetes Gateway API v__GATEWAY_API_VERSION__ (Standard channel)
# Uses FluxCD Kustomization. CRDs only (GatewayClass, Gateway, HTTPRoute, etc.)
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: gateway-api
spec:
namespace: default

description: >-
Kubernetes Gateway API v__GATEWAY_API_VERSION__ — Standard channel CRDs (GatewayClass, Gateway, HTTPRoute, GRPCRoute, ReferenceGrant).

kustomization:
interval: 30m
prune: true
sourceRef:
kind: GitRepository
name: otterscale-module-operator
namespace: otterscale-system
path: config/modules/gateway-api
12 changes: 12 additions & 0 deletions config/modules/gitrepository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Shared GitRepository for addon ModuleTemplates (CDI, KubeVirt, etc.)
# All addon components deploy from this repo's config/modules/*/deploy paths.
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: otterscale-module-operator
namespace: otterscale-system
spec:
interval: 24h
url: https://github.com/otterscale/module-operator
ref:
tag: __MODULE_OPERATOR_VERSION__
1 change: 1 addition & 0 deletions config/modules/gpu-operator/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
25.10.1
8 changes: 8 additions & 0 deletions config/modules/gpu-operator/helmrepository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: nvidia-gpu-operator
namespace: otterscale-system
spec:
interval: 24h
url: https://nvidia.github.io/gpu-operator
77 changes: 77 additions & 0 deletions config/modules/gpu-operator/moduletemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# ModuleTemplate — NVIDIA GPU Operator blueprint
#
# Deploys NVIDIA GPU Operator for driver, DCGM, DCGM Exporter, Node Status Exporter.
# Device plugin and vGPU are disabled (use HAMI for GPU sharing).
#
# Apply:
# kubectl apply -f moduletemplate.yaml
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: gpu-operator
spec:
namespace: gpu-operator

description: >-
Deploys NVIDIA GPU Operator (driver 580.x, DCGM, DCGM Exporter, Node Status Exporter).
Device plugin disabled for use with HAMI vGPU.

helmRelease:
interval: 30m

chart:
spec:
chart: gpu-operator
version: "__GPU_OPERATOR_VERSION__"
sourceRef:
kind: HelmRepository
name: nvidia-gpu-operator
namespace: otterscale-system
interval: 24h

install:
remediation:
retries: 3

upgrade:
remediation:
retries: 3
force: false

timeout: 30m

values:
cdi:
enabled: false

driver:
enabled: true
version: "580.105.08"
upgradePolicy:
autoUpgrade: false

# Offline version
# driver:
# version: "580"
# usePrecompiled: true

devicePlugin:
enabled: false

operator:
cleanupCRD: true
upgradeCRD: false

dcgm:
enabled: true

dcgmExporter:
enabled: true
serviceMonitor:
enabled: true

nodeStatusExporter:
enabled: true

vgpuDeviceManager:
enabled: false
1 change: 1 addition & 0 deletions config/modules/hami/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2.8.0
8 changes: 8 additions & 0 deletions config/modules/hami/helmrepository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: hami
namespace: otterscale-system
spec:
interval: 24h
url: https://project-hami.github.io/HAMi/
75 changes: 75 additions & 0 deletions config/modules/hami/moduletemplate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# ModuleTemplate — HAMI (Heterogeneous AI Computing Virtualization) blueprint
#
# Deploys HAMI for GPU sharing (vGPU) with NVIDIA GPU Operator.
# Requires GPU Operator to be installed first (driver + nvidia-validations).
#
# Apply:
# kubectl apply -f moduletemplate.yaml
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: hami
spec:
# HAMI scheduler and device plugin typically run in kube-system
namespace: kube-system

description: >-
Deploys HAMI for GPU sharing (vGPU) with device split, memory/core scaling.
Integrates with NVIDIA GPU Operator via gpuOperatorToolkitReady.

helmRelease:
interval: 30m

dependsOn:
- name: gpu-operator
namespace: gpu-operator

chart:
spec:
chart: hami
version: "__HAMI_VERSION__"
sourceRef:
kind: HelmRepository
name: hami
namespace: otterscale-system
interval: 24h

install:
remediation:
retries: 3

upgrade:
remediation:
retries: 3
force: false

timeout: 30m

values:
global:
imageTag: "v__HAMI_VERSION__"
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "31992"
# managedNodeSelectorEnable: false
# managedNodeSelector:
# usage: "nvidia.com/gpu.workload.config"

scheduler:
service:
type: ClusterIP

devicePlugin:
enabled: true
gpuOperatorToolkitReady:
enabled: true
hostPath: "/run/nvidia/validations"
deviceSplitCount: 10
deviceMemoryScaling: 1
deviceCoreScaling: 1
service:
type: ClusterIP
runtimeClassName: "nvidia"
nvidiaNodeSelector:
gpu: null
nvidia.com/gpu.workload.config: "vgpu"
1 change: 1 addition & 0 deletions config/modules/istio/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.29.1
8 changes: 8 additions & 0 deletions config/modules/istio/helmrepository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: istio
namespace: otterscale-system
spec:
interval: 24h
url: https://istio-release.storage.googleapis.com/charts
42 changes: 42 additions & 0 deletions config/modules/istio/moduletemplate-base.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# ModuleTemplate — Istio base (CRDs)
#
# Installs Istio base chart (CRDs). Must be applied before istiod.
# See: https://istio.io/latest/docs/setup/install/helm/
#
# Apply order: base → istiod
apiVersion: module.otterscale.io/v1alpha1
kind: ModuleTemplate
metadata:
name: istio-base
spec:
namespace: istio-system

description: >-
Installs Istio base chart (CRDs). Required before istiod.

helmRelease:
interval: 30m

chart:
spec:
chart: base
version: "__ISTIO_VERSION__"
sourceRef:
kind: HelmRepository
name: istio
namespace: otterscale-system
interval: 24h

install:
remediation:
retries: 3

upgrade:
remediation:
retries: 3
force: false

timeout: 10m

values:
defaultRevision: default
Loading
Loading