Repository navigation
fix: stop EduIDEPVCPending firing for every idle workspace #37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -360,9 +360,21 @@ spec: | |
| device". Checking free space will not show the problem. | ||
| runbook_url: {{ $runbook }} | ||
|
|
||
| {{- /* | ||
| Narrowed to claims a pod is actually waiting on. | ||
|
|
||
| Most storage classes bind `WaitForFirstConsumer`, so a workspace volume | ||
| with no running session sits Pending indefinitely and that is correct, | ||
| not a fault - it binds when a pod first mounts it. Alerting on Pending | ||
| alone fired permanently for every idle workspace on the first cluster | ||
| this was deployed to. The join requires some pod to reference the claim, | ||
| which is what makes a stuck Pending a real problem. | ||
| */}} | ||
| - alert: EduIDEPVCPending | ||
| expr: |- | ||
| kube_persistentvolumeclaim_status_phase{namespace=~"{{ $ns }}", phase="Pending"} == 1 | ||
| (kube_persistentvolumeclaim_status_phase{namespace=~"{{ $ns }}", phase="Pending"} == 1) | ||
| and on (namespace, persistentvolumeclaim) | ||
| kube_pod_spec_volumes_persistentvolumeclaims_info{namespace=~"{{ $ns }}"} | ||
|
Comment on lines
+363
to
+377
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: printf '%s\n' '--- repository conventions ---'
find /tmp/coderabbit-repo-knowledge/eduide-eduide-helm-68f1d859 -type f -name '*.md' -maxdepth 3 -print
printf '%s\n' '--- alert template ---'
sed -n '350,400p' charts/eduide-cluster/templates/monitoring/prometheusrule.yaml
printf '%s\n' '--- related metric usage ---'
rg -n -C 3 'kube_pod_spec_volumes_persistentvolumeclaims_info|EduIDEPVCPending|waiting on the PVC|Pending PVC' charts/eduide-cluster/templatesRepository: EduIDE/EduIDE-Helm Length of output: 7611 🏁 Script executed: cat /tmp/coderabbit-repo-knowledge/eduide-eduide-helm-68f1d859/conventions/charts-eduide-cluster-templates-monitoring.md
printf '%s\n' '--- metric references and alert wording ---'
rg -n -C 5 'kube_pod_spec_volumes_persistentvolumeclaims_info|EduIDEPVCPending|waiting on it|session that needs it' .Repository: EduIDE/EduIDE-Helm Length of output: 7322 Align the alert wording with the metric condition.
🤖 Prompt for AI AgentsSource: MCP tools |
||
| for: 10m | ||
| labels: | ||
| severity: warning | ||
|
|
@@ -371,8 +383,11 @@ spec: | |
| summary: 'A workspace volume will not provision in {{ "{{" }} $labels.namespace {{ "}}" }}' | ||
| description: >- | ||
| PersistentVolumeClaim {{ "{{" }} $labels.persistentvolumeclaim {{ "}}" }} in | ||
| {{ "{{" }} $labels.namespace {{ "}}" }} has been Pending for 10 minutes, so the session | ||
| that needs it cannot start. | ||
| {{ "{{" }} $labels.namespace {{ "}}" }} has been Pending for 10 minutes while a pod is | ||
| waiting on it, so the session that needs it cannot start. | ||
| A workspace volume with no pod is a different thing and does not alert: | ||
| most storage classes bind on first consumer, so an idle workspace is | ||
| Pending by design. | ||
| The storage class is the first thing to check - it is a cluster property | ||
| set once in clusters/<name>.yaml, and a claim naming a class the cluster | ||
| does not offer stays Pending forever without any other symptom. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Align
appVersionwith the release version.This chart now has
version: 2.2.1but keepsappVersion: "1.2.0". The release train checks both fields against the requested release version, so a 2.2.1 release will reject this chart. Update the release metadata before merging.🤖 Prompt for AI Agents