feat: Alert Knob - #205
feat: Alert Knob#205sinapah wants to merge 6 commits into
Conversation
CI resolves black 24.8.0 (Python 3.8) while local dev on newer interpreters resolves black 26.x; the two styles disagree on multiline string literals passed directly as call arguments. Assign such configs to local variables before passing them to from_yaml(), which both styles format identically.
47696f8 to
5666488
Compare
| "AlertRulesCustomization", | ||
| "AlertRulesCustomizationError", |
There was a problem hiding this comment.
Look at canonical/prometheus-k8s-operator#857 to see how these are intended to be used.
There was a problem hiding this comment.
Would it help to have a feature file?
|
|
||
| ## Overview | ||
|
|
||
| This module provides :class:`AlertRulesCustomization`, a pure transformation helper that |
There was a problem hiding this comment.
Would it also work with recording rules?
| ``MetricsConsumer.alerts`` produce) and an admin-provided YAML customization config, and | ||
| returns the modified rules in the same format. | ||
|
|
||
| The customization config supports two top-level keys: |
There was a problem hiding this comment.
| The customization config supports two top-level keys: | |
| The customization config supports the following top-level keys: |
| group_name, | ||
| identifier, | ||
| ", ".join(changes), | ||
| ) |
There was a problem hiding this comment.
Should we impose an annotation indicating the rule went through AK? E.g.
annotations:
_transformed_by_cos_lib: "expr changed; labels changed"There was a problem hiding this comment.
As discussed in real time, we seem to lean towards not doing this. At least not just yet.
WDYT @marcusboden? Would you say having this annotation would be helpful?
There was a problem hiding this comment.
Hi @sinapah. It might actually be helpful. But it's not too important. But we can also bring it in in a later iteration.
* feat: alert knob using pydantic * fix: comments
3 participants
Issue
See canonical/prometheus-k8s-operator#857 for a test of how this feature would be implemented and used.
COS charms (prometheus-k8s, loki-k8s, mimir-coordinator, loki-coordinator) process alert rules provided by relations, applying Juju topology matchers and validation automatically. However, there is no standardized way for administrators to fine-tune these rules once deployed. This PR addresses the need for "Day-2" flexibility, allowing operators to patch or remove specific relation-derived alerts or inject custom rules without disrupting built-in monitoring or recording rules.
Solution
Adds a new
cosl.rules_customizationmodule with a single public class:AlertRulesCustomization.from_yaml(config_string): parses the admin's YAML config and validates the schema at parse time.AlertRulesCustomization.apply(relation_alerts): pure transformation that takes aMapping[str, OfficialRuleFileFormat](the same dict relation libraries already produce) and returns aDict[str, OfficialRuleFileFormat]in the same shape, with remove/patch/add applied.The config supports three optional top-level keys:
Operations are applied in order: remove → patch. Inputs are never mutated (deep-copied). Validation of the resulting rule expressions is the charm's responsibility after calling apply().
Checklist
project.versionfield in thepyproject.tomlfile.Context
Testing Instructions
Upgrade Notes