-
Notifications
You must be signed in to change notification settings - Fork 138
feat(backups): backup api rework #1873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
9e94a69
41646b2
d7ae321
5924c48
ba04063
79bd3ad
02ace2e
272d2b7
e3aab24
f254c5f
f6641c1
2cca1bc
0e05578
ffde02c
8755497
73d6e30
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -100,13 +100,13 @@ Describe **when**, **how**, and **where** to back up a specific managed applicat | |
| ```go | ||
| type PlanSpec struct { | ||
| // Application to back up. | ||
| // If apiGroup is not specified, it defaults to "apps.cozystack.io". | ||
| ApplicationRef corev1.TypedLocalObjectReference `json:"applicationRef"` | ||
|
|
||
| // Where backups should be stored. | ||
| StorageRef corev1.TypedLocalObjectReference `json:"storageRef"` | ||
|
|
||
| // Driver-specific BackupStrategy to use. | ||
| StrategyRef corev1.TypedLocalObjectReference `json:"strategyRef"` | ||
| // BackupClassName references a BackupClass that contains strategy and other parameters (e.g. storage reference). | ||
| // The BackupClass will be resolved to determine the appropriate strategy and parameters | ||
| // based on the ApplicationRef. | ||
| BackupClassName string `json:"backupClassName"` | ||
|
|
||
| // When backups should run. | ||
| Schedule PlanSchedule `json:"schedule"` | ||
|
|
@@ -145,12 +145,12 @@ Core Plan controller: | |
| * Create a `BackupJob` in the same namespace: | ||
|
|
||
| * `spec.planRef.name = plan.Name` | ||
| * `spec.applicationRef = plan.spec.applicationRef` | ||
| * `spec.storageRef = plan.spec.storageRef` | ||
| * `spec.strategyRef = plan.spec.strategyRef` | ||
| * `spec.triggeredBy = "Plan"` | ||
| * `spec.applicationRef = plan.spec.applicationRef` (normalized with default apiGroup if not specified) | ||
| * `spec.backupClassName = plan.spec.backupClassName` | ||
| * Set `ownerReferences` so the `BackupJob` is owned by the `Plan`. | ||
|
|
||
| **Note:** The `BackupJob` controller resolves the `BackupClass` to determine the appropriate strategy and parameters, based on the `ApplicationRef`. The strategy template is processed with a context containing the `Application` object and `Parameters` from the `BackupClass`. | ||
|
|
||
| The Plan controller does **not**: | ||
|
|
||
| * Execute backups itself. | ||
|
|
@@ -159,17 +159,64 @@ The Plan controller does **not**: | |
|
|
||
| --- | ||
|
|
||
| ### 4.2 Storage | ||
| ### 4.2 BackupClass | ||
|
|
||
| **Group/Kind** | ||
| `backups.cozystack.io/v1alpha1, Kind=BackupClass` | ||
|
|
||
| **Purpose** | ||
| Define a class of backup configurations that encapsulate strategy and parameters per application type. `BackupClass` is a cluster-scoped resource that allows admins to configure backup strategies and parameters in a reusable way. | ||
|
|
||
| **Key fields (spec)** | ||
|
|
||
| ```go | ||
| type BackupClassSpec struct { | ||
| // Strategies is a list of backup strategies, each matching a specific application type. | ||
| Strategies []BackupClassStrategy `json:"strategies"` | ||
| } | ||
|
|
||
| type BackupClassStrategy struct { | ||
| // StrategyRef references the driver-specific BackupStrategy (e.g., Velero). | ||
| StrategyRef corev1.TypedLocalObjectReference `json:"strategyRef"` | ||
|
|
||
| // Application specifies which application types this strategy applies to. | ||
| // If apiGroup is not specified, it defaults to "apps.cozystack.io". | ||
| Application ApplicationSelector `json:"application"` | ||
|
|
||
| // Parameters holds strategy-specific parameters, like storage reference. | ||
| // Common parameters include: | ||
| // - backupStorageLocationName: Name of Velero BackupStorageLocation | ||
| // +optional | ||
| Parameters map[string]string `json:"parameters,omitempty"` | ||
| } | ||
|
|
||
| **API Shape** | ||
| type ApplicationSelector struct { | ||
| // APIGroup is the API group of the application. | ||
| // If not specified, defaults to "apps.cozystack.io". | ||
| // +optional | ||
| APIGroup *string `json:"apiGroup,omitempty"` | ||
|
|
||
| // Kind is the kind of the application (e.g., VirtualMachine, MySQL). | ||
| Kind string `json:"kind"` | ||
| } | ||
| ``` | ||
|
|
||
| TBD | ||
| **BackupClass resolution** | ||
|
|
||
| **Storage usage** | ||
| * When a `BackupJob` or `Plan` references a `BackupClass` via `backupClassName`, the controller: | ||
| 1. Fetches the `BackupClass` by name. | ||
| 2. Matches the `ApplicationRef` against strategies in the `BackupClass`: | ||
| * Normalizes `ApplicationRef.apiGroup` (defaults to `"apps.cozystack.io"` if not specified). | ||
| * Finds a strategy where `ApplicationSelector` matches the `ApplicationRef` (apiGroup and kind). | ||
| 3. Returns the matched `StrategyRef` and `Parameters`. | ||
| * Strategy templates (e.g., Velero's `backupTemplate.spec`) are processed with a context containing: | ||
| * `Application`: The application object being backed up. | ||
| * `Parameters`: The parameters from the matched `BackupClassStrategy`. | ||
|
|
||
| * `Plan` and `BackupJob` reference `Storage` via `TypedLocalObjectReference`. | ||
| * Drivers read `Storage` to know how/where to store or read artifacts. | ||
| * Core treats `Storage` spec as opaque; it does not directly talk to S3 or buckets. | ||
| **Parameters** | ||
|
|
||
| * Parameters are passed via `Parameters` in the `BackupClass` (e.g., `backupStorageLocationName` for Velero). | ||
| * The driver uses these parameters to resolve the actual resources (e.g., Velero's `BackupStorageLocation` CRD). | ||
|
|
||
| --- | ||
|
|
||
|
|
@@ -189,16 +236,13 @@ type BackupJobSpec struct { | |
| PlanRef *corev1.LocalObjectReference `json:"planRef,omitempty"` | ||
|
|
||
| // Application to back up. | ||
| // If apiGroup is not specified, it defaults to "apps.cozystack.io". | ||
| ApplicationRef corev1.TypedLocalObjectReference `json:"applicationRef"` | ||
|
|
||
| // Storage to use. | ||
| StorageRef corev1.TypedLocalObjectReference `json:"storageRef"` | ||
|
|
||
| // Driver-specific BackupStrategy to use. | ||
| StrategyRef corev1.TypedLocalObjectReference `json:"strategyRef"` | ||
|
|
||
| // Informational: what triggered this run ("Plan", "Manual", etc.). | ||
| TriggeredBy string `json:"triggeredBy,omitempty"` | ||
| // BackupClassName references a BackupClass that contains strategy and related parameters | ||
| // The BackupClass will be resolved to determine the appropriate strategy and parameters | ||
| // based on the ApplicationRef. | ||
| BackupClassName string `json:"backupClassName"` | ||
| } | ||
| ``` | ||
|
|
||
|
|
@@ -223,7 +267,9 @@ type BackupJobStatus struct { | |
| * Each driver controller: | ||
|
|
||
| * Watches `BackupJob`. | ||
| * Reconciles runs where `spec.strategyRef.apiGroup/kind` matches its **strategy type(s)**. | ||
| * Resolves the `BackupClass` referenced by `spec.backupClassName`. | ||
| * Matches the `ApplicationRef` against strategies in the `BackupClass` to find the appropriate strategy. | ||
| * Reconciles runs where the resolved strategy's `apiGroup/kind` matches its **strategy type(s)**. | ||
| * Driver responsibilities: | ||
|
|
||
| 1. On first reconcile: | ||
|
|
@@ -232,7 +278,12 @@ type BackupJobStatus struct { | |
| * Set `status.phase = Running`. | ||
| 2. Resolve inputs: | ||
|
|
||
| * Read `Strategy` (driver-owned CRD), `Storage`, `Application`, optionally `Plan`. | ||
| * Resolve `BackupClass` from `spec.backupClassName`. | ||
| * Match `ApplicationRef` against `BackupClass` strategies to get `StrategyRef` and `Parameters`. | ||
| * Read `Strategy` (driver-owned CRD) from `StrategyRef`. | ||
| * Read `Application` from `ApplicationRef`. | ||
| * Extract parameters from `Parameters` (e.g., `backupStorageLocationName` for Velero). | ||
| * Process strategy template with context: `Application` object and `Parameters` from `BackupClass`. | ||
| 3. Execute backup logic (implementation-specific). | ||
| 4. On success: | ||
|
|
||
|
|
@@ -264,13 +315,14 @@ Represent a single **backup artifact** for a given application, decoupled from a | |
| type BackupSpec struct { | ||
| ApplicationRef corev1.TypedLocalObjectReference `json:"applicationRef"` | ||
| PlanRef *corev1.LocalObjectReference `json:"planRef,omitempty"` | ||
| StorageRef corev1.TypedLocalObjectReference `json:"storageRef"` | ||
| StrategyRef corev1.TypedLocalObjectReference `json:"strategyRef"` | ||
| TakenAt metav1.Time `json:"takenAt"` | ||
| DriverMetadata map[string]string `json:"driverMetadata,omitempty"` | ||
| } | ||
| ``` | ||
|
|
||
| **Note:** Parameters are not stored directly in `Backup`. Instead, they are resolved from `BackupClass` parameters when the backup was created. The storage location is managed by the driver (e.g., Velero's `BackupStorageLocation`) and referenced via parameters in the `BackupClass`. | ||
|
|
||
| **Key fields (status)** | ||
|
|
||
| ```go | ||
|
|
@@ -290,7 +342,8 @@ type BackupStatus struct { | |
| * Creates a `Backup` in the same namespace (typically owned by the `BackupJob`). | ||
| * Populates `spec` fields with: | ||
|
|
||
| * The application, storage, strategy references. | ||
| * The application reference. | ||
| * The strategy reference (resolved from `BackupClass` during `BackupJob` execution). | ||
| * `takenAt`. | ||
| * Optional `driverMetadata`. | ||
| * Sets `status` with: | ||
|
|
@@ -306,6 +359,8 @@ type BackupStatus struct { | |
| * Anchor `RestoreJob` operations. | ||
| * Implement higher-level policies (retention) if needed. | ||
|
|
||
| **Note:** Parameters are resolved from `BackupClass` when the `BackupJob` is created. The driver uses these parameters to determine where to store backups. The storage location itself is managed by the driver (e.g., Velero's `BackupStorageLocation` CRD) and is not directly referenced in the `Backup` resource. When restoring, the driver resolves the storage location from the original `BackupClass` parameters or from the driver's own metadata. | ||
|
|
||
|
Comment on lines
+362
to
+363
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Clarify how restore-time storage is deterministically resolved. RestoreJob only references Also applies to: 411-418 🤖 Prompt for AI Agents |
||
| --- | ||
|
|
||
| ### 4.5 RestoreJob | ||
|
|
@@ -353,13 +408,13 @@ type RestoreJobStatus struct { | |
| * Determines effective: | ||
|
|
||
| * **Strategy**: `backup.spec.strategyRef`. | ||
| * **Storage**: `backup.spec.storageRef`. | ||
| * **Storage**: Resolved from driver metadata or `BackupClass` parameters (e.g., `backupStorageLocationName` stored in `driverMetadata` or resolved from the original `BackupClass`). | ||
| * **Target application**: `spec.targetApplicationRef` or `backup.spec.applicationRef`. | ||
| * If effective strategy’s GVK is one of its supported strategy types → driver is responsible. | ||
| 3. Behaviour: | ||
|
|
||
| * On first reconcile, set `status.startedAt` and `phase = Running`. | ||
| * Resolve `Backup`, `Storage`, `Strategy`, target application. | ||
| * Resolve `Backup`, storage location (from driver metadata or `BackupClass`), `Strategy`, target application. | ||
| * Execute restore logic (implementation-specific). | ||
| * On success: | ||
|
|
||
|
|
@@ -414,8 +469,10 @@ The Cozystack backups core API: | |
| * Uses a single group, `backups.cozystack.io`, for all core CRDs. | ||
| * Cleanly separates: | ||
|
|
||
| * **When & where** (Plan + Storage) – core-owned. | ||
| * **When** (Plan schedule) – core-owned. | ||
| * **How & where** (BackupClass) – central configuration unit that encapsulates strategy and parameters (e.g., storage reference) per application type, resolved per BackupJob/Plan. | ||
| * **Execution** (BackupJob) – created by Plan when schedule fires, resolves BackupClass to get strategy and parameters, then delegates to driver. | ||
| * **What backup artifacts exist** (Backup) – driver-created but cluster-visible. | ||
| * **Execution lifecycle** (BackupJob, RestoreJob) – shared contract boundary. | ||
| * **Restore lifecycle** (RestoreJob) – shared contract boundary. | ||
| * Allows multiple strategy drivers to implement backup/restore logic without entangling their implementation with the core API. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -57,13 +57,10 @@ type BackupSpec struct { | |
| // +optional | ||
| PlanRef *corev1.LocalObjectReference `json:"planRef,omitempty"` | ||
|
|
||
| // StorageRef refers to the Storage object that describes where the backup | ||
| // artifact is stored. | ||
| StorageRef corev1.TypedLocalObjectReference `json:"storageRef"` | ||
|
|
||
| // StrategyRef refers to the driver-specific BackupStrategy that was used | ||
| // to create this backup. This allows the driver to later perform restores. | ||
| StrategyRef corev1.TypedLocalObjectReference `json:"strategyRef"` | ||
| // This references a cluster-scoped resource, so it does not include a namespace. | ||
| StrategyRef TypedClusterObjectReference `json:"strategyRef"` | ||
|
Comment on lines
+62
to
+63
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Revert this. A local object reference does not have a namespace. Neither does a cluster-scoped object. TypedLocalObjectReference fits the use-case perfectly. |
||
|
|
||
| // TakenAt is the time at which the backup was taken (as reported by the | ||
| // driver). It may differ slightly from metadata.creationTimestamp. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| // Package v1alpha1 defines backups.cozystack.io API types. | ||
| // | ||
| // Group: backups.cozystack.io | ||
| // Version: v1alpha1 | ||
| package v1alpha1 | ||
|
|
||
| import ( | ||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
| "k8s.io/apimachinery/pkg/runtime" | ||
| ) | ||
|
|
||
| func init() { | ||
| SchemeBuilder.Register(func(s *runtime.Scheme) error { | ||
| s.AddKnownTypes(GroupVersion, | ||
| &BackupClass{}, | ||
| &BackupClassList{}, | ||
| ) | ||
| return nil | ||
| }) | ||
| } | ||
|
|
||
| // +kubebuilder:object:root=true | ||
| // +kubebuilder:resource:scope=Cluster | ||
| // +kubebuilder:subresource:status | ||
|
|
||
| // BackupClass defines a class of backup configurations that can be referenced | ||
| // by BackupJob and Plan resources. It encapsulates strategy and storage configuration | ||
| // per application type. | ||
| type BackupClass struct { | ||
| metav1.TypeMeta `json:",inline"` | ||
| metav1.ObjectMeta `json:"metadata,omitempty"` | ||
|
|
||
| Spec BackupClassSpec `json:"spec,omitempty"` | ||
| Status BackupClassStatus `json:"status,omitempty"` | ||
| } | ||
|
|
||
| // +kubebuilder:object:root=true | ||
|
|
||
| // BackupClassList contains a list of BackupClasses. | ||
| type BackupClassList struct { | ||
| metav1.TypeMeta `json:",inline"` | ||
| metav1.ListMeta `json:"metadata,omitempty"` | ||
| Items []BackupClass `json:"items"` | ||
| } | ||
|
|
||
| // BackupClassSpec defines the desired state of a BackupClass. | ||
| type BackupClassSpec struct { | ||
| // Strategies is a list of backup strategies, each matching a specific application type. | ||
| Strategies []BackupClassStrategy `json:"strategies"` | ||
| } | ||
|
|
||
| // BackupClassStrategy defines a backup strategy for a specific application type. | ||
| type BackupClassStrategy struct { | ||
| // StrategyRef references the driver-specific BackupStrategy (e.g., Velero). | ||
| // This references a cluster-scoped resource, so it does not include a namespace. | ||
| StrategyRef TypedClusterObjectReference `json:"strategyRef"` | ||
|
|
||
| // Application specifies which application types this strategy applies to. | ||
| Application ApplicationSelector `json:"application"` | ||
|
|
||
| // Parameters holds strategy-specific and storage-specific parameters. | ||
| // Common parameters include: | ||
| // - backupStorageLocationName: Name of Velero BackupStorageLocation | ||
| // +optional | ||
| Parameters map[string]string `json:"parameters,omitempty"` | ||
| } | ||
|
|
||
| // TypedClusterObjectReference contains enough information to let you locate a | ||
| // cluster-scoped typed resource. It does not include a namespace because | ||
| // cluster-scoped resources do not have namespaces. | ||
| type TypedClusterObjectReference struct { | ||
| // APIGroup is the group for the resource being referenced. | ||
| // If APIGroup is not specified, the specified Kind must be in the core API group. | ||
| // For any other third-party types, APIGroup is required. | ||
| // +optional | ||
| APIGroup *string `json:"apiGroup,omitempty"` | ||
|
|
||
| // Kind is the type of resource being referenced. | ||
| Kind string `json:"kind"` | ||
|
|
||
| // Name is the name of resource being referenced. | ||
| Name string `json:"name"` | ||
| } | ||
|
|
||
| // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. | ||
| func (in *TypedClusterObjectReference) DeepCopyInto(out *TypedClusterObjectReference) { | ||
| *out = *in | ||
| if in.APIGroup != nil { | ||
| in, out := &in.APIGroup, &out.APIGroup | ||
| *out = new(string) | ||
| **out = **in | ||
| } | ||
| } | ||
|
|
||
| // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TypedClusterObjectReference. | ||
| func (in *TypedClusterObjectReference) DeepCopy() *TypedClusterObjectReference { | ||
| if in == nil { | ||
| return nil | ||
| } | ||
| out := new(TypedClusterObjectReference) | ||
| in.DeepCopyInto(out) | ||
| return out | ||
| } | ||
|
|
||
| // ApplicationSelector specifies which application types a strategy applies to. | ||
| type ApplicationSelector struct { | ||
| // APIGroup is the API group of the application. | ||
| // If not specified, defaults to "apps.cozystack.io". | ||
| // +optional | ||
| APIGroup *string `json:"apiGroup,omitempty"` | ||
|
|
||
| // Kind is the kind of the application (e.g., VirtualMachine, MySQL). | ||
| Kind string `json:"kind"` | ||
| } | ||
|
|
||
| // BackupClassStatus defines the observed state of a BackupClass. | ||
| type BackupClassStatus struct { | ||
| // Conditions represents the latest available observations of a BackupClass's state. | ||
| // +optional | ||
| Conditions []metav1.Condition `json:"conditions,omitempty"` | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use headings instead of bold text for section titles (MD036).
markdownlint flags these as emphasis used for headings. Consider switching to proper heading levels.
📝 Suggested fix
Also applies to: 204-204, 216-216
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)
170-170: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
🤖 Prompt for AI Agents