diff --git a/README.md b/README.md index e3c7748..8abbed1 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,15 @@ ## Overview -Argo CD Resource Tracker is a tool which dynamically updates the resource inclusion settings of ArgoCD by modifying the `argocd-cm` ConfigMap. It tracks resources created by ArgoCD applications, retrieves their relationships, and ensures that ArgoCD watches only those resources managed by its applications. This reduces the number of watched resources, optimizing API server load and cache memory usage. +Argo CD Resource Tracker is a **CLI tool** that helps manage ArgoCD's resource inclusion settings. It retrieves resources created by ArgoCD applications and their relationships, and ensures that ArgoCD watches only those resources managed by its applications. This reduces the number of watched resources, optimizing API server load and cache memory usage. + +> **Note:** Currently available as a CLI tool. A Kubernetes controller mode is planned for future releases to enable continuous, automated resource tracking. ## Problem Statement ArgoCD’s current implementation watches all resources in the cluster cache, leading to excessive watch connections. In Kubernetes clusters with a large number of CRDs (~200), this results in client-side throttling due to high API server load. -Static configuration settings like `resource.inclusions` and `resource.exclusions` require users to define which resource types to manage in advance, making it inflexible. This project provides a dynamic solution to watch only resources actively managed by ArgoCD applications. +Static configuration settings like `resource.inclusions` and `resource.exclusions` require users to define which resource types to manage in advance, making it inflexible. This project provides a solution to watch only resources actively managed by ArgoCD applications. ## Motivation @@ -20,69 +22,24 @@ Static configuration settings like `resource.inclusions` and `resource.exclusion ## How It Works -* Continuously retrieve all ArgoCD applications. - -* Process each application by fetching its target objects from the repo-server. - -* Check if the resource relationships exist in resource-relation-lookup ConfigMap. - -* If resource relations are missing: - - * Dynamically obtain the resource relations and update resource-relation-lookup ConfigMap and resource inclusion settings in argocd-cm ConfigMap. +* It can process a single ArgoCD application or all applications in a namespace. -* If resource relations are present: - - * Check if the inclusion settings need an update. - - * Update the inclusion settings if necessary, otherwise skip. +* Processes each application by fetching its target objects from the ArgoCD repo-server. +* Retrieves all the relations of each ArgoCD application using one of two strategies: + * **Dynamic (default)**: Uses owner references to recursively discover parent-child relationships. + * **Graph**: Uses the Cyphernetes library to query resource relationships via graph queries. ## Benefits -* Dynamic resource management: Automatically tracks and manages resource inclusions settings in `argocd-cm` ConfigMap. +* Helps track and manage resource inclusions settings in `argocd-cm` ConfigMap. -* Optimized API interactions: Prevents unnecessary API calls and throttling. +* Optimised API interactions by preventing unnecessary API calls and throttling. -* Reduced operational overhead: Eliminates the need for manual resource inclusion configuration. +* Reduced operational overhead by eliminating the need for manually finding the resource relations. ## Installation & Usage -See [doc](docs/installation.md) for installation and usage instructions. - -This project aims to make Argo CD more efficient and scalable by reducing unnecessary resource watches. Contributions and feedback are welcome! - -There are 2 methods of determining the nested children of an Argo CD Application. - -### Approach 1 : Using the cyphernetes graph querying library. - -This is the most efficient way of determining the nested children of one or more Argo CD Application. - -If querying for all the applications, it is efficient to use the `--global` flag which is efficient and reduces the number of calls made to the Kubernetes API server. - -``` shell -argocd-resource-tracker run-query --global --tracking-method label --loglevel info -``` - -By enabling the flag, `--update-enabled`, it is possible to calculate the `resource.inclusions` and `resource.exclusions` and update the `argocd-cm` config map present in the namespace -determined by the parameter `--argocd-namespace` (argocd by default). - -```shell -argocd-resource-tracker run-query --global --tracking-method label --loglevel info --update-enabled true -``` - -Based on the resource tracking method configured in Argo CD, one can specify either the `label` tracking method or the `annotation` tracking method using the `--tracking-method` parameter. -```shell -argocd-resource-tracker run-query --global --tracking-method annotation --loglevel info --update-enabled true -``` - -### Approach 2 : Using the Argo CD repo server - -In this approach, the command makes a grpc call to the Argo CD Repo server that retrieves the immediate children, then recursively going through its owner references to determine the parent-child relationships. -Any previously known relationships are not executed again to reduce the number of calls to the Kubernetes API server. - -Eg: - -```shell +* **[Installation Guide](docs/installation.md)** - Installation instructions and quick start examples +* **[Command Reference](docs/reference.md)** - Complete command-line options and flags documentation -argocd-resource-tracker run --interval 2m --loglevel info --repo-server argocd-repo-server.argocd.svc.cluster --repo-server-timeout-seconds 60 \ ---repo-server-plaintext true --argocd-namespace argocd -``` \ No newline at end of file +This project aims to make Argo CD more efficient and scalable by reducing unnecessary resource watches. Contributions and feedback are welcome! \ No newline at end of file diff --git a/cmd/main.go b/cmd/main.go index a41fae5..4a30873 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -8,7 +8,7 @@ import ( // newRootCommand implements the root command of argocd-resource-tracker func newRootCommand() error { - var rootCmd = &cobra.Command{ + rootCmd := &cobra.Command{ Use: "argocd-resource-tracker", Short: "Argo CD Resource Tracker", Long: "Argo CD Resource Tracker is a tool which analyzes the resource inclusions settings based on the resources managed by Argo Applications", diff --git a/cmd/tracker_cmd.go b/cmd/tracker_cmd.go index bd880ef..ce883cc 100644 --- a/cmd/tracker_cmd.go +++ b/cmd/tracker_cmd.go @@ -2,16 +2,10 @@ package main import ( "context" + "errors" "fmt" "strings" - "github.com/anandf/resource-tracker/pkg/analyzer" - dynamicbackend "github.com/anandf/resource-tracker/pkg/analyzer/dynamic" - graphbackend "github.com/anandf/resource-tracker/pkg/analyzer/graph" - "github.com/anandf/resource-tracker/pkg/common" - "github.com/anandf/resource-tracker/pkg/env" - "github.com/anandf/resource-tracker/pkg/kube" - "github.com/anandf/resource-tracker/pkg/version" argocdcommon "github.com/argoproj/argo-cd/v3/common" kubeutil "github.com/argoproj/argo-cd/v3/util/kube" "github.com/avitaltamir/cyphernetes/pkg/core" @@ -21,6 +15,14 @@ import ( "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" "k8s.io/client-go/tools/clientcmd" + + "github.com/anandf/resource-tracker/pkg/analyzer" + dynamicbackend "github.com/anandf/resource-tracker/pkg/analyzer/dynamic" + graphbackend "github.com/anandf/resource-tracker/pkg/analyzer/graph" + "github.com/anandf/resource-tracker/pkg/common" + "github.com/anandf/resource-tracker/pkg/env" + "github.com/anandf/resource-tracker/pkg/kube" + "github.com/anandf/resource-tracker/pkg/version" ) type queryCLIConfig struct { @@ -45,7 +47,7 @@ func NewAnalyzeCommand() *cobra.Command { Use: "analyze", Short: "Analyze resource relationships and dependencies for ArgoCD applications", Long: "Analyze resource relationships and dependencies for ArgoCD applications. Can process a single app or all apps.", - RunE: func(cmd *cobra.Command, args []string) error { + RunE: func(_ *cobra.Command, _ []string) error { log.Infof("%s %s starting [loglevel:%s]", version.BinaryName(), version.Version(), @@ -60,7 +62,7 @@ func NewAnalyzeCommand() *cobra.Command { // Require --app when --all-apps is false, to avoid silently analyzing all apps. if !cfg.allApps && cfg.applicationName == "" { - return fmt.Errorf("application name is required to analyze a single application") + return errors.New("application name is required to analyze a single application") } // Support app specified as "namespace/name" similar to `argocd app get ns/name`. @@ -127,7 +129,7 @@ func NewAnalyzeCommand() *cobra.Command { cmd.Flags().StringVarP(&cfg.argocdNamespace, "namespace", "n", "argocd", "ArgoCD namespace") cmd.Flags().StringVar(&cfg.kubeConfig, "kubeconfig", "", "Path to kubeconfig file for cluster access") cmd.Flags().BoolVar(&cfg.allApps, "all-apps", false, "Analyze all applications in the namespace") - cmd.Flags().StringVar(&cfg.strategy, "strategy", "graph", "Analysis strategy: 'dynamic' (OwnerRef walking) or 'graph' (Cyphernetes)") + cmd.Flags().StringVar(&cfg.strategy, "strategy", "dynamic", "Analysis strategy: 'dynamic' (OwnerRef walking) or 'graph' (Cyphernetes)") return cmd } @@ -137,7 +139,7 @@ func printInclusions(groupedKinds *common.GroupedResourceKinds) { log.Errorf("error generating resource.inclusions: %s", resourceInclusionString) return } - log.Infof("resource.inclusions: |\n%s", resourceInclusionString) + log.Infof("\nresource.inclusions: |\n%s", resourceInclusionString) } func ensureRepoServerAddress(restCfg *rest.Config, namespace, current string) (string, error) { diff --git a/cmd/version.go b/cmd/version.go index 1f2d06a..7761e0a 100644 --- a/cmd/version.go +++ b/cmd/version.go @@ -3,17 +3,18 @@ package main import ( "fmt" - "github.com/anandf/resource-tracker/pkg/version" "github.com/spf13/cobra" + + "github.com/anandf/resource-tracker/pkg/version" ) // newVersionCommand implements "version" command func newVersionCommand() *cobra.Command { var short bool - var versionCmd = &cobra.Command{ + versionCmd := &cobra.Command{ Use: "version", Short: "Display version information", - RunE: func(cmd *cobra.Command, args []string) error { + RunE: func(cmd *cobra.Command, _ []string) error { out := cmd.OutOrStdout() if !short { fmt.Fprintf(out, "%s\n", version.Useragent()) diff --git a/cmd/version_test.go b/cmd/version_test.go index b3fc117..ef7f616 100644 --- a/cmd/version_test.go +++ b/cmd/version_test.go @@ -5,6 +5,7 @@ import ( "testing" "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" "github.com/anandf/resource-tracker/pkg/version" ) @@ -44,7 +45,7 @@ func TestNewVersionCommand(t *testing.T) { cmd.SetOut(buf) cmd.SetArgs(tt.args) err := cmd.Execute() - assert.NoError(t, err) + require.NoError(t, err) assert.Equal(t, tt.expected, buf.String()) }) } diff --git a/docs/installation.md b/docs/installation.md index 5df8a71..7e367f8 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,78 +1,77 @@ -# Quick Start Guide +# Installation and Quick Start Guide ## Installation -It is required to run Argo CD Resource Tracker in the same Kubernetes namespace where Argo CD is installed. +Argo CD Resource Tracker is currently available as a CLI tool. A Kubernetes controller mode is planned for future releases. -### Follow these steps to install argocd-resource-tracker: -``` +### Binary Installation + +Download the latest release binary for your platform from the [releases page](https://github.com/anandf/resource-tracker/releases) or build from source: + +```bash git clone https://github.com/anandf/resource-tracker.git cd resource-tracker -kubectl apply -f manifest/install.yaml -n argocd +make build ``` -## resource-relation-lookup ConfigMap +The binary will be available at `dist/argocd-resource-tracker`. -Argo CD Resource Tracker utilizes the resource-relation-lookup ConfigMap to optimize resource tracking and reduce unnecessary API queries. This ConfigMap stores discovered parent-child relationships between Kubernetes resources, allowing Argo CD Resource Tracker to efficiently track dependencies without repeatedly querying the API server. +## Quick Start -### resource-relation-lookup ConfigMap Structure +### Analyze a Single Application -``` -apiVersion: v1 -data: - apps_DaemonSet: apps_ControllerRevision,core_Pod - apps_Deployment: apps_ReplicaSet - apps_ReplicaSet: core_Pod - apps_StatefulSet: apps_ControllerRevision,core_Pod - core_Node: coordination.k8s.io_Lease,core_Pod - core_Service: discovery.k8s.io_EndpointSlice -kind: ConfigMap -metadata: - name: resource-relation-lookup - namespace: argocd +**Basic usage (default: dynamic strategy):** +```shell +argocd-resource-tracker analyze --app argocd/my-app ``` -### How Argo CD Resource Tracker Uses the resource-relation-lookup ConfigMap +### Analyze All Applications -* When processing an Argo CD application, the Resource Tracker first checks the resource-relation-lookup ConfigMap to determine if the resource relationships have already been discovered. +```shell +argocd-resource-tracker analyze --all-apps --namespace argocd +``` -* If the necessary relationships exist in the ConfigMap, Argo CD directly utilizes them, avoiding redundant API queries. +### Using Graph Strategy -* If no relation is found in the ConfigMap, the Argo CD Resource Tracker queries all objects in the API server and, using owner references, discovers the relationships dynamically. +```shell +argocd-resource-tracker analyze --app argocd/my-app --strategy graph +``` -* Once new relationships are identified, they are added to the ConfigMap to ensure that subsequent applications do not need to query the API server again, reducing API load and improving performance. +### Output Format -### Cluster-wide Inclusion Pattern -To ensure Argo CD watches all relevant resources across multiple clusters, the clusters: ['*'] wildcard is used in the resource.inclusions setting: -``` -- apiGroups: +The command outputs `resource.inclusions` YAML: + +```yaml +resource.inclusions: | +- apigroups: - apps kinds: - - ReplicaSet - Deployment + - StatefulSet + - DaemonSet + clusters: + - '*' +- apigroups: + - "" + kinds: + - Service + - ConfigMap + - ServiceAccount + - Pod + clusters: + - '*' +- apigroups: + - rbac.authorization.k8s.io + kinds: + - Role + - RoleBinding + - ClusterRole + - ClusterRoleBinding clusters: - '*' - ``` -This wildcard enables inclusion settings to apply globally across all clusters managed by Argo CD. Due to the ConfigMap size limitation (1MB), defining inclusion rules per cluster individually is not scalable. Using the wildcard reduces redundancy and keeps the configuration compact. - -***Trade-off:*** - -While this ensures that all necessary resources managed by Argo CD are watched, it may also include a few unrelated resources in clusters where those relationships don’t exist. - -**Example:** - -**In Cluster A:** -- **Resource Kind:** `Deployment` - - **Children:** `ReplicaSets`, `Services` - -**In Cluster B:** -- **Resource Kind:** `Deployment` - - **Children:** `ReplicaSets`, `Services`, `Secrets` - -Since resource relationships can differ across clusters, using `clusters: ['*']` ensures that all possible resource relationships are covered, even when they vary across clusters, without the need for maintaining separate configurations for each cluster. +This output can be copied into ArgoCD's `argocd-cm` ConfigMap to configure resource inclusions. -For detailed configuration options and command-line parameters, please refer to the -[Configuration and Command Line Reference](./reference.md). +For complete command-line reference, see [Configuration and Command Line Reference](./reference.md). diff --git a/docs/reference.md b/docs/reference.md index b7d6cd5..d12da6f 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -1,126 +1,43 @@ -# Configuration and command line reference +# Configuration and Command Line Reference -The `argocd-resource-tracker` provides some command line parameters to control the -behaviour of its operations. The following is a list of available parameters -and their description. +The `argocd-resource-tracker` CLI provides command-line parameters to control its behavior. This document describes all available commands and flags. -## Command "version" +## Command: `analyze` ### Synopsis -`argocd-image-updater version` +```shell +argocd-resource-tracker analyze [flags] +``` ### Description -Prints out the version of the binary and exits. - -## Command "run-query" - -### Synopsis - -`argocd-resource-tracker run-query [flags]` - -### Description - -Finds the resource kinds that Argo CD manages by running a [Cyphernetes](https://cyphernet.es) graph query using either label or annotation tracking. - -### Flags - -**--interval** - -Interval for how often to check for updates. -Default: 5m - -**--loglevel** - -Sets the log level. Options: trace, debug, info, warn, error. -Default: info - -**--tracking-method** - -Sets the log level. Options: trace, debug, info, warn, error. -Default: label - -**--kubeconfig** - -Full path to the kube client configuration (e.g., ~/.kube/config). - -**--app-name** - -Name of the app whose children needs to be queried. If left empty, all Argo CD Applications would be considered for the query. -Default: "" - -**--app-namespace** - -Namespace in which the applications will be queried and nested children would be considered. If left empty, all namespaces are considered for the query. -Default: "" - -**--argocd-namespace** - -Namespace in which the Argo CD control plane components are running and the `argocd-cm` config map is present -Default: "argocd" - -**--global** - -An efficient way of querying children when children for all applications across all namespaces needs to be queried. -Default: "true" -Allowed Values: "true" or "false" - -**--direct-update** - -If this flag is enabled, then the `resource.inclusions` and `resource.exclusions` properties in `argocd-cm` config map gets updated automatically. -Default: "false" -Allowed Values: "true" or "false" - -**--once** - -If this flag is enabled, then the command would be run only once and if disabled, the command would run continuously in a loop. -Default: "false" -Allowed Values: "true" or "false" - -## Command "run" - -### Synopsis - -`argocd-resource-tracker run [flags]` - -### Description - -Runs the Argo CD Resource Tracker, possibly in an endless loop. +Analyzes resource relationships and dependencies for ArgoCD applications. Can process a single application or all applications in a namespace. Outputs `resource.inclusions` YAML that can be used to configure ArgoCD's `argocd-cm` ConfigMap. ### Flags -**--interval** - -Interval for how often to check for updates. -Default: 2m - -**--repo-server** - -Argo CD repo server address. (default "argocd-repo-server:8081") - -**--repo-server-timeout-seconds** - -Timeout in seconds for repo server RPC calls. -Default: 60 - -**--repo-server-plaintext** - -If specified, use an unencrypted HTTP connection to the ArgoCD API instead of TLS. - -**--repo-server-strict-tls** +| Flag | Short | Default | Description | +|------|-------|---------|-------------| +| `--app` | | | Application name (required for single app analysis). Supports `namespace/name` syntax (e.g., `argocd/my-app`) | +| `--app-namespace` | `-N` | `argocd` | Namespace where the application is located | +| `--all-apps` | | `false` | Analyze all applications in the namespace | +| `--strategy` | | `dynamic` | Analysis strategy: `dynamic` (OwnerRef walking) or `graph` (Cyphernetes) | +| `--namespace` | `-n` | `argocd` | ArgoCD namespace (where ArgoCD is installed) | +| `--kubeconfig` | | | Path to kubeconfig file. If not provided, uses in-cluster config or `~/.kube/config` | +| `--repo-server` | | | Repo server address. If empty, CLI will port-forward to `argocd-repo-server` service | +| `--repo-server-plaintext` | | `false` | Use unencrypted HTTP connection to repo-server (instead of TLS) | +| `--repo-server-strict-tls` | | `false` | Enable strict TLS validation for repo-server connection | +| `--repo-server-timeout-seconds` | | `60` | Timeout in seconds for repo-server RPC calls | +| `--loglevel` | | `info` | Log level: `trace`, `debug`, `info`, `warn`, or `error` | -If specified, enables strict TLS validation for the repo server connection. -**-loglevel** +### Strategy Options -Sets the log level. Options: trace, debug, info, warn, error. -Default: info +#### Dynamic Strategy (Default) -**--kubeconfig** +The dynamic strategy uses the Kubernetes discovery client to list all API resource kinds (including CRDs) on the cluster and builds an in-memory parent→child cache from owner references. For each resource returned by the Argo CD repo-server (the application’s immediate children), it looks up that kind in the cache, then recursively looks up each child kind until the full dependency tree is built. -Full path to the kube client configuration (e.g., ~/.kube/config). +#### Graph Strategy -**--argocd-namespace** +The Graph strategy uses the [Cyphernetes](https://cyphernet.es) library to run graph based relationship queries against the cluster. For each resource returned by the Argo CD repo-server (the application’s immediate children), it runs a Cyphernetes query to find that resource’s children, then repeats this process recursively for each child until the full dependency tree is built. -Namespace where ArgoCD runs. If not specified, uses the current namespace. diff --git a/go.mod b/go.mod index 083e645..82577f6 100644 --- a/go.mod +++ b/go.mod @@ -9,10 +9,12 @@ require ( github.com/sirupsen/logrus v1.9.3 github.com/spf13/cobra v1.9.1 github.com/stretchr/testify v1.10.0 + golang.org/x/sync v0.13.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/apiextensions-apiserver v0.32.2 k8s.io/apimachinery v0.32.2 k8s.io/client-go v0.32.2 + k8s.io/kube-aggregator v0.32.2 ) require ( @@ -133,7 +135,6 @@ require ( golang.org/x/crypto v0.37.0 // indirect golang.org/x/net v0.39.0 // indirect golang.org/x/oauth2 v0.28.0 // indirect - golang.org/x/sync v0.13.0 // indirect golang.org/x/sys v0.32.0 // indirect golang.org/x/term v0.31.0 // indirect golang.org/x/text v0.24.0 // indirect @@ -154,7 +155,6 @@ require ( k8s.io/component-helpers v0.32.2 // indirect k8s.io/controller-manager v0.0.0 // indirect k8s.io/klog/v2 v2.130.1 // indirect - k8s.io/kube-aggregator v0.32.2 // indirect k8s.io/kube-openapi v0.0.0-20250304201544-e5f78fe3ede9 // indirect k8s.io/kubectl v0.32.2 // indirect k8s.io/kubernetes v1.32.2 // indirect diff --git a/operator/common.go b/operator/common.go index b66a654..ccedf63 100644 --- a/operator/common.go +++ b/operator/common.go @@ -4,16 +4,13 @@ import ( "context" "encoding/base64" "encoding/json" + "errors" "fmt" "os" "os/signal" "syscall" "time" - "github.com/anandf/resource-tracker/pkg/argocd" - "github.com/anandf/resource-tracker/pkg/common" - "github.com/anandf/resource-tracker/pkg/graph" - "github.com/anandf/resource-tracker/pkg/kube" argocdcommon "github.com/argoproj/argo-cd/v3/common" log "github.com/sirupsen/logrus" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -22,6 +19,11 @@ import ( "k8s.io/client-go/dynamic/dynamicinformer" "k8s.io/client-go/rest" "k8s.io/client-go/tools/cache" + + "github.com/anandf/resource-tracker/pkg/argocd" + "github.com/anandf/resource-tracker/pkg/common" + "github.com/anandf/resource-tracker/pkg/graph" + "github.com/anandf/resource-tracker/pkg/kube" ) const ( @@ -46,7 +48,6 @@ type BaseControllerConfig struct { type BaseController struct { dynamicClient dynamic.Interface - restConfig *rest.Config previousGroupedKinds common.GroupedResourceKinds queryServers map[string]*graph.QueryServer argoCDClient argocd.ArgoCD @@ -55,7 +56,7 @@ type BaseController struct { func newBaseController(cfg *BaseControllerConfig) (*BaseController, error) { if cfg.updateResourceKind != ConfigMapResourceKind && cfg.updateResourceName != ArgoCDResourceKind { - return nil, fmt.Errorf("invalid update-resource-kind, valid values are ConfigMap and ArgoCD") + return nil, errors.New("invalid update-resource-kind, valid values are ConfigMap and ArgoCD") } restConfig, err := kube.GetKubeConfig(cfg.kubeConfig) if err != nil { @@ -89,10 +90,10 @@ func newBaseController(cfg *BaseControllerConfig) (*BaseController, error) { return nil, err } for _, clusterConfig := range clusterConfigs { - if len(clusterConfig.Host) == 0 { + if clusterConfig.Host == "" { continue } - queryServer, err := graph.NewQueryServer(clusterConfig, trackingMethod, true) + queryServer, err := graph.NewQueryServer(clusterConfig, trackingMethod) if err != nil { return nil, err } @@ -100,7 +101,6 @@ func newBaseController(cfg *BaseControllerConfig) (*BaseController, error) { } return &BaseController{ dynamicClient: dynamicClient, - restConfig: restConfig, queryServers: queryServerMap, argoCDClient: argoClient, }, nil @@ -118,14 +118,14 @@ func initApplicationInformer(dynamicClient dynamic.Interface, executor Executabl // Add event handlers _, err := informer.AddEventHandler(cache.ResourceEventHandlerFuncs{ - AddFunc: func(obj interface{}) { + AddFunc: func(obj any) { unstructuredObj := obj.(*unstructured.Unstructured) log.Infof("Object Added: %s/%s", unstructuredObj.GetNamespace(), unstructuredObj.GetName()) if err := executor.execute(); err != nil { log.Error(err) } }, - UpdateFunc: func(oldObj, newObj interface{}) { + UpdateFunc: func(oldObj, newObj any) { oldUnstructured := oldObj.(*unstructured.Unstructured) newUnstructured := newObj.(*unstructured.Unstructured) log.Infof("Object Updated: %s/%s (ResourceVersion: %s -> %s)", @@ -135,7 +135,7 @@ func initApplicationInformer(dynamicClient dynamic.Interface, executor Executabl log.Error(err) } }, - DeleteFunc: func(obj interface{}) { + DeleteFunc: func(obj any) { unstructuredObj := obj.(*unstructured.Unstructured) log.Infof("Object Deleted: %s/%s", unstructuredObj.GetNamespace(), unstructuredObj.GetName()) if err := executor.execute(); err != nil { @@ -201,7 +201,7 @@ func listClusterConfigs(dynamicClient dynamic.Interface, argocdNS string) ([]*re if err != nil { return nil, fmt.Errorf("failed to unmarshal cluster config: %w", err) } - if len(kubeConfig.Host) == 0 { + if kubeConfig.Host == "" { log.Errorf("ignoring kubeconfig with empty host for cluster secret '%s/%s'", secret.GetNamespace(), secret.GetName()) continue } diff --git a/operator/graphquery.go b/operator/graphquery.go index e9959cf..b35aaf9 100644 --- a/operator/graphquery.go +++ b/operator/graphquery.go @@ -5,12 +5,13 @@ import ( "strings" "time" - "github.com/anandf/resource-tracker/pkg/common" - "github.com/anandf/resource-tracker/pkg/env" - "github.com/anandf/resource-tracker/pkg/version" "github.com/avitaltamir/cyphernetes/pkg/core" log "github.com/sirupsen/logrus" "github.com/spf13/cobra" + + "github.com/anandf/resource-tracker/pkg/common" + "github.com/anandf/resource-tracker/pkg/env" + "github.com/anandf/resource-tracker/pkg/version" ) type GraphQueryControllerConfig struct { @@ -25,10 +26,10 @@ type GraphQueryController struct { // newGraphQueryCommand implements "runQuery" command which executes a cyphernetes graph query against a given kubeconfig func newGraphQueryCommand() *cobra.Command { cfg := &GraphQueryControllerConfig{} - var runQueryCmd = &cobra.Command{ + runQueryCmd := &cobra.Command{ Use: "run-query", Short: "Runs the resource-tracker which executes a graph based query to fetch the dependencies", - RunE: func(cmd *cobra.Command, args []string) error { + RunE: func(_ *cobra.Command, _ []string) error { log.Infof("%s %s starting [loglevel:%s, interval:%s]", fmt.Sprintf("%s-%s", version.BinaryName(), "operator"), version.Version(), diff --git a/operator/main.go b/operator/main.go index 85b6501..8586623 100644 --- a/operator/main.go +++ b/operator/main.go @@ -8,7 +8,7 @@ import ( // newRootCommand implements the root command of argocd-resource-tracker func newRootCommand() error { - var rootCmd = &cobra.Command{ + rootCmd := &cobra.Command{ Use: "argocd-resource-tracker-operator", Short: "Dynamically update resource.inclusions based on the resources managed by Argo Applications", } diff --git a/pkg/analyzer/dynamic/backend.go b/pkg/analyzer/dynamic/backend.go index e0fd8a8..30cd6a2 100644 --- a/pkg/analyzer/dynamic/backend.go +++ b/pkg/analyzer/dynamic/backend.go @@ -2,18 +2,19 @@ package dynamicbackend import ( "context" - "fmt" + "errors" "sync" + "github.com/argoproj/argo-cd/v3/pkg/apis/application/v1alpha1" + "github.com/emirpasic/gods/sets/hashset" + log "github.com/sirupsen/logrus" + "golang.org/x/sync/errgroup" + "github.com/anandf/resource-tracker/pkg/analyzer" "github.com/anandf/resource-tracker/pkg/argocd" "github.com/anandf/resource-tracker/pkg/common" "github.com/anandf/resource-tracker/pkg/dynamic" "github.com/anandf/resource-tracker/pkg/kube" - "github.com/argoproj/argo-cd/v3/pkg/apis/application/v1alpha1" - "github.com/emirpasic/gods/sets/hashset" - log "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" ) // Backend implements the analysis using OwnerRefs and the dynamic resource graph logic. @@ -33,7 +34,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G logger.Info("Starting dynamic analysis backend...") if opts.KubeConfig == nil { - return nil, fmt.Errorf("dynamic backend: KubeConfig is nil in Options") + return nil, errors.New("dynamic backend: KubeConfig is nil in Options") } // Initialize ArgoCD high-level client against the control-plane cluster. @@ -65,7 +66,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G for i := range appsList { app := appsList[i] apps = append(apps, &app) - missingResources, err := ac.GetResourcesFromApplicationStatus(ctx, &app) + missingResources, err := ac.GetResourcesFromApplicationStatus(&app) if err != nil { logger.WithError(err).Error("Error getting missing resources from application conditions") continue @@ -73,16 +74,14 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G logger.Debugf("Found %d missing resources from application conditions", len(missingResources)) groupedKinds.MergeResourceInfos(missingResources) } - - } else { - // Analyze a single app + } else { // Analyze a single app logger.Infof("Getting application %q", opts.TargetApp) app, err := ac.GetApplication(opts.TargetApp) if err != nil { return nil, err } apps = []*v1alpha1.Application{app} - missingResources, err := ac.GetResourcesFromApplicationStatus(ctx, app) + missingResources, err := ac.GetResourcesFromApplicationStatus(app) if err != nil { return nil, err } @@ -91,7 +90,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G } // Use the v2 implementation based on errgroup for concurrency and cancellation. - appChildren := analyzeWithDynamicTracker(opts.KubeConfigPath, ctx, apps, ac, rt, logger) + appChildren := analyzeWithDynamicTracker(ctx, opts.KubeConfigPath, apps, ac, rt, logger) groupedKinds.MergeResourceInfos(appChildren) return &groupedKinds, nil } @@ -99,11 +98,11 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G // analyzeWithDynamicTracker analyzes the applications concurrently using errgroup // and returns the computed list of resources. func analyzeWithDynamicTracker( - kubeconfigPath string, ctx context.Context, + kubeconfigPath string, apps []*v1alpha1.Application, - ac argocd.ArgoCD, // Passed in dependency - rt *dynamic.DynamicTracker, // Passed in dependency + ac argocd.ArgoCD, + rt *dynamic.DynamicTracker, logger *log.Entry, ) []*common.ResourceInfo { var ( @@ -133,7 +132,7 @@ func analyzeWithDynamicTracker( server := app.Spec.Destination.Server if server == "" { if app.Spec.Destination.Name == "" { - err := fmt.Errorf("both destination server and name are empty") + err := errors.New("both destination server and name are empty") appLogger.WithError(err).Error("Destination missing") return nil } @@ -167,7 +166,7 @@ func analyzeWithDynamicTracker( appLogger.Error("Resource mapper for cluster was not created; ensure Applications have valid .spec.destination and Argo CD has access") return nil } - childManifests, err := ac.GetApplicationChildManifests(ctx, app, kubeconfigPath, server) + childManifests, err := ac.GetApplicationChildManifests(ctx, app) if err != nil { appLogger.WithError(err).Error("Error getting child manifests") return nil @@ -200,9 +199,7 @@ func analyzeWithDynamicTracker( rt.CacheMu.RUnlock() if len(stillMissingKeys) > 0 { appLogger.WithFields(log.Fields{ - "cluster": server, - "missingResources": stillMissingKeys, - "count": len(stillMissingKeys), + "cluster": server, }).Info("Syncing cache for missing resources") rt.EnsureSyncedSharedCacheOnHost(ctx, server) // Add direct resources as leaf nodes (empty children set) if they're still not in cache @@ -228,6 +225,9 @@ func analyzeWithDynamicTracker( }) } // Wait for all workers to complete. - g.Wait() + err := g.Wait() + if err != nil { + logger.WithError(err).Error("Error analyzing applications") + } return appChildren } diff --git a/pkg/analyzer/graph/backend.go b/pkg/analyzer/graph/backend.go index 251c345..6d7c3de 100644 --- a/pkg/analyzer/graph/backend.go +++ b/pkg/analyzer/graph/backend.go @@ -2,16 +2,18 @@ package graphbackend import ( "context" + "errors" "fmt" "sync" + "github.com/argoproj/argo-cd/v3/pkg/apis/application/v1alpha1" + log "github.com/sirupsen/logrus" + "github.com/anandf/resource-tracker/pkg/analyzer" "github.com/anandf/resource-tracker/pkg/argocd" "github.com/anandf/resource-tracker/pkg/common" "github.com/anandf/resource-tracker/pkg/graph" "github.com/anandf/resource-tracker/pkg/kube" - "github.com/argoproj/argo-cd/v3/pkg/apis/application/v1alpha1" - log "github.com/sirupsen/logrus" ) // It caches one QueryServer per destination cluster. @@ -37,7 +39,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G logger.Info("Starting graph (Cyphernetes) analysis backend...") if opts.KubeConfig == nil { - return nil, fmt.Errorf("graph backend: KubeConfig is nil in Options") + return nil, errors.New("graph backend: KubeConfig is nil in Options") } // Initialize ArgoCD client against the control-plane cluster. @@ -73,7 +75,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G if qs, err := b.getQueryServerForApp(ctx, argoCDClient, argoApp, opts.KubeConfigPath, trackingMethod, appLogger); err != nil { appLogger.WithError(err).Error("Error getting query server for destination cluster") } else { - appChildren, err := argoCDClient.GetApplicationChildManifests(ctx, argoApp, opts.KubeConfigPath, "") + appChildren, err := argoCDClient.GetApplicationChildManifests(ctx, argoApp) if err != nil { appLogger.WithError(err).Error("Error getting application children") } else { @@ -93,7 +95,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G // Always try to augment with resources inferred from Application.status, // even if graph traversal failed. - resources, err := argoCDClient.GetResourcesFromApplicationStatus(ctx, argoApp) + resources, err := argoCDClient.GetResourcesFromApplicationStatus(argoApp) if err != nil { appLogger.WithError(err).Error("Error getting resources from application status") } else { @@ -109,7 +111,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G appLogger := logger.WithField("applicationName", argoApp.Name) appLogger.Info("Processing application") appLogger.Debugf("Querying Argo CD application %q", argoApp.Name) - appChildren, err := argoCDClient.GetApplicationChildManifests(ctx, &argoApp, opts.KubeConfigPath, "") + appChildren, err := argoCDClient.GetApplicationChildManifests(ctx, &argoApp) if err != nil { appLogger.WithError(err).Error("Error getting application children") } @@ -132,7 +134,7 @@ func (b *Backend) Execute(ctx context.Context, opts analyzer.Options) (*common.G } // Always try to augment with resources inferred from Application.status, // even if graph traversal failed. - resources, err := argoCDClient.GetResourcesFromApplicationStatus(ctx, &argoApp) + resources, err := argoCDClient.GetResourcesFromApplicationStatus(&argoApp) if err != nil { appLogger.WithError(err).Error("Error getting resources from application status") continue @@ -188,7 +190,7 @@ func (b *Backend) getQueryServerForApp( return nil, fmt.Errorf("failed to build rest.Config for cluster %q: %w", server, err) } - qs, err := graph.NewQueryServer(restCfg, trackingMethod, true) + qs, err := graph.NewQueryServer(restCfg, trackingMethod) if err != nil { return nil, fmt.Errorf("failed to create query server for cluster %q: %w", server, err) } diff --git a/pkg/analyzer/interface.go b/pkg/analyzer/interface.go index 26d5d86..a77b1ba 100644 --- a/pkg/analyzer/interface.go +++ b/pkg/analyzer/interface.go @@ -3,8 +3,9 @@ package analyzer import ( "context" - "github.com/anandf/resource-tracker/pkg/common" "k8s.io/client-go/rest" + + "github.com/anandf/resource-tracker/pkg/common" ) // Options holds the configuration required to run an analysis. diff --git a/pkg/argocd/application.go b/pkg/argocd/application.go index a09abbc..1f23557 100644 --- a/pkg/argocd/application.go +++ b/pkg/argocd/application.go @@ -6,13 +6,8 @@ import ( "fmt" "regexp" - "github.com/anandf/resource-tracker/pkg/common" - "github.com/anandf/resource-tracker/pkg/graph" - "github.com/anandf/resource-tracker/pkg/kube" - "github.com/anandf/resource-tracker/pkg/repo" "github.com/argoproj/argo-cd/v3/pkg/apis/application/v1alpha1" "github.com/argoproj/argo-cd/v3/pkg/client/clientset/versioned" - "github.com/argoproj/argo-cd/v3/util/argo" "github.com/argoproj/argo-cd/v3/util/db" "github.com/argoproj/argo-cd/v3/util/settings" log "github.com/sirupsen/logrus" @@ -22,6 +17,11 @@ import ( "k8s.io/client-go/dynamic" "k8s.io/client-go/rest" "k8s.io/client-go/util/retry" + + "github.com/anandf/resource-tracker/pkg/common" + "github.com/anandf/resource-tracker/pkg/graph" + "github.com/anandf/resource-tracker/pkg/kube" + "github.com/anandf/resource-tracker/pkg/repo" ) const ( @@ -35,9 +35,9 @@ type ArgoCD interface { GetApplication(name string) (*v1alpha1.Application, error) GetAppProject(app *v1alpha1.Application) (*v1alpha1.AppProject, error) GetApplicationClusterServerByName(ctx context.Context, clusterName string) (string, error) - GetResourcesFromApplicationStatus(ctx context.Context, application *v1alpha1.Application) ([]*common.ResourceInfo, error) + GetResourcesFromApplicationStatus(application *v1alpha1.Application) ([]*common.ResourceInfo, error) GetAllMissingResources() ([]*common.ResourceInfo, error) - GetApplicationChildManifests(ctx context.Context, application *v1alpha1.Application, kubeconfig string, server string) ([]*common.ResourceInfo, error) + GetApplicationChildManifests(ctx context.Context, application *v1alpha1.Application) ([]*common.ResourceInfo, error) GetTrackingMethod() (string, error) GetAppCluster(ctx context.Context, server string) (*v1alpha1.Cluster, error) GetCurrentResourceInclusions(gvr *schema.GroupVersionResource, resourceName, resourceNamespace string) (string, error) @@ -50,8 +50,6 @@ type argocd struct { kubeClient *kube.KubeClient dynamicClient dynamic.Interface applicationClientSet versioned.Interface - queryServers map[string]*graph.QueryServer - trackingMethod v1alpha1.TrackingMethod repoServerManager *repo.RepoServerManager settingsManager *settings.SettingsManager applicationNamespace string @@ -64,12 +62,6 @@ func NewArgoCD(config *rest.Config, argocdNS string, applicationNS string, repoS if err != nil { return nil, fmt.Errorf("could not create K8s client: %w", err) } - qsMap := make(map[string]*graph.QueryServer) - qs, err := graph.NewQueryServer(config, graph.TrackingMethodLabel, false) - if err != nil { - return nil, fmt.Errorf("could not create query server: %w", err) - } - qsMap[config.Host] = qs settingsMgr := settings.NewSettingsManager(context.Background(), resourceTrackerConfig.KubeClient.Clientset, argocdNS) dynamicClient, err := dynamic.NewForConfig(config) if err != nil { @@ -84,8 +76,6 @@ func NewArgoCD(config *rest.Config, argocdNS string, applicationNS string, repoS kubeClient: resourceTrackerConfig.KubeClient, dynamicClient: dynamicClient, applicationClientSet: resourceTrackerConfig.ApplicationClientSet, - queryServers: qsMap, - trackingMethod: argo.GetTrackingMethod(settingsMgr), settingsManager: settingsMgr, repoServerManager: repoServerManager, applicationNamespace: applicationNS, @@ -153,14 +143,14 @@ func (a *argocd) UpdateResourceInclusions(gvr *schema.GroupVersionResource, reso return retry.RetryOnConflict(retry.DefaultRetry, func() error { resource, err := a.dynamicClient.Resource(*gvr).Namespace(resourceNamespace).Get(ctx, resourceName, metav1.GetOptions{}) if err != nil { - return fmt.Errorf("error fetching ConfigMap: %v", err) + return fmt.Errorf("error fetching ConfigMap: %w", err) } if err := unstructured.SetNestedField(resource.Object, resourceInclusionYaml, getResourceInclusionsHierarchy(gvr)...); err != nil { - return fmt.Errorf("failed to set resource.inclusions value: %v", err) + return fmt.Errorf("failed to set resource.inclusions value: %w", err) } if err := unstructured.SetNestedField(resource.Object, "", getResourceExclusionsHierarchy(gvr)...); err != nil { - return fmt.Errorf("failed to set resource.inclusions value: %v", err) + return fmt.Errorf("failed to set resource.exclusions value: %w", err) } // exclude all resources that are not explicitly excluded. unstructured.RemoveNestedField(resource.Object, "data", "resource.exclusions") @@ -180,7 +170,7 @@ func (a *argocd) UpdateResourceInclusions(gvr *schema.GroupVersionResource, reso func (a *argocd) GetCurrentResourceInclusions(gvr *schema.GroupVersionResource, resourceName, resourceNamespace string) (string, error) { argocdCM, err := a.dynamicClient.Resource(*gvr).Namespace(resourceNamespace).Get(context.Background(), resourceName, metav1.GetOptions{}) if err != nil { - return "", fmt.Errorf("error fetching ConfigMap: %v", err) + return "", fmt.Errorf("error fetching ConfigMap: %w", err) } resourceInclusionsYaml, found, err := unstructured.NestedString(argocdCM.Object, getResourceInclusionsHierarchy(gvr)...) if err != nil { @@ -199,10 +189,7 @@ func getMissingResources(obj *v1alpha1.Application) ([]*common.ResourceInfo, err if err != nil { return nil, err } - missingResources, err := getResourcesFromConditions(conditions) - if err != nil { - return nil, err - } + missingResources := getResourcesFromConditions(conditions) return missingResources, nil } @@ -229,7 +216,7 @@ func getExcludedResourceConditions(statusConditions []v1alpha1.ApplicationCondit // getResourcesFromConditions returns the resources that are missing to be managed reported in status.conditions // of an Argo CD Application -func getResourcesFromConditions(conditions []metav1.Condition) ([]*common.ResourceInfo, error) { +func getResourcesFromConditions(conditions []metav1.Condition) []*common.ResourceInfo { regex := regexp.MustCompile(ExcludedResourceWarningMsgPattern) results := make([]*common.ResourceInfo, 0, len(conditions)) for _, condition := range conditions { @@ -250,10 +237,10 @@ func getResourcesFromConditions(conditions []metav1.Condition) ([]*common.Resour } } } - return results, nil + return results } -func (a *argocd) GetResourcesFromApplicationStatus(ctx context.Context, application *v1alpha1.Application) ([]*common.ResourceInfo, error) { +func (a *argocd) GetResourcesFromApplicationStatus(application *v1alpha1.Application) ([]*common.ResourceInfo, error) { missingResources, err := getMissingResources(application) if err != nil { return nil, fmt.Errorf("error getting missing resources: %w", err) @@ -315,12 +302,12 @@ func (a *argocd) GetAppCluster(ctx context.Context, server string) (*v1alpha1.Cl return cluster, nil } -func (a *argocd) GetApplicationChildManifests(ctx context.Context, application *v1alpha1.Application, kubeconfig string, server string) ([]*common.ResourceInfo, error) { +func (a *argocd) GetApplicationChildManifests(ctx context.Context, application *v1alpha1.Application) ([]*common.ResourceInfo, error) { appProject, err := a.GetAppProject(application) if err != nil { return nil, fmt.Errorf("error getting app project: %w", err) } - childManifests, err := a.repoServerManager.GetApplicationChildManifests(ctx, application, appProject, kubeconfig) + childManifests, err := a.repoServerManager.GetApplicationChildManifests(ctx, application, appProject) if err != nil { return nil, fmt.Errorf("error getting child manifests: %w", err) } diff --git a/pkg/common/common.go b/pkg/common/common.go index 4475e16..34ee725 100644 --- a/pkg/common/common.go +++ b/pkg/common/common.go @@ -9,16 +9,21 @@ import ( ) type Void struct{} -type ResourceInfoSet map[ResourceInfo]Void -type ResourceInfo struct { - Kind string - Group string - Name string - Namespace string -} -type Kinds map[string]Void -type GroupedResourceKinds map[string]Kinds +type ( + ResourceInfoSet map[ResourceInfo]Void + ResourceInfo struct { + Kind string + Group string + Name string + Namespace string + } +) + +type ( + Kinds map[string]Void + GroupedResourceKinds map[string]Kinds +) type ResourceInclusionEntry struct { APIGroups []string `json:"apiGroups,omitempty"` @@ -84,10 +89,10 @@ func (g *GroupedResourceKinds) Equal(other *GroupedResourceKinds) bool { return false } for otherGroupName, otherKinds := range *other { - if _, ok := (*g)[otherGroupName]; !ok { + currentKinds, ok := (*g)[otherGroupName] + if !ok { return false } - currentKinds, _ := (*g)[otherGroupName] if !currentKinds.Equal(&otherKinds) { return false } @@ -110,19 +115,20 @@ func (g *GroupedResourceKinds) FromYaml(resourceInclusionsYaml string) error { return err } for _, resourceInclusion := range existingResourceInclusionsInCM { - for _, apiGroup := range resourceInclusion.APIGroups { - group := apiGroup - if group == "" { - group = "core" - } - for _, kind := range resourceInclusion.Kinds { - if (*g)[group] == nil { - (*g)[group] = make(map[string]Void) - } - (*g)[group][kind] = Void{} + if len(resourceInclusion.APIGroups) == 0 { + continue + } + // check only the first item in apiGroup + apiGroup := resourceInclusion.APIGroups[0] + group := apiGroup + if group == "" { + group = "core" + } + for _, kind := range resourceInclusion.Kinds { + if (*g)[group] == nil { + (*g)[group] = make(map[string]Void) } - // break after the first item in apiGroup list - break + (*g)[group][kind] = Void{} } } return nil diff --git a/pkg/dynamic/dynamic.go b/pkg/dynamic/dynamic.go index 5f511d6..b61d610 100644 --- a/pkg/dynamic/dynamic.go +++ b/pkg/dynamic/dynamic.go @@ -51,7 +51,6 @@ func (rt *DynamicTracker) GetClusterSyncLock(server string) *sync.Mutex { // EnsureSyncedSharedCacheOnHost ensures the shared cache is synced on the given server. func (rt *DynamicTracker) EnsureSyncedSharedCacheOnHost(ctx context.Context, server string) { - mapper, ok := rt.ResourceMapperStore[server] if !ok || mapper == nil { rt.logger.Warningf("No mapper for host %s", server) diff --git a/pkg/dynamic/resourcegraph.go b/pkg/dynamic/resourcegraph.go index 4484b52..2333ea2 100644 --- a/pkg/dynamic/resourcegraph.go +++ b/pkg/dynamic/resourcegraph.go @@ -6,22 +6,23 @@ import ( "reflect" "strings" - "github.com/anandf/resource-tracker/pkg/common" "github.com/emirpasic/gods/sets/hashset" log "github.com/sirupsen/logrus" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apiextensionsclient "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset" apiextensionsinformer "k8s.io/apiextensions-apiserver/pkg/client/informers/externalversions" - k8sErrors "k8s.io/apimachinery/pkg/api/errors" - v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/client-go/discovery" "k8s.io/client-go/dynamic" "k8s.io/client-go/rest" "k8s.io/client-go/tools/cache" + + "github.com/anandf/resource-tracker/pkg/common" ) -//TODO: Rename this pkg +// TODO: Rename this pkg type ResourceMapper struct { DiscoveryClient discovery.DiscoveryInterface @@ -132,9 +133,9 @@ func NewResourceMapper(destinationConfig *rest.Config) (*ResourceMapper, error) } // Set up event handlers - crdInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{ + _, err = crdInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{ AddFunc: rm.addToResourceList, - UpdateFunc: func(oldObj, newObj interface{}) { + UpdateFunc: func(oldObj, newObj any) { oldCRD, ok1 := oldObj.(*apiextensionsv1.CustomResourceDefinition) newCRD, ok2 := newObj.(*apiextensionsv1.CustomResourceDefinition) if !ok1 || !ok2 { @@ -149,10 +150,13 @@ func NewResourceMapper(destinationConfig *rest.Config) (*ResourceMapper, error) rm.addToResourceList(newObj) }, }) + if err != nil { + return nil, fmt.Errorf("failed to add CRD event handler: %w", err) + } return rm, nil } -func (r *ResourceMapper) addToResourceList(obj interface{}) { +func (r *ResourceMapper) addToResourceList(obj any) { crd, ok := obj.(*apiextensionsv1.CustomResourceDefinition) if !ok { log.Errorf("Failed to convert object to CRD") @@ -165,10 +169,10 @@ func (r *ResourceMapper) addToResourceList(obj interface{}) { } // Check if the CRD is namespaced - if !(crd.Spec.Scope == apiextensionsv1.NamespaceScoped) { + if crd.Spec.Scope != apiextensionsv1.NamespaceScoped { gv := fmt.Sprintf("%s/%s", crd.Spec.Group, version.Name) key := GetResourceKey(gv, crd.Spec.Names.Kind) - //log.Infof("Adding cluster scoped resource: %s", key) + // log.Infof("Adding cluster scoped resource: %s", key) r.ClusterScopedResources.Add(key) continue } @@ -179,13 +183,6 @@ func (r *ResourceMapper) addToResourceList(obj interface{}) { Resource: crd.Spec.Names.Plural, // CRD resources are named using the `plural` field } - // Log whether we're updating or adding a new CRD version - if r.ResourceList.Contains(gvr) { - // log.Debugf("Updating existing CRD version: %s/%s (%s)", gvr.Group, gvr.Version, gvr.Resource) - } else { - //log.Infof("Adding new CRD version: %s/%s (%s)", gvr.Group, gvr.Version, gvr.Resource) - } - r.ResourceList.Add(gvr) // Add the served version of CRD to ResourceList } @@ -244,6 +241,7 @@ func GetResourceKey(groupVersion, kind string) string { } return fmt.Sprintf("%s_%s", group, kind) } + func (r *ResourceMapper) StartInformer() { log.Info("Starting informer for cluster ", r.ClusterHostname) r.InformerFactory.Start(context.Background().Done()) @@ -261,12 +259,12 @@ func (r *ResourceMapper) GetClusterResourcesRelation(ctx context.Context) (map[s } var continueToken string for { - resourceList, err := r.DynamicClient.Resource(gvr).Namespace(v1.NamespaceAll).List(ctx, v1.ListOptions{ + resourceList, err := r.DynamicClient.Resource(gvr).Namespace(metav1.NamespaceAll).List(ctx, metav1.ListOptions{ Limit: 250, Continue: continueToken, }) if err != nil { - if !k8sErrors.IsNotFound(err) { + if !apierrors.IsNotFound(err) { log.Errorf("Failed to list resource %s: %v", gvr, err) } break diff --git a/pkg/env/env.go b/pkg/env/env.go index 1afa401..d0b2ae7 100644 --- a/pkg/env/env.go +++ b/pkg/env/env.go @@ -12,7 +12,6 @@ import ( func GetStringVal(envVar string, defaultValue string) string { if val := os.Getenv(envVar); val != "" { return val - } else { - return defaultValue } + return defaultValue } diff --git a/pkg/env/env_test.go b/pkg/env/env_test.go index 4c30dc5..d111901 100644 --- a/pkg/env/env_test.go +++ b/pkg/env/env_test.go @@ -1,7 +1,6 @@ package env import ( - "os" "testing" ) @@ -9,8 +8,7 @@ func TestGetStringVal(t *testing.T) { t.Run("returns environment variable value when set", func(t *testing.T) { envVar := "TEST_ENV_VAR" expectedValue := "test_value" - os.Setenv(envVar, expectedValue) - defer os.Unsetenv(envVar) + t.Setenv(envVar, expectedValue) result := GetStringVal(envVar, "default_value") if result != expectedValue { @@ -31,8 +29,7 @@ func TestGetStringVal(t *testing.T) { t.Run("returns default value when environment variable is empty", func(t *testing.T) { envVar := "EMPTY_ENV_VAR" defaultValue := "default_value" - os.Setenv(envVar, "") - defer os.Unsetenv(envVar) + t.Setenv(envVar, "") result := GetStringVal(envVar, defaultValue) if result != defaultValue { diff --git a/pkg/graph/query.go b/pkg/graph/query.go index a90294e..d1075fc 100644 --- a/pkg/graph/query.go +++ b/pkg/graph/query.go @@ -1,17 +1,22 @@ package graph import ( + "context" "fmt" "os" "strings" - "github.com/anandf/resource-tracker/pkg/common" "github.com/avitaltamir/cyphernetes/pkg/core" "github.com/avitaltamir/cyphernetes/pkg/provider" "github.com/avitaltamir/cyphernetes/pkg/provider/apiserver" log "github.com/sirupsen/logrus" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/discovery" "k8s.io/client-go/rest" + aggregator "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset" + + "github.com/anandf/resource-tracker/pkg/common" ) const ( @@ -65,33 +70,22 @@ var ( } leafKinds = map[string]bool{ - "ConfigMap": true, - "Secret": true, - "ServiceAccount": true, - "Namespace": true, - } - - defaultIncludedResources = common.ResourceInfoSet{ - common.ResourceInfo{ - Kind: "ConfigMap", - Group: "", - }: common.Void{}, - common.ResourceInfo{ - Kind: "Secret", - Group: "", - }: common.Void{}, - common.ResourceInfo{ - Kind: "ServiceAccount", - Group: "", - }: common.Void{}, - common.ResourceInfo{ - Kind: "Pod", - Group: "", - }: common.Void{}, - common.ResourceInfo{ - Kind: "Namespace", - Group: "", - }: common.Void{}, + "Role": true, + "RoleBinding": true, + "ClusterRole": true, + "ClusterRoleBinding": true, + "ConfigMap": true, + "Secret": true, + "ServiceAccount": true, + "Namespace": true, + "PersistentVolume": true, + "PersistentVolumeClaim": true, + "Endpoints": true, + "EndpointSlice": true, + "NetworkPolicy": true, + "Ingress": true, + "Route": true, + "SecurityContextConstraints": true, } ) @@ -104,7 +98,7 @@ type QueryServer struct { VisitedKinds map[common.ResourceInfo]bool } -func NewQueryServer(restConfig *rest.Config, trackingMethod string, loadCustomRules bool) (*QueryServer, error) { +func NewQueryServer(restConfig *rest.Config, trackingMethod string) (*QueryServer, error) { // Create the API server provider p, err := apiserver.NewAPIServerProviderWithOptions(&apiserver.APIServerProviderConfig{ Kubeconfig: restConfig, @@ -123,29 +117,8 @@ func NewQueryServer(restConfig *rest.Config, trackingMethod string, loadCustomRu fieldAMatchCriteria = AnnotationTrackingCriteria comparison = core.StringContains } - if loadCustomRules { - for _, knownResourceKind := range p.(*apiserver.APIServerProvider).GetKnownResourceKinds() { - if blackListedKinds[knownResourceKind] || leafKinds[knownResourceKind] { - log.Infof("skipping resource kind: %s", knownResourceKind) - continue - } - relationshipTypeName := strings.ToUpper(fmt.Sprintf("%s_%s_%s", "ARGOAPP_OWN", tracker, knownResourceKind)) - if strings.Index(relationshipTypeName, ".") != -1 { - relationshipTypeName = strings.Replace(relationshipTypeName, ".", "_", -1) - } - core.AddRelationshipRule(core.RelationshipRule{ - KindA: strings.ToLower(knownResourceKind), - KindB: "applications.argoproj.io", - Relationship: core.RelationshipType(relationshipTypeName), - MatchCriteria: []core.MatchCriterion{ - { - FieldA: fieldAMatchCriteria, - FieldB: "$.metadata.name", - ComparisonType: comparison, - }, - }, - }) - } + if isOpenShiftCluster(restConfig) { + log.Info("OpenShift cluster detected, adding OpenShift specific rules") addOpenShiftSpecificRules() } // Create query executor with the provider @@ -161,7 +134,6 @@ func NewQueryServer(restConfig *rest.Config, trackingMethod string, loadCustomRu Comparison: comparison, VisitedKinds: make(map[common.ResourceInfo]bool), }, nil - } func (q *QueryServer) GetApplicationChildResources(name, namespace string) (common.ResourceInfoSet, error) { @@ -175,10 +147,6 @@ func (q *QueryServer) GetApplicationChildResources(name, namespace string) (comm func (q *QueryServer) GetNestedChildResources(resource *common.ResourceInfo) (common.ResourceInfoSet, error) { allLevelChildren := make(common.ResourceInfoSet) - for resInfo := range defaultIncludedResources { - allLevelChildren[resInfo] = common.Void{} - q.VisitedKinds[resInfo] = true - } allLevelChildren, err := q.depthFirstTraversal(resource, allLevelChildren) if err != nil { return nil, err @@ -189,31 +157,31 @@ func (q *QueryServer) GetNestedChildResources(resource *common.ResourceInfo) (co // getChildren returns the immediate direct child of a given node by doing a graph query. func (q *QueryServer) getChildren(parentResourceInfo *common.ResourceInfo) ([]*common.ResourceInfo, error) { if leafKinds[parentResourceInfo.Kind] || blackListedKinds[parentResourceInfo.Kind] { - log.Infof("skipping leaf or blacklisted resource: %v", parentResourceInfo) + log.Debugf("skipping leaf or blacklisted resource: %v", parentResourceInfo) return nil, nil } visitedKindKey := common.ResourceInfo{Kind: parentResourceInfo.Kind, Group: parentResourceInfo.Group} if _, ok := q.VisitedKinds[visitedKindKey]; ok { - log.Infof("skipping resource %v as kind already visited", parentResourceInfo) + log.Debugf("skipping resource %v as kind already visited", parentResourceInfo) return nil, nil } - unambiguousKind := parentResourceInfo.Kind + var unambiguousKind string if parentResourceInfo.Group == "" { unambiguousKind = fmt.Sprintf("%s.%s", "core", parentResourceInfo.Kind) + } else { + pluralKind := strings.ToLower(parentResourceInfo.Kind) + "s" + unambiguousKind = fmt.Sprintf("%s.%s", pluralKind, parentResourceInfo.Group) } // Get the query string - queryStr := fmt.Sprintf("MATCH (p: %s) -> (c) RETURN c.kind, c.apiVersion, c.metadata.namespace", parentResourceInfo.Kind) + queryStr := fmt.Sprintf("MATCH (p: %s) -> (c) RETURN c.kind, c.apiVersion, c.metadata.namespace", unambiguousKind) if parentResourceInfo.Name != "" { - queryStr = fmt.Sprintf("MATCH (p: %s{name:\"%s\"}) -> (c) RETURN c.kind, c.apiVersion, c.metadata.namespace", unambiguousKind, parentResourceInfo.Name) + queryStr = fmt.Sprintf("MATCH (p: %s{name:%q}) -> (c) RETURN c.kind, c.apiVersion, c.metadata.namespace", unambiguousKind, parentResourceInfo.Name) } queryResult, err := q.executeQuery(queryStr, parentResourceInfo.Namespace) if err != nil { return nil, err } - results, err := extractResourceInfo(queryResult, "c") - if err != nil { - return nil, err - } + results := extractResourceInfo(queryResult, "c") q.VisitedKinds[visitedKindKey] = true return results, nil } @@ -225,6 +193,12 @@ func (q *QueryServer) executeQuery(queryStr, namespace string) (*core.QueryResul if err != nil { return nil, err } + // If namespace is empty, set all namespaces to true + if namespace == "" { + core.AllNamespaces = true + } else { + core.AllNamespaces = false + } // Execute the query against the Kubernetes API. queryResult, err := q.Executor.Execute(ast, namespace) @@ -262,31 +236,15 @@ func (q *QueryServer) depthFirstTraversal(info *common.ResourceInfo, visitedNode return visitedNodes, nil } -// AddRuleForResourceKind adds the rule for a new resource kind that was added -func (q *QueryServer) AddRuleForResourceKind(resourceKind string) { - core.AddRelationshipRule(core.RelationshipRule{ - KindA: strings.ToLower(resourceKind), - KindB: "applications", - Relationship: core.RelationshipType(strings.ToUpper(fmt.Sprintf("%s_%s_%s", "ARGOAPP_OWN", q.Tracker, resourceKind))), - MatchCriteria: []core.MatchCriterion{ - { - FieldA: q.FieldAMatchCriteria, - FieldB: "$.metadata.name", - ComparisonType: q.Comparison, - }, - }, - }) -} - // extractResourceInfo extracts the ResourceInfo from a given query result and variable name. -func extractResourceInfo(queryResult *core.QueryResult, variable string) ([]*common.ResourceInfo, error) { +func extractResourceInfo(queryResult *core.QueryResult, variable string) []*common.ResourceInfo { child := queryResult.Data[variable] if child == nil { - return nil, nil + return nil } - resourceInfoList := make([]*common.ResourceInfo, 0, len(child.([]interface{}))) - for _, meta := range child.([]interface{}) { - info, ok := meta.(map[string]interface{}) + resourceInfoList := make([]*common.ResourceInfo, 0, len(child.([]any))) + for _, meta := range child.([]any) { + info, ok := meta.(map[string]any) if !ok { continue } @@ -308,7 +266,7 @@ func extractResourceInfo(queryResult *core.QueryResult, variable string) ([]*com Group: group, Name: info["name"].(string), } - metadata, ok := info["metadata"].(map[string]interface{}) + metadata, ok := info["metadata"].(map[string]any) if !ok { continue } @@ -318,7 +276,7 @@ func extractResourceInfo(queryResult *core.QueryResult, variable string) ([]*com } resourceInfoList = append(resourceInfoList, &resourceInfo) } - return resourceInfoList, nil + return resourceInfoList } // addOpenShiftSpecificRules adds rules that are specific to OpenShift CustomResources @@ -336,3 +294,21 @@ func addOpenShiftSpecificRules() { }, }) } + +func isOpenShiftCluster(restConfig *rest.Config) bool { + aggregatorClient, err := aggregator.NewForConfig(restConfig) + if err != nil { + return false + } + gv := schema.GroupVersion{ + Group: "config.openshift.io", + Version: "v1", + } + if err = discovery.ServerSupportsVersion(aggregatorClient, gv); err != nil { + // check if the API is registered + _, err = aggregatorClient.ApiregistrationV1().APIServices(). + Get(context.TODO(), fmt.Sprintf("%s.%s", gv.Version, gv.Group), metav1.GetOptions{}) + return err == nil + } + return true +} diff --git a/pkg/kube/kubernetes.go b/pkg/kube/kubernetes.go index 6dbc3a0..fde7290 100644 --- a/pkg/kube/kubernetes.go +++ b/pkg/kube/kubernetes.go @@ -37,6 +37,11 @@ func NewKubernetesClient(ctx context.Context, client kubernetes.Interface, names // NewKubernetesClientFromConfig creates a new Kubernetes client object from given // rest.Config object. func NewKubernetesClientFromConfig(ctx context.Context, namespace string, kubeConfig *rest.Config) (*ResourceTrackerKubeClient, error) { + // Default namespace if caller doesn't provide one. This mirrors kubectl/client-go + // behavior and keeps callers/tests predictable. + if namespace == "" { + namespace = "default" + } clientset, err := kubernetes.NewForConfig(kubeConfig) if err != nil { return nil, err @@ -58,7 +63,7 @@ func GetKubeConfig(kubeconfigPath string) (*rest.Config, error) { restConfig, err := rest.InClusterConfig() if err != nil && !errors.Is(err, rest.ErrNotInCluster) { - return nil, fmt.Errorf("failed to create config: %v", err) + return nil, fmt.Errorf("failed to create config: %w", err) } // If the binary is not being run inside a kubernetes cluster, @@ -72,7 +77,7 @@ func GetKubeConfig(kubeconfigPath string) (*rest.Config, error) { kubeConfig := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides) restConfig, err = kubeConfig.ClientConfig() if err != nil { - return nil, fmt.Errorf("failed to create config: %v", err) + return nil, fmt.Errorf("failed to create config: %w", err) } } return restConfig, nil @@ -94,7 +99,7 @@ func RestConfigFromCluster(c *v1alpha1.Cluster, kubeconfigPath string) (*rest.Co if strings.Contains(c.Server, "kubernetes.default.svc") { localCfg, err := rest.InClusterConfig() if err != nil && !errors.Is(err, rest.ErrNotInCluster) { - return nil, fmt.Errorf("failed to create config: %v", err) + return nil, fmt.Errorf("failed to create config: %w", err) } if localCfg == nil { if kubeconfigPath != "" { @@ -111,7 +116,6 @@ func RestConfigFromCluster(c *v1alpha1.Cluster, kubeconfigPath string) (*rest.Co } cfg = localCfg } else { - switch { case c.Config.AWSAuthConfig != nil: // EKS via argocd-k8s-auth (same contract as Argo CD) @@ -176,7 +180,7 @@ func RestConfigFromCluster(c *v1alpha1.Cluster, kubeconfigPath string) (*rest.Co cfg.Timeout = v1alpha1.K8sServerSideTimeout cfg.QPS = v1alpha1.K8sClientConfigQPS cfg.Burst = v1alpha1.K8sClientConfigBurst - v1alpha1.SetK8SConfigDefaults(cfg) + _ = v1alpha1.SetK8SConfigDefaults(cfg) return cfg, nil } diff --git a/pkg/kube/kubernetes_test.go b/pkg/kube/kubernetes_test.go index 2027779..49f4152 100644 --- a/pkg/kube/kubernetes_test.go +++ b/pkg/kube/kubernetes_test.go @@ -28,6 +28,7 @@ func Test_NewKubernetesClientFromConfig(t *testing.T) { assert.Equal(t, "argocd", client.KubeClient.Namespace) }) } + func Test_NewKubernetesClient(t *testing.T) { t.Run("Create new Kubernetes client with valid inputs", func(t *testing.T) { mockClientset := &kubernetes.Clientset{} // Mock clientset diff --git a/pkg/repo/repo.go b/pkg/repo/repo.go index d08c87e..7db7e34 100644 --- a/pkg/repo/repo.go +++ b/pkg/repo/repo.go @@ -11,34 +11,28 @@ import ( "github.com/argoproj/argo-cd/v3/util/db" "github.com/argoproj/argo-cd/v3/util/env" "github.com/argoproj/argo-cd/v3/util/io" - kubeutil "github.com/argoproj/argo-cd/v3/util/kube" "github.com/argoproj/argo-cd/v3/util/settings" "github.com/argoproj/argo-cd/v3/util/tls" - "github.com/argoproj/gitops-engine/pkg/utils/kube" log "github.com/sirupsen/logrus" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" ) -type clusterAPIDetails struct { - APIVersions string - APIResources []kube.APIResourceInfo -} - type RepoServerManager struct { db db.ArgoDB settingsMgr *settings.SettingsManager repoClientset apiclient.Clientset - kubectl kube.Kubectl - controllerNS string } -func NewRepoServerManager(kubeConfig *rest.Config, - controllerNamespace string, repoServerAddress string, +func NewRepoServerManager( + kubeConfig *rest.Config, + controllerNamespace string, + repoServerAddress string, repoServerTimeoutSeconds int, repoServerPlaintext bool, - repoServerStrictTLS bool) (*RepoServerManager, error) { + repoServerStrictTLS bool, +) (*RepoServerManager, error) { clientSet, err := kubernetes.NewForConfig(kubeConfig) if err != nil { return nil, err @@ -50,9 +44,10 @@ func NewRepoServerManager(kubeConfig *rest.Config, StrictValidation: repoServerStrictTLS, } if !tlsConfig.DisableTLS && tlsConfig.StrictValidation { + appConfigPath := env.StringFromEnv(common.EnvAppConfigPath, common.DefaultAppConfigPath) pool, err := tls.LoadX509CertPool( - fmt.Sprintf("%s/reposerver/tls/tls.crt", env.StringFromEnv(common.EnvAppConfigPath, common.DefaultAppConfigPath)), - fmt.Sprintf("%s/reposerver/tls/ca.crt", env.StringFromEnv(common.EnvAppConfigPath, common.DefaultAppConfigPath)), + appConfigPath+"/reposerver/tls/tls.crt", + appConfigPath+"/reposerver/tls/ca.crt", ) if err != nil { return nil, fmt.Errorf("failed to load tls certs: %w", err) @@ -60,18 +55,15 @@ func NewRepoServerManager(kubeConfig *rest.Config, tlsConfig.Certificates = pool } repoClientset := apiclient.NewRepoServerClientset(repoServerAddress, repoServerTimeoutSeconds, tlsConfig) - kubectl := kubeutil.NewKubectl() return &RepoServerManager{ db: dbInstance, settingsMgr: settingsMgr, repoClientset: repoClientset, - kubectl: kubectl, - controllerNS: controllerNamespace, }, nil } // GetApplicationChildManifests fetches manifests and filters direct child resources -func (r *RepoServerManager) GetApplicationChildManifests(ctx context.Context, application *appsv1alpha1.Application, proj *appsv1alpha1.AppProject, kubeconfig string) ([]*unstructured.Unstructured, error) { +func (r *RepoServerManager) GetApplicationChildManifests(ctx context.Context, application *appsv1alpha1.Application, proj *appsv1alpha1.AppProject) ([]*unstructured.Unstructured, error) { // Fetch Helm repositories helmRepos, err := r.db.ListHelmRepositories(ctx) if err != nil { @@ -123,6 +115,26 @@ func (r *RepoServerManager) GetApplicationChildManifests(ctx context.Context, ap if err != nil { return nil, fmt.Errorf("error getting ref sources: %w", err) } + // Resolve destination cluster to get the real kube version + var kubeVersion string + var apiVersions []string + dest := application.Spec.Destination + if dest.Server != "" || dest.Name != "" { + server := dest.Server + if server == "" && dest.Name != "" { + servers, err := r.db.GetClusterServersByName(ctx, dest.Name) + if err == nil && len(servers) > 0 { + server = servers[0] + } + } + if server != "" { + cluster, err := r.db.GetCluster(ctx, server) + if err == nil { + kubeVersion = cluster.Info.ServerVersion + apiVersions = cluster.Info.APIVersions + } + } + } targetObjs := make([]*unstructured.Unstructured, 0) for i, source := range sources { repo, err := r.db.GetRepository(ctx, source.RepoURL, proj.Name) @@ -141,6 +153,8 @@ func (r *RepoServerManager) GetApplicationChildManifests(ctx context.Context, ap ApplicationSource: &source, EnabledSourceTypes: enabledSourceTypes, KustomizeOptions: kustomizeOptions, + KubeVersion: kubeVersion, + ApiVersions: apiVersions, HelmRepoCreds: permittedHelmCredentials, HasMultipleSources: application.Spec.HasMultipleSources(), RefSources: refSources,