operator refactor to use uv and ops

Author: sciabarracom

DEVEL.md

@@ -20,78 +20,116 @@
 
 # Apache OpenServerless Operator
 
-NOTE: this readme is outdated and needs to be updated, not everyhing is still relevant.
+In this README we explain how to do development with the Operator
 
----
+The operator is a collection of modules managed with ops. 
 
-In this readme there are information for developers.
+The operator itself is an ops plugin to invoke some of functions from the command line.
 
-We describe how to build and test the operator in our development environment
+To work with it, install ops and clone the source code from the top level, so you get the operator code under `openserverless/operator-op`
 
-Please refer to the [website](https://openserverless.apache.org) for user information.
+```
+curl -sL bit.ly/get-ops | bash
+git clone https://github.com/apache/openserverless --recurse-submodules
+cd openserverless
+```
 
-## How to build and use an operator image
+To be able to work with the operator you need a Kubernetes cluster and a Working configuration. 
 
-Ensure you have satisfied the prerequisites below. Most notably, you need to use our development virtual machine and you
-need write access to a GitHub repository.
+You can easily create one with kind:
 
-Once you have satisfied the prerequisites, you can build an image you can use in the development machine.
+```
+# destroy the old one
+ops op clu kind destroy
+# create a new cluster
+ops op clu kind create
+```
 
-Build an image with:
+If you want to use kubectl directory use `ops util kubeconfig` 
+to export the kind configurat to `~/.kube/config`. WARNING it overwrites yout existing one.
 
-```shell
-task build
+## Configuration
+
+You need a full configuration to be able to work with the operator. 
+
+You can create an actual configuration on the cluster skipping the launch of the operator:
+
+For example this is a config of slim mode:
+
+```
+# configure slim mode for example 
+ops config slim
+# configure
+ops setup kubernetes configure
 ```
 
-Please note that it will build the image locally and push in an internal registry, even if it is name is
-`ghcr.io/${GITHUB_USER}/openserverless-operator`.
+## Execute the operator as a cli plugin
 
-To be able to build, the task `build` will commit and push all your changes and then build the operator from the public
-sources in your local k3s.
+Many modules are now executable as `ops op nuvolaris`  commands.
 
-It will also show the logs for the latest build.
+For example: `ops op nuv etcd`
 
-You can then deploy it with:
+It shows the subcommand with `create [<replicas>]` and `delete` 
 
-```shell
-task deploy
+## Working on the cli
+
+You can also test and work on the cli. Try this:
+
+```
+ops op cli
 ```
 
-Once you have finished with development you can create a public image with `task publish` that will publish the tag and
-trigger a creation of the image.
+Initialize the configuration:
 
-## Prerequisites
+```
+import nuvolaris.operator_util as operator_util
+owner = kube.get("wsk/controller")
+operator_util.config_from_spec(owner['spec'])
+```
 
-1. Please set up and use a development VM [as described here](https://github.com/apache/openserverless)
+Create the components:
 
-2. With VSCode, access the development VM, open the workspace `openserverless/openserverless.code-workspace` and then
-   open a terminal with `operator` subproject: this will enable the `nix` environment with direnv (provided by the VM).
+```
+import nuvolaris.etcd as etcd
+msg = etcd.create(owner)
+```
 
-3. Create a fork of `github.com/apache/openserverless-operator`
+## How to publish the operator image
 
-4. Copy .env.dist in .env and put your GitHub username in it
+When you are satisfied with your development you can publish the image.
 
-5. Since the build requires you push your sources in your repo, you need the credentials to access it. The fastest way
-   is
-   to [create a personal token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
+First, install [task](http://taskfile.dev/docs/installation).
 
-6. Now set up a remote to access your repo and set it as your default upstream branch.
+You need to setup some environment variables. Copy .env.dist in .env and put your GitHub username in it
 
-```
-git remote add fork https://<GITHUB_USERNAME>:<GITHUB_TOKEN>@github.com/<GITHUB_USERNAME>/openserverless-operator
-git branch -u https://github.com/<GITHUB_USERNAME>/openserverless-operator
+Since the build requires you push your sources in your repo, you need the credentials to access it. The fastest way is
+to [create a personal token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
+
+Build an image with:
+
+```shell
+task build
 ```
 
-That's it. Now you can use `task build` to build the image.
+Please note that it will build the image locally and push in an internal registry, even if it is name is
+`ghcr.io/${GITHUB_USER}/openserverless-operator`.
+
+To be able to build, the task `build` will commit and push all your changes and then build the operator from the public
+sources in your local k3s.
 
-7. Deploy the operator
+It will also show the logs for the latest build.
 
-To deploy a testing configuration of the Apache OpenServerless operator execute the command
+You can then deploy it with:
 
 ```shell
-task all
+task deploy
 ```
 
-The operator instance will be configured applying the `test/k3s/whisk.yaml` template.
-All the components are activated except TLS and MONITORING.
+Once you have finished with development you can create a public image with `task publish` that will publish the tag and
+trigger a creation of the image.
 
+Once the image is publicly available you have to put in in `opsroot.json` to use it:
+
+```
+https://github.com/apache/openserverless-task/blob/9d2227b87196be9b487673d4d1f8202c2ec354f2/opsroot.json#L8
+```

README.md

@@ -0,0 +1,7 @@
+# three node (two workers) cluster config
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+- role: control-plane
+- role: worker
+- role: worker

clusters/kind/kind.yaml

@@ -32,8 +32,8 @@ tasks:
   create: 
     desc: create
     cmds:
-    - kind create cluster --name "{{.NAME}}"
-    - kind get kubeconfig --name="{{.NAME}}" >{{.KUBEFILE}}
+    - kind create cluster --name "{{.NAME}}" #--config kind.yaml
+    - kind get kubeconfig --name="{{.NAME}}"  >{{.KUBEFILE}}
     - cp "{{.KUBEFILE}}" "$OPS_TMP/kubeconfig"
     - kubectl --namespace=ingress-nginx  --kubeconfig "{{.KUBEFILE}}" apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml 
 

clusters/kind/opsfile.yml

@@ -27,7 +27,6 @@
 
 from nuvolaris.util import get_etcd_replica
 
-
 def create(owner=None):
     logging.info("create etcd")
     data = util.get_etcd_config_data()
@@ -173,4 +172,23 @@ def patch(status, action, owner=None):
         if  action == 'create':
             status['whisk_create']['etcd']='error'
         else:            
-            status['whisk_create']['etcd']='error'
+            status['whisk_create']['etcd']='error'
+
+import fire
+from  nuvolaris.operator_util import LoadConfig
+
+class Etcd(LoadConfig): 
+    def create(self, replicas: int = 0):
+        """ 
+        [<replicas>]
+        """
+        if replicas > 0:
+            print("setting replicas to:", replicas)
+            cfg.put("etcd.replicas", replicas)
+        print(create(self.owner))
+
+    def delete(self):
+       print(delete(self.owner))
+
+if __name__ == "__main__":
+    fire.Fire(Etcd)

nuvolaris/etcd.py

@@ -15,7 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 #
-import logging
+import sys, logging
 import nuvolaris.openwhisk as openwhisk
 import nuvolaris.kube as kube
 import nuvolaris.userdb_util as userdb
@@ -132,4 +132,15 @@ def patch_operator_status(status,component, status_code):
         logging.debug(f"patching component {component} with code = {status_code} from event handler {status}")
         status['whisk_create'][component]=status_code
     except Exception as e:
-        logging.error('*** failed to patch_operator_status: %s' % e)  
+        logging.error('*** failed to patch_operator_status: %s' % e)
+
+
+# base class to load config from the kubernetes cluster
+class LoadConfig:
+    def __init__(self):
+        self.owner = kube.get("wsk/controller")
+        if not self.owner:
+            print("cannot find a config in kubernetes\nplease use ops setup kubernetes configure")
+            sys.exit(1)
+        spec = self.owner.get("spec")
+        config_from_spec(spec)

nuvolaris/operator_util.py

@@ -0,0 +1,11 @@
+version: '3'
+
+
+tasks:
+
+  etcd:
+    silent: true
+    desc: etcd
+    dir: ..
+    cmds:
+      - uv run nuvolaris/etcd.py {{.CLI_ARGS}}

nuvolaris/opsfile.yml

@@ -33,6 +33,7 @@
 import nuvolaris.kube as kube
 import nuvolaris.template as template
 
+
 # Implements truncated exponential backoff from
 # https://cloud.google.com/storage/docs/retry-strategy#exponential-backoff
 def nuv_retry(deadline_seconds=120, max_backoff=5):
@@ -842,4 +843,3 @@ def get_registry_config_data():
     registry_affinity_tolerations_data(data)
     return data
 
-

nuvolaris/util.py

@@ -24,22 +24,27 @@ vars:
 
 tasks:
 
+  setup:
+    silent: true
+    desc: setup the environment
+    cmds:
+      - uv venv
+      - uv pip install -r pyproject.toml
+
   cli:
     silent: true
-    desc: run cli commands
+    desc: launch a cli for interactive development
     cmds:
       - |
         export IPYTHONDIR="$(pwd)"
         export PYTHONPATH="$(pwd)"
         uv run ipython
 
-  setup:
-    silent: true
-    desc: setup the environment
-    cmds:
-      - uv venv
-      - uv pip install -r pyproject.toml
 
   clusters:
     silent: true
-    desc: setup the environment
+    desc: install clusters
+
+  nuvolaris:
+    silent: true
+    desc: install nuvolaris services

opsfile.yml

@@ -41,6 +41,7 @@ dependencies = [
     "redis>=5.0.4,<6",
     "bcrypt>=4.2.1,<5",
     "python-dotenv>=1.1.1",
+    "fire>=0.7.1",
 ]
 
 [project.scripts]