AIS chart orchestration and user experience discussion

[dean-taylor]

ref. https://helm.sh/docs/howto/charts_tips_and_tricks/#complex-charts-with-many-dependencies

[dean-taylor]

Originally the XNAT helm chart was configured as a central umbrella chart to incorporate the evolving complexity of the AIS project. This chart incorporates the Postgres and XNAT web sub-charts ATM.

To date many of the features have demonstrated a tightly coupled relationship as XNAT as the central management service and user entry point.

Helm design points to take into consideration.

1. Each sub-chart is an independent and self contained product on its own.
2. The relationship and interaction between an umbrella chart and sub-charts has to be taken into account. There are well documented designs in the wild however AIS is more complex than many services currently available. How Helm passes parameters from the values file and Templates interact within the hierarchy is key.
3. A powerful and mostly overlooked function is that Helm will query the API resources prior to deployment. This allows the Helm chart to make deployment decisions based on Cluster state and functionality.

[dean-taylor]

Recent XNAT design decision. Currently in pre-release the xnat and xnat-web sub chart was consolidated. This was due to the development and deployment overheads but more importantly to allow more control over automated scripts and template support. In this instance the xnat and xnat-web, with sub-charts such as Postres and supposedly upcoming sub-charts had a more tightly coupled and interdependency than first anticipated. This made the design decision to keep the xnat-web as a sub-chart less appealing.

[RyanPSullivan7]

I think XNAT should be renamed to "AIS Node" and should include

• XNAT
• PostgreSQL
• CTP
• JupyterHub
• CVMFS Client (if not already included in JupyterHub)
• OpenCost
• Prometheus
• K8s Cron Jobs
• Globus

OpenCost, Prometheus, and some of the k8s Cron Jobs will feed into national reporting of usage as required by the NIF funding. (Users, Data Volumes, pipeline run stats, Neurodesk run stats)

It may include the ServiceMesh to ensuring consistent network controls on pipelines and jupyter instances, but we need to have a discussion and agreement on the technology choice

The following AIS Edge deployments would make sense as separate Helm Charts

• CTP
• AIS Uploader
• NVFlare/FLERA/AIS-FL

[dean-taylor]

IMO. From a user experience standpoint, having a consolidated deployment from a central (Umbrella) helm chart.
Pros.

• Allow the interoperation of these infrastructure components to be managed in an opinionated way through the Umbrella chart. This would constitute a more switch like deployment such as xnat.cvmfs.enabled=true, xnat.fed_learn.enabled=true, ...
• Complexity for each site/deployment can be managed, leveraging the consolidated experience of the AIS group via the Helm templating service.
• Evolution of the AIS project requires less logistic overheads as each site follows a software style release cycle process.
• A simplified Helm release process fits with automations such as GitOps for large scale complex and multi-deployment implementations.

Cons.

• The central Values.yaml file will become a potential monolithic file unless some of the options can be consolidated into an enable/disable and opinionated outcome.

Using an opinionated deployment (The AIS way) a site can deploy in a well tested pattern, however, if this does not suit a specific use case there is still the option to disable a particular sub-chart xnat.cvmf.enabled=false and configure the component separately using the sub-charts Helm chart directly.

[aswinnarayanan]

We could have a collection of charts (or sub-charts) for ais-node and a collection for ais-edge. Ops can deploy the charts and also maintain separate values.yaml files (or sub-chart values)

[franksc]

Some strong positives noted by @dean-taylor above. It is reasonable for an AIS umbrella chart to enforce an opinionated 'AIS way' without myriad options for customisation bulking out the values file etc, as part of the value-add of this chart providing for orchestrated deployment of multiple services.