Import past meeting notes from disscussions

users/2023/20231129.md

@@ -0,0 +1,16 @@
+---
+date: 2023-11-29
+type: users
+attendees:
+  - Olivier M.(DGFIP)
+  - Moncef L.(DGFIP)
+  - Mehdi B.(DGFIP)
+  - Idir I. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- **Kickoff** : begin the design and implementation
+- first technology to integrate : jupyterhub
+- Participation to the KubeCon Paris 2024
+- Update of the github repository with new logo: https://github.com/OKDP
+- website domain reserved : okdp.io

users/2023/20231206.md

@@ -0,0 +1,19 @@
+---
+date: 2023-12-06
+type: users
+attendees:
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- begin the study of the first technology to integrate : jupyterhub
+- separation between OKDP and its requirements in an extra project like kube, minio, longhorn, etc.
+- clone used projects into a separate repository in ordre to keep their history
+- start with a github jupyterhub-draft project in the okdp repo then design an okdp project to centralize all integrated technologies
+- maintaining Issues/PRs + 2 approves like TDP only for official projects and not draft
+- creating discussions for decision-making on current topics
+- next step: preparation of local environments via kind, vagrant, etc.

users/2024/20240117.md

@@ -0,0 +1,18 @@
+---
+date: 2024-01-17
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Nicolas V. (DGFIP)
+  - Rémi PERRIER (ENEDIS)
+---
+
+- new partner joined okdp meetings : Rémi PERRIER - ENEDIS
+- prepare okdp presentation for the TOSIT day (February 9th at BPCE)
+- work in progress : 1. OKDP WebSite update with more details about okdp and its roadmap 2. build of jupyterhub/jupyterlab images 3. choice between ArgoCD and FluxCD for CD implementation

users/2024/20240228.md

@@ -0,0 +1,23 @@
+---
+date: 2024-02-28
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- Back to TOSIT Data Day Event
+- Front OKDP : Angular or React can be used for the front dev and SpringBoot for the back
+- Superset: integrate its integration into the roadmap for dataViz
+- Go-lang : can be used for transverse automation layers for OKDP
+- OKDP Web site up to date with last version by Moncef L.
+- Spark: advanced Metastore, Spark-ui and spark-hs integration by Idir I.
+- Spark: FluxCD deployment integration by Serge A.
+- FluxCD : OKDP integration done by Serge A. with some features in progress : security, multi-tenancy, etc.
+- ArgoCD can be used for applicative deployments
+- Trino : first integration in progress by Moncef L.

users/2024/20240306.md

@@ -0,0 +1,22 @@
+---
+date: 2024-03-06
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Edouard ROUSSEAUX (EDF)
+---
+
+- RoadMap (to be published via the web site) updated with integration of superset :
+  1.  Spark integration
+  2.  Metastore integration (hive metastore in progress)
+  3.  Trino integration (design and requirements)
+  4.  SuperSets integration (to-do)
+  5.  Sanbox implementation (to-do)
+- Implementation of an authentication module for spark history server (demo to be planned for tosit members)
+- implementation of github discussions with decisions and technical details

users/2024/20240313.md

@@ -0,0 +1,28 @@
+---
+date: 2024-03-13
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Edouard R. (EDF)
+  - Lionel H. (Orange)
+  - Pierre S. (Adaltas)
+---
+
+- Olivier M.: okdp roadmap to be updated
+- Mehdi B.: new project to manage authentication in Spark History and its release management
+- Pierre S.: topics to share for new contributions
+- Pierre S.: Proposal for other DEV environments with other kubernetes distributions
+- Lionel H.: Orange contributions:
+  - Spark HS to be tested in the Orange context
+  - dev of an UI to list spark apps via APIs kube + rex to be planned
+  - share Spark images with the community
+- presentation to be planned :
+  - March 27th: SparkHistory and okdp projects
+  - April 3: build of spark images and Spark History
+  - April 4: okdp demo for TOSIT members

users/2024/20240320.md

@@ -0,0 +1,23 @@
+---
+date: 2024-03-20
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Nicolas V. (DGFIP)
+  - Pierre S. (Adaltas)
+---
+
+- Olivier M.: confirmation of the following meetings: Thursday April 11 for the demo from 9 a.m. to 10 a.m.
+- Olivier M.: DGFIP is looking for alternatives to array (potentially supersets)
+- Moncef L.: Update roadmap on the web-site and remove of react
+- Moncef L.: implementation of the Helm chart for Spark History Server
+- Serge A.: implementation of the Helm chart for Hive Metastore and postgres
+- Idir I.: build of spark images
+- Mehdi & Chung: finalizing the setup of the test environment with Kind
+- Moncef: creation of an OKDP channel on the TOSIT mattermost to discuss current technical subjects

users/2024/20240327.md

@@ -0,0 +1,27 @@
+---
+date: 2024-03-27
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Nicolas V. (DGFIP)
+  - Pierre S. (Adaltas)
+  - Lionel H. (Orange)
+---
+
+- Lionel H.: working on a build container for constant dev env.
+- Lionel H.: proposal for a PR on the Spark History authentication project to make the identifier configurable for the JWT token (between sub and email)
+- Olivier H.: Participation in the Big Data & AI event with a stand in November 2024.
+- Pierre S.: working on multi-tenancy feature in SuperSet
+- Serge A.: hive metastore integration and release management
+- Serge A.: using FluxCD to deploy requirements (cert manager, metalLb, etc.) in Gitops mode
+- Idir I. : build of Spark images
+- Moncef L.: build factory
+- Mehdi B.: preparation of OKDP slides
+- Lionel H.: contribution in the form of PR on the Spark build with: dependency checking + Log4j patch + jre deployment for jdk 11 and 17 + kerberos deployment + tests + observability configuration
+- Moncef L.: creation of an OKDP sub-channel for specific discussions

users/2024/20240516.md

@@ -0,0 +1,18 @@
+---
+date: 2024-05-16
+type: users
+attendees:
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Lionel H. (Orange)
+---
+
+- Moncef L.: integration of Trino in progress via a Helm chart
+- Idir I.: integration of Superset with a focus on security
+- Serge A.: implementation of the OKDP sandbox via FluxCD
+- Moncef L.: preparation of the June demo in progress
+- Moncef L.: integration of the Spark History authentication module at the Helm chart level
+- Lionel H.: successful testing and integration of the Spark auth module in the Orange context

users/2024/20240522.md

@@ -0,0 +1,20 @@
+---
+date: 2024-05-22
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Stephane B. (Orange)
+  - Lionel H. (Orange)
+---
+
+- Olivier M.: preparation of the presentation for the Data & AI Leaders
+- Confirmation of participation in Tosit Day on Thursday 13/06 for an OKDP presentation (Olivier M., Idir I. and Mehdi B. on site and Moncef L. remotely)
+- Olivier M.: contact BPCE and Arkea on a potential collaboration on Trino
+- Olivier M.: OKDP internal presentation on 07/06 and slides to be reviewed by Mehdi B. and Moncef L.
+- Moncef L.: preparation of the demo and progress on current topics: Trino (Moncef L.), Sandbox via FluxCD (Serge A.) and Superset (Idir I.)
+- Stephane B. and Lionel H.: integration of the spark HS authentication module and review of the integration with OIDC in order to avoid declaring a wlidcard for SparkUI redirects
+- Lionel H.: internal validation for Orange's participation in the OKDP presentation

users/2024/20240529.md

@@ -0,0 +1,21 @@
+---
+date: 2024-05-29
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Lionel H. (Orange)
+  - Edouard R. (EDF)
+  - Nicolas V. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- Olivier M.: preparation of the presentation for the Data & AI Leaders
+- Olivier M.: OKDP internal presentation on 07/06 and slides to be reviewed by Mehdi B. and Moncef L.
+- Olivier M.: collaboration with an inter-ministerial group around superset
+- Edouard R.: proposal for a demo of EDF's containerized Data platform on 06/19/24
+- Olivier M.: ongoing update of the website by Mehdi B. with new events (in progress)
+- Lionel H.: creation of a new github issue ( followed by Idir I. ) around Spark History Server and its authentication module in order to make it more generic

users/2024/20240605.md

@@ -0,0 +1,24 @@
+---
+date: 2024-06-05
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Lionel H. (Orange)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- Update of the OKDP web site with various events (PR ongoing)
+- Presentation of OKDP to Arkéa on Thursday, 30/05 (done)
+- Presentation of EDF's proprietary solution on Spark, Wednesday, 19/06/2024.
+- Testing of the spark authentication module by new users
+- Github issue created by Orange on the Authentication/Ldap (IDP) part on GitHub (ongoing)
+- DINUM (BlueHats workshop), Friday, June 7, 2024, from 11:00 to 12:30 (Jupyter, Spark + Spark History authentication module, SandBox, SuperSet, end-to-end build and deployment, etc.)
+- TOSiT-Day on the afternoon of 13/06, fill in the framadate (done)
+- Roundtable at the Data & AI Leaders on 27 and 28/11 (in progress)
+- Packaging SuperSet by Idir, publication and documentation to follow
+- Build OKDP: Orange will approach DIODE (DGFiP) to work and provide feedback on the Build part
+- Participation of partners (BPCE and Orange) in the Hoche Data & AI, Data Mesh, Data Cloud Platform, Data Fabric & Architectures Meetups

users/2024/20240612.md

@@ -0,0 +1,18 @@
+---
+date: 2024-06-12
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- Update the OKDP website with our various events (done)
+- DINUM (BlueHats workshop): Friday, June 7, 2024, from 11:00 AM to 12:30 PM (Jupyter, Spark + Spark History authentication module, SandBox, SuperSet, end-to-end build and deployment, etc.). There were 40 participants (done)
+- TOSiT-Day on 06/13 in the afternoon.
+- SuperSet: Meeting on Tuesday, 06/18, for a new collaboration on the support and REX part
+- Presentation of EDF’s proprietary solution on Spark, Wednesday, 06/19/2024.
+- TODO : MLOPS: integration of kubeflow, MLFlow, Onyxia, other UIs, etc.

users/2024/20240703.md

@@ -0,0 +1,14 @@
+---
+date: 2024-07-03
+type: users
+attendees:
+  - Serge A. (DGFIP)
+  - Idir I. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Lionel H. (Orange)
+---
+
+- Serge A.: integration of topolvm with vgs management via helm chart for the sandbox
+- Idir I.: create github repo for hive/superset images and their sources
+- Lionel H.: propose a solution to secure Docker images (currently being tested on Orange side)

users/2024/20240717.md

@@ -0,0 +1,16 @@
+---
+date: 2024-07-17
+type: users
+attendees:
+  - Nicolas V. (DGFIP)
+  - Idir I. (DGFIP)
+  - Serge A. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Lionel H. (Orange)
+---
+
+- Idir: split helm charters in github (done)
+- Idir: start the architecture choices on the front part with a preference for go-lang for its watcher support
+- Serge A.: integration of topolvm for minio/sandbox (done)
+- Serge A.: automation of minio resources creation in declarative mode (in progress)

users/2024/20240724.md

@@ -0,0 +1,18 @@
+---
+date: 2024-07-24
+type: users
+attendees:
+  - Nicolas V. (DGFIP)
+  - Idir I. (DGFIP)
+  - Serge A. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Olivier M. (DGFIP)
+  - Guillaume M. (DGFIP)
+---
+
+- Updating slides with new UI features
+- preparation of the internal presentation with the new roadmap
+- Discussion with other UX design teams
+- Minio integration and automatic resource management in progress
+- split Helm charts on github finalized

users/2024/20240731.md

@@ -0,0 +1,19 @@
+---
+date: 2024-07-31
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Nicolas V. (DGFIP)
+  - Idir I. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Lionel H. (Orange)
+---
+
+- Idir I.: Split Helm charts (done)
+- Idir I.: REST API in Golang + Angular for the UI (in progress)
+- Serge A.: MinIO and declarative resource management + RBAC (bucket + policies) (in progress)
+- Moncef L.: Take over the Kubernetes clusters topic on private cloud
+- Lionel H.: New ticket on the Spark authentication module

users/2024/20240911.md

@@ -0,0 +1,9 @@
+---
+date: 2024-09-11
+type: users
+---
+
+- Idir I.: Develop some Angular models for the OKDP UI (in progress)
+- Serge A.: MinIO and declarative resource management + RBAC (bucket + policies) (in progress)
+- Lionel H.: New [ticket ](https://github.com/OKDP/okdp-spark-auth-filter/issues/34)on the Spark authentication module
+- Idir I. : Choosing the backend for the UI, potentially in GoLang.

users/2024/20240918.md

@@ -0,0 +1,28 @@
+---
+date: 2024-09-18
+type: users
+attendees:
+  - Olivier M. (DGFIP)
+  - Idir I. (DGFIP)
+  - Nicolas V (DGFIP)
+  - Mehdi B. (DGFIP)
+  - Chung N. (DGFIP)
+  - Guillaume M. (DGFIP)
+  - Moncef L. (DGFIP)
+  - Serge A. (DGFIP)
+  - Lionel H. (Orange)
+---
+
+- Backend choice for the UI:
+  1. GlassKube: https://glasskube.dev/
+  2. Timoni: https://timoni.sh/
+  3. Open Component Model OCM: https://ocm.software/
+  4. Open Application Model (Microsoft, Alibaba): https://oam.dev/ -> kubevela: https://kubevela.io/
+  5. kusionstack: https://www.kusionstack.io/
+- Criteria:
+  1. Integration with fluxcd as the first selection criterion
+  2. A user-friendly frontend
+  3. Multi-cluster support
+  4. CNCF incubation
+- Serge A.: Minio integration in progress for resource management + LDAP integration
+- Lionel H.: Spark issue in progress