From 9bda5fffffe200fbe9158b8d108fd49afea32be2 Mon Sep 17 00:00:00 2001 From: ThanKarab Date: Mon, 15 Jun 2026 18:35:49 +0300 Subject: [PATCH 1/2] doc: 9.1 README updates --- README.md | 44 ++++++++++++++++++++++++------------- documentation/Components.md | 10 ++++----- 2 files changed, 34 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 98905a1..fdcd00d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ A powerful federated data processing and analysis system that preserves patient # Table of Content - [Preamble](#preamble) -- [9.0 Release](#mip-90-release--major-updates) +- [9.1 Release](#mip-91-release--major-updates) - [Components](#components) - [Deployment](#deployment) - [Federated Analysis Algorithms](#federated-analysis-algorithms) @@ -22,24 +22,38 @@ This information is evolving along with the MIP so please make sure you consult In the following sections, links and references to useful information is made available. -# MIP 9.0 Release – Major Updates +# MIP 9.1 Release – Major Updates -## **Platform-UI - New user interface** -The new user interface has been developed from scratch, to provide the user a more native experience to federated learning. -The algorithm visualizations have also been reworked to provide a more meaningful view in the results. +## **Broader federated analytics** -## **Platform-Backend - Improvements to support the new user interface** -Other than required improvements to support the new user interface, the backend has also rewritten the communication -flow with the engine. +MIP 9.1 expands the range of analyses that can be performed across distributed +clinical datasets without moving patient-level data. The release adds support +for more advanced statistical workflows, including survival analysis, +association testing, histogram exploration, outlier reporting, and mixed-effects +modeling, while refreshing the documentation for the core analysis portfolio. -## **Exaflow – Federated Engine Revamp** +## **Improved analysis workflow** -Exaflow, the former Exareme2 engine, underwent major improvements, allowing for the integration of other federated learning engines. +The analysis experience has been refined to make experiment setup and result +review clearer. Users get better guidance when selecting variables and +configuring algorithms, clearer feedback when an analysis cannot run with the +selected inputs, and improved result views with more consistent tables, charts, +labels, and export actions. -Closely integrated in exaflow is exareme3, our federated learning engine, using duckdb for data loading, grpc for -communications and an aggregation server, allowing for a new paradigm in federated learning. +## **Built-in data preparation options** -Flower is also integrated, allowing for flower-developed algorithm integration as-is. +MIP 9.1 introduces more data preparation options directly into the analysis +workflow. Missing-value handling, outlier handling, and longitudinal +transformations can be configured before running an analysis, reducing the need +for manual preparation outside the platform. + +## **More maintainable and deployment-ready platform** + +The platform has been updated with newer component versions, refreshed +deployment defaults, and reviewed documentation for deployment, data management, +architecture, and onboarding. The release also improves alignment between the +user interface, backend services, and federated analysis engine, making the +platform easier to operate and evolve. # Components @@ -56,7 +70,7 @@ The MIP comes with a single code base but with two modes of deployment. One for This includes documentation on existing algorithm federation approach as well as information related to creating a new algorithm. - [Available federated analysis algorithms](documentation/algorithms.md) -- [Exaflow Analytic Engine](https://github.com/madgik/exaflow/tree/1.0.0) +- [Exaflow Analytic Engine](https://github.com/madgik/exaflow/tree/1.1.0) # Data Management @@ -68,7 +82,7 @@ A detailed user guide for Data Quality Control tool can be found here: - [Data Quality Control Tool Guide](https://github.com/HBPMedical/DataQualityControlTool/wiki) Data Catalog is a component of the Medical Informatics Platform (MIP) for the EBrains. It enables seamless management, visualization, and access to data models and medical conditions. - - [Data Catalog Guide](https://github.com/Medical-Informatics-Platform/datacatalog/tree/1.2.1) + - [Data Catalog Guide](https://github.com/Medical-Informatics-Platform/datacatalog) # High Level Description diff --git a/documentation/Components.md b/documentation/Components.md index fc943c0..b13592f 100644 --- a/documentation/Components.md +++ b/documentation/Components.md @@ -1,12 +1,12 @@ -## Components in MIP version 9.0 +## Components in MIP version 9.1 These are the repositories with all the independent components that MIP is composed of: - The "Deployment Stack", which enable the installation of the MIP both locally and in a federation is part of the current repo. [Deployment Package](../deployment) - The "MIP Infrastructure", MIP KaaS deployment infrastructure repo based on ArgoCD deployments [https://github.com/Medical-Informatics-Platform/mip-infra](https://github.com/Medical-Informatics-Platform/mip-infra) -- The "Platform-UI" is the interface of MIP: [https://github.com/Medical-Informatics-Platform/platform-ui/tree/1.0.1](https://github.com/Medical-Informatics-Platform/platform-ui/tree/1.0.1) -- The "Platform Backend API" supports the Web App : [https://github.com/Medical-Informatics-Platform/platform-backend/tree/9.1.1](https://github.com/Medical-Informatics-Platform/platform-backend/tree/9.1.1) -- The "Exaflow Analysis Engine" offers federated analysis capabilities: [https://github.com/madgik/exaflow/tree/1.0.0](https://github.com/madgik/exaflow/tree/1.0.0) +- The "Platform-UI" is the interface of MIP: [https://github.com/Medical-Informatics-Platform/platform-ui/tree/1.2.0](https://github.com/Medical-Informatics-Platform/platform-ui/tree/1.2.0) +- The "Platform Backend API" supports the Web App : [https://github.com/Medical-Informatics-Platform/platform-backend/tree/9.3.0](https://github.com/Medical-Informatics-Platform/platform-backend/tree/9.3.0) +- The "Exaflow Analysis Engine" offers federated analysis capabilities: [https://github.com/madgik/exaflow/tree/1.1.0](https://github.com/madgik/exaflow/tree/1.1.0) #### Additional Standalone Tools: - Quality Control Tools: [https://github.com/HBPMedical/DataQualityControlTool/tree/v.4.1](https://github.com/HBPMedical/DataQualityControlTool/tree/v.4.1) -- DataCatalog: [https://github.com/Medical-Informatics-Platform/datacatalog/tree/1.2.1](https://github.com/Medical-Informatics-Platform/datacatalog/tree/1.2.1) +- DataCatalog: [https://github.com/Medical-Informatics-Platform/datacatalog/tree/2.0.0](https://github.com/Medical-Informatics-Platform/datacatalog/tree/2.0.0) From 71420b0d948883515f13aabfca0247ef7c07ff55 Mon Sep 17 00:00:00 2001 From: ThanKarab Date: Mon, 15 Jun 2026 18:38:04 +0300 Subject: [PATCH 2/2] doc: Client facing README --- README.md | 84 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 43 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index fdcd00d..a03228a 100644 --- a/README.md +++ b/README.md @@ -1,100 +1,102 @@ -# Technical Documentation for the Medical Informatics Platform (MIP) +# Medical Informatics Platform (MIP) -A powerful federated data processing and analysis system that preserves patient privacy. More info on the [MIP Website](https://ebrains.eu/data-tools-services/medical-analytics/medical-informatics-platform) +A federated analytics platform for clinical research networks that need to +collaborate on sensitive health data without centralizing patient-level records. +More information is available on the [MIP Website](https://ebrains.eu/data-tools-services/medical-analytics/medical-informatics-platform). -# Table of Content +# Table of Contents -- [Preamble](#preamble) +- [About MIP](#about-mip) - [9.1 Release](#mip-91-release--major-updates) -- [Components](#components) +- [What MIP Includes](#what-mip-includes) - [Deployment](#deployment) - [Federated Analysis Algorithms](#federated-analysis-algorithms) - [Data Management](#data-management) -- [High Level Description](#high-level-description) - [Architecture](#architecture) +- [Onboarding](#onboarding) -# Preamble +# About MIP -This repository is an index for a collection of documents and other sources of information related to the Medical Informatics Platform. The intended audience comprises developers, technical deployment and support teams, and anyone else with a deep technical interest in the functioning of the MIP. Its purpose is to facilitate access to a range of information necessary to represent the current state of the MIP. It should provide adequate material for suitably qualified staff to understand how the MIP works, and to develop, deploy and operate the MIP. +The Medical Informatics Platform enables privacy-preserving analysis across +distributed clinical datasets. Hospitals and data providers keep patient-level +data within their local governance boundaries, while researchers can run +federated statistical and machine-learning analyses across participating sites. -This information is evolving along with the MIP so please make sure you consult the document version that is relevant to the indented, or preferably, latest version of the Medical Informatics Platform. - -In the following sections, links and references to useful information is made available. +This repository collects the technical, deployment, data-management, and +architecture documentation needed to understand, deploy, and operate MIP. # MIP 9.1 Release – Major Updates ## **Broader federated analytics** MIP 9.1 expands the range of analyses that can be performed across distributed -clinical datasets without moving patient-level data. The release adds support -for more advanced statistical workflows, including survival analysis, -association testing, histogram exploration, outlier reporting, and mixed-effects -modeling, while refreshing the documentation for the core analysis portfolio. +clinical datasets without moving patient-level data. The release adds support for +advanced statistical workflows such as survival analysis, association testing, +histogram exploration, outlier reporting, and mixed-effects modeling, while +refreshing the documentation for the core analysis portfolio. -## **Improved analysis workflow** +## **Smoother analysis workflow** The analysis experience has been refined to make experiment setup and result -review clearer. Users get better guidance when selecting variables and -configuring algorithms, clearer feedback when an analysis cannot run with the -selected inputs, and improved result views with more consistent tables, charts, -labels, and export actions. +review clearer. Users get better guidance when selecting variables, clearer +feedback when an analysis cannot run with the selected inputs, and improved +result views with more consistent tables, charts, labels, and export actions. -## **Built-in data preparation options** +## **Built-in data preparation** MIP 9.1 introduces more data preparation options directly into the analysis workflow. Missing-value handling, outlier handling, and longitudinal transformations can be configured before running an analysis, reducing the need for manual preparation outside the platform. -## **More maintainable and deployment-ready platform** +## **Deployment-ready platform** -The platform has been updated with newer component versions, refreshed -deployment defaults, and reviewed documentation for deployment, data management, -architecture, and onboarding. The release also improves alignment between the -user interface, backend services, and federated analysis engine, making the -platform easier to operate and evolve. +The platform has been updated with refreshed deployment defaults and reviewed +documentation for deployment, data management, architecture, and onboarding, +making MIP easier to operate and evolve across research infrastructures. -# Components +# What MIP Includes -The main [MIP building blocks](documentation/Components.md) are listed along with the respective repositories that host them. +MIP combines a web interface, backend services, federated analysis engine, +deployment tooling, and supporting data-management tools. The main +[MIP building blocks](documentation/Components.md) are listed with the +repositories that host them. # Deployment -The MIP comes with a single code base but with two modes of deployment. One for local usage only, and one that enables the creation of a federation of nodes. Information on the different deployment approaches can be found in the following location: +MIP supports both local development deployments and Kubernetes-based deployments +for production-like or federated installations. - [Deployment Documentation](deployment) # Federated Analysis Algorithms -This includes documentation on existing algorithm federation approach as well as information related to creating a new algorithm. +The algorithm documentation describes the available federated analyses and links +to the underlying Exaflow analytic engine documentation. - [Available federated analysis algorithms](documentation/algorithms.md) - [Exaflow Analytic Engine](https://github.com/madgik/exaflow/tree/1.1.0) # Data Management -For all details relating to the Data Factory, how to manage your data and process it for use with the MIP, please consult the following document +The data-management documentation explains how datasets and metadata are prepared +for use in MIP. - [Data Management Guide](documentation/MIP_Data_management_documentation.md) -A detailed user guide for Data Quality Control tool can be found here: +A detailed user guide for the Data Quality Control tool can be found here: - [Data Quality Control Tool Guide](https://github.com/HBPMedical/DataQualityControlTool/wiki) -Data Catalog is a component of the Medical Informatics Platform (MIP) for the EBrains. It enables seamless management, visualization, and access to data models and medical conditions. +Data Catalog is a component of MIP for EBRAINS. It enables management, +visualization, and access to data models and medical conditions. - [Data Catalog Guide](https://github.com/Medical-Informatics-Platform/datacatalog) -# High Level Description - -For a high-level description of the MIP please consult: - -- [The MIP: A powerful federated data processing and analysis system that preserves patient privacy](https://ebrains.eu/data-tools-services/medical-analytics/medical-informatics-platform) on EBRAINS research infrastructure - # Architecture - [High-level view of the architecture](documentation/Architecture.md), the main building blocks and data flows. -# Onboarding guidance +# Onboarding - [Onboarding to the Medical Informatics Platform MIP](https://wiki.ebrains.eu/bin/view/Collabs/onboarding-to-the-mip/) on EBRAINS Collaboratory