What is mzb-suite?

mzb-suite is an image processing pipeline for lab images of macrozoobenthos (MZB), partially automating data extraction from images.

What can it be used for?

• Segment individual MZB organisms from large-pane images into individual clips (unsupervised).
• Extract total length of individual organisms (unsupervised), and both length and head width for selected taxa (supervised).
• Use pre-trained Machine Learning (ML) models to predict coarse-grained identity for selected of taxa.
• Assisted ML model re-training on other taxa using user-provided annotations.
• Assists in organising data in a folder structure that is easy to navigate, and that can be used for further analysis.
• Worked examples and documentation are provided to adapt the pipeline to users' projects.

Who can use it?

Anyone that wants to process images of MZB or other organisms acquired in a lab setting, with fixed focal length, uniform background and lighting conditions. The pipeline can handle small amounts of noise in the images, but any debris similar in size to the organisms of interest will not be filtered out, making this pipeline unsuitable for images taken in the field.

---

Get started

You can get the trained models and some demo data here: https://doi.org/10.5281/zenodo.17581222

Please see the documentation, it explains everything relating to the package. You can jump to directly to sections here:

1. Installation

2. Workflow and Models

3. Examples

4. Processing scripts

5. mzbsuite module reference

---

Project info

Contributors & contacts

• Luca Pegoraro (WSL) - luca.pegoraro@wsl.ch
• Michele Volpi (SDSC) - mivolpi@ethz.ch

Full authors and contribution details in list in this file.

Issues & feature requests

If you encounter a reproducible bug, please prepare a MWE and open an Issue here where we can track it.

Development time is limited for this project, so no major new features are being implemented at the moment. You are of course welcome to open a pull request, we will try to examine it quickly!

How to cite

coming soon...

Changelog

Find it here.

ToDo list

Features under consideration

• Add original image file name as a separate column in all csv outputs
• Add options to provide multiple px/mm conversion rates, one for each image (i.e. parse csv file as dictionary to pass as arguments)
• Add module for evaluations, and for generating plots
• Change scale exclusion parameter in configs & documentation so that you select square where scale is to exclude, instead of selecting the pixels to keep in the image. -- changed config file and docs

General

• Move taxonomy file location onto running parameters from configuration file
• Clean unused workflow sh files and comments within them
• worflows/full_pipeline_custom.sh has no execute permission in repo
• Reduce images in example dataset
• Set up push mirror GitLab → GitHub (without LFS files)
• Renku workflows for the different modules (or add renku run ... in front of command in sh scripts) → not sure about this one (yet)

Functionality and data

• Double check that excluding the millimetre/colour scale in images works properly in cv2, x and y are flipped compared to most other implementations!
• Add the Dübendorf data use case to check consistency for multiple projects?
• Make example dataset for supervised skeletonization model finetuning
• Update workflows/run_finetune_skeletonization.sh accordingly

JupyterLab via Renku

• JupyterLab screws up all the filepaths… Working dir is home/jovyan/work
• Notebooks not picking up correct conda env in JupyterLab...
• Cannot run .sh script in the console directly if in cwd???
• Notebooks don’t pick up conda env…
• Cannot load correct kernel (with mzbsuite) in Jupyter notebooks from Renku interactive session...
• Finding and changing filepaths in Jupyter notebooks is difficult for users…

Documentation

• Put docs on ReadTheDocs and/or GitHub Pages
  • Configure ReadTheDocs integration
  • Configure automated docs build in GitHub Pages from mirror repo -- unsupported
  • See if you can pull the commit name and reconstruct the docker image name on renku dynamically in the documentation (source/files/installing.rst)...
  • Otherwise just grab a recent one that build correctly and stick with that
• Add "Quickstart" section with tutorial in the documentation
• in the documentation, docs/source/files/workflow_models.rst, add section called "Supervised Skeleton Prediction" and explain model architectures used for supevised skeleton prediction (this should also fix Sphinx build warnings for missing refs).
• Merge ToDo in README.md and Evernote.

Notebooks

• model retraining not working in notebook interactive environment...
• compress long code blocks?
• direct links to documentation within markdown cells in notebooks
• make notebook for supervised skeletons finetuning
  • conflict with Jupyter notebook environment...
• Fix notebooks outputs:
  • segmentation.ipynb replace plots in-place while running instead of generating new ones.
  • skeletonizatn_unsupervised.ipynb replace plots in-place while running instead of generating new ones.
  • skeletonization_supervised_inference.ipynb returns empty predictions in notebook...
  • classification_finetune.ipynb last cell (actually retraining the model) might not be compatible with an interactive environment...

Figures to make for paper

• Class (im)balance for flume MZB samples (classification and skeletons)
• Accuracy for classification model - results/project_portable_flume/class_convnext-small-v0_validation_set
• Accuracy for supervised skeletonization model (length and head width) - results/project_portable_flume/skseg_mit-b2-v1_validation_set
• Accuracy for unsupervised skeletonization (length)

---

OLD COMPLETED TODO ITEMS - v0.1.0 - Not in any priority

• ALIGN ALL LEARNING SET IMAGES TO NEW PIPELINE, from pngs to jpgs
• Check measures of supervised skeletonizations (length and width) and compare to manual annotations
• LICENSE and AUTHORS and CITATION placeholders
• update the skeleton files: image blobs are now named differently!
• Fix how save folders are passed for the supervised skeletonization
• check env and pandas in it use, build setupy.py
• Add notebooks for plotting of results, images, etc.
• Check all docstrings and potentially build documentation into html
• Add a README.md to the data folder
• Spend some time in thinking whether it is better to have one big config file, or one config file per module, or one config file per script.
• check wandb accounts and api for loggers (added support for tensorboard, running locally)