Mkdocs Website

.github/workflows/docs.yml

@@ -0,0 +1,43 @@
+name: Build and deploy documentation
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install ffmpeg for notebook animations
+        run: sudo apt-get update && sudo apt-get install -y ffmpeg
+      - name: Install package and docs dependencies
+        run: pip install ".[docs]"
+      - name: Build documentation
+        run: mkdocs build
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -6,6 +6,7 @@
 __pycache__
 /large_files/
 *.h5
+!/docs/data/*.h5
 venv
 profile*
 *-checkpoint.ipynb

.pre-commit-config.yaml

@@ -11,4 +11,8 @@ repos:
   rev: v4.5.0
   hooks:
     - id: check-added-large-files
-      args: ['--maxkb=2000'] 
+      args: ['--maxkb=2000']
+- repo: https://github.com/kynan/nbstripout
+  rev: 0.8.1
+  hooks:
+    - id: nbstripout 

README.md

@@ -31,7 +31,7 @@ The `Experiment` objects may be saved to a standardized HDF5-backed format for l
 
 Learn more about the containers in the following example notebook.
 
-[container_objects.ipynb](example_notebooks/container_objects.ipynb)
+[container_objects.ipynb](https://electronsandstuff.github.io/ParetoBench/container_objects/)
 
 
 ## Plotting
@@ -48,13 +48,13 @@ Tools for plotting the data from multi-objective optimization algorithms are als
 
 
 See more information in the following notebooks.
-- [plotting_populations.ipynb](example_notebooks/plotting_populations.ipynb)
-- [plotting_histories.ipynb](example_notebooks/plotting_histories.ipynb)
+- [plotting_populations.ipynb](https://electronsandstuff.github.io/ParetoBench/plotting/plotting_populations/)
+- [plotting_histories.ipynb](https://electronsandstuff.github.io/ParetoBench/plotting/plotting_histories/)
 
 <table><tr>
-<td><img src="assets/plotting-decision-vars-bounds.png" width="250"/></td>
-<td><img src="assets/plotting-2d-history-attainment.png" width="250"/></td>
-<td><img src="assets/plotting-3d-attainment-surface.png" width="250"/></td>
+<td><img src="docs/assets/plotting-decision-vars-bounds.png" width="250"/></td>
+<td><img src="docs/assets/plotting-2d-history-attainment.png" width="250"/></td>
+<td><img src="docs/assets/plotting-3d-attainment-surface.png" width="250"/></td>
 </tr></table>
 
 ## Benchmark Problems
@@ -84,14 +84,14 @@ When possible, the benchmark problems include analytical Pareto fronts.
 
 <table>
 <tr>
-<td><img src="assets/pareto_front_zdt1.png" width="250"/></td>
-<td><img src="assets/pareto_front_zdt2.png" width="250"/></td>
-<td><img src="assets/pareto_front_zdt3.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_zdt1.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_zdt2.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_zdt3.png" width="250"/></td>
 </tr>
 <tr>
-<td><img src="assets/pareto_front_wfg1.png" width="250"/></td>
-<td><img src="assets/pareto_front_wfg2.png" width="250"/></td>
-<td><img src="assets/pareto_front_cf5.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_wfg1.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_wfg2.png" width="250"/></td>
+<td><img src="docs/assets/pareto_front_cf5.png" width="250"/></td>
 </tr>
 </table>
 

example_notebooks/duplicate_detection.ipynb → docs/analysis/duplicate_detection.ipynb

@@ -10,7 +10,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -20,18 +20,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Number of individuals: 6\n",
-      "Number of unique individuals: 3\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Create a population with exact duplicates and calculate the number of unique individuals\n",
     "pop = pb.Population(f=np.array([[1, 2, 3], [1, 2, 3], [1, 2, 3], [2, 3, 4], [3, 4, 5], [1, 2, 3]], dtype=float))\n",
@@ -41,18 +32,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Number of individuals: 6\n",
-      "Number of unique individuals: 3\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Create some individuals with noise in the last few digits\n",
     "pop = pb.Population(\n",

docs/analysis/inverted_generational_distance_analysis.ipynb

@@ -0,0 +1,103 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Analysis of Inverted Generational Distance\n",
+    "This notebook demonstrates how optimization algorithm benchmarking data can be loaded from disk, metrics (such as inverted generational distance) can be calculated for all reported populations, and the resulting data analyzed for comparisons between experiements. We demonstrate the process by comparing three algorithms against eachother on a single test problem. The resulting comparisons are displayed and it is shown how the table can be exported to latex for publication."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import paretobench as pb"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%time\n",
+    "# Evaluate the IGD metric on all populations stored in three saved experiments\n",
+    "df = pb.eval_metrics_experiments(\n",
+    "    [\"../data/NSGAII.h5\", \"../data/CPSMOEA.h5\", \"../data/ParEGO.h5\"],\n",
+    "    pb.InvertedGenerationalDistance(),\n",
+    "    n_procs=8,\n",
+    ")\n",
+    "df.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Calculate aggregate statistics of the metric on each history object including individuals up until a budget of `max_feval`\n",
+    "# function evaluations has been exceeded. Use the Wilcoxon rank sum test to compare algorithms against ParEGO.\n",
+    "dfa = pb.aggregate_metrics_feval_budget(df, max_feval=300, wilcoxon_idx=2)\n",
+    "dfa.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Turn the aggregate stats into a table comparing metric values for each problem between algorithms.\n",
+    "dfc = pb.construct_metric_comparison_table(\n",
+    "    dfa,\n",
+    "    problem_params=\"n\",\n",
+    "    mean_fmt_kwargs={\"precision\": 3, \"exp_digits\": 1},\n",
+    "    std_fmt_kwargs={\"precision\": 3, \"exp_digits\": 1},\n",
+    ")\n",
+    "dfc"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Demonstrate printing to latex\n",
+    "print(pb.comparison_table_to_latex(dfc))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}