diff --git a/README.md b/README.md index 7d5bff3..f2b69b9 100644 --- a/README.md +++ b/README.md @@ -378,17 +378,17 @@ x_max = X.max(axis=0) x_min = X.min(axis=0) X = (X - x_min) / (x_max - x_min) -# Instantiate the OI object -OI = OverlapIndex() +# Instantiate the OI object with a reproducible centroid fit +OI = OverlapIndex(kmeans_kwargs={"random_state": 0}) # Calculate the Overlap Index OI.fit(X, y) print(OI.index) - -# Output: -# 0.9266666666666666 ``` +The exact fitted score depends on backend settings and library versions. Set a +random seed, as above, whenever a result must be repeatable. + Additional runnable examples are available in the `examples/` directory. --- @@ -605,4 +605,4 @@ This package is intended for researchers and practitioners working on: The source code is licensed under the GNU Affero General Public License v3.0 or later (AGPLv3-or-later). Commercial licenses are available; please -contact the maintainer through GitHub. \ No newline at end of file +contact the maintainer through GitHub. diff --git a/docs/backends/index.md b/docs/backends/index.md index 9500c02..71383a9 100644 --- a/docs/backends/index.md +++ b/docs/backends/index.md @@ -31,6 +31,9 @@ using the same backend and settings. - Only Fuzzy and Hypersphere ARTMAP preserve learned state across `partial_fit` calls. - Set backend random seeds when results need to be reproducible. +- For very large ARTMAP runs, a lower `rho` in roughly the `0.5` to `0.7` + range can reduce prototype growth and improve runtime. Treat that range as a + starting point and validate the resulting resolution on representative data. ```{toctree} :hidden: diff --git a/docs/concepts.md b/docs/concepts.md index 8c1ea5f..43a6f12 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -14,6 +14,12 @@ copied once for every positive label during prototype fitting. For an ARTMAP backend, supervised incremental learning creates and updates the label-owned prototypes directly. +The scoring layer is not tied to a single prototype geometry: centroid, +landmark-ball, Fuzzy ART, and hypersphere representations all expose the same +class-owned best-matching-unit interface. Geometry still matters to the +measured value, so comparisons should keep the backend and its resolution +fixed. + `predict(X)` returns these global prototype IDs. It is therefore useful for inspecting the fitted representation, but it does **not** return predicted class labels. @@ -35,6 +41,10 @@ rows and denominators differ. On ordinary single-label data, $N_{a,b}$ is the support of $a$. On multi-label data, it contains only rows where $a$ is present and $b$ is absent. +In an incremental ARTMAP run these activation counts are updated as labeled +samples arrive. Offline backends compute the same bookkeeping after fitting +their class-owned prototypes to the supplied batch. + ## Per-label and global aggregation The per-label score is its worst evaluable competitor: diff --git a/docs/continuous_targets.md b/docs/continuous_targets.md index d52f6b0..df52d7b 100644 --- a/docs/continuous_targets.md +++ b/docs/continuous_targets.md @@ -28,6 +28,15 @@ The estimator is offline-first. MiniBatchKMeans, KMeans, and BallCover are supported; `partial_fit` refits on the supplied batch and does not retain continuous-target state across calls. +KMeans and MiniBatchKMeans also accept SciPy sparse feature matrices. Sparse +inputs remain in CSR form through prototype fitting, adjacency scoring, and +permutation-null refits; continuous targets remain dense numeric arrays. + +`random_state` seeds target-cell construction, projection directions, +permutation sampling, and the selected feature backend. An explicit +`random_state` inside `kmeans_kwargs` or `ballcover_kwargs` takes precedence +for that backend. + ## The fitting pipeline 1. Scale the continuous target columns according to `target_scaling`. @@ -82,6 +91,37 @@ Use `adjacency_mode="hard_top1"` for strict single-competitor scoring or backwards comparisons. Keep adjacency settings constant when comparing representations. +## Continuous behavior gallery + +The main synthetic gallery follows three genuinely continuous regression +problems: a smooth latent signal with increasing observation fidelity, a +folded latent trajectory that is progressively unfolded in feature space, and +a continuous covariate that is gradually recovered. It uses eight target +cells, rather than reducing each problem to a high-versus-low split, and +strict nearest-competitor adjacency so the ideal target-ordered endpoints +approach the `1.0` anchor. + +![ContinuousOverlapIndex separation sweeps](../img/continuous_overlap_sweeps.png) + +Regenerate it from the repository root with: + +```bash +poetry run python examples/visualize_continuous_overlap_sweeps.py +``` + +An additional gallery shows a heteroscedastic target field becoming cleaner +and a multivariate oscillator target observed with increasing fidelity: + +![Additional ContinuousOverlapIndex sweeps](../img/continuous_overlap_additional_sweeps.png) + +```bash +poetry run python examples/visualize_continuous_overlap_additional_sweeps.py +``` + +The example scripts use six refit permutations per score to keep the complete +sweeps practical to reproduce. Increase `n_null_permutations` when adapting +them for final quantitative reporting. + ## Permutation nulls - `null_mode="refit_permutation"` rebuilds target cells and feature prototypes @@ -117,6 +157,10 @@ Useful fitted attributes include: - `target_cell_ids_`, `target_cover_`, and `target_distance_` for resolved target-space choices. +The prototype target attributes retain empirical target measures rather than +reducing every prototype to only a mean or variance. This is what allows the +configured Wasserstein distance to compare richer local target distributions. + Continuous and discrete scores use related interpretation anchors but different calibrations. Do not directly compare an OI from one estimator with a COI from the other. diff --git a/docs/diagnostics.md b/docs/diagnostics.md index 4b77cd3..5dfbbd6 100644 --- a/docs/diagnostics.md +++ b/docs/diagnostics.md @@ -83,12 +83,30 @@ This is expected. KMeans, MiniBatchKMeans, and BallCover refit on only the provided batch. Combine the desired training rows and call `fit`, or select an ARTMAP backend when true incremental state is required. +### State reset and explicit continuation + +Full `fit(X, y)` and `score(X, y)` calls always construct a fresh backend. +The lower-level `fit_offline(..., reset_state=False)` form is accepted only +when explicitly continuing an ARTMAP backend. KMeans, MiniBatchKMeans, and +BallCover reject it because global prototype IDs cannot be safely accumulated +across independent offline refits. `ContinuousOverlapIndex` is offline-first +and always resets its fitted state. + ### Indicator labels are not the expected names Indicator columns become integer labels `0..n_labels-1`. Maintain an external column-to-name mapping when original names are needed, or pass collections of named labels instead. +## Parameter validation + +Count, neighborhood, projection, permutation, threshold, and chunk-size +parameters require genuine integers where documented; booleans, strings, and +fractional values are not silently coerced. Radii and temperatures must be +finite and positive. Class-specific dictionaries such as `kmeans_k`, +`ballcover_k`, and `ballcover_radius` must provide a valid entry for every +observed label. + ## Performance checks - Use MiniBatchKMeans as the first choice for large offline datasets. diff --git a/docs/getting_started.md b/docs/getting_started.md index d958cc8..caa5db1 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -21,6 +21,13 @@ python -m pip install "overlapindex[art]" OverlapIndex supports Python 3.9 through 3.14. The ART extra is not imported by the default offline backends. +To install the latest development source directly from GitHub: + +```bash +python -m pip install \ + "git+https://github.com/NiklasMelton/OverlapIndex.git@develop" +``` + ## A first classification score The normal workflow is: prepare features, construct the estimator, fit, then @@ -50,6 +57,35 @@ print("per label:", dict(oi.singleton_index)) calculation can be written as `score = oi.add_batch(X, y)` when a direct float return is more convenient. +## Iris walkthrough + +This compact example performs the complete normalization and fitting sequence +on the Iris dataset: + +```python +import numpy as np +from sklearn.datasets import load_iris +from overlapindex import OverlapIndex + +iris = load_iris() +X = iris.data.astype(np.float64) +y = iris.target.astype(np.int64) + +x_min = X.min(axis=0) +x_max = X.max(axis=0) +X = (X - x_min) / (x_max - x_min) + +oi = OverlapIndex( + kmeans_kwargs={"random_state": 0}, +).fit(X, y) +print(oi.index) +``` + +The exact fitted score depends on backend settings and library versions. Set a +random seed, as above, whenever a result must be repeatable; interpret the +value using the anchors in {doc}`concepts` rather than as a fixed expected +constant. + ## Preprocessing The package validates input but does not apply a general-purpose feature @@ -99,3 +135,14 @@ When comparing embeddings, layers, or preprocessing choices: Continue with {doc}`concepts` for the scoring mechanics or {doc}`backends/index` for backend selection. + +## Build the documentation locally + +The published guide uses the same warning-strict Sphinx command that can be +run from the repository root: + +```bash +python -m pip install -r docs/requirements.txt +python -m pip install -e . +sphinx-build --fail-on-warning --keep-going -b html docs docs/_build/html +``` diff --git a/docs/index.md b/docs/index.md index 119471f..2956ab6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -62,8 +62,6 @@ traced back to the labels that produced it. - {doc}`use_cases` — apply OI to representation comparison, streaming, and dataset analysis. - {doc}`api` — browse the complete source-generated API. -- {doc}`readme` — read the repository README in full without leaving the - documentation site. ```{note} `MiniBatchKMeans` is the default and the recommended starting point for batch @@ -84,5 +82,4 @@ continuous_targets diagnostics use_cases api -readme ``` diff --git a/docs/readme.md b/docs/readme.md deleted file mode 100644 index f55bebc..0000000 --- a/docs/readme.md +++ /dev/null @@ -1,10 +0,0 @@ -# Complete project README - -This page includes the repository README verbatim so that installation notes, -examples, figures, parameter summaries, outputs, audience information, and -licensing guidance are available without leaving Read the Docs. - -```{include} ../README.md -:relative-images: -:heading-offset: 1 -``` diff --git a/docs/use_cases.md b/docs/use_cases.md index 7442f34..4a40926 100644 --- a/docs/use_cases.md +++ b/docs/use_cases.md @@ -37,8 +37,35 @@ Use the fitted diagnostics to understand a summary score: - For multi-label targets, `unevaluable_pairs_` and `unevaluable_labels_` identify comparisons that lacked suitable positive/negative rows. +## Synthetic separation behavior + +The following experiment sweeps two synthetic populations from fully +interleaved to well separated. Gaussian clouds and vertical bars vary their +center distance; concentric rings vary the difference between their radii. +Each response curve is the mean across repeated deterministic draws, and the +shaded band is one standard deviation. + +![Discrete OverlapIndex separation sweeps](../img/discrete_overlap_sweeps.png) + +Regenerate the figure from the repository root with: + +```bash +poetry run python examples/visualize_discrete_overlap_sweeps.py +``` + +The script writes `img/discrete_overlap_sweeps.png` by default and accepts +`--output PATH` to select another destination. + ## Common use cases +### Evolving clustering validation + +OI can serve as an incremental cluster-validity measure when labels or cluster +assignments identify the groups whose overlap should be monitored. ARTMAP +backends update the prototype representation online; offline backends evaluate +a complete supplied partition. In either case, report prototype growth and +resolution alongside the score because they influence the measured overlap. + ### Representation and embedding comparison Compute OI on embeddings from multiple feature extractors, model layers, or @@ -60,6 +87,12 @@ def separation_score(embeddings, labels): Normalize every representation consistently and keep `kmeans_k` fixed when comparing results. +The same procedure applies to backbone evaluation for transfer learning: fit +OI on embeddings from candidate feature extractors using identical downstream +samples, labels, preprocessing, and OI settings. A higher score indicates +better label separation in that representation; it does not by itself measure +downstream task accuracy. + ### Dataset diagnostics Use per-label and pairwise results to find classes that are difficult to @@ -135,3 +168,13 @@ When publishing or tracking a result, record: See {doc}`backends/index` to choose and configure a backend, and {doc}`api` for the complete estimator interface. + +## Intended users and licensing + +The package is intended for researchers and practitioners working in +incremental or continual learning, clustering validation, representation +learning, and transfer learning. + +OverlapIndex is licensed under the GNU Affero General Public License v3.0 or +later (AGPL-3.0-or-later). Commercial licenses are available; contact the +maintainer through the [project's GitHub page](https://github.com/NiklasMelton/OverlapIndex).