Coincidex

Transfer-aware, replay-free continual learning for PyTorch.

Most continual-learning methods either forget (plain fine-tuning) or isolate and forfeit transfer (fixed random masks — XdG / SupSup / PackNet, or one adapter per task). Coincidex's context-gate does both at once: similar contexts activate overlapping sparse sub-networks (→ forward transfer) while dissimilar contexts activate disjoint ones (→ retention) — with no replay buffer.

It is a small, dependency-light drop-in layer (torch + numpy only). The whole idea is one mechanism:

hidden = relu(W1 · x) · burst,   burst = HardThreshold( σ(basal) · σ(apical) − θ )
         basal = bottom-up evidence (the input drive),   apical = top-down context (a task/domain descriptor)

Because the burst is a hard 0/1 gate, a gated-off unit receives exactly zero gradient — so its old-task weights are structurally frozen (retention) with no rehearsal data stored. Because similar contexts produce overlapping bursts, related tasks share capacity (transfer); dissimilar contexts produce disjoint bursts, so unrelated tasks stay isolated.

---

Why it exists (the gap it fills)

Continual learning has a stability↔plasticity trade-off. The popular families sit on the extremes:

Method family	Retention	Transfer between related tasks	Stores past data?
Dense fine-tuning	✗ forgets	✓ (but overwrites)	no
Fixed random masks (XdG / SupSup / PackNet)	✓	✗ forfeits by construction	no
One adapter/LoRA per task	✓	✗ (each task isolated)	no
Replay / rehearsal	✓	~	yes (a buffer)
Coincidex (this)	✓	✓ graded, learned from context	no

Coincidex is the middle ground: it learns which tasks are similar (from a context vector) and shares capacity accordingly, without a replay buffer. See Benchmarks for the measured trade-off.

---

Install

pip install coincidex

or from source:

git clone https://github.com/rakib-nyc/coincidex
cd coincidex
pip install -e .

Requires Python ≥ 3.9, torch ≥ 1.12, numpy. CPU is fine — the benchmarks in this repo run on a laptop.

---

Quickstart

import coincidex as cx

# A domain-incremental benchmark: 12 tasks, 5 classes, in 4 similarity "families".
cfg   = cx.PermutedConfig(D=64, K=5, T=12, ctx_dim=16, families=4, family_swaps=20)
bench = cx.PermutedContinual(cfg)

# The transfer-aware gate (rho = target active fraction / sparsity).
model = cx.RateCoinCL(D=64, H=96, K=5, ctx_dim=16, rho=0.15)
print(cx.run_continual_full(model, bench))
# -> {'ACC': 0.98, 'BWT': -0.02, 'FWT_within': +0.57, 'FWT_cross': +0.05, 'peak_mean': 1.0}

# Compare to an isolation-only baseline (fixed random masks, XdG-style):
xdg = cx.XdGCL(D=64, H=96, K=5, T=12, active_frac=0.25)
print(cx.run_continual_full(xdg, bench))
# -> similar retention, but far lower FWT_within (it cannot transfer between related tasks)

Metrics returned by run_continual_full:

• ACC — mean accuracy over all tasks at the end of the stream.
• BWT — backward transfer / retention (final − peak; higher / less-negative = remembers more).
• FWT_within / FWT_cross — forward transfer to a new task from same-family vs different-family tasks (zero-shot accuracy before training the new task, minus chance). A large within ≫ cross gap means the gate is sharing with related tasks and isolating unrelated ones — the property Coincidex is built for.
• peak_mean — mean per-task accuracy right after training it (an underfit guard: if this isn't high, the tasks weren't learned and retention/transfer numbers are meaningless).

---

Use it on your own data

There are two ways to use Coincidex.

1. As a layer in your own training loop

RateCoinCL is a normal nn.Module. Its forward signature is forward(x, t, ctx):

model = cx.RateCoinCL(D=feat_dim, H=256, K=num_classes, ctx_dim=16, rho=0.15)
logits = model(x, t, ctx)     # x:(B, feat_dim)  t:int task/domain index  ctx:(ctx_dim,) context vector
loss   = F.cross_entropy(logits, y) + model.aux_loss()   # aux_loss() keeps the burst rate near rho (0 for other models)

• x — your inputs, ideally features from a (small/weak) frozen backbone. See Where it works.
• ctx — a context descriptor for the current task/domain: a hand-given task id embedded as a vector, a domain descriptor, or something inferred from the input (e.g. a small classifier's output, or the domain's feature-centroid). Similar contexts → overlapping bursts → transfer. This is the one thing you must supply.
• t — the integer task index (used only for logging / by some baselines).

2. Wrap your data as a benchmark and use the evaluator

run_continual / run_continual_full accept any object exposing this small protocol:

class MyStream:
    cfg  = ...            # object with .D .K .T .ctx_dim (ints)
    family = [...]        # list of length T: family id per task (use list(range(T)) if none)
    def context(self, t):            -> Tensor (ctx_dim,)      # context descriptor for task t
    def task(self, t, n, seed):      -> (x:Tensor(n,D), y:Tensor(n,))   # a batch for task t; seed==99991 means the test split

Then: cx.run_continual_full(model, MyStream()). PermutedContinual in this repo is a reference implementation.

Choosing the gate variant

• RateCoinCL(..., rho) — recommended default. A learnable threshold held at a target burst rate rho (the fraction of units active, like active_frac), so you tune an interpretable sparsity knob, not an opaque θ.
• QuantileCoinCL(..., rho) — no learnable threshold at all: a per-batch quantile picks the top-rho fraction of units. Self-calibrating and robust across feature scales; slightly weaker retention than RateCoin.
• CoincidenceCL(..., theta) — the raw form with a frozen threshold theta; use only if you want to set the threshold directly.

---

Benchmarks

The synthetic numbers below reproduce exactly with python examples/benchmark_vs_baselines.py (3 seeds, CPU). The real-data and quantization numbers are from the research record (BENCHMARKS.md) and are not reproducible from this minimal package (they need cached image features / a separate experiment). Metric convention: BWT = retention (higher better), FWT_within = transfer (higher better); all models reach per-task peak ≈ 1.0, so the differences are retention/transfer, not underfitting.

Domain-incremental, synthetic (12 tasks, 4 families, 3 seeds): the gate Pareto-dominates fixed-random isolation — it matches its retention while transferring ~2× more (these are the reproduce-command numbers):

model	BWT (retention)	FWT_within (transfer)
RateCoinCL (ours)	−0.03	+0.46
XdGCL (isolation-only)	−0.01	+0.25
Dense (naive)	−0.41	+0.47 (transfers, but forgets)

Domain-incremental on REAL frozen features (ImageNet MobileNetV2, 3 seeds): the win holds on real features — RateCoin −0.035 / +0.696 vs XdG −0.018 / +0.569 (transfer edge +0.13, retention parity). The domains here are feature-shifts on real features; a natural image-corruption variant gives a similar transfer edge (~+0.12).

Robustness: the transfer edge survives INT8/INT4 weight quantization (100% of full-precision at INT8), shown in simulation (weight fake-quantization — on-hardware validation is future work). The gate can also be trained by a local, backprop-free three-factor rule (feedback-alignment + burst-gated plasticity) with the transfer-with-isolation edge intact.

---

Where it works (and where it doesn't)

We tested this rigorously and report the boundaries up front. This is not a universal continual learner.

✓ Works (use it here):

• Domain / covariate shift — the task stays the same (shared label space) and the input distribution shifts and recurs (permuted / corrupted / seasonal / per-domain). This is the regime it is built for.
• Small or weak models — its advantage is largest when the model is weak enough that naive fine-tuning would forget. (On very strong backbones there is little to gain; see below.)
• When you want transfer between related tasks and isolation of unrelated ones, with no data stored.

✗ Does not help (don't use it here):

• Class-incremental — genuinely new classes appearing over time (distinct labels). Here contextual similarity does not imply solution compatibility, so overlap becomes interference; isolation methods win.
• Strong frozen backbones that already generalize — if a big frozen model already handles your shifting contexts zero-shot, a continual adapter (any adapter) adds little; the gate is "swamped".
• As a retention-only method vs a simple dense baseline — under mild drift naive fine-tuning barely forgets, so the retention advantage is small. The robust, repeatable edge is transfer vs isolation-based methods.

The honest one-line summary: transfer-with-isolation on small/weak models under real domain shift, replay-free.

---

How it works (a bit more detail)

Each unit's activation is gated by a coincidence-burst: it fires only when the bottom-up drive σ(basal) and a top-down σ(apical) context signal coincide above a threshold. The threshold is held at a target sparsity rho (interpretable — the fraction of active units), which is essential: a drifting/over-broad threshold causes over-sharing and forgetting. The gate is trained with a straight-through surrogate gradient (fast-sigmoid) so the hard forward pass stays discrete while backprop still flows. The design is inspired by two-compartment (basal + apical) pyramidal neurons and dendritic gating, but the value is measured, not biological — it is a practical routing rule for continual learning.

---

Reproduce the results

pip install -e ".[dev]"
python examples/quickstart.py                 # the 60-second demo
python examples/benchmark_vs_baselines.py     # the full bake-off vs XdG / Dense / soft-gate, multi-seed
pytest                                        # unit tests

---

Status

Research / beta (v0.2). The mechanism and benchmarks are validated as described above; the API may change before v1.0. It is provided as is, with no warranty and no liability, under the Apache License 2.0 (see LICENSE). Issues, reproductions, and PRs are welcome.

Planned directions (each a future edition, not promised): the local backprop-free training mode as a first-class API, quantized/edge deployment helpers, more real-data benchmarks (DomainNet, drift streams), and inferred-context utilities so you don't have to supply the context vector by hand.

---

Citation

If you use Coincidex in your work, please cite it:

@software{islam2026coincidex,
  author  = {Islam, Muhammad Rakibul},
  title   = {Coincidex: Transfer-aware, replay-free continual learning for PyTorch},
  year    = {2026},
  url     = {https://github.com/rakib-nyc/coincidex}
}

License

Apache License 2.0 — Copyright 2026 Muhammad Rakibul Islam. See LICENSE and NOTICE.