HF-HMA

HF-HMA is a small weekend research experiment about choosing among a finite set of candidate policies during long-horizon tasks with noisy feedback.

Instead of averaging candidate action distributions together, it selects one candidate at each state using a weighted Hellinger medoid. The weights stay fixed during the rollout and are updated once the feedback sequence is complete.

Why try it?

Averaging policies can repeatedly inject a small amount of a poor candidate throughout a long trajectory. HF-HMA asks whether the evidence represented by the weighted candidate set can instead be rounded to one actual candidate at each state without paying a statistical cost that grows explicitly with the trajectory length.

The central deterministic inequality is:

D_medoid <= 4 * sum_i w_i D_i

Here, D_medoid is the product squared Hellinger defect of the feedback laws selected by the medoid indices along a fixed state sequence, and D_i is the corresponding defect of candidate i. This is a statement about feedback-law agreement, not policy behavior by itself.

Combined with a square-root likelihood update, the technical notes derive the following conditional risk bound when a separate, horizon-uniform transfer inequality is available. With probability at least 1 - delta, it holds simultaneously for every finite T:

sum_{t<=T} Risk_t <= 4 * C_transfer * log(1 / (w_1(*) delta))

There is no explicit episode-length term in this expression. The result is conditional on the assumptions in METHOD.md, especially full candidate probability access and a feedback-to-behavior transfer bound whose constant does not depend on rollout horizon. A horizon-dependent transfer constant would reintroduce horizon dependence.

Try it

The code requires Python 3.10 or newer, uses only the standard library, and is intended to be read, run, and modified rather than installed as a package.

python3 hf_hma.py
python3 check_hf_hma.py

The second command runs 79,677 exhaustive deterministic cases, 103,500 fractional-grid cases, a fixed-seed random stress test, an exact evidence identity check, a basic end-to-end update check, and malformed-input rejection checks. These checks can find counterexamples; they cannot prove the derivation.

Files

• hf_hma.py is the dependency-free categorical reference implementation.
• check_hf_hma.py stress-tests the medoid product inequality with bounded memory.
• METHOD.md contains the setup, derivation, assumptions, and prior-art boundary.
• images/ contains the three Swarm Research graphics.

Important limits

HF-HMA is not a general reliability layer for arbitrary agents. The current result assumes:

• a finite candidate set containing the true feedback model with positive prior mass;
• evaluable candidate probability distributions, not text-only or sample-only access;
• conditionally independent feedback labels along each learner-generated rollout;
• weights frozen during a rollout and a fixed medoid tie rule;
• a separately justified, horizon-uniform transfer bound before feedback agreement is interpreted as horizon-independent behavioral performance.

Exact selection also requires O(N^2) pairwise distance work per state for N candidates.

Status and novelty

This is a research snapshot, not a peer-reviewed result or an actively maintained library. The derivation and code have passed local checks that my MacBook memory can handle, but they have not received independent expert review.

Hellinger distance, weighted medoids, fractional likelihood updates, and Bayesian evidence bounds are all established ideas. This repository does not establish whether the particular composition or product inequality is novel, and it makes no novelty or priority claim. Counterexamples and earlier matching work are welcome.

This was a model-assisted weekend research experiment and was curated for release by Blackscale Technologies as a Swarm Research item in case it is useful for the community. It is separate from the Swarm product and does not require or integrate with Swarm.

No ongoing maintenance is planned. Fork it, adapt it, test it, or use any part that is helpful.

License

Released under the MIT License.