Mm rework lingfeng into Dev

[wei-lingfeng]

Summary

This merges mm_rework_lingfeng into dev. #25 documents the original set of semantic/behavioral changes that differentiate this branch from mm_rework — uncertainty handling (error propagation vs. weighted variance), Parallax's point requirement, weight-calculation precision, fit_star_idxs overwrite behavior, and the absolute_sigma default — see that PR for the full list and a worked example.

On top of that baseline, this branch also carries a substantial follow-on pass of correctness, performance, and memory fixes, summarized below.

Correctness fixes

• NaN propagation (97e2c7e): a non-finite m0 could never be excluded by the magnitude-range check (NaN comparisons are always False), letting bad reference stars corrupt the magnitude-zeropoint transform; fit_motion_models' default t0 could silently come out NaN for some multi-epoch stars from a masked/plain-average mismatch; combine_lists now falls back to a nominal unweighted mean instead of discarding a star's only valid measurement when every epoch's uncertainty is invalid.
• Magnitude weighting consistency (cb7dbd3): fixed an inverted weights_col check that made fit_motion_models always average magnitudes unweighted; routed "simple" (≤1 valid epoch) stars in update_ref_table_aggregates through the same corrected weighting so both paths agree. Single-epoch stars now get a real (if large) m0_err instead of a forced inf. This resolves the test_MosaicSelfRef_vel failure noted as a known issue in Update mm_rework to match mm_rework_lingfeng for comparison #25.
• Stale singular-fit bug (9a0f8c5): absolute_sigma=False's chi2 rescaling could turn a singular fit's correct inf error into nan (inf * sqrt(nan) == nan); fixed by reapplying the singular/insufficient-data override unconditionally at the end.
• Silent inf→finite bug in combine_lists (eba8435): removed a fragile "patch a fake weight in, then remember to force the error back to inf" pattern in favor of deriving std directly from real weights only (naturally inf via 1/0, no override needed).
• False-positive StarList warning (2bc7926): every pickle round-trip (astropy passes columns positionally to __init__) incorrectly warned about missing required arguments.
• Stale motion-model params (9287e51): a star that dropped to a simpler motion model (e.g. Linear → Fixed, since epoch counts aren't monotonic across align() iterations) kept stale params (vx/vy) from its old model indefinitely; now reset to fill_value/inf right after reclassification.

Performance & memory

• Vectorized batch fitting for Fixed (cea7916), Linear (9a0f8c5), and a trivial batch path for Empty (9a0f8c5) — fit_motion_models now fits a whole group of stars in one vectorized pass instead of one star at a time, whenever a motion model supports it. On a real dataset, processes=3 dropped from 7.07s to 2.68s (matching serial's 2.66s) once all three common models were batched, since a multiprocessing pool no longer needs to spin up just to run an O(1) fill for Empty stars.
• numpy.ma indexing bottleneck (506c2cf): ~38% faster fit_motion_models by avoiding numpy.ma's slow generic per-element indexing; reused one multiprocessing pool across the whole call instead of spawning one per motion-model group.
• StarTable construction (9287e51): gained an opt-in copy=False, and now builds all columns in a single constructor call instead of many add_column() calls (~1.2s / +4.6GB → ~0.001s / ~0GB for ~29 columns at 1.4M rows).
• vstack double-copy (6aa6d38): adding new-star rows built a whole parallel table and vstack-ed it onto the growing ref_table, transiently holding old + new + concatenated data for every column at once — roughly doubling peak memory on every "add new stars" step. Now concatenates columns directly and drops old references immediately.
• fit_motion_models per-star overhead (9287e51, this session): eliminated an O(N_stars) data-prep cost that ran regardless of how few stars actually needed refitting (now sliced down first); removed a redundant double-copy in combine_lists/fit_motion_models's array prep; a per-star fixed-params dict is now built lazily only for stars whose motion model actually needs it (skipped for ~84% of stars — those already handled by the vectorized batch path — in one benchmark).
• KDTree threading + multiprocessing threshold (this session): match() gained a workers parameter (default 1) for scipy.spatial.KDTree.query_ball_point; align.py exposes it as match_workers (default 1, since production runs typically share a machine — opt in explicitly for the speedup). Verified this doesn't affect the delicate dm_min == dr_min tie-break, which depends on within-list neighbor order (workers=1 vs workers=-1 give identical, order-preserved neighbor lists, checked against dense/duplicate-point edge cases and a full end-to-end run with all 37 ref_table output columns byte-identical). Also added a configurable mp_star_threshold (default 100,000): a multiprocessing pool for motion-model fitting is only spun up when the number of stars actually requiring the non-vectorized path meets this threshold, even if processes > 1 was requested — measured break-even for that fixed pool-spawn/IPC overhead was between 20,000 and 100,000 stars on a 10-core machine.

Other

• inherit_n_detect (14e7f00): new MosaicSelfRef/MosaicToRef parameter (default True) so a star's n_detect reflects the total number of raw detections it represents across nested alignment layers, not just 1 per input starlist.
• Minor: removed a misleading tqdm progress bar over a handful-of-motion-model-types loop (2d07f94).
• Cherry-picked three docs/packaging commits from mm_rework that hadn't yet made it into this branch: a pyproject.toml fix (66ac298, originally 4b2da35), adding the flystar modules to the Sphinx docs (7b3fb01, originally 34611d4), and a docs formatting cleanup (a1b2409, originally 92b44cd).

Documentation

• Read the Docs (5f00827): added .readthedocs.yaml. Getting a working build also required fixing three real, pre-existing bugs unrelated to RTD specifically — they'd have broken a fresh pip install . for anyone on a current Python: pyproject.toml's [build-system] pinned cython==0.29.14 (leftover astropy-template boilerplate for a package with zero .pyx/C extensions), which imports the stdlib cgi module removed in Python 3.13, breaking install outright; the declared dependencies were missing scipy, matplotlib, tqdm, joblib, and pandas, all imported unconditionally at module level; and setup.cfg's github_project was still the template default (astropy/astropy), which conf.py uses unconditionally to build doc issue-links. Verified via two independent fresh-venv builds (editable and the exact non-editable pip install .[docs] RTD runs) — sphinx-build succeeds, 92 warnings, all pre-existing docstring formatting nits unrelated to this change.
  • Docs are currently configured to build from wei-lingfeng/flystar rather than this repo directly — importing MovingUniverseLab/flystar into Read the Docs needs repo-admin access (to add its webhook) that isn't available on this account. The fork tracks this branch 1:1, so content is identical; re-pointing Read the Docs at the org repo later (once an org owner grants access) needs no other changes.

Validation

Each change was validated individually — synthetic fuzz tests against reference implementations (thousands to tens of thousands of cases per change), byte-for-bit output comparison on real multi-thousand and multi-million-star datasets, and dedicated new unit tests where behavior changed. See individual commit messages for specifics.

Full test suite passes except two pre-existing, environment/data-dependent failures unrelated to any change here: test_masked_cols (missing test fixture file) and test_generic_match (test data contains no finite values).