Repository navigation
Audit real Qwen memory offload and expose native storage costs - #58
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The independent-row quality experiment had no same-checkpoint native storage and complete-model cost audit. Micro Qwen fixtures could verify an interface, but could not establish the cost of serving those trained memories.
Add a row/nonrow-delta-only native exporter that avoids loading a dense Qwen backbone, verified CPU query workers that never read the source dense table, a fixed block-size diagnostic, and separate-process source/native Qwen workers with selected-GPU occupancy gates. Preserve the source vocabulary, original local-head addressing, FP32 bytes, nonrow delta and frozen-base fingerprints. Record anonymous/file-backed RSS separately.
The committed evidence covers Qwen3-0.6B and Qwen3.5-4B at 20,877 edits, seeds 3/7/11. All source/XOR and seed-3 DATrie-layout comparisons pass: 512 paired complete-prefix next-token logit hashes are identical, and every original row is verified. Native opening removes 70.913/70.110 MiB of CUDA allocated memory; observed complete-prefix latency is similar. Full-model startup peak RSS is essentially unchanged. The default native artifact is slightly larger than the original FP32 rows; registered-key exact structural merges are zero and the additional DATrie index has a substantial cost. Retain all negative results, block-size candidates and the first occupancy-blocked GPU reports.
Validation: local suite passed 257 tests (44 skipped), the expanded tiny real Qwen integration checks native bundle reopen and source/native complete outputs, and the GPU runs and standard-library archive verifier passed. CI now runs both the storage/model interface fixture and the new evidence verifier:
python3 -S docs/qwen_native_identity_evidence_20261109/verify.py MEFT_COUNTERFACT_SOURCE="$PWD/artifacts/identity-native-check-20261108/upstream" OMP_NUM_THREADS=1 HF_HUB_OFFLINE=1 artifacts/identity-native-check-20261108/env/bin/python -m unittest discover -s testsLimits: frozen-backbone counterfactual editing, roughly 74 MB memory tables, related chronological prefixes, warm caches, batch one and no KV cache. Qwen3.5 uses the same Torch fallback in both variants. No natural co-training cosine, full-model pretraining, generated-answer quality, optimized-serving superiority or billion-row extrapolation is established. Source weights/token keys are not redistributed; scalar/hash evidence and exact runtime snapshots are included. The measured 8-byte-per-row identity forwarding map is a concrete next optimization.