fix(qwen35): retain construction owners until publication - #735
Conversation
The rollback ledger stored DeviceBuffer::alias() views, which free_tensor refuses to release. Stage real GpuTensor owners and publish them only after every mandatory/optional allocation succeeds; unwind failures in reverse. GPU fault regression on gfx1151/HIP7.2: warm54 allocations; fail the final allocation and retry. Before:107 fresh allocations (53 buffers lost). After:54, full warm-pool reuse. Default qwen35 lib tests191 passed/1 ignored; the ignored GPU fault test was explicitly run and passed. Scoped clippy completed with existing warnings; crate maps43/43, ratchets21/21 pass. Rebuilt CLI/daemon, verified qwen3.6:27b DFlash factual+5-turn chain and qwen3.6:35b-a3b AR factual:7 turns stop, no empty/runaway/attractor outputs. Matched128-token DFlash request is byte-identical before/after (both hit that explicit cap); separate512-token run completes at148 tokens. Evidence scope: first independent G4 allocation-error fix, not G4 milestone acceptance, numerical/state admission, physical multi-GPU, or performance.
bd653c5 to
229f5f5
Compare
|
Landed into beta as800ddf740: original exclusive229f5f54 cherry-picked with -x after #731, avoiding duplicate prefill commit7b599a84. Independent static review approved. Exact composed candidate246a26f passed all4 GPU rollback regressions on gfx1201/HIP7.15 in a unique Cargo target (an earlier stale shared-target run was discarded). Dense canonical XT+DFlash battery5/5 and chain5/5 passed with decoded text inspected; Ornith MQ4R AR battery5/5 passed at2048 tokens with replay-route-proof logging enabled. Initial256-token Ornith run had2 reasoning-budget-truncated/empty turns and is NOT a passing receipt. Candidate-to-landed source diff contains only separately reviewed #738/#740 tooling fixes. Scope caveat: nested MoE constructors can still leak before returning an owner; that predates and lies outside this bounded returned-owner publication fix. #736 remains held for separate leaf-copy/AWQ atomicity defects. |
|
Decoded output receipts for the #731+#735 composed candidate (not a performance comparison). All three accepted runs inspected manually; original JSON retained under /home/kaden/ClaudeCode/warpfront/_pr_smokes/. CLI MD5 b95f46207e8629c46deccadf67599b35; daemon ee68c70fdb50200c663b38a63ab77772. Canonical XT DFlash battery (256 tokens)Turn1, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=43ca0d15712d3dfb777b51ae76d8fd5f Decoded assistant answerdef merge_sorted(a, b):
"""Merge two sorted lists into a single sorted list."""
result = []
i, j = 0, 0
while i < len(a) and j < len(b):
if a[i] <= b[j]:
result.append(a[i])
i += 1
else:
result.append(b[j])
j += 1
result.extend(a[i:])
result.extend(b[j:])
return resultTurn2, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=640e0fd4f55996cb175a422f0a12cef5 Decoded assistant answerStep 1: Find the distance for the first part. [ [ Step 2: Find the distance for the second part. [ Step 3: Add the distances together. [ Final answer: 210 miles Turn3, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8f66b4c97988825bd8e7840aaf44357e Decoded assistant answerEarth's axis is tilted about 23.5 degrees relative to its orbit around the Sun. As Earth orbits the Sun, different hemispheres receive more direct sunlight and longer days at different times of the year. This variation in sunlight intensity and day length causes the seasons. Turn4, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8fe0ad36f61bcf4992cc9df81cdf3817 Decoded assistant answerElias, the lighthouse keeper, found a glass bottle balanced on the black rocks after the storm. Inside was not a message, but a tiny silver key, warm as if it had been held just now. He walked to the locked sea door he had never dared to open. The key fit, and beyond it the sea glowed with lights he had only ever dreamed of. Turn5, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8bed8e2d056dc1d47dccae9d32dbecf4 Decoded assistant answer
Canonical XT DFlash chain (2048 tokens)Turn1, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=43ca0d15712d3dfb777b51ae76d8fd5f Decoded assistant answerdef merge_sorted(a, b):
"""Merge two sorted lists into a single sorted list."""
result = []
i, j = 0, 0
while i < len(a) and j < len(b):
if a[i] <= b[j]:
result.append(a[i])
i += 1
else:
result.append(b[j])
j += 1
result.extend(a[i:])
result.extend(b[j:])
return resultTurn2, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=640e0fd4f55996cb175a422f0a12cef5 Decoded assistant answerStep 1: Distance at 60 mph for 2.5 hours Step 2: Distance at 40 mph for 1.5 hours Step 3: Total distance Final answer: 210 miles Turn3, finish=stop, cached=256, empty=False, attractor=False, runaway=False, prompt_md5=8f66b4c97988825bd8e7840aaf44357e Decoded assistant answerEarth's axial tilt of about 23.5 degrees causes different hemispheres to receive varying angles of sunlight throughout the year. As Earth orbits the Sun, the tilted axis makes one hemisphere lean toward the Sun, producing summer, while the other leans away, producing winter. This cycle repeats annually, creating the pattern of spring, summer, autumn, and winter. Turn4, finish=stop, cached=256, empty=False, attractor=False, runaway=False, prompt_md5=8fe0ad36f61bcf4992cc9df81cdf3817 Decoded assistant answerEvery morning, Elias checked the rocks below his lighthouse for barnacles, broken glass, and the usual driftwood. One gray dawn, he found a brass compass tied to a bottle, its needle still trembling despite the tide. The glass label bore his own name, written in his late father's hand. He turned it over, and on the reverse was tomorrow's date in fresh ink. Turn5, finish=stop, cached=256, empty=False, attractor=False, runaway=False, prompt_md5=8bed8e2d056dc1d47dccae9d32dbecf4 Decoded assistant answer
Ornith MQ4R AR battery (2048 tokens)Turn1, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=43ca0d15712d3dfb777b51ae76d8fd5f Decoded assistant answerHere's a clean implementation using the classic two-pointer "merge" approach (the same step merge sort uses): def merge_sorted(a, b):
"""Merge two sorted lists into a single sorted list without using sorted()."""
result = []
i = j = 0
while i < len(a) and j < len(b):
if a[i] <= b[j]:
result.append(a[i])
i += 1
else:
result.append(b[j])
j += 1
# Append whatever remains (one of the lists will be exhausted already)
result.extend(a[i:])
result.extend(b[j:])
return resultHow it works:
Example: print(merge_sorted([1, 3, 5], [2, 4, 6])) # [1, 2, 3, 4, 5, 6]
print(merge_sorted([], [1, 2])) # [1, 2]
print(merge_sorted([1, 2], [2, 3])) # [1, 2, 2, 3]This runs in O(n + m) time, which is optimal since every element must be visited at least once. Turn2, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=640e0fd4f55996cb175a422f0a12cef5 Decoded assistant answerTrain Distance CalculationStep 1: First leg of the tripStep 2: Second leg of the tripStep 3: Add both distances togetherFinal AnswerThe train traveled a total of 210 miles. Turn3, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8f66b4c97988825bd8e7840aaf44357e Decoded assistant answerThe seasons on Earth are primarily caused by the approximately 23.5-degree tilt of Earth's axis relative to its orbital path around the Sun. Because this axis remains pointed in a fixed direction as Earth orbits, different hemispheres are angled toward or away from the Sun at different times of the year. When a hemisphere is tilted toward the Sun, it receives more direct sunlight and longer days, producing summer, while a tilt away from the Sun creates the colder conditions of winter. Turn4, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8fe0ad36f61bcf4992cc9df81cdf3817 Decoded assistant answerFor twenty years, old Tomás had patrolled the same black rocks at low tide, knowing every splintered hull and torn net the sea chose to return. But on this bitter November morning, something glimmered among the barnacled stones that no storm could have carried. He knelt in the freezing spray and brushed away the seaweed to reveal a small wooden chest, its hinges fused by salt and its lid carved with the name of a ship he had buried decades ago. Inside, wrapped in oilcloth, lay a letter addressed in his own young man's hand, and a single silver ring he had thrown into the dark the night he left home. Turn5, finish=stop, cached=0, empty=False, attractor=False, runaway=False, prompt_md5=8bed8e2d056dc1d47dccae9d32dbecf4 Decoded assistant answer
|
G4.1b — Qwen35 construction ownership
Depends on #731. This PR is stacked on its prefill-scratch ownership fix; once #731 lands, this diff reduces to the remaining decode-batch and per-layer construction transactions.
Changes
Qwen35DecodeBatchStateoutput tensors as actual owners until complete publication; no borrowed aliases participate in rollbackload_layervariantsWeightBackend::free_tensorrollback seam for HFQ and PaRo backendsFinal-head evidence
Head:
229f5f54f8771e41eb81dd35cb18a5361f6a6ae0(rebased onto currentbeta).cargo test -p hipfire-runtime --lib: 609 passedcargo test -p hipfire-arch-qwen35 --lib: 191 passed, 4 ignoredcargo build --release: passeddrafter=dflashfor all requests253c7ac50857fe6d0e10fb0d2c5e35c0) coherentlyFinal artifacts:
hipfiremd5:09cc1504492ff0628608a58434ce7c60daemonmd5:c6db7ff700b3b17b48461736e6f584f5e1c9480a3fa54ad0a0f25ca18105510a7b6df2a4ee1c8d933f0a52e187d1860bedde51ec1dac0f2bd42cff5ef1cb8944Scoped
cargo clippy --no-depsexits successfully with the repository's existing warning set. The advisory workspace CI Clippy job fails on the pre-existingapprox_constantlint inexamples/test_dflash_hidden_scatter_gfx1100.rs; required build, unit-test, rustfmt, cargo-deny, and gate jobs pass.