Repository navigation
Fix example rounding in 2 challenges and align example tests with Example 1 in 4 - #389
kunal-mansukhani wants to merge 1 commit into
Conversation
- 5_softmax / 40_batch_normalization: round shown outputs correctly (0.245, ±1.225). - 52_silu, 14_multi_agent_sim, 61_rope_embedding, 64_weight_dequantization: generate_example_test() now uses the Example 1 inputs from challenge.html.
There was a problem hiding this comment.
- edits existing challenge grader/tests challenges/easy/52_silu/challenge.py
- edits existing challenge grader/tests challenges/hard/14_multi_agent_sim/challenge.py
- edits existing challenge grader/tests challenges/medium/61_rope_embedding/challenge.py
- edits existing challenge grader/tests challenges/medium/64_weight_dequantization/challenge.py
shxjames
left a comment
There was a problem hiding this comment.
Reviewed c4d9acd. Both rounding fixes and all four example-test alignments check out. The diff is limited to those challenge.html lines and the four generate_example_test() functions; reference_impl, tolerances, functional/perf tests, and starters are unchanged.
Rounding (HTML only). Independent fp64 matches reference_impl, and 3-decimal rounding is now consistent:
- 5_softmax Example 1: 0.090031, 0.244728, 0.665241 →
[0.090, 0.245, 0.665]. The old 0.244 was truncated. Example 2 is unchanged and already correctly rounded. - 40_batch_normalization Example 1: ±1.224743 →
±1.225. The old ±1.224 was truncated. Example 2 already matches.
Example-test alignment. Each generate_example_test() now uses the HTML Example 1 inputs (and shapes/params). Reference and fp64 outputs match the published HTML, and every example dict still includes the output buffer and all signature params:
- 52_silu:
N=3,[0.5, 1.0, -0.5]→[0.311230, 0.731059, -0.188770]. The HTML prints0.3112295/-0.1887705(pre-existing mixed precision; within 5e-7 of fp64). - 14_multi_agent_sim: agents
[0,0,1,0, 3,4,0,-1]→[1,0,1,0, 3,3,0,-1]. Distance is exactly 5.0, so the strict< rrule means no neighbors and the update is just position += velocity, matching the HTML. - 61_rope_embedding:
M=2, D=4with the Example 1 Q/Cos/Sin (already half-split) →[[1,2,3,4],[-1,-1,1,1]]. - 64_weight_dequantization:
4×4,TILE_SIZE=2→ the exact integer HTML output.
Pre-existing, non-blocking: 64's constraints say TILE_SIZE ∈ {16,32,64,128}, but Example 1 (and now the example test) uses 2. Functional tests still use the allowed sizes.
CI is green. I have not approved.
Follow-up to the example-output audit in #388. This PR is separate from #388 and branched from
main. It makes two kinds of change: it fixes two rounding slips inchallenge.html, and it makesgenerate_example_test()use the same inputs as Example 1 in four challenges. Nothing changes inreference_impl, tolerances, functional or performance tests, or starters.1. Rounding fixes in
challenge.htmlreference_impl(CPU fp32)[0.090, 0.244, 0.665][0.090, 0.245, 0.665][[-1.224, -1.224], [0.0, 0.0], [1.224, 1.224]][[-1.225, -1.225], [0.0, 0.0], [1.225, 1.225]]Both examples keep their 3-decimal precision. The old values were truncated instead of rounded.
2.
generate_example_test()now matches Example 1In these four challenges the HTML was already correct, but the example test used different inputs. The test now uses the HTML inputs.
N=4, input[1, 2, 3, 4]N=3, input[0.5, 1.0, -0.5][0.311230, 0.731059, -0.188770][0,0,1,0, 5,0,0,1][0,0,1,0, 3,4,0,-1][1,0,1,0, 3,3,0,-1]M=1024, D=64M=2, D=4with Q/Cos/Sin from Example 1[[1,2,3,4],[-1,-1,1,1]]256x256,TILE_SIZE=1284x4,TILE_SIZE=2with X/S from Example 1[[5,5,10,10],[5,5,10,10],[8,8,2,2],[8,8,2,2]]The Example 1 Cos/Sin tables in 61 already have equal halves, which is the
_half_splitlayout, so they're used as-is._half_splitis still used by the functional tests.Verification
generate_example_test()throughreference_implon CPU (device="cpu") and compared the result with the HTML Example 1 output at the precision it's printed. All six challenges match. Every other example in those challenges still matches too.black --check,isort --check-onlyandflake8pass locally on the changed files.