Tests

perf: Optimize detokenize buffer sizing for CJK-heavy outputs #772

Job	Run time
build-and-test (macos-14, 3.13, -DLLAMA_METAL=off, (No Metal))	2m 44s
build-and-test (windows-latest, 3.13)	7m 52s
build-and-test (windows-latest, 3.9)	9m 30s
build-and-test (ubuntu-latest, 3.9)	3m 33s
build-and-test (macos-14, 3.14, -DLLAMA_METAL=on -DGGML_METAL_USE_BF16=on -DGGML_METAL_EMBED_LIBR...	2m 31s
build-and-test (ubuntu-latest, 3.14)	3m 26s
build-and-test (ubuntu-latest, 3.13)	2m 50s
build-and-test (macos-14, 3.13, -DLLAMA_METAL=on -DGGML_METAL_USE_BF16=on -DGGML_METAL_EMBED_LIBR...	3m 34s
build-and-test (macos-14, 3.14, -DLLAMA_METAL=off, (No Metal))	2m 35s
build-and-test (windows-latest, 3.14)	7m 29s
build-and-test (macos-14, 3.9, -DLLAMA_METAL=off, (No Metal))	3m 11s
build-and-test (macos-14, 3.9, -DLLAMA_METAL=on -DGGML_METAL_USE_BF16=on -DGGML_METAL_EMBED_LIBRA...	3m 56s
	53m 11s

Provide feedback