Issue with azure benchmarking tool on passing input tokens greater than 30k

[rajm3180]

We are trying to run our benchmarking exercise using the benchmarking tool for gpt4o but getting below error on passing the input context greater than 30k from the command line

Command:
python -m benchmark.bench load --temperature 0.0 --shape-profile custom --deployment 'deployment name' --max-tokens 200 --context-tokens 35000 --api-version 2024-02-01 --rate 10 --duration 600 https://genai-stg-westus3-1.openai.azure.com/

Error:
getting error as "There aren't enough words to choose from. Cannot generate 15991 word(s)

[michaeltremeer]

Hey @rajm3180 , this repo is no longer maintained and it is suggested to use the improved fork here: https://aka.ms/ImprovedAOAIBenchmarkingTool.

The fork has already fixed this issue and it is actively maintained, with a large number of fixes and improvements compared to this repo. Make sure to read the README to understand all of the updates that exist.