In the paper https://arxiv.org/abs/2405.03709, the table shows that ScenicNL achieves the best performance. However, when using the gen_scenic tool, I’m not sure which command corresponds to ScenicNL. Specifically, I’m unclear about which option to select for the llm_prompt_type parameter.
Based on the gen_scenic --help output, the available options for llm_prompt_type are as follows:
--llm_prompt_type [predict_zero_shot|predict_few_shot|predict_scenic_tutorial|predict_python_api|predict_python_api_oneline|predict_lmql|predict_few_shot_with_rag|predict_few_shot_hyde|predict_few_shot_hyde_tot|predict_tot_then_hyde|predict_tot_then_split|predict_tot_into_nl|expert_discussion|expert_synthesis|ast_feedback|predict_lmql_to_hyde|predict_lmql_retry|predict_lmql_tot_retry]
Unfortunately, I couldn’t find a clear explanation in the documentation about which option corresponds to ScenicNL. Could you please clarify the following:
What is the specific command to use for ScenicNL?
Which value should be set for llm_prompt_type to correspond to ScenicNL?
If there are any additional parameters or configurations required, could you please provide details?
Thank you for your assistance!
In the paper https://arxiv.org/abs/2405.03709, the table shows that ScenicNL achieves the best performance. However, when using the gen_scenic tool, I’m not sure which command corresponds to ScenicNL. Specifically, I’m unclear about which option to select for the llm_prompt_type parameter.
Based on the gen_scenic --help output, the available options for llm_prompt_type are as follows:
--llm_prompt_type [predict_zero_shot|predict_few_shot|predict_scenic_tutorial|predict_python_api|predict_python_api_oneline|predict_lmql|predict_few_shot_with_rag|predict_few_shot_hyde|predict_few_shot_hyde_tot|predict_tot_then_hyde|predict_tot_then_split|predict_tot_into_nl|expert_discussion|expert_synthesis|ast_feedback|predict_lmql_to_hyde|predict_lmql_retry|predict_lmql_tot_retry]
Unfortunately, I couldn’t find a clear explanation in the documentation about which option corresponds to ScenicNL. Could you please clarify the following:
What is the specific command to use for ScenicNL?
Which value should be set for llm_prompt_type to correspond to ScenicNL?
If there are any additional parameters or configurations required, could you please provide details?
Thank you for your assistance!