Add files via upload - #251
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces task-specific prompt routing and a mixed context length strategy to optimize data annotation using Qwen3-4B. Key changes include category-specific prompt builders, a task-type router, and dynamic context length selection (30k/16k/8k) based on the task and sample index. Feedback on these changes highlights a critical bug where examples_str caching prevents the mixed context length strategy from executing correctly beyond the first sample. Additionally, the reviewer recommended passing a pre-loaded tokenizer to avoid redundant reloading, removing unused imports and dead code (such as the uncalled build_prompt_cot function), and correcting the return type annotation of annotate_ascend.
| if examples_str is None: | ||
| examples_str = select_examples(icl_examples, task_description, text2annotate) | ||
| # 使用混合上下文长度策略：传递task_id和sample_idx | ||
| examples_str = select_examples(icl_examples, task_description, text2annotate, task_id, sample_idx) | ||
| input_prompt = prompt.replace("[[EXAMPLES]]\n\n", examples_str+'\n\n') |
There was a problem hiding this comment.
The mixed context length strategy is currently broken because examples_str is cached as a single string initialized to None outside the loop. As a result, select_examples is only called once (at sample_idx == 0), meaning the 30k context length is used for all subsequent samples, and the 8k optimization is never triggered.
To fix this, we can repurpose examples_str as a dictionary cache to store both the large (30k/16k) and small (8k) versions of the few-shot examples, and select the appropriate one based on the current sample_idx.
| if examples_str is None: | |
| examples_str = select_examples(icl_examples, task_description, text2annotate) | |
| # 使用混合上下文长度策略：传递task_id和sample_idx | |
| examples_str = select_examples(icl_examples, task_description, text2annotate, task_id, sample_idx) | |
| input_prompt = prompt.replace("[[EXAMPLES]]\n\n", examples_str+'\n\n') | |
| if not isinstance(examples_str, dict): | |
| examples_str = {} | |
| cache_key = "large" if (task_id == 8 or sample_idx < 50) else "small" | |
| if cache_key not in examples_str: | |
| examples_str[cache_key] = select_examples(icl_examples, task_description, text2annotate, task_id, sample_idx, qwen_tokenizer) | |
| input_prompt = prompt.replace("[[EXAMPLES]]\n\n", examples_str[cache_key] + '\n\n') |
| return examples_str | ||
|
|
||
| def select_examples(all_examples: list[dict], task_description: str, text2annotate: str) -> str: | ||
| def select_examples(all_examples: list[dict], task_description: str, text2annotate: str, task_id: int = None, sample_index: int = 0) -> str: |
There was a problem hiding this comment.
To avoid redundantly reloading the tokenizer on every call to select_examples (and to respect the --tokenizer_path argument configured in main.py), allow passing an optional pre-loaded tokenizer instance.
| def select_examples(all_examples: list[dict], task_description: str, text2annotate: str, task_id: int = None, sample_index: int = 0) -> str: | |
| def select_examples(all_examples: list[dict], task_description: str, text2annotate: str, task_id: int = None, sample_index: int = 0, tokenizer: AutoTokenizer = None) -> str: |
| # 初始化Qwen3-4B的tokenizer（自动下载/加载千问3-4B的分词器） | ||
| # 若本地已下载模型，可替换为本地路径，如 "./qwen3-4b" | ||
| tokenizer = AutoTokenizer.from_pretrained("/share/project/wuhaiming/spaces/data_agent/OpenSeek-main/openseek/competition/LongContext-ICL-Annotation/src/Qwen3-4B", trust_remote_code=True) | ||
| tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True) |
There was a problem hiding this comment.
If a pre-loaded tokenizer is passed to the function, reuse it instead of loading a new one from the hardcoded path.
| tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True) | |
| if tokenizer is None: | |
| tokenizer = AutoTokenizer.from_pretrained("/root/Qwen3-4B", trust_remote_code=True) |
| # from method import build_prompt, select_examples, annotate | ||
|
|
||
| from method import build_prompt, select_examples | ||
| from method import build_prompt, select_examples, build_prompt_cot, build_prompt_by_task_type |
There was a problem hiding this comment.
The imports build_prompt and build_prompt_cot are not used anywhere in main.py. Removing them keeps the imports clean and avoids confusion.
| from method import build_prompt, select_examples, build_prompt_cot, build_prompt_by_task_type | |
| from method import select_examples, build_prompt_by_task_type |
| return prediction | ||
|
|
||
| def annotate_ascend(input_prompt:str)->list[str]: | ||
| def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]: |
There was a problem hiding this comment.
The return type annotation of annotate_ascend is currently list[str], but the function actually returns a single string (either the raw model output or the extracted answer from count_answer). Update the type annotation to str to reflect the actual return type.
| def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]: | |
| def annotate_ascend(input_prompt:str, task_id:int=None)->str: |
| ) | ||
| return prompt | ||
|
|
||
| def build_prompt_cot(task_description: str, text2annotate: str, task_id: int) -> str: |
There was a problem hiding this comment.
The function build_prompt_cot is defined and imported in main.py, but it is never called anywhere in the codebase. If Chain-of-Thought prompting is intended for Task 3 (and potentially Task 8), it should be integrated into the routing function build_prompt_by_task_type. Otherwise, if it is no longer needed, it should be removed to avoid dead code.
1 participant
No description provided.