Merge pull request #9 from Aucannot/codex/review-skill-implementation-for-ai-paper-summary

feat: add paper-reading workflow and subcategories to ai-auto-note-publisher

Author: Aucannot

README.md

@@ -56,7 +56,9 @@ python3 skills/ai-auto-note-publisher/scripts/create_ai_auto_note.py \
   --repo-root .
 ```
 
-可选：如果你要覆盖自动分类，可传 `--subcategory`（例如 `rag`、`agents`、`general`）。
+可选：如果你要覆盖自动分类，可传 `--subcategory`（例如 `rag`、`agents`、`paper-reading`、`productivity`、`general`）。
+
+论文场景中，脚本支持从 `arXiv:2501.12345` 这类文本自动规范化为 `https://arxiv.org/abs/2501.12345`。
 
 生成并提交后，如需自动推送 PR 到本仓库（需要已安装并登录 `gh`）：
 

skills/ai-auto-note-publisher/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: ai-auto-note-publisher
-description: 自动把学习对话沉淀成博客笔记并提交到当前仓库。当用户提到“学习总结/自动归档/上传笔记/ai auto note”时使用。统一写入 categories: ["ai-auto-note", "<auto-subcategory>"]，并按子目录自动归类。
+description: 自动把学习对话沉淀成博客笔记并提交到当前仓库。当用户提到“学习总结/自动归档/上传笔记/ai auto note/论文共读/论文总结”时使用。统一写入 categories: ["ai-auto-note", "<auto-subcategory>"]，并按子目录自动归类。
 ---
 
 # AI Auto Note Publisher
@@ -9,23 +9,27 @@ description: 自动把学习对话沉淀成博客笔记并提交到当前仓库
 
 ## 何时使用
 
-当用户希望把与 AI 学习的结论自动沉淀成文档，并上传到仓库时。
+当用户希望把与 AI 学习（尤其是论文共读）的结论自动沉淀成文档，并上传到仓库时。
 
 ## 输出规范
 
 - 路径：`_posts/ai-auto-note/<subcategory>/YYYY-MM-DD-title.md`
 - Front matter 固定：
   - `categories: ["ai-auto-note", "<subcategory>"]`
-  - `tags: ["ai-auto-note", "<subcategory>"]`
+  - `tags`: 至少包含 `"ai-auto-note"` 与 `<subcategory>`
 - 正文结构至少包含：
   - `## 背景`
   - `## 结论速记`
   - `## 后续行动`
+- 若子分类是 `paper-reading`，额外包含：
+  - `## 论文信息`
+  - `## 方法与实验要点`
 
 ## 快速流程
 
-1. 准备学习记录文本（例如 `tmp/notes.txt`）。
-2. 运行脚本生成文章：
+1. 先和用户完成论文/主题讨论，确认有稳定结论。
+2. 准备学习记录文本（例如 `tmp/notes.txt`）。
+3. 运行脚本生成文章：
 
 ```bash
 python3 skills/ai-auto-note-publisher/scripts/create_ai_auto_note.py \
@@ -37,22 +41,26 @@ python3 skills/ai-auto-note-publisher/scripts/create_ai_auto_note.py \
 python3 skills/ai-auto-note-publisher/scripts/create_ai_auto_note.py \
   --title "你的标题" \
   --input tmp/notes.txt \
-  --subcategory rag \
+  --subcategory paper-reading \
   --repo-root .
 ```
 
-3. 检查生成内容是否符合语义；必要时微调标题与要点。
-4. 提交到仓库。
-5. 推送分支并自动创建 PR（需 gh CLI 已登录）。
+4. 检查生成内容是否符合语义；若是论文笔记，补全“核心任务/主要贡献”等占位字段。
+5. 提交到仓库。
+6. 推送分支并自动创建 PR（需 gh CLI 已登录）。
 
 ## 子分类策略
 
-脚本会按关键词自动分类（如 `llm`、`prompting`、`rag`、`agents` 等），并优先做英文关键词边界匹配（避免 `evaluation` 误命中等情况）。
+脚本会按关键词自动分类（如 `llm`、`prompting`、`rag`、`agents`、`paper-reading`、`productivity` 等），并优先做英文关键词边界匹配（避免 `evaluation` 误命中等情况）。
 
 如未命中关键词，默认分类为 `general`。
 
 详细映射见：`references/subcategory-rules.md`。
 
+标签策略：脚本会去重并在输入稀疏时补充稳定兜底标签（如 `paper`、`reading-notes`）。
+
+脚本会自动从原文提取论文标题（如 `Title:` / `论文标题:` / `《...》`）与 arXiv 链接（支持 `https://arxiv.org/...` 和 `arXiv:2501.12345`）；未识别时保留“待补充”。
+
 ## 自动推送 PR 到本仓库
 
 准备 PR 描述文件（例如 `tmp/pr-body.md`）后执行：

skills/ai-auto-note-publisher/references/subcategory-rules.md

@@ -11,10 +11,18 @@
 - `training`: finetune, sft, rlhf, 训练, 蒸馏
 - `evaluation`: benchmark, eval, 评测, 幻觉, hallucination
 - `deployment`: 部署, serving, latency, 吞吐, inference
+- `paper-reading`: paper, 论文, arxiv, method, 实验, ablation, baseline, sota
+- `productivity`: 复盘, 学习计划, todo, 行动项, 习惯, 时间管理
 - 默认: `general`
 
 ## 调整建议
 
 - 如果某个主题常出现误分类，优先新增关键词而不是新增大量新类。
 - 英文关键词建议使用“完整词”匹配（如 `eval` 只匹配独立单词），减少误命中。
 - 保持二级分类数量可控（建议 < 12）。
+
+
+## 元信息提取补充
+
+- arXiv 链接支持两种来源：完整 URL（`https://arxiv.org/abs/...`）和编号格式（`arXiv:2501.12345`）。
+- 若识别到编号格式，生成文章时会规范化为 `https://arxiv.org/abs/<id>`。

skills/ai-auto-note-publisher/scripts/create_ai_auto_note.py

@@ -18,6 +18,17 @@
     "training": ["finetune", "sft", "rlhf", "训练", "蒸馏"],
     "evaluation": ["benchmark", "eval", "评测", "幻觉", "hallucination"],
     "deployment": ["部署", "serving", "latency", "吞吐", "inference"],
+    "paper-reading": [
+        "paper",
+        "论文",
+        "arxiv",
+        "method",
+        "实验",
+        "ablation",
+        "baseline",
+        "sota",
+    ],
+    "productivity": ["复盘", "学习计划", "todo", "行动项", "习惯", "时间管理"],
 }
 
 VALID_SUBCATEGORIES = set(SUBCATEGORY_KEYWORDS) | {"general"}
@@ -38,8 +49,30 @@
     "inference",
     "latency",
     "serving",
+    "paper",
+    "arxiv",
+    "method",
+    "ablation",
+    "baseline",
+    "sota",
+    "todo",
 }
 
+METADATA_PREFIXES = (
+    "title:",
+    "title：",
+    "paper title:",
+    "paper title：",
+    "论文标题:",
+    "论文标题：",
+    "链接:",
+    "链接：",
+    "link:",
+    "link：",
+    "arxiv:",
+    "arxiv：",
+)
+
 
 def slugify(text: str) -> str:
     slug = re.sub(r"[^a-zA-Z0-9\u4e00-\u9fff\s-]", "", text).strip().lower()
@@ -78,11 +111,24 @@ def classify_subcategory(text: str) -> str:
     return best if scores[best] > 0 else "general"
 
 
+def _is_metadata_line(line: str) -> bool:
+    lower = line.strip().lower()
+    if lower.startswith(METADATA_PREFIXES):
+        return True
+
+    # Skip pure arXiv URL lines, but keep explanatory sentences that merely contain a URL.
+    pure_arxiv_url = re.fullmatch(
+        r"https?://arxiv\.org/(?:abs|pdf)/\d{4}\.\d{4,5}(?:v\d+)?/?",
+        lower,
+    )
+    return pure_arxiv_url is not None
+
+
 def summarize_points(content: str, max_points: int = 6) -> list[str]:
     lines = [ln.strip(" -\t") for ln in content.splitlines() if ln.strip()]
     points = []
     for line in lines:
-        if len(line) < 8:
+        if len(line) < 8 or _is_metadata_line(line):
             continue
         points.append(line)
         if len(points) >= max_points:
@@ -121,6 +167,8 @@ def _collect_tags(subcategory: str, source_text: str) -> list[str]:
         "evaluation": ["eval", "benchmark", "评测", "hallucination"],
         "training": ["sft", "rlhf", "finetune", "训练", "蒸馏"],
         "deployment": ["inference", "serving", "部署", "latency"],
+        "paper-reading": ["paper", "arxiv", "method", "实验", "ablation"],
+        "productivity": ["复盘", "行动项", "计划", "时间管理", "学习方法"],
     }
 
     for tag, markers in topic_tags.items():
@@ -129,7 +177,58 @@ def _collect_tags(subcategory: str, source_text: str) -> list[str]:
         if any(_keyword_hit(normalized_text, marker) for marker in markers):
             tags.append(tag)
 
-    return tags[:5]
+    # Deduplicate while preserving order.
+    deduped_tags = list(dict.fromkeys(tags))
+
+    # Add stable fallback tags to keep tags informative even on sparse inputs.
+    fallback_tags = {
+        "paper-reading": ["paper", "reading-notes"],
+        "productivity": ["learning-method", "action-items"],
+    }
+    for fallback in fallback_tags.get(subcategory, ["learning-notes"]):
+        if fallback not in deduped_tags:
+            deduped_tags.append(fallback)
+        if len(deduped_tags) >= 3:
+            break
+
+    return deduped_tags[:5]
+
+
+def _extract_field_value(line: str, field: str) -> str | None:
+    pattern = rf"^\s*{re.escape(field)}\s*[:：]\s*(.+?)\s*$"
+    match = re.match(pattern, line, flags=re.IGNORECASE)
+    if match:
+        value = match.group(1).strip()
+        return value or "待补充"
+    return None
+
+
+def _extract_paper_title(source_text: str) -> str:
+    lines = [ln.strip() for ln in source_text.splitlines() if ln.strip()]
+    title_fields = ("title", "paper title", "论文标题")
+    for line in lines:
+        for field in title_fields:
+            value = _extract_field_value(line, field)
+            if value is not None:
+                return value
+
+    quoted = re.search(r"[《\"]([^》\"]{8,160})[》\"]", source_text)
+    if quoted:
+        return quoted.group(1).strip()
+
+    return "待补充"
+
+
+def _extract_arxiv_link(source_text: str) -> str:
+    url_match = re.search(r"https?://arxiv\.org/(?:abs|pdf)/\d{4}\.\d{4,5}(?:v\d+)?", source_text, re.IGNORECASE)
+    if url_match:
+        return url_match.group(0)
+
+    id_match = re.search(r"arxiv\s*[:：]\s*(\d{4}\.\d{4,5}(?:v\d+)?)", source_text, re.IGNORECASE)
+    if id_match:
+        return f"https://arxiv.org/abs/{id_match.group(1)}"
+
+    return "待补充"
 
 
 def build_post(
@@ -147,9 +246,29 @@ def build_post(
         "",
         "本篇为学习对话自动沉淀的笔记，保留结论和可复用要点。",
         "",
+    ]
+
+    if subcategory == "paper-reading":
+        paper_title = _extract_paper_title(source_text)
+        arxiv_link = _extract_arxiv_link(source_text)
+        body.extend(
+            [
+                "## 论文信息",
+                "",
+                f"- 论文标题：{paper_title}",
+                "- 核心任务：待补充",
+                "- 主要贡献：待补充",
+                f"- 链接：{arxiv_link}",
+                "",
+                "## 方法与实验要点",
+                "",
+            ]
+        )
+
+    body.extend([
         "## 结论速记",
         "",
-    ]
+    ])
     body.extend([f"- {point}" for point in bullets])
     body.extend(
         [
@@ -167,7 +286,7 @@ def build_post(
             f'title: "{_yaml_escaped(title)}"',
             f"date: {date.isoformat()} 09:00:00 +0800",
             f'categories: ["{CATEGORY}", "{subcategory}"]',
-            "tags: [" + ", ".join(f'"{tag}"' for tag in tags) + "]",
+            "tags: [" + ", ".join(f'\"{tag}\"' for tag in tags) + "]",
             "---",
             "",
         ]
@@ -197,7 +316,7 @@ def main() -> None:
     parser.add_argument(
         "--subcategory",
         default=None,
-        help="Force specific subcategory (e.g. rag/agents/general)",
+        help="Force specific subcategory (e.g. rag/agents/paper-reading/productivity/general)",
     )
 
     args = parser.parse_args()

skills/ai-auto-note-publisher/scripts/test_create_ai_auto_note.py

@@ -0,0 +1,85 @@
+import sys
+from pathlib import Path
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+if str(SCRIPT_DIR) not in sys.path:
+    sys.path.insert(0, str(SCRIPT_DIR))
+
+import datetime as dt
+import unittest
+
+from create_ai_auto_note import (
+    _extract_arxiv_link,
+    _extract_paper_title,
+    build_post,
+    classify_subcategory,
+    summarize_points,
+)
+
+
+class AutoNoteTests(unittest.TestCase):
+    def test_classify_paper_reading(self):
+        text = "这篇paper在arxiv上发布，包含ablation实验和baseline对比"
+        self.assertEqual(classify_subcategory(text), "paper-reading")
+
+    def test_classify_productivity(self):
+        text = "今天复盘学习计划，明确todo和行动项"
+        self.assertEqual(classify_subcategory(text), "productivity")
+
+    def test_extract_paper_metadata(self):
+        text = "Title: Test-Time Scaling\n链接：https://arxiv.org/abs/2501.12345"
+        self.assertEqual(_extract_paper_title(text), "Test-Time Scaling")
+        self.assertEqual(_extract_arxiv_link(text), "https://arxiv.org/abs/2501.12345")
+
+
+    def test_extract_arxiv_id_format(self):
+        text = "该论文编号 arXiv:2501.12345v2"
+        self.assertEqual(_extract_arxiv_link(text), "https://arxiv.org/abs/2501.12345v2")
+
+    def test_extract_paper_title_with_fullwidth_colon(self):
+        text = "论文标题：使用 CoT 增强推理能力"
+        self.assertEqual(_extract_paper_title(text), "使用 CoT 增强推理能力")
+
+    def test_build_post_paper_reading_contains_sections(self):
+        source = """Title: Test-Time Scaling
+我们讨论了method与ablation结果。
+链接：https://arxiv.org/abs/2501.12345
+下一步复现实验。"""
+        _, post = build_post(
+            title="论文测试",
+            source_text=source,
+            date=dt.date(2026, 3, 16),
+            forced_subcategory="paper-reading",
+        )
+        self.assertIn("## 论文信息", post)
+        self.assertIn("- 论文标题：Test-Time Scaling", post)
+        self.assertIn("- 链接：https://arxiv.org/abs/2501.12345", post)
+        self.assertIn("## 方法与实验要点", post)
+        # metadata lines should not be duplicated as bullets in summary
+        self.assertNotIn("- Title: Test-Time Scaling", post)
+        self.assertNotIn("- 链接：https://arxiv.org/abs/2501.12345", post.split("## 结论速记", 1)[1])
+
+    def test_summarize_points_keeps_explanatory_line_with_arxiv_url(self):
+        source = """我们参考了 https://arxiv.org/abs/2501.12345 并总结了关键实验结论。"""
+        points = summarize_points(source)
+        self.assertEqual(points, ["我们参考了 https://arxiv.org/abs/2501.12345 并总结了关键实验结论。"])
+
+
+    def test_build_post_includes_fallback_tags_for_sparse_input(self):
+        _, post = build_post(
+            title="简短记录",
+            source_text="仅有一行简短内容。",
+            date=dt.date(2026, 3, 16),
+            forced_subcategory="paper-reading",
+        )
+        self.assertIn('tags: ["ai-auto-note", "paper-reading", "paper"', post)
+    def test_summarize_points_skips_metadata_lines(self):
+        source = """Title: A
+链接：https://arxiv.org/abs/2501.12345
+这是有效总结内容，长度足够。"""
+        points = summarize_points(source)
+        self.assertEqual(points, ["这是有效总结内容，长度足够。"])
+
+
+if __name__ == "__main__":
+    unittest.main()

skills/ai-auto-note/SKILL.md

@@ -9,13 +9,15 @@ description: "Summarize completed AI learning conversations into publishable Jek
 
 Convert finalized learning conversations into structured, publication-ready notes under `_posts/`, with consistent metadata and stable categorization.
 
+> For repository automation and PR creation, prefer using `skills/ai-auto-note-publisher` scripts after drafting content.
+
 ## Workflow
 
 1. Confirm the source material is conclusion-ready.
 2. Extract key conclusions, evidence, and next actions.
 3. Map the note to one subcategory using `references/subcategory-taxonomy.md`.
 4. Generate a new post file from `assets/post-template.md`.
-5. Save the file to `_posts/YYYY-MM-DD-ai-auto-note-<slug>.md`.
+5. Save the file to `_posts/ai-auto-note/<subcategory>/YYYY-MM-DD-<slug>.md`.
 6. Keep `categories` in this exact order:
    - `ai-auto-note`
    - `<subcategory>`