Skip to content

feat: exam-memory V2 — numpy-based semantic retrieval, question bank, knowledge sources #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Tenstu merged 1 commit into from
Jun 17, 2026
Merged

feat: exam-memory V2 — numpy-based semantic retrieval, question bank, knowledge sources #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 28 additions & 4 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,34 @@ Thumbs.db
input.txt
transformer-forward-pass.md
transformer-review.md
.tmp/

# 考试经验沉淀系统 - 运行时个人数据（代码和配置应被跟踪）
exam_memory/experiences/
exam_memory/user_profile.json
exam_memory/vectorstore/
shared/exam_memory/experiences/
shared/exam_memory/bank/*.md
!shared/exam_memory/bank/README.md
shared/exam_memory/user_profile.json
shared/exam_memory/vectorstore/

# V2 临时验证脚本（不提交）
_extract.py
_test_v2.py
_verify_v2.py
_verify_result.json

# V2 review regression tests（dev-only，不同步 clean-main）
tests/test_chunking.py
tests/test_fts_store.py
tests/test_hybrid_search.py
tests/test_security.py
tests/test_server.py
tests/test_vector_store.py

# 个人备考数据（保留目录框架，忽略实际数据文件）
# HANDOFF.md 作为模板跟踪（每轮备考时填写具体数据，提交前应清空）
daily/*
!daily/README.md
shared/daily/*.md

progress/choice-questions/*
!progress/choice-questions/.gitkeep
Expand All @@ -49,13 +61,25 @@ progress/task-board/*
!progress/task-board/.gitkeep
progress/reviews/*
!progress/reviews/.gitkeep
shared/progress/**
!shared/progress/README.md
!shared/progress/**/.gitkeep
targets/*/progress/**
!targets/*/progress/**/.gitkeep
targets/*/mistake_log.md
targets/*/mock_exam_log.md

# MCP 与个人记忆工具（仅本地，不入 git）
.mcp.json
.claude/
.codex/
.serena/
.chatmem/
.mempalace/

# 个人分支私有文件（dev-only，不允许合并到 clean-main）
skills/branch-ops.md
docs/branch-workflow.md
skills/harness-dev-flow/
skills/dev-review-flow/
docs/
prompts/review-fix-session-prompt.md
97 changes: 54 additions & 43 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Pass-LLM-with-LLM
# pass-llm-with-llm

> Use LLM to pass LLM exams — a Claude Code Skills + MCP powered AI exam preparation engine

Expand Down Expand Up @@ -34,19 +34,23 @@ Built for AI/算法岗位笔试 preparation, but the core Skill Pipeline is exam
| Model Provider | Any provider supported by Claude Code (Claude API, third-party, local) |
| Python | 3.10+ (only for exam-memory MCP Server) |

This project was developed using the **Claude Code VS Code extension** with third-party model providers (Xiaomi mimo-v2.5-pro, StepFun step-3.7-flash). The Skill Pipeline is model-agnostic — any capable model works. See [Environment Support](docs/environment-support.md) for details.
This project was developed using the **Claude Code VS Code extension** with third-party model providers. The Skill Pipeline is model-agnostic — any capable model works.

### Install

```bash
git clone https://github.com/Tenstu/Pass-LLM-with-LLM.git
cd Pass-LLM-with-LLM
git clone https://github.com/Tenstu/pass-llm-with-llm.git
cd pass-llm-with-llm
```

### Configure Your Target Exam

Edit `HANDOFF.md` with your exam name, date, and daily study hours. Update `targets/{target}/sources/` with your exam's historical patterns.

### Use

1. Open the project in Claude Code
2. **First time?** Say "init" or "初始化" — the init-guide skill will collect your exam target, date, and daily study hours, then configure `HANDOFF.md`, `sources/`, `AGENTS.md`, and user profile automatically
2. **First time?** Say "init" or "初始化" to launch the onboarding guide — it collects your exam target, date, and scope
3. Daily use: read `START_HERE.md` for session bootstrap
4. For algorithm problems: `Skill(skill="solve-skeleton")`
5. For diagnosis: `Skill(skill="solve-analyze")`
Expand All @@ -55,22 +59,21 @@ cd Pass-LLM-with-LLM
### Startup Order

```
git clone → cd Pass-LLM-with-LLM
git clone → cd pass-llm-with-llm
│
├── pip install mcp # optional: for exam-memory MCP server
│
├── edit .mcp.json # register exam-memory (see docs/mcp-setup-guide.md)
├── edit .mcp.json # register exam-memory, pointing to shared/exam_memory/server.py
│
├── open in Claude Code
│ │
│ ├── first time → say "init" → init-guide auto-configures everything
│ │ (HANDOFF.md, sources/, AGENTS.md, user profile)
│ ├── first time → say "init" → init-guide Skill walks you through setup
│ │
│ └── daily use → read START_HERE.md → Skill Pipeline
│
└── (optional) configure external MCPs: ChatMem, mempalace, onefind
these are environment-level, not project-bundled
see docs/mcp-setup-guide.md for references
configure these in your Claude Code environment if needed
```

### MCP Dependencies
Expand All @@ -84,7 +87,7 @@ This project **bundles one MCP server** (`exam-memory`) and **references externa
| mempalace | No | Structured knowledge storage | [External install](https://github.com/MemPalace/mempalace) |
| onefind | No | Local knowledge base retrieval | [External install](https://github.com/iawnfoanaowt/OneFind) |

All skills degrade gracefully to local-only mode when MCP is unavailable. See [MCP Setup Guide](docs/mcp-setup-guide.md) for full configuration instructions.
All skills degrade gracefully to local-only mode when MCP is unavailable. To enable the bundled server, register `.mcp.json` with a stdio command that runs `shared/exam_memory/server.py`.

## Skill Reference

Expand Down Expand Up @@ -112,7 +115,7 @@ All skills with "Optional" MCP degrade gracefully to local-only mode when MCP is
| init-guide | Remembers previous onboarding attempts; avoids re-collecting known info |
| solve-analyze | Links diagnosis history across sessions for pattern recognition |

Install ChatMem and register it in your Claude Code global config. See [MCP Setup Guide](docs/mcp-setup-guide.md).
Install ChatMem and register it in your Claude Code global config.

### MemPalace Enhancement (Optional)

Expand All @@ -133,21 +136,21 @@ Best paired with review-tracker (knowledge graph for coverage gaps) and exam-ass
| Use Case | How It Helps |
|----------|-------------|
| Obsidian notes | Search your existing ML/algorithm notes for related concepts when practicing |
| Zotero library | Retrieve reference papers for Transformer, GNN, Diffusion topics in `llm/` cheatsheets |
| Zotero library | Retrieve reference papers for Transformer, GNN, Diffusion topics in `shared/cheatsheets/` or `targets/{target}/cheatsheets/` |
| Hybrid search | Combine lexical + semantic search across all local sources |

Best paired with choice-q-create (search notes for question material), exam-assistant (retrieve references during explanation), and review-tracker (check if your notes cover the required topics). Not directly called by any bundled skill — use OneFind tools manually via Claude Code.

#### OneFind + exam-memory: Complementary Search Layers

OneFind's **folder source** can index `exam_memory/experiences/` for semantic search. However, OneFind is designed as a **read-only retrieval** layer — it cannot replace exam-memory's write-through pipeline (save experience → vectorize → store atomically). The recommended setup:
OneFind's **folder source** can index `shared/exam_memory/experiences/` for semantic search. However, OneFind is designed as a **read-only retrieval** layer — it cannot replace exam-memory's write-through pipeline (save experience → vectorize → store atomically). The recommended setup:

| Layer | Role | Write | Read |
|-------|------|:-----:|:----:|
| `exam-memory` MCP | Experience CRUD + error counting + user profiling | Yes (save, update) | Yes (list, filter by type) |
| OneFind folder source | Semantic search overlay on experience files | No (index refresh only) | Yes (semantic + keyword) |

**Setup**: Configure OneFind's `folder_library` to point at `exam_memory/experiences/`, then use `onefind_search` with `target="folder"` for semantic retrieval of past experiences. After saving new experiences via MCP, trigger `onefind_index_refresh` to pick up changes.
**Setup**: Configure OneFind's `folder_library` to point at `shared/exam_memory/experiences/`, then use `onefind_search` with `target="folder"` for semantic retrieval of past experiences. After saving new experiences via MCP, trigger `onefind_index_refresh` to pick up changes.

## Roadmap

Expand Down Expand Up @@ -184,7 +187,7 @@ Upgrade `exam-memory` from keyword matching to semantic search:
## Directory Structure

```
Pass-LLM-with-LLM/
pass-llm-with-llm/
AGENTS.md # Project rules, Component Map, Skill Pipeline
START_HERE.md # Session bootstrap + Skill invocation guide
HANDOFF.md # Session handoff template
Expand All @@ -201,39 +204,47 @@ Pass-LLM-with-LLM/
exam-assistant.md # MCP-backed exam assistant
review-tracker.md # Progress aggregation

algorithms/ # OJ code assets
python_oj_template.py # Utility function library
solutions_batch.py # Exam problem solution collection
practice/ # Practice problems by topic
mistake_log.md # WA/TLE error patterns
topic_checklist.md # Topic coverage tracking

llm/ # AI/ML quick-reference notes
llm_core_cheatsheet.md # Transformer, SFT/LoRA, RLHF, RAG, KV cache
gnn_diffusion_cheatsheet.md
math_fundamentals.md
ai_lab_context.md # Target company/lab background

exam_memory/ # Custom MCP Server (optional)
server.py # 6 MCP tools for experience persistence
experiences/ # Experience files (YAML frontmatter + Markdown)
user_profile.json # User strengths/weaknesses/preferences

daily/ # Daily plans (YYYY-MM-DD.md)
progress/ # Progress tracking by category
sources/ # External reference materials
targets/ # Target-specific exam content
ai-lab/
exam_config.md # Exam format and scoring parameters
cheatsheets/ # Target-specific AI/ML quick-reference notes
daily/ # Target-specific daily plans
progress/ # Choice rounds, study planning, exam analysis, task board
prompts/ # Target-specific prompt templates
sources/ # Historical patterns and target-specific references
pdd-algo/
exam_config.md # PDD algorithm exam configuration
python_oj_template.py # Utility function library
solutions_batch.py # Exam problem solution collection
practice/ # Practice problems by topic
solutions/ # Individual solution write-ups
mistake_log.md # WA/TLE error patterns
topic_checklist.md # Topic coverage tracking
progress/ # Target-specific progress tracking

shared/ # Cross-target shared content
cheatsheets/ # Generic LLM/ML/project quick-reference notes
daily/ # Shared daily plans (YYYY-MM-DD.md)
exam_memory/ # Custom MCP server and experience store
server.py # MCP tools for experience persistence
experiences/ # Experience files (YAML frontmatter + Markdown)
user_profile.json # User strengths/weaknesses/preferences
progress/ # Shared progress/task-board files
prompts/ # Generic prompt templates

algorithms/ # Legacy stub; active OJ assets live under targets/
exam_memory/ # Legacy stub; active MCP code lives under shared/exam_memory/
progress/ # Legacy stub; active progress lives under shared/ or targets/
prompts/ # Prompt templates
docs/ # Design documents, setup guides, and license
mcp-setup-guide.md # MCP configuration: exam-memory + external MCP references
```

## Adapting to Other Exams

The framework defaults to AI Lab exam format but is designed to be reconfigured:
The framework defaults to the configured target in `HANDOFF.md` and is designed to be reconfigured:

1. Replace `sources/ai_lab_history_problems.md` with your target exam's pattern analysis
2. Update `skills/choice-q-create.md` question generation parameters
3. Swap `llm/` notes with your domain's core knowledge
1. Add or replace files under `targets/{target}/sources/` with your target exam's pattern analysis
2. Update `targets/{target}/exam_config.md` with question counts, scoring, and timing
3. Put target-specific notes under `targets/{target}/cheatsheets/`; keep reusable notes under `shared/cheatsheets/`
4. Adjust `AGENTS.md` Exam Format table

## Contributing
Expand Down
Loading