Skip to content

Commit a63f9fe

Browse files
committed
docs: update daily report with Session 5 — LanceDB fix, ingestion, LLM switch, Rust modules
1 parent ebb13f7 commit a63f9fe

1 file changed

Lines changed: 74 additions & 0 deletions

File tree

uet_web/Docs_software_design/daily_reports/2026-03-20.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,3 +284,77 @@ POST /chat {"prompt": "UET คืออะไร"} → 200 OK
284284
8. ✅ Safe __init__.py imports
285285
9. ✅ API key security (.env, not hardcoded)
286286
10. ✅ End-to-end verified (server starts, LLM responds, memory works)
287+
288+
---
289+
290+
## Session 5: LanceDB Fix, Knowledge Ingestion, LLM Switch, Rust Modules (same day)
291+
292+
### LanceDB Vector Schema Fix
293+
294+
The `OmegaSearch` engine was failing with:
295+
```
296+
RuntimeError: Data type is not a vector (FixedSizeListArray), but Float64
297+
```
298+
299+
**Root cause**: Vectors stored as `list<double>` (variable-length) instead of `fixed_size_list<float>[N]` (fixed-length) that LanceDB requires for ANN search.
300+
301+
**Fix**: Created `research_uet/knowledge_base/migrate_lance_schema.py` that:
302+
1. Reads all 5609 rows from old table
303+
2. Builds explicit PyArrow schema with `pa.list_(pa.float32(), N)`
304+
3. Drops old table, creates new with proper vector types
305+
4. Verifies vector search works
306+
307+
Result: `semantic_vec: fixed_size_list<float>[8]`, `uet_vec: fixed_size_list<float>[20]`
308+
309+
### Knowledge Ingestion
310+
311+
Created `uet_agents/ingest_docs.py` — bulk ingestion script that reads all markdown docs and feeds them to the Semantic Engine.
312+
313+
- **116 documents** from `research_uet/Docs/` and `research_uet/Doc/`
314+
- **5984 chunks**, **18085 vocab terms** (up from 7 test chunks)
315+
- Supports both API mode and direct mode
316+
317+
### LLM Model Switch
318+
319+
Switched from `qwen/qwen-2.5-7b-instruct` (mixed Chinese mid-response) to `z-ai/glm-4.7-flash`:
320+
- Responds in Thai consistently ✅
321+
- References source data accurately ✅
322+
- Handles reasoning model format (`content` + `reasoning` fields)
323+
- $0.06/M input tokens (cheap)
324+
325+
Also fixed: `config.toml` with API keys was tracked in Git → untracked and added to `.gitignore`
326+
327+
### Rust Modules Integration
328+
329+
Added 4 under-development Rust modules to workspace:
330+
331+
| Module | Files | Status |
332+
|--------|-------|--------|
333+
| `uet_economic` | 6 source files (difficulty, issuance, portfolio) | ✅ Compiles |
334+
| `uet_governance` | 6 source files (voting, proposals, policy) | ✅ Compiles |
335+
| `uet_market` | 7 source files (AMM, price discovery) | ✅ Compiles |
336+
| `uet_oracle` | 5 source files (verifiers, registry) | ✅ Compiles |
337+
338+
Fixed: Cargo.toml dependency paths (`../uet_chain``../../uet_chain`)
339+
340+
### OmegaSearch Test Results
341+
342+
```
343+
Query: "galaxy rotation curve dark matter"
344+
Result 1: Cluster Virial Data (Ω-dist: 0.0000, axiom overlap: 100%)
345+
Result 2: Chandra A133 (Ω-dist: 0.0000, axiom overlap: 100%)
346+
Result 3: Pioneer Anomaly Data (Ω-dist: 0.0000, scale match: ✅)
347+
```
348+
349+
### Complete task list for today
350+
351+
1. ✅ Git cleanup, Docker/Prisma, CI/CD, repo hygiene
352+
2. ✅ Agent infra (CORS, Dockerfile, compose, env vars)
353+
3. ✅ API key security (.env, config.toml untracked)
354+
4. ✅ LLM integration verified end-to-end
355+
5. ✅ Knowledge ingestion (116 docs → 5984 chunks)
356+
6. ✅ LLM model switch (GLM 4.7 Flash)
357+
7. ✅ LanceDB vector schema fix (fixed_size_list)
358+
8. ✅ OmegaSearch verified working
359+
9. ✅ 4 Rust modules added to workspace and compiling
360+
10. ✅ Dependency paths fixed for all modules

0 commit comments

Comments
 (0)