🚗 NIO-Weaver

NIO-Weaver is a graph-augmented analytics pipeline that converts natural-language CAN analysis requests into executable SQL workflows.

Item	Description
Domain	Vehicle CAN signal analytics
Input	Natural-language analysis request
Output	Parsed signals, intermediate artifacts, and executable SQL
Core stack	LLM + embedding retrieval + graph reasoning + SQL generation

🗺️ Quick Navigation

• Architecture
• Project Structure
• Requirements
• Quick Start
• Configuration Guide
• Common Workflows
• Outputs
• Troubleshooting

🏗️ Architecture

1. Build a domain graph from signal metadata.
2. Parse user intent and relevant entities from a query.
3. Retrieve candidate signals and context.
4. Generate SQL with decomposition and iterative refinement.
5. Optionally run candidate voting to select the best final SQL.

🗂️ Project Structure

NIO-Weaver/
├─ LLMAsAnalyst.py              # Main orchestration class
├─ main.py                      # Example pipeline script
├─ prompts/
│  └─ prompt/
│     ├─ prompt.yaml            # Core prompts
│     ├─ decompose.yaml         # SQL decomposition prompts
│     └─ finalvote.yaml         # Candidate SQL evaluation prompts
├─ src/
│  ├─ config.yaml               # Runtime configuration
│  ├─ graphbuild.py             # Graph building pipeline
│  ├─ signalparser.py           # Query/signal parsing pipeline
│  ├─ sqlgenerate.py            # SQL generation pipeline
│  ├─ Models/                   # LLM + embedding implementations
│  ├─ GraphBuilderModules/      # Graph-building submodules
│  ├─ SignalSelectModules/      # Signal-retrieval submodules
│  └─ utils/                    # Shared utilities
└─ README.md

✅ Requirements

🧩 Runtime

• Python 3.10+ (recommended)
• Neo4j
• LLM provider credentials

📚 Python Dependencies

pyyaml, python-dotenv, pandas, tqdm, openai, anthropic, tiktoken, sentence-transformers, aiohttp, neo4j

🚀 Quick Start

1) 🔧 Initialize

from LLMAsAnalyst import LLMAsAnalyst

analyst = LLMAsAnalyst()

2) 🕸️ Build Graph

analyst.buildgraph()

3) 🧠 Parse a Single Query

query = "Analyze MAI activation events under specific driving conditions."
analyst.signalparse(query)

4) 🧾 Generate SQL in Batch

analyst.batch_generate_sql()
analyst.batch_iterative_sql_generation()

5) 🗳️ Final Candidate Voting

analyst.batch_final_sql_vote()

⚙️ Configuration Guide

Configuration file: src/config.yaml

Section	Purpose
llm.*	LLM provider/model/API settings
embedding_model.*	Embedding provider/model/API settings
neo4j.*	Graph database connection
entity_extraction	Entity extraction behavior
entity_linking	Entity linking behavior
signal_parse	Signal parsing and retrieval settings
sql_generate	SQL generation settings
log_files.*	Log output paths

Before running:

1. Fill api_key and model in config.
2. Confirm Neo4j connection settings.
3. Keep prompt files under prompts/prompt/ unless you also update loader logic.

🛠️ Common Workflows

📥 Batch Query Parsing

analyst.batch_process_queries(
    input_file="data/input/queries.xlsx",
    checkpoint_dir="data/output/batch_results/checkpoints"
)

🧪 Batch Debug from Mid-Pipeline

analyst.batch_debug_queries(
    input_file="data/input/queries.xlsx",
    checkpoint_dir="data/process/debug_checkpoints"
)

🔁 One-Query Iterative SQL Generation

result = analyst.single_iterative_sql_generation(
    query_index=0,
    retry_times=5,
    max_iterations=3,
    top_k_additional=6
)

📦 Outputs

Typical outputs are generated under data/output/:

• parsed signal artifacts
• SQL generation results
• iterative checkpoints
• evaluation/voting outputs

Logs are written under logs/ according to src/config.yaml.

📝 Operational Notes

1. Prompt YAML files are runtime dependencies. Keep syntax valid.
2. Batch APIs with checkpointing are recommended for long jobs.
3. Validate generated SQL in your target engine before production usage.
4. In production, avoid logging raw user-sensitive content.

🧯 Troubleshooting

Symptom	Likely Cause	Action
ModuleNotFoundError	Missing dependency	Install required Python packages
Neo4j connection failed	Invalid neo4j.* settings	Verify URI/user/password
LLM request failed	Invalid provider or API config	Check llm.provider/model/api_key/api_base
Prompt loading failed	Missing prompt files	Confirm prompts/prompt/*.yaml exists