A multi-agent Python bot that answers analytical questions about the Bitext Customer Support LLM Chatbot Training Dataset. Ask questions in plain English — the bot routes them to the right agent, runs Python code against the data where needed, and returns a clean answer.
Features session persistence (LangGraph + SQLite), follow-up query resolution, a per-user profile system, and an MCP server for tool integration.
This bot is a dataset analysis assistant, not a customer-facing chatbot. It is designed to let you explore and interrogate the Bitext dataset interactively:
- "How many unique intents are there?"
- "Show me 5 examples of the cancel_order intent."
- "Show me 5 more." ← follow-up resolved automatically
- "Summarize what customers complain about."
- "What do you remember about me?" ← user profile recall
All inference runs through the Nebius API — no local GPU required.
Each user turn passes through a two-node LangGraph graph before returning an answer:
User query
│
▼
┌─────────────────────────────────────────────────────────────┐
│ chat node │
│ │
│ 1. Detect profile-recall query? ─── yes ──► return profile │
│ │ no │
│ 2. Follow-up resolver │
│ (rewrite "show 3 more" → "show examples 4-6 of X") │
│ │ │
│ 3. Orchestrator ── classifies query ──► routing decision │
│ │ │
│ ├─ STRUCTURED ──► Coding Agent │
│ │ generates + executes pandas code │
│ │ formats result │
│ │ │
│ ├─ UNSTRUCTURED ► Coding Agent (extract samples) │
│ │ │ │
│ │ Summarization Agent │
│ │ writes narrative answer │
│ │ │
│ └─ OUT_OF_SCOPE ► Out-of-scope Agent │
│ polite decline + redirect │
└─────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────┐
│ update_profile node │
│ LLM extracts new facts │
│ merges into profile file│
└──────────────────────────┘
│
▼
Answer returned + state checkpointed to sessions.db
sessions.db ← LangGraph SQLite checkpoint (conversation history)
profiles/
{session_id}.json ← distilled user profile (name, topics, preferences)
Both files survive process restarts. The same session_id restores both.
| Item | Value |
|---|---|
| Source | bitext/Bitext-customer-support-llm-chatbot-training-dataset via HuggingFace |
| Size | ~26,872 rows (quality filter flags="B") |
| Columns | flags, instruction, category, intent, response |
| Intents | 27 (e.g. cancel_order, track_order, payment_issue, get_refund) |
| Categories | 11 (e.g. REFUND, SHIPPING, ACCOUNT, PAYMENT) |
Used by all agents and supporting components.
Why Llama 3.3 70B:
- Strong instruction-following for code generation and classification tasks
- Reliable structured output (clean Python, single-word classifications, JSON facts)
- OpenAI-compatible API — no SDK changes needed
- Cost-effective at Nebius pricing for a 70B-class model
- Version 3.3 has improved tool-calling and instruction adherence over 3.1
Why not a smaller model (e.g. 7B/8B): Code generation quality degrades significantly at smaller sizes. The coding agent must produce correct pandas expressions from natural language, which requires strong reasoning and deep knowledge of the pandas API.
| Library | Purpose |
|---|---|
datasets |
Load Bitext dataset directly from HuggingFace |
pandas |
In-memory DataFrame that all queries run against |
openai |
Python SDK — talks to Nebius OpenAI-compatible endpoint |
python-dotenv |
Load NEBIUS_API_KEY and base_url from .env |
tenacity |
Exponential backoff for Nebius API rate-limit errors |
langgraph |
State graph, add_messages reducer, graph compilation |
langgraph-checkpoint-sqlite |
SQLite-backed checkpoint saver for session persistence |
langchain-core |
RunnableConfig — passes thread_id into graph nodes |
fastmcp |
MCP server framework — exposes tools to MCP clients |
task3/
├── .env # NEBIUS_API_KEY + base_url (never commit)
├── requirements.txt
├── run.py # Interactive loop with session ID support
├── app.py # Streamlit chat UI
├── mcp_server.py # FastMCP server — 4 MCP tools
├── sessions.db # LangGraph SQLite checkpoint (auto-created)
├── profiles/ # Per-user profile files (auto-created)
│ └── {session_id}.json
├── PLAN.md
└── src/
├── nebius_client.py # Shared OpenAI client pointed at Nebius
├── loader.py # HuggingFace dataset → pandas DataFrame
├── graph.py # LangGraph state graph + SQLite checkpointer
├── followup_resolver.py # Rewrites follow-up queries as self-contained
├── orchestrator.py # Classifies query and routes to correct agent
├── coding_agent.py # Generates + safely executes pandas code
├── summarization_agent.py # Writes narrative answers from extracted data
├── out_of_scope_agent.py # Handles off-topic queries
└── profile_manager.py # Load / save / update / format user profiles
- Python 3.10+
- A Nebius API key with access to
meta-llama/Llama-3.3-70B-Instruct
pip install -r requirements.txtCreate .env in the working directory root:
NEBIUS_API_KEY=your_key_here
base_url=https://api.studio.nebius.ai/v1/python run.pyOn first run you are assigned a new session ID. Pass that ID on the next run to restore your conversation and profile.
+------------------------------------------+
| Bitext Dataset Q&A Bot (multi-agent) |
| Type your question, or 'quit' to exit |
+------------------------------------------+
Session ID (press Enter for new session): ← press Enter or paste a previous ID
New session: 3f2a1b...
Loaded 26,872 rows.
Session: 3f2a1b...
You: show me 3 examples from the REFUND category
[router] STRUCTURED
Bot: Here are 3 examples from the REFUND category: ...
You: show me 3 more
[resolved] show me 3 examples from the REFUND category starting at index 3
[router] STRUCTURED
Bot: Here are 3 more examples...
You: what do you remember about me?
Bot: I don't have any profile information about you yet...
You: quit
Session saved: 3f2a1b...
Goodbye.
Resuming a previous session:
Session ID (press Enter for new session): 3f2a1b...
Resuming session: 3f2a1b... (4 previous turns)
Last Q: show me 3 examples from the REFUND category
Last A: Here are 3 examples from the REFUND category...
streamlit run app.pyOpens at http://localhost:8501. The dataset loads once on first request and is
cached for the lifetime of the server process.
![Streamlit UI layout]
| Area | What it shows |
|---|---|
| Sidebar — Session | Current session ID, New session button, Restore-by-ID input |
| Sidebar — Profile | Live profile panel — updates after every turn |
| Sidebar — Dataset | Row count, intent count, category count |
| Chat area | Full conversation history, restored from checkpoint on page load |
| Routing badge | Collapsible expander per reply — shows 🟢 Structured / 🔵 Unstructured / ⚪ Out of scope and any resolved follow-up query |
python src/main.py "distribution of intents"
python src/main.py "what delivery issues do customers report?"# stdio transport — connect from any MCP client
python mcp_server.py
# Browser-based MCP Inspector (interactive testing)
fastmcp dev mcp_server.pyClaude Desktop config (claude_desktop_config.json):
{
"mcpServers": {
"bitext-bot": {
"command": "python",
"args": ["<absolute-path-to-working-directory>/mcp_server.py"]
}
}
}Use FastMCP's built-in async client to call tools from any Python script.
The client launches mcp_server.py as a subprocess and communicates over stdio.
# client_example.py
import asyncio
from fastmcp import Client
async def main():
async with Client("mcp_server.py") as client:
# List available tools
tools = await client.list_tools()
print("Available tools:", [t.name for t in tools])
# → ['query_dataset', 'summarize_topic', 'get_dataset_stats', 'get_user_profile']
# Tool 1 — instant schema overview (no LLM)
stats = await client.call_tool("get_dataset_stats")
print(stats.content[0].text)
# Tool 2 — structured query
result = await client.call_tool(
"query_dataset",
{"query": "how many rows are in the REFUND category?"}
)
print(result.content[0].text)
# → "There are 2,992 rows in the REFUND category."
# Tool 3 — narrative summary
summary = await client.call_tool(
"summarize_topic",
{"topic": "what cancellation issues do customers face?"}
)
print(summary.content[0].text)
# Tool 4 — user profile
profile = await client.call_tool(
"get_user_profile",
{"session_id": "your-session-id"}
)
print(profile.content[0].text)
asyncio.run(main())Run from the working directory root (so .env is found):
python client_example.pyEach
call_toolreturns aCallToolResult;.content[0].textholds the string result. The server loads the dataset on first connection — expect a few seconds on the first call.
The coding agent generates and runs pandas code, then formats the result.
| Example question | What happens |
|---|---|
| How many unique categories? | df['category'].nunique() |
| List all intents | df['intent'].unique() |
| Distribution of intents | df['intent'].value_counts() |
| Show 5 examples of cancel_order | filter + head(5) |
| How many rows in the PAYMENT category? | filter + shape[0] |
| How many refund-related rows? | str.contains('refund') filter |
| Example question | What happens |
|---|---|
| Summarize what customers complain about | sample complaint rows → narrative |
| What delivery issues do customers face? | filter delivery rows → narrative |
| Describe account-related problems | extract account subset → narrative |
| What kinds of refund complaints exist? | refund sample + counts → narrative |
The follow-up resolver detects reference words ("more", "those", "again", etc.) and rewrites the query into a self-contained question using the last two turns.
| Turn | Query | Resolved to |
|---|---|---|
| 1 | Show 3 examples from REFUND | (unchanged) |
| 2 | Show 3 more | Show 3 examples from REFUND starting at index 3 |
| 3 | Show 3 more | Show 3 examples from REFUND starting at index 6 |
| 1 | How many rows in PAYMENT? | (unchanged) |
| 2 | What about REFUND? | How many rows in the REFUND category? |
| Query | Response |
|---|---|
| What do you remember about me? | Returns stored profile |
| What do you know about me? | Returns stored profile |
| My profile | Returns stored profile |
| Example | Response |
|---|---|
| Write me a poem | Decline + suggested question |
| What's the weather in Paris? | Decline + suggested question |
The bot can suggest what to explore next based on your profile and conversation history.
How to use:
- Ask "What should I query next?", "Recommend something", "What else can I ask?", or similar
- The bot reads your profile and recent conversation, then suggests a tailored query
- You can confirm it, refine it, or reject it
State machine:
Idle (pending_suggestion = None)
│
├─ User asks for recommendation ──► Suggest query
│ (pending_suggestion = "suggested query")
│
└─ Suggested state
│
├─ User: "yes" / "go ahead" ────────► Execute the suggested query
│ (pending_suggestion = None)
│
├─ User: "but..." / "maybe..." ─────► Adjust suggestion
│ (pending_suggestion = "adjusted query")
│
├─ User: "no" / "skip" ─────────────► Reject & return to idle
│ (pending_suggestion = None)
│
└─ User: [independent query] ───────► Treat as new question, clear pending
(pending_suggestion = None)
Example flow:
You: What should I query next?
Bot: Based on your interest in REFUND issues, here's a suggestion:
"How many refund requests mention delivery problems?"
Reason: You've explored cancellations and refunds, but haven't yet looked
at intersection of delivery + refunds.
**Here's a suggestion:**
> How many refund requests mention delivery problems?
Shall I run it, or would you like to adjust it?
You: Show me examples instead of just the count
Bot: How about this instead:
"Show 5 examples of refund requests mentioning delivery problems"
Reason: Changed from aggregation to sampling.
**How about this instead:**
> Show 5 examples of refund requests mentioning delivery problems
Shall I run it, or would you like to adjust it?
You: Yes, run it
[recommendation] confirmed — executing
Bot: Here are 5 examples of refund requests mentioning delivery issues:
...
In the Streamlit UI: When a suggestion is pending, a confirmation panel appears above the chat input:
ℹ️ Pending: "How many unique customers complained about cancellations?"
[✅ Yes, run it] Or type your adjustment below ↓
Click ✅ Yes, run it to execute, or type a message to refine or reject.
Conversation state is persisted using LangGraph with a SQLite checkpointer.
sessions.dbis created automatically in the working directory root on first run- Every turn's messages are checkpointed after the
update_profilenode completes - Passing the same
session_idon a new run restores the full message history - The pandas DataFrame is never stored in the checkpoint — it is loaded fresh from HuggingFace on each startup and captured by closure in the graph nodes
# How sessions work internally
config = {"configurable": {"thread_id": session_id}}
graph.invoke(
{"messages": [{"role": "user", "content": query}]},
config=config, # LangGraph loads + saves checkpoint scoped to this ID
)Each session builds a persistent profile stored separately from conversation history.
| Field | Example |
|---|---|
name |
"Nir" |
topics_of_interest |
["REFUND", "SHIPPING"] |
preferences |
["prefers tables over bullet points"] |
other_facts |
["data analyst"] |
After every turn the update_profile graph node runs a lightweight LLM call to
extract new facts from the latest exchange. The LLM outputs NO_UPDATE (fast path)
when nothing noteworthy was said, or a JSON object with only the new fields to merge.
Facts accumulate across turns — the profile is never reset between sessions.
Profiles are stored as profiles/{session_id}.json, independent of sessions.db.
They survive process restarts and can be read directly by the MCP get_user_profile tool.
Queries containing phrases like "what do you remember about me", "my profile",
or "what do you know about me" are intercepted in the chat node before reaching
the orchestrator, and answered directly from the profile file.
| Agent | LLM calls | Capabilities | Max iterations |
|---|---|---|---|
| Orchestrator | 1 | Classifies query text — no data access | 1 |
| Coding Agent | 1–2 (code gen) + 1 (format) | ast.parse, sandboxed exec, pandas |
2 code-gen attempts |
| Summarization Agent | 1 | Reads extracted data string only | 1 |
| Out-of-scope Agent | 1 | No data access | 1 |
| Follow-up Resolver | 0–1 | Reads recent history — no data access | 1 (skipped if no follow-up signals) |
| Profile Updater | 1 | Reads last exchange — writes profile file | 1 (outputs NO_UPDATE most turns) |
attempt 1: generate_code(query) → execute_code(code, df)
│
┌─────────────┴────────────┐
success error
│ │
format answer attempt 2: generate_code(query + error)
→ execute_code(code2, df)
Hard limit: 2 code-generation attempts. If both fail the error string is passed to the formatter, which explains what went wrong.
| Capability | Orchestrator | Coding Agent | Summarization | Out-of-scope | Follow-up Resolver | Profile Updater |
|---|---|---|---|---|---|---|
| LLM inference | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Read DataFrame | — | ✓ | — | — | — | — |
Execute Python (exec) |
— | ✓ | — | — | — | — |
| Read extracted data string | — | — | ✓ | — | — | — |
| Read conversation history | — | — | — | — | ✓ | — |
| Read / write profile file | — | — | — | — | — | ✓ |
| Internet / OS / filesystem | — | — | — | — | — | — |
The coding agent's
execruns with__builtins__replaced by a safe whitelist.open(),os,subprocess, and all network calls are blocked inside generated code.
mcp_server.py exposes 4 tools via the Model Context Protocol.
| Tool | Parameters | LLM | Description |
|---|---|---|---|
get_dataset_stats |
(none) | No | Row count, all categories, all intents — instant schema overview |
query_dataset |
query: str |
Yes | Structured analytical query — counts, lists, distributions, examples |
summarize_topic |
topic: str |
Yes | Narrative summary of a topic, intent, or category |
get_user_profile |
session_id: str |
No | Retrieve the stored profile for a session |
get_dataset_stats()
→ "Rows: 26,872 Categories (11): ACCOUNT, CANCEL ... Intents (27): cancel_order ..."
query_dataset("how many rows are in the REFUND category?")
→ "There are 2,992 rows in the REFUND category."
summarize_topic("what cancellation issues do customers face?")
→ "Customers face issues with canceling orders and understanding fees ..."
get_user_profile("my-session-id")
→ "Here's what I know about you: Name: Nir Topics: REFUND, SHIPPING ..."
- Dataset scope only — the bot answers questions about the Bitext dataset. It has no knowledge of your company's actual policies or customer data.
- Code execution is sandboxed but not timed — very large extractions are truncated at 4,000 characters before the formatting step.
- Routing is LLM-based — borderline queries (e.g. "what types of X exist") may occasionally route to STRUCTURED instead of UNSTRUCTURED. The answer is still useful either way.
- Profile capture is best-effort — the LLM extracts facts from natural conversation; if a user never shares personal details, the profile stays empty.
- Follow-up resolution uses the last 2 turns only — very long chains of follow-ups may lose earlier context.