feat: v0.4.0 — add streaming, explain, project scaffolding, session persistence, CI/CD

.github/workflows/ci.yml

@@ -0,0 +1,50 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+env:
+  CARGO_TERM_COLOR: always
+
+jobs:
+  fmt:
+    name: Format
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: rustfmt
+      - run: cargo fmt -- --check
+
+  clippy:
+    name: Lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy
+      - uses: Swatinem/rust-cache@v2
+      - run: cargo clippy -- -D warnings
+
+  test:
+    name: Test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: Swatinem/rust-cache@v2
+      - run: cargo test
+
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: Swatinem/rust-cache@v2
+      - run: cargo build --release

.gitignore

@@ -56,6 +56,4 @@ out/
 generated/
 logs/
 *.py
-
-
-.github
+CLAUDE.md

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "python-maker-bot"
-version = "0.3.0"
+version = "0.4.0"
 edition = "2021"
 
 [dependencies]

FEATURES.md

@@ -77,21 +77,15 @@ To make this stand out for a senior-year portfolio, consider moving from a "func
     - **Configuration**: `enable_dashboard = true` and `dashboard_port = 3000` in `pymakebot.toml`
     - New `/dashboard` REPL command shows the dashboard URL
 
-7. "Chat with Data" (Local RAG)
-    Currently, the bot generates code based only on LLM training data. Real-world AI systems augment generation with user-provided context.
+7. ~~"Chat with Data" (Local RAG)~~ ✅ **DONE**
+    ~~Currently, the bot generates code based only on LLM training data.~~
 
-    Feature: Allow users to load custom documentation (PDF, TXT, CSV) via a `/context <file_path>` command. The bot embeds these documents locally and injects relevant chunks into the LLM prompt when generating code.
-
-    Why it impresses recruiters: RAG (Retrieval-Augmented Generation) is the hottest topic in AI/ML. Building it from scratch demonstrates deep technical mastery and practical LLM knowledge.
-
-    Implementation Plan:
-    - **Ingestion**: Add a `/context <file_path>` REPL command that accepts local files (PDF, TXT, CSV).
-    - **Text Extraction**: Parse files into chunks (e.g., ~256 tokens per chunk) using existing dependencies or a lightweight library.
-    - **Embedding**: Use the `fastembed` crate to convert text chunks into vector embeddings locally (no API calls).
-    - **Storage**: Store embeddings in memory as a simple `Vec<(String, Vec<f32>)>` or use a lightweight vector DB like Qdrant for persistence.
-    - **Retrieval**: When the user asks a question, embed their query and retrieve the top 3 most similar chunks using cosine similarity.
-    - **Injection**: Prepend these chunks to the system prompt: `"Context: [Chunk 1] [Chunk 2] [Chunk 3]. Use this context to answer: [User Prompt]"`.
-    - **Configuration**: Add `enable_rag = true` option in `pymakebot.toml` to toggle RAG on/off.
+    Implemented in `src/rag.rs` with the `/context <file_path>` REPL command. Supports TXT, MD, and CSV files:
+    - **Text chunking** with configurable chunk size and 10% overlap
+    - **Embedding generation** via provider-specific APIs (HuggingFace, Ollama, OpenAI-compatible)
+    - **In-memory vector store** with cosine similarity retrieval
+    - **Automatic RAG injection** into prompts when `enable_rag = true`
+    - Configurable: `enable_rag`, `rag_embedding_model`, `rag_chunk_size` in `pymakebot.toml`
 
 8. Multi-File Project Generation (Scaffolding)
     Today, the bot returns single scripts. Professional engineering requires generating complete project structures.
@@ -124,6 +118,40 @@ To make this stand out for a senior-year portfolio, consider moving from a "func
 
 
 
+9. ~~GitHub Actions CI/CD Pipeline~~ ✅ **DONE**
+    Automated quality gates on every push and pull request:
+    - `cargo fmt --check` — formatting verification
+    - `cargo clippy -- -D warnings` — zero-warning lint policy
+    - `cargo test` — full test suite execution
+    - Uses Rust stable on ubuntu-latest with dependency caching
+
+10. ~~Code Explanation Mode~~ ✅ **DONE**
+    The bot can now explain generated code step-by-step via `/explain` REPL command and dashboard endpoint:
+    - Dedicated explanation system prompt for educational breakdowns
+    - Explains: overview, step-by-step walkthrough, key concepts, potential improvements
+    - Dashboard "Explain" button for one-click explanations
+
+11. ~~Conversation Persistence~~ ✅ **DONE**
+    Sessions can be saved and loaded across restarts:
+    - `/session save <name>` — serialize conversation history to JSON
+    - `/session load <name>` — restore a previous session
+    - `/session list` — browse saved sessions
+    - Dashboard endpoints for session persistence
+
+12. ~~Streaming LLM Responses~~ ✅ **DONE**
+    Real-time token-by-token code generation:
+    - REPL: characters appear as the model generates them
+    - Dashboard: Server-Sent Events (SSE) for live streaming
+    - Configurable via `use_streaming = true` in `pymakebot.toml`
+    - Graceful fallback to non-streaming on error
+
+13. ~~Multi-File Project Generation~~ ✅ **DONE**
+    Generate entire project structures, not just single scripts:
+    - `/project <prompt>` scaffolds a complete project directory
+    - LLM outputs structured JSON with file paths and contents
+    - Writes to `generated/<project_name>/` with proper directory structure
+    - Dashboard endpoint for project generation
+
 ## Possible Optimizations
 
 Optimization Tip: Creating a fresh venv inside the ephemeral container for every execution is secure but computationally expensive (slow).

README.md

@@ -1,4 +1,4 @@
-# 🤖 Python Maker Bot v0.3.0
+# 🤖 Python Maker Bot v0.4.0
 
 **An AI-Powered Python Code Generator Built with Rust**
 
@@ -29,6 +29,11 @@ A Rust-based interactive shell that leverages AI language models to generate, re
 - **API Retry with Backoff**: Automatic retries with exponential backoff on network errors, rate limits, and server errors
 - **Execution Timeout**: Configurable timeout kills runaway scripts (Captured mode only)
 - **Conversation History Limit**: Automatically trims old messages to keep context manageable
+- **Streaming Responses** ⚡: Real-time token-by-token code generation in REPL and dashboard via SSE
+- **Code Explanation** 📖: `/explain` command and dashboard button for step-by-step code breakdowns
+- **Multi-File Projects** 📁: `/project` command generates entire project structures with multiple files
+- **Session Persistence** 💾: Save and load conversation sessions across restarts
+- **CI/CD Pipeline** ✅: GitHub Actions with automated formatting, linting, and testing
 - **Script Management**: List and re-run previously generated scripts anytime
 - **Dependency Detection**: Automatically detects non-standard library imports
 - **Auto-Installation**: Prompts to install required packages via pip (or auto-installs with config)
@@ -108,6 +113,11 @@ use_docker = true
 | `/lint` | Lint the last generated code with ruff |
 | `/security` | Run security scan (bandit) on last code |
 | `/dashboard` | Show dashboard URL (if enabled) |
+| `/explain` | Explain the last generated code step-by-step |
+| `/project <prompt>` | Generate a multi-file project structure |
+| `/session save <name>` | Save current session to disk |
+| `/session load <name>` | Load a saved session |
+| `/session list` | List all saved sessions |
 
 ### Example Session
 
@@ -382,18 +392,6 @@ fn main() -> anyhow::Result<()> {
 ```
 
 
-## 📚 Documentation
-
-Complete guides available:
-
-- **[DEMO_EXAMPLES.md](DEMO_EXAMPLES.md)** - Battle-tested examples perfect for demos and presentations
-- **[DEFENSE_CHEATSHEET.md](DEFENSE_CHEATSHEET.md)** - Quick reference for project defense/presentations
-- **[QUICK_REFERENCE.md](QUICK_REFERENCE.md)** - Command cheat sheet and feature overview
-- **[INTERACTIVE_MODE.md](INTERACTIVE_MODE.md)** - Technical deep dive into interactive execution
-- **[EXAMPLES.md](EXAMPLES.md)** - Usage examples and patterns
-- **[FIX_SUMMARY.md](FIX_SUMMARY.md)** - Technical implementation details
-- **[ARCHITECTURE_DIAGRAM.md](ARCHITECTURE_DIAGRAM.md)** - Visual system diagrams
-
 ---
 
 ## 📝 License
@@ -419,7 +417,26 @@ MIT License - see LICENSE file for details
 
 ## 🔄 Version History
 
-### v0.3.0 (Current — February 2026)
+### v0.4.0 (Current — April 2026)
+- ⚡ **Streaming LLM Responses**: Real-time token-by-token code generation in both REPL and web dashboard
+  - REPL: characters appear as the model generates them (typing effect)
+  - Dashboard: Server-Sent Events (SSE) endpoint for live streaming
+  - Configurable via `use_streaming = true` in `pymakebot.toml`
+  - Graceful fallback to non-streaming on error
+- 📖 **Code Explanation Mode**: `/explain` REPL command and dashboard "Explain" button
+  - Sends generated code to the LLM with a dedicated explanation system prompt
+  - Returns structured breakdown: overview, step-by-step walkthrough, key concepts, improvements
+- 📁 **Multi-File Project Generation**: `/project <prompt>` command scaffolds complete project structures
+  - LLM outputs structured JSON with file paths and contents
+  - Writes to `generated/<project_name>/` with proper directory hierarchy
+  - Security: validates paths (no `..`, no absolute paths)
+- 💾 **Conversation Persistence**: Save and load sessions across restarts
+  - `/session save <name>` — serialize conversation history, code, and metadata to JSON
+  - `/session load <name>` — restore a previous session
+  - `/session list` — browse saved sessions with timestamps
+- ✅ **CI/CD Pipeline**: GitHub Actions workflow with `cargo fmt`, `cargo clippy`, and `cargo test`
+
+### v0.3.0 (February 2026)
 - 🌐 **Web Dashboard**: Real-time browser-based dashboard running alongside the CLI REPL
   - Code generation via the web UI (same LLM & config as the REPL)
   - Script history sidebar with click-to-view source