diff --git a/RESULTS.md b/RESULTS.md index 1053acf7..f4adca3c 100644 --- a/RESULTS.md +++ b/RESULTS.md @@ -1,67 +1,203 @@ -# Smoke-Pipeline Results — Index +# Validation and Measured Results -Every project with a non-trivial runtime ships a `smoke/run_smoke.py` that exercises the code path end-to-end on synthetic or in-repo data and writes a report into the project's `reports/` directory. This index links each one. +This file separates measured outputs from reproducible execution paths. A passing test, a smoke pipeline, a portable reference implementation, and a real-model metric are different forms of evidence and are reported separately. -> **Reading note:** The committed reports are *placeholder shells* with the expected structural shape (qualitative ranking, identity-check rows, expected metric ranges) but **not** filled-in numbers. The regenerate command in each report file produces the actual numbers — they're left out of source control because they drift run-to-run with random seeds. The shape of the output is what's stable. Real figures for the two flagship pipelines are recorded under **Verified runs** below. +## Audited Repository Inventory -## Verified runs (real figures, not placeholders) +Inventory audited on 2026-07-27 against [`.github/workflows/ci.yml`](./.github/workflows/ci.yml): -The two flagship RL pipelines were run locally on **2026-06-02** — these are actual outputs. Regenerate any time with the commands in the table below. +- **10 Python project entries** +- **6 Rust crates** +- **1 Terraform validation target** +- **17 CI targets in total** +- **8 tracked `smoke/run_smoke.py` pipelines** + +The counts above describe the tracked repository and CI matrix. They do not include local ignored or untracked work. + +## 1. Verified Measured Runs + +### RAG Assistant: embedding retrieval versus cross-encoder re-ranking + +- **Run date:** 2026-07-26 +- **Environment:** CPU; `sentence-transformers` 5.6.1; `transformers` 5.14.1; `torch` 2.13.0 +- **Corpus:** three checked-in sample documents +- **Cases:** five checked-in retrieval cases +- **Metric cutoff:** k = 3 + +| Configuration | recall@3 | MRR | +|---|---:|---:| +| Embedding retrieval | 1.000 | 0.900 | +| Cross-encoder re-ranked | 1.000 | 1.000 | +| Observed delta | +0.000 | +0.100 | + +The re-ranker moved one relevant result from rank 2 to rank 1. This is a scoped smoke result, not evidence of a general quality lift. [Issue 18](https://github.com/lmdixon23/my_dev_projects/issues/18) tracks the required discriminative benchmark. + +Regenerate the comparison: + +```bash +cd ai_engineering/rag_assistant +python -m smoke.run_reranker_eval +``` ### Regularized Operator Zoo -Both regularizer identities hold to machine precision on `q = [1.0, 2.0, 0.5], beta = 1.0`: +- **Run date:** 2026-06-02 +- **Environment:** local CPU; NumPy and Matplotlib project requirements; exact package versions were not recorded in the retained run evidence +- **Input:** `q = [1.0, 2.0, 0.5]`, `beta = 1.0` + +- Maximum policy-to-gradient residual across the five operators: **1.8e-07** +- Maximum conjugate-identity difference: **2.2e-16** -- **Identity 1** (gradient form, `pi* = grad Omega*(q)`): max `||pi - grad Omega*||_2` = **1.8e-07** (Tsallis); the other closed-form operators agree to ~1e-10. -- **Identity 2** (conjugate form, `Omega*(q) = - Omega(pi)`): max difference = **2.2e-16** across entropy, KL-to-uniform, KL-to-anchor, Tsallis, and Renyi. +The tested identities are: -### GRPO Minimal — reproduces Shao et al. 2024, Figure 5 +```text +pi* = grad Omega*(q) +Omega*(q) = - Omega(pi*) +``` -Final accuracy on the synthetic verifiable-reward task (mean of 3 seeds, 1200 steps each): +Regenerate: + +```bash +cd ai_engineering/rlvr/regularized_operator_zoo +python -m smoke.run_smoke +``` + +### GRPO Minimal + +- **Run date:** 2026-06-02 +- **Environment:** local CPU; NumPy and Matplotlib project requirements; exact package versions were not recorded in the retained run evidence +- **Task:** synthetic categorical verifiable-reward task +- **Protocol:** mean of 3 seeds, 1,200 steps per seed | Method | Final accuracy | -|---|---| +|---|---:| | RFT | 0.181 | | Online RFT | 0.945 | | GRPO+OS | 0.990 | -Qualitative ranking **RFT < Online RFT < GRPO+OS** matches the paper. +The measured qualitative ordering is **RFT < Online RFT < GRPO+OS**. The project targets the ordering associated with Figure 5 of Shao et al. 2024, not the paper's absolute benchmark values. + +Regenerate from a shell with the sibling operator library on `PYTHONPATH`: + +```bash +cd ai_engineering/rlvr/grpo_minimal +PYTHONPATH="$PYTHONPATH:$(realpath ../regularized_operator_zoo)" \ + python -m smoke.run_smoke +``` + +## 2. Reproducible Smoke Pipelines + +These tracked projects provide a `smoke/run_smoke.py` entry point. A smoke run exercises a bounded end-to-end path; it does not automatically establish external validity or production readiness. + +| Project | Command | Evidence produced | +|---|---|---| +| [RAG Assistant](./ai_engineering/rag_assistant/) | `cd ai_engineering/rag_assistant && python -m smoke.run_smoke` | Ingest, retrieve, and retrieval metrics on checked-in documents | +| [Agent Toolkit](./ai_engineering/agent_toolkit/) | `cd ai_engineering/agent_toolkit && python -m smoke.run_smoke` | Deterministic agent/tool execution and trace output | +| [LLM Eval Harness](./ai_engineering/llm_eval_harness/) | `cd ai_engineering/llm_eval_harness && python -m smoke.run_smoke` | Evaluation-suite execution and report generation | +| [Regularized Operator Zoo](./ai_engineering/rlvr/regularized_operator_zoo/) | `cd ai_engineering/rlvr/regularized_operator_zoo && python -m smoke.run_smoke` | Operator identity checks and beta-sweep artifacts | +| [GRPO Minimal](./ai_engineering/rlvr/grpo_minimal/) | `cd ai_engineering/rlvr/grpo_minimal && PYTHONPATH="$PYTHONPATH:$(realpath ../regularized_operator_zoo)" python -m smoke.run_smoke` | Three-method synthetic training comparison | +| [Image Captioning](./machine_learning/image_captioning_cnn_rnn_tpu/) | `cd machine_learning/image_captioning_cnn_rnn_tpu && python -m smoke.run_smoke` | CPU-compatible end-to-end captioning path | +| [Image Classification](./machine_learning/image_classification/) | `cd machine_learning/image_classification && python -m smoke.run_smoke` | Synthetic train, evaluate, persist, and inference path | +| [Predictive Maintenance](./machine_learning/predictive_maintenance/) | `cd machine_learning/predictive_maintenance && python -m smoke.run_smoke` | Synthetic telemetry, model training, persistence, and inference path | + +Generated reports are local runtime artifacts unless a project README explicitly identifies a retained verified run. + +## 3. Test-Suite or Validation-Only Projects + +These projects are validated primarily through deterministic tests, compilation, or configuration validation rather than a separate smoke report. + +### Python + +| Project | CI command | Boundary | +|---|---|---| +| [NLP Text Summarization CLI](./ai_engineering/nlp_text_summarization_api/) | `cd ai_engineering/nlp_text_summarization_api && python -m unittest discover tests` | Network-free HTTP-path, persistence, concurrency, and retry tests; no retained summary-quality benchmark | + +The RAG-to-evaluation bridge is also enforced separately: + +```bash +cd ai_engineering/llm_eval_harness +PYTHONPATH="$PYTHONPATH:$(realpath ../rag_assistant)" \ + python -m pytest tests/test_bridge_to_rag.py -v +``` + +The current bridge baseline is **2 passing tests**. + +### Rust + +The CI matrix contains six crates: + +1. [Cross-Chain Atomic Bridge](./blockchain_protocols/rust_cross_chain_atomic_bridge/) +2. [Decentralized Voting](./blockchain_protocols/rust_decentralized_voting/) +3. [Quadratic Voting and Liquid Democracy](./blockchain_protocols/rust_quadratic_voting/) +4. [PoS and ZKP Voting](./blockchain_protocols/rust_pos_zkp_voting/) +5. [PoA and ZKP Voting](./blockchain_protocols/rust_poa_zkp_voting/) +6. [DeFi Lending Protocol](./blockchain_protocols/rust_defi_lending_protocol/) + +For each crate, CI runs: + +```bash +cargo build --verbose +cargo test --verbose +cargo clippy --all-targets --no-deps -- -W clippy::all +``` + +Clippy is advisory in the current workflow. These repositories model protocol behavior in-process and do not claim deployed-chain security or consensus. + +### Terraform + +[SSE Coexistence Testing](./network_security/sse_coexistence_testing/) is checked with: + +```bash +cd network_security/sse_coexistence_testing/terraform +terraform init -backend=false +terraform validate +``` + +Validation checks Terraform configuration structure. It does not deploy AWS resources or verify a live network environment. -## Smoke reports — regenerate locally +## 4. Portable Reference Implementations -Each runnable project's `smoke/run_smoke.py` writes a report into its `reports/` folder. Those report files are **gitignored** (their numbers drift with random seeds), so they are not committed — run the command to produce them in a local clone. For committed, real figures see **Verified runs** above. +### Sales Data ETL -| Project | Regenerate command | -|---|---| -| RAG Assistant | `cd ai_engineering/rag_assistant && python -m smoke.run_smoke` | -| Agent Toolkit | `cd ai_engineering/agent_toolkit && python -m smoke.run_smoke` | -| LLM Eval Harness | `cd ai_engineering/llm_eval_harness && python -m smoke.run_smoke` | -| Regularized Operator Zoo | `cd ai_engineering/rlvr/regularized_operator_zoo && python -m smoke.run_smoke` | -| GRPO Minimal (reproduces Shao et al. 2024 Figure 5) | `cd ai_engineering/rlvr/grpo_minimal && PYTHONPATH=$PYTHONPATH:$(realpath ../regularized_operator_zoo) python -m smoke.run_smoke` | -| Image Classification | `cd machine_learning/image_classification && python -m smoke.run_smoke` | -| Image Captioning (CNN+RNN) | `cd machine_learning/image_captioning_cnn_rnn_tpu && python -m smoke.run_smoke` | -| Sentiment Analysis (BERT) | `cd machine_learning/sentiment_analysis_transfer_learning && python -m smoke.run_smoke` | -| Predictive Maintenance | `cd machine_learning/predictive_maintenance && python -m smoke.run_smoke` | -| Sales Data ETL (Python reference) | uses `create_sales_data.py` then `sales_etl.py` | +The [Sales Data ETL](./data_engineering/sales_data_etl_ssis/) project contains an SSIS deliverable and a Python/SQLite reference path. Linux CI verifies the portable reference, not the Windows SSIS package. -That's all 10 runnable projects — full coverage. +Portable execution: -## Projects without a smoke pipeline +```bash +cd data_engineering/sales_data_etl_ssis +python create_sales_data.py --rows 500 +python python_reference_etl/sales_etl.py \ + --source sales_data.csv \ + --target sqlite:///salesdatadb.sqlite +python -m unittest discover python_reference_etl +``` -The following projects don't have a `smoke/` directory because their full test suite already exercises the code path end-to-end without external resources: +This path validates the transformation, rejection, load, and logging contract without claiming that GitHub Actions executed the `.dtsx` package. -- **All five Rust blockchain projects** — `cargo test` runs the unit + integration tests on every push. -- **The Terraform SSE coexistence test** — `terraform validate` runs in the CI Terraform job; the deployment harness has its own mocked-network unit tests. -- **The ink! counter prototype** — `cargo test` covers it. +## Audit Commands -## What "placeholder report" means +Recount the CI matrix and tracked smoke scripts from the repository root: -Each committed `reports/*.md` (or `.html`) under the projects above contains: +```bash +python - <<'PY_AUDIT' +from pathlib import Path +import re +import subprocess -- A `_Last regenerated: PLACEHOLDER_` line — clearly flags that the file has not been filled in by a real run yet. -- The exact command to regenerate it. -- The **structural shape** of the expected output — qualitative ranking, identity-check rows, expected metric ranges. +ci = Path('.github/workflows/ci.yml').read_text(encoding='utf-8') +python_projects = re.findall(r'^\s*- project:\s*([^\n]+)$', ci, re.M) +rust_crates = re.findall(r'^\s*- (blockchain_protocols/[^\n]+)$', ci, re.M) +terraform_targets = len(re.findall(r'^\s*- run: terraform validate\s*$', ci, re.M)) +tracked = subprocess.check_output(['git', 'ls-files'], text=True).splitlines() +smoke_scripts = sorted(path for path in tracked if path.endswith('/smoke/run_smoke.py')) -It does **not** contain absolute numerical results that would drift between runs. After you run the regenerate command, the file is overwritten with the real run's output (including a real timestamp and the actual numbers). +print('Python CI projects:', len(python_projects)) +print('Rust CI crates:', len(rust_crates)) +print('Terraform validation targets:', terraform_targets) +print('Tracked smoke scripts:', len(smoke_scripts)) +for path in smoke_scripts: + print(' ', path) +PY_AUDIT +``` -This pattern lets a reviewer who is browsing the repo on GitHub see *what kind of output the smoke pipeline produces* without having to run the code first. It's the "you can read about my results before installing my code" affordance. +The release gate also checks relative links, stale claims, whitespace, RAG tests, the RAG-to-eval bridge, and the exact three-file change boundary. diff --git a/ai_engineering/rag_assistant/README.md b/ai_engineering/rag_assistant/README.md index 4ffd55dc..62af7f7f 100644 --- a/ai_engineering/rag_assistant/README.md +++ b/ai_engineering/rag_assistant/README.md @@ -2,225 +2,226 @@ ## Overview -**RAG_Assistant** is a Retrieval-Augmented Generation system written in Python. It chunks documents, embeds them with sentence-transformers, indexes them in FAISS, retrieves relevant context for a user question, and generates a cited answer using the OpenAI Chat Completions API. It ships with a CLI, a Flask serving layer, a retrieval-quality evaluation harness, and a self-contained smoke pipeline so reviewers can verify the whole flow without an OpenAI key. +RAG Assistant is a modular Retrieval-Augmented Generation system written in Python. It chunks documents, embeds them, stores normalized vectors in FAISS or a NumPy fallback, retrieves relevant context, optionally re-ranks a larger candidate pool with a cross-encoder, and generates cited answers through the OpenAI Chat Completions API. + +The repository includes a CLI, a Flask service, retrieval metrics, deterministic network-free tests, a basic smoke pipeline, and a baseline-versus-re-ranked comparison command. + +## Verified Snapshot + +A CPU run completed on 2026-07-26 with `sentence-transformers` 5.6.1, `transformers` 5.14.1, and `torch` 2.13.0: + +| Configuration | recall@3 | MRR | +|---|---:|---:| +| Embedding retrieval | 1.000 | 0.900 | +| Cross-encoder re-ranked | 1.000 | 1.000 | +| Observed delta | +0.000 | +0.100 | + +The re-ranker moved one relevant result from rank 2 to rank 1. This is a five-case smoke measurement over three small documents. It demonstrates that the comparison path works; it does not establish a general retrieval-quality lift. [Issue 18](https://github.com/lmdixon23/my_dev_projects/issues/18) tracks the required 30-50 case discriminative benchmark. ## Key Features -- **Chunker**: Paragraph-first splitting with a sentence-level fallback for very long paragraphs, plus configurable character-based overlap so answers aren't split across chunk boundaries. -- **Pluggable Embedder**: `Embedder` protocol with two implementations. `SentenceTransformerEmbedder` is the default (MiniLM 384-d); `HashEmbedder` is deterministic, zero-dep, and what the tests use so CI never has to download model weights. -- **FAISS Vector Store with NumPy Fallback**: `IndexFlatIP` over L2-normalized embeddings; falls back to NumPy cosine search if FAISS is unavailable. -- **Cited Answers, Refuses on Empty Retrieval**: The generator instructs the model to cite sources in `[source: ...]` brackets and returns a clear "no context" response when retrieval is empty — fixing the single most common RAG failure mode. -- **Retrieval Eval Harness**: `eval_retrieval(retriever, cases, k=)` reports **recall@k** and **MRR (mean reciprocal rank)** with per-case breakdowns. -- **CLI + Flask API + Docker**: `cli.py {ingest,ask,serve,eval}`; `app.py` exposes `/ask` and `/health`; `Dockerfile` builds a serve-able image. -- **Smoke Pipeline**: `python -m smoke.run_smoke` ingests three included sample docs, runs the eval cases, and writes `reports/smoke_eval.md` — no OpenAI key needed. +- **Paragraph-first chunking**: Long paragraphs fall back to sentence-level splitting, with configurable character overlap. +- **Pluggable embeddings**: `SentenceTransformerEmbedder` is the default; `HashEmbedder` provides deterministic, network-free tests. +- **FAISS with NumPy fallback**: Normalized vectors use inner-product search, equivalent to cosine similarity after normalization. +- **Optional cross-encoder re-ranking**: Embedding retrieval remains the default. An opt-in cross-encoder re-scores a configurable candidate pool before the final top-k is returned. +- **Score provenance**: Re-ranked responses expose both the final cross-encoder `score` and the original embedding `retrieval_score`. +- **Cited generation with an empty-context guard**: The generator requests source tags and refuses to produce a grounded answer when retrieval returns no context. +- **Retrieval evaluation**: `eval_retrieval` reports recall@k, MRR, and per-case results. +- **CLI, Flask, and Docker surfaces**: Ingestion, asking, evaluation, serving, and container execution use the same pipeline components. +- **Reproducible comparison**: `python -m smoke.run_reranker_eval` records aggregate and per-case changes between embedding-only and re-ranked retrieval. +- **Network-free validation**: The RAG suite contains 19 tests across 4 files. A separate two-test bridge verifies compatibility with the sibling LLM Eval Harness. ## Architecture -Standard Python package layout. Each pipeline stage is a separate class behind an interface, so swapping out the vector store (Chroma, Pinecone), embedder (OpenAI embeddings, Cohere), or generator (Anthropic, local Llama) is a one-class change. - ```mermaid flowchart TD - subgraph ingest["Ingest — offline, no API key needed"] - D["Documents (sample_docs/)"] --> C["Chunker — paragraph-first + overlap"] - C --> EM["Embedder — MiniLM 384-d / HashEmbedder"] - EM --> VS[("FAISS IndexFlatIP + NumPy fallback, persisted to ./index/")] + subgraph ingest["Ingest - offline"] + D["Documents"] --> C["Chunker"] + C --> E["Embedder"] + E --> V[("FAISS or NumPy vector store")] end - subgraph ask["Ask — query time"] - Q["Question"] --> RET["Retriever — Embedder + VectorStore"] - VS --> RET - RET -->|"top-k chunks"| GEN["Generator — OpenAI Chat Completions"] - RET -->|"empty retrieval"| NC["No-context answer (guard)"] - GEN --> ANS["Cited answer with source tags"] + subgraph query["Query time"] + Q["Question"] --> R["Embedding retriever"] + V --> R + R -->|"default top-k"| G["Generator"] + R -->|"optional candidate pool"| X["Cross-encoder re-ranker"] + X -->|"final top-k"| G + R -->|"empty"| N["No-context response"] + G --> A["Cited answer"] end - subgraph evalh["Eval harness"] - EC["eval_cases.json"] --> RET - RET --> MET["recall@k + MRR"] + subgraph evaluation["Evaluation"] + T["Labeled cases"] --> R + R --> M["recall@k and MRR"] + B["Baseline versus re-ranked runner"] --> M end ``` -``` +## File Map + +```text rag/ - chunker.py Paragraph-first chunker with sentence fallback - embedder.py Embedder protocol + SentenceTransformer + HashEmbedder - vector_store.py FAISS IndexFlatIP + NumPy fallback + JSON sidecar - retriever.py Embedder + VectorStore wrapper - generator.py OpenAI client with cited-answer prompt + empty-retrieval guard - eval.py recall@k + MRR - pipeline.py RAGPipeline.from_env() + ingest() + ask() + save()/load() -cli.py ingest / ask / serve / eval subcommands -app.py Flask /ask + /health -Dockerfile python:3.11-slim with index/ baked in -sample_docs/ 3 sample .md docs + eval_cases.json -smoke/run_smoke.py End-to-end pipeline + reports/smoke_eval.md -tests/ 3 test files, no network required + chunker.py Document and chunk types; paragraph-first splitting + embedder.py Embedder protocol, HashEmbedder, SentenceTransformerEmbedder + vector_store.py FAISS IndexFlatIP with NumPy fallback and persistence + retriever.py Embedding retrieval and optional candidate-pool expansion + reranker.py Reranker protocol and CrossEncoderReranker + generator.py OpenAI generation, source formatting, empty-context guard + eval.py EvalCase, recall@k, MRR, and per-case results + pipeline.py End-to-end ingest, retrieve, optional re-rank, and generate +cli.py ingest, ask, serve, and eval commands +app.py Flask /ask and /health endpoints +sample_docs/ Three sample documents and five labeled cases +smoke/run_smoke.py Embedding-only end-to-end smoke evaluation +smoke/run_reranker_eval.py Baseline-versus-re-ranked CPU comparison +reports/ Generated local reports; values may vary by environment +tests/ + test_chunker.py + test_generator.py + test_pipeline.py + test_reranker.py +Dockerfile requirements.txt .env.example ``` -## Example Usage - -The pipeline runs in four stages: - -- **Ingest**: Documents are loaded, chunked, embedded, and stored in a FAISS index that's persisted to disk under `./index/`. -- **Retrieve**: A user question is embedded and matched against the index; the top-k chunks are returned with cosine-similarity scores. -- **Generate**: The retrieved chunks are inserted into a system-prompted Chat Completions call; the model produces an answer citing the source of each claim. -- **Evaluate**: A JSON file of `{question, relevant_sources}` cases is replayed against the retriever; recall@k and MRR are reported per case and in aggregate. - ## Getting Started ### Prerequisites -- **Python 3.10+**. -- **OpenAI API Key** (for the generation step only — retrieval works locally). +- Python 3.10+ +- An OpenAI API key only for answer generation +- Downloadable sentence-transformers model weights for the default embedder and real cross-encoder comparison ### Installation -Clone the repository and navigate to the project directory: - ```bash git clone https://github.com/lmdixon23/my_dev_projects.git cd my_dev_projects/ai_engineering/rag_assistant -pip install -r requirements.txt +python -m venv .venv +# Windows PowerShell: .\.venv\Scripts\Activate.ps1 +# macOS or Linux: source .venv/bin/activate +python -m pip install -r requirements.txt cp .env.example .env -# Edit .env and set OPENAI_API_KEY. ``` -### Running +Set `OPENAI_API_KEY` in `.env` only when generation is required. + +### Ingest and Ask ```bash -# Index the included sample docs python cli.py ingest --docs-dir ./sample_docs --store ./index -# Ask a question -python cli.py ask --store ./index --question "What is RAG?" +# Embedding-only retrieval +python cli.py ask --store ./index --question "What is RAG?" -k 5 -# Serve over HTTP -python cli.py serve --store ./index --port 8080 -curl -X POST http://localhost:8080/ask \ - -H "Content-Type: application/json" \ - -d '{"question": "What is RAG?"}' - -# Container -docker build -t rag-assistant . -docker run -p 8080:8080 -e OPENAI_API_KEY=$OPENAI_API_KEY rag-assistant - -# Smoke pipeline (no OpenAI key required) -python -m smoke.run_smoke +# Optional cross-encoder re-ranking +python cli.py ask \ + --store ./index \ + --question "What is RAG?" \ + --rerank \ + --candidate-k 20 \ + -k 5 ``` -### Testing +### Evaluate Retrieval ```bash -python -m pytest tests/ # no network, no API key -``` - -## Technical Specifications +# Embedding-only evaluation +python cli.py eval \ + --store ./index \ + --cases ./sample_docs/eval_cases.json \ + -k 3 -- **Language**: Python 3.10+ -- **Embeddings**: `sentence-transformers/all-MiniLM-L6-v2` (384-d), pluggable -- **Vector Store**: FAISS `IndexFlatIP` over L2-normalized vectors, NumPy fallback -- **Generator**: OpenAI Chat Completions (`gpt-4o-mini` default), pluggable -- **Eval**: `recall@k`, MRR with per-case JSON output -- **Serving**: Flask (`/ask`, `/health`), Dockerfile checked in -- **Test Coverage**: 11 tests across 3 files; HTTP path mocked via `httpx.MockTransport`; embedder defaults to `HashEmbedder` in tests so no model download is required +# Re-ranked evaluation +python cli.py eval \ + --store ./index \ + --cases ./sample_docs/eval_cases.json \ + --rerank \ + --candidate-k 20 \ + -k 3 -## Results +# Reproducible baseline comparison +python -m smoke.run_reranker_eval +``` -Retrieval quality on the checked-in eval set (`sample_docs/eval_cases.json`, k=3), produced by `python -m smoke.run_smoke` (no API key required; full per-case breakdown written to `reports/smoke_eval.md`). +The comparison writes `reports/reranker_eval.md` with aggregate metrics, deltas, and per-case reciprocal-rank changes. -| Embedder | recall@3 | MRR | Cases | -|---|---|---|---| -| `all-MiniLM-L6-v2` (sentence-transformers) | 1.000 | 0.900 | 5 | -| `HashEmbedder` (no-deps fallback) | _run `smoke.run_smoke` without sentence-transformers installed to populate_ | | 5 | +### Serve over HTTP -This is a 5-case smoke eval (`sample_docs/eval_cases.json`), not a large benchmark — it verifies the retrieval path end-to-end and gives a reproducible number, not a leaderboard score. MRR 0.900 reflects one of the five questions ranking its relevant chunk 2nd instead of 1st. Regenerate with `python -m smoke.run_smoke`; the full per-case table is written to `reports/smoke_eval.md`. +Embedding-only retrieval remains the service default: -## What This Project Demonstrates +```bash +python cli.py serve --store ./index --port 8080 +``` -- A RAG architecture built around **pluggable components** behind small interfaces. Replacing the vector store or embedder is one class, not a rewrite. -- **Network-free unit tests** for both the retrieval path (`HashEmbedder`) and the generation path (`httpx.MockTransport`). -- **Empty-retrieval guard** in the generator — the single most important defense against RAG hallucination, and the one most homemade implementations miss. -- **Honest evaluation**: ships an actual `recall@k` + MRR eval harness with checked-in cases, instead of just claiming "high quality". -- **Reproducible smoke pipeline**: `python -m smoke.run_smoke` exercises ingest + retrieve + eval end-to-end, no key required, and writes a markdown report a reviewer can read in 30 seconds. +To enable service-side re-ranking, set: -## Optional Cross-Encoder Re-ranking +```text +RERANKER_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2 +RERANKER_CANDIDATES=20 +``` -Embedding retrieval remains the default. Add `--rerank` to `ask` or `eval` to -retrieve a larger candidate pool and re-score it with -`cross-encoder/ms-marco-MiniLM-L-6-v2`: +Then send a request: ```bash -python cli.py ask \ - --store ./index \ - --question "What is RAG?" \ - --rerank \ - --candidate-k 20 \ - -k 5 +curl -X POST http://localhost:8080/ask \ + -H "Content-Type: application/json" \ + -d '{"question": "What is RAG?"}' +``` -python cli.py eval \ - --store ./index \ - --cases ./sample_docs/eval_cases.json \ - --rerank \ - --candidate-k 20 \ - -k 3 +## Testing + +```bash +python -m pytest tests/ -v ``` -For the Flask service, set `RERANKER_MODEL` and optionally -`RERANKER_CANDIDATES`. The JSON response exposes both the final cross-encoder -`score` and the original `retrieval_score`. +The current RAG suite contains **19 tests across 4 files**. The tests use deterministic embeddings, an injected fake cross-encoder, and `httpx.MockTransport`, so they require no model download, network call, or API key. -Run the baseline comparison with: +The cross-project bridge is run from the sibling harness: ```bash -python -m smoke.run_reranker_eval +cd ../llm_eval_harness +PYTHONPATH="$PYTHONPATH:$(realpath ../rag_assistant)" \ + python -m pytest tests/test_bridge_to_rag.py -v ``` -The generated `reports/reranker_eval.md` records baseline and re-ranked -recall@k and MRR, their deltas, and per-case reciprocal-rank changes. The -current five-case suite is saturated, so the observed delta is provisional -until the expanded benchmark in issue #18 is available. +The bridge contains **2 tests** and checks both a successful grading contract and a detected mismatch. -### Measured Smoke Result +## Result Interpretation -A CPU run using `sentence-transformers/all-MiniLM-L6-v2` and -`cross-encoder/ms-marco-MiniLM-L-6-v2` produced: +The five-case result is useful for release verification because it confirms that: -| Configuration | recall@3 | MRR | -|---|---:|---:| -| Embedding only | 1.000 | 0.900 | -| Cross-encoder re-ranked | 1.000 | 1.000 | -| Observed delta | +0.000 | +0.100 | +1. the baseline and re-ranked paths execute against the same corpus and labels; +2. the candidate-pool and score-provenance logic are wired correctly; +3. aggregate and per-case deltas are reported reproducibly. -The re-ranker moved the approximate-nearest-neighbor question from rank 2 to -rank 1. This records the observed result on the checked-in five-case smoke -suite; it does not establish a general quality lift. Issue #18 remains the -required follow-up for a discriminative benchmark. +It is not large or difficult enough to support a broad comparative claim. The next evidence gate is [issue 18](https://github.com/lmdixon23/my_dev_projects/issues/18), which requires confusable sources, hard negatives, category slices, recall@1, recall@3, MRR, deterministic and real-model baselines, and regression checks. -## Scope +## Scope and Limitations -- The chunker is character-based and language-agnostic, which is good portability but slightly worse than tokenization-aware chunking for very long contexts. -- No re-ranking layer (cross-encoder / cohere-rerank). Adding one is roughly a hundred lines and would typically lift MRR by 0.05–0.10. -- The generator uses single-shot completions; no streaming endpoint is exposed. -- "Multilingual" is supported by swapping the embedder model name (`paraphrase-multilingual-MiniLM-L12-v2`) but is not the default. +- The checked-in benchmark has five cases over three small documents and is intentionally described as a smoke evaluation. +- The chunker is character-based rather than token-aware. +- Cross-encoder re-ranking is optional and requires external model weights for a real-model run; CI tests the control flow with an injected deterministic model. +- Retrieval is dense-only. BM25, hybrid retrieval, and reciprocal-rank fusion are not implemented. +- Generation is single-shot; the Flask service does not expose streaming responses. +- The default embedding model is English-oriented. Multilingual retrieval has not been benchmarked in this repository. +- The project is a compact reference system, not a multi-tenant production service. ## Future Enhancements -Priority order: grow the eval set so the metrics discriminate, then layer quality and infra levers. - -1. **Larger eval set**: The 5-case smoke set is saturated (MiniLM already scores recall@3 1.000 / MRR 0.900 — see Results), so it can no longer discriminate between retrieval approaches. Expand to 30–50 labeled cases so the metrics become a meaningful baseline to beat. -2. **Re-ranker**: Add a cross-encoder (`cross-encoder/ms-marco-MiniLM-L-6-v2`) over the top candidates before generation, and report the MRR delta on the (expanded) eval set (Scope flags the missing re-ranker; measure the lift rather than assert it). -3. **Token-aware chunking ablation**: Compare the current character-based chunker against a tokenizer-aware splitter on the eval set, turning the Scope chunking caveat into a measurement. -4. **Hybrid Search**: Add BM25 as a secondary retriever, merge with reciprocal rank fusion, and report before/after. -5. **Streaming Responses**: Switch the Flask `/ask` to a Server-Sent Events stream so the UI can render the answer as it arrives. -6. **Multi-tenant Index**: Support multiple namespaces in one store for SaaS use cases. (Lowest priority — infra, not retrieval quality.) +1. **Complete issue 18**: Build the 30-50 case discriminative benchmark before changing retrieval algorithms. +2. **Token-aware chunking ablation**: Compare the current splitter against a tokenizer-aware alternative on the expanded benchmark. +3. **Hybrid retrieval**: Add BM25 and reciprocal-rank fusion only after the benchmark can measure gains and regressions. +4. **Streaming responses**: Add a Server-Sent Events path for generation. +5. **Namespace isolation**: Add explicit index namespaces after retrieval quality and evaluation coverage are stable. ## References - Lewis, P., et al. (2020). *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.* arXiv:2005.11401. - Karpukhin, V., et al. (2020). *Dense Passage Retrieval for Open-Domain Question Answering.* arXiv:2004.04906. -- Johnson, J., Douze, M., & Jégou, H. (2017). *Billion-scale similarity search with GPUs.* arXiv:1702.08734. (FAISS) -- Reimers, N., & Gurevych, I. (2019). *Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.* arXiv:1908.10084. +- Johnson, J., Douze, M., and Jegou, H. (2017). *Billion-scale similarity search with GPUs.* arXiv:1702.08734. +- Reimers, N., and Gurevych, I. (2019). *Sentence-BERT.* arXiv:1908.10084. -Licensed under the [MIT License](https://github.com/lmdixon23/my_dev_projects/blob/main/LICENSE). +Licensed under the [MIT License](../../LICENSE). diff --git a/ai_engineering/rlvr/README.md b/ai_engineering/rlvr/README.md new file mode 100644 index 00000000..00816ef9 --- /dev/null +++ b/ai_engineering/rlvr/README.md @@ -0,0 +1,132 @@ +# RLVR Companion Implementations + +## Purpose + +This directory contains compact, runnable companion implementations for reinforcement learning from verifiable rewards and regularized policy operators. Its purpose is to turn mathematical claims into inspectable code, tests, plots, and bounded CPU experiments. + +The directory currently has two projects: + +| Project | Role | +|---|---| +| [Regularized Operator Zoo](./regularized_operator_zoo/) | Implements five regularized greedy operators and numerically checks the gradient and conjugate identities that connect regularization to policy updates | +| [GRPO Minimal](./grpo_minimal/) | Places RFT, Online RFT, and GRPO+OS on one training skeleton and measures their qualitative ordering on a synthetic verifiable-reward task | + +## How the Projects Connect + +The projects share KL-regularization mathematics, but the current GRPO Minimal update does not call the operator-zoo loss helper during optimization. `grpo_minimal.methods` imports `kl_anchor_term` from `operator_zoo.losses` at module load, while `_grad_logits_for_one_prompt` computes the KL gradient directly. + +`grpo_minimal/tests/test_bridge_to_zoo.py` checks that the imported symbol remains available and separately verifies the KL-to-anchor closed form against `KLToAnchor`. The present connection is therefore a tested mathematical and module-level bridge, not runtime delegation of the GRPO update to an operator-zoo helper. + +Conceptually: + +```text +regularizer and convex conjugate identities + | + v +shared KL-to-anchor formula and tested import + | + v +direct KL-gradient implementation in GRPO Minimal +``` + +This creates a narrow, explicit relationship between the mathematical software and the empirical post-training comparison without overstating code reuse. + +## Verified Evidence + +The retained measured runs are indexed in [`RESULTS.md`](../../RESULTS.md). + +### Regularized Operator Zoo + +Run date: 2026-06-02. For `q = [1.0, 2.0, 0.5]` and `beta = 1.0`: + +- maximum policy-to-gradient residual: **1.8e-07**; +- maximum conjugate-identity difference: **2.2e-16**. + +The run covers negative entropy, KL-to-uniform, KL-to-anchor, Tsallis/sparsemax, and chi-squared/Pearson regularization. + +### GRPO Minimal + +Run date: 2026-06-02. Mean final accuracy across three seeds and 1,200 steps: + +| Method | Final accuracy | +|---|---:| +| RFT | 0.181 | +| Online RFT | 0.945 | +| GRPO+OS | 0.990 | + +The result supports the qualitative ordering **RFT < Online RFT < GRPO+OS** on the repository's synthetic task. It does not reproduce the absolute GSM8K or MATH values from DeepSeekMath. + +## Running the Operator Zoo + +```bash +cd ai_engineering/rlvr/regularized_operator_zoo +python -m venv .venv +# Windows PowerShell: .\.venv\Scripts\Activate.ps1 +# macOS or Linux: source .venv/bin/activate +python -m pip install -r requirements.txt +python -m pytest tests/ -v +python -m smoke.run_smoke +``` + +The smoke run writes the identity report and policy-versus-beta figure under `reports/`. + +## Running GRPO Minimal + +### macOS or Linux + +```bash +cd ai_engineering/rlvr/grpo_minimal +python -m venv .venv +source .venv/bin/activate +python -m pip install -r requirements.txt +export PYTHONPATH="$PYTHONPATH:$(realpath ../regularized_operator_zoo)" +python -m pytest tests/ -v +python -m smoke.run_smoke +``` + +### Windows PowerShell + +```powershell +Set-Location ai_engineering\rlvr\grpo_minimal +python -m venv .venv +.\.venv\Scripts\Activate.ps1 +python -m pip install -r requirements.txt +$env:PYTHONPATH = (Resolve-Path ..\regularized_operator_zoo).Path +python -m pytest tests\ -v +python -m smoke.run_smoke +``` + +## Relationship to the RLVR Operator Series + +These projects are companion implementations for a separately managed RLVR Operator Series. The article sources, frozen review packets, publication assets, and release process remain outside `my_dev_projects`. + +This separation is intentional: + +- this repository contains runnable portfolio code and repository-level validation; +- the series repository owns article claims, frozen boundaries, independent review, and publication artifacts; +- changes here must not silently revise frozen article claims; +- the series source must not be absorbed into this directory. + +## Scope and Limitations + +- Both projects are pedagogical research implementations, not production RL training infrastructure. +- The operator zoo works on small discrete action simplexes rather than full language-model vocabularies. +- GRPO Minimal imports the operator-zoo KL helper but currently computes its update gradient directly; the bridge is mathematical and module-level rather than delegated computation. +- GRPO Minimal uses a tabular categorical policy, not a transformer. +- The synthetic task has a deterministic 0/1 verifier and does not model long-form sequence generation. +- GRPO+PS, iterative reference refresh, large-model training, distributed execution, and GPU benchmarking are not implemented. +- The retained numerical results are local CPU measurements. Exact package versions were not recorded for the 2026-06-02 runs. + +## Planned Verifier Direction + +A future bounded design spike will examine how verifier properties affect learning on small symbolic, mathematical, and executable-code tasks. Candidate verifier conditions include accurate, sparse, dense, noisy, biased, and exploitable reward signals. + +That work is not implemented in this directory and is not part of v1.1.0. Its gate requires a discriminative CPU path, multiple seeds, reward-to-correctness diagnostics, bounded compute, and a design that does not duplicate GRPO Minimal or the separately managed Operator Series. + +## References + +- Geist, M., Scherrer, B., and Pietquin, O. (2019). *A Theory of Regularized Markov Decision Processes.* arXiv:1901.11275. +- Vieillard, N., et al. (2020). *Leverage the Average: an Analysis of KL Regularization in Reinforcement Learning.* arXiv:2003.14089. +- Shao, Z., et al. (2024). *DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.* arXiv:2402.03300. + +Licensed under the [MIT License](../../LICENSE).