From 1655d033029e1f6b17df1fb841871ec2ddd6d821 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:36:35 +0530 Subject: [PATCH 01/16] Add files via upload --- README.md | 372 ++------------------------------ STATIC_PLAYGROUND_DEPLOYMENT.md | 69 ++++++ index.html | 366 +++++++++++++++++++++++++++++++ test_public_api.py | 105 +++++++++ 4 files changed, 558 insertions(+), 354 deletions(-) create mode 100644 STATIC_PLAYGROUND_DEPLOYMENT.md create mode 100644 index.html create mode 100644 test_public_api.py diff --git a/README.md b/README.md index 3853084..ba07b7c 100644 --- a/README.md +++ b/README.md @@ -1,365 +1,29 @@ -# πŸͺ‘ RepoNeedle - -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/) -[![CI](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml) -[![Hugging Face Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-RepoNeedle--Bench-blue)](https://huggingface.co/datasets/RepoNeedle/RepoNeedle-Bench) -[![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Space-RepoNeedle--Playground-indigo)](https://huggingface.co/spaces/RepoNeedle/RepoNeedle-Playground) - -**RepoNeedle** is an open-source, token-budget-aware repository context retrieval system designed to help AI coding agents discover the most critical files for a coding task under strict context-token constraints. - ---- - -## ⚑ The Upstream Context Problem in AI Coding - -Autonomous coding agents (e.g., Claude Code, Aider, OpenHands, Cline) face a severe upstream bottleneck: **Context Acquisition**. - -When an agent tackles an issue, bug report, or test failure, passing the entire repository into the prompt is cost-prohibitive or exceeds context windows. Existing approaches face persistent failure modes: -- **Dense Embedding Models**: Rank files without considering file size (tokens), repository topology (call/import graphs), or task intent. They frequently retrieve large, irrelevant files that exhaust the context window. -- **RepoMap / PageRank Centrality**: Heavily prioritizes architectural hubs, but is completely blind to test files (which are leaf nodes with zero in-degree in import graphs) and cannot route specific error frames. -- **Context Bloat & Token Waste**: Studies like ContextBench demonstrate that coding agents pull massive amounts of irrelevant noise into context, degrading reasoning quality via "lost-in-the-middle" effects. -- **Selective Hallucination**: When a query cannot be answered within the repository (e.g. out-of-scope or counterfactual tasks), standard retrievers still force top-$k$ hallucinated files into context instead of abstaining. - -RepoNeedle solves this upstream bottleneck by combining: -1. **Multi-Signal Hybrid Scoring**: Field-weighted BM25 + Subword Semantic Vectors + AST Repository Graph Centrality. -2. **Workflow-Adaptive Routing**: Automatic classification of query intent (`trace2code`, `code2test`, `edit2ripple`, `comment2context`, `selective`) with dynamic weight tuning. -3. **Inverted Test-to-Target Mapping**: Dependency graph inversion and naming convention heuristics to locate regression test suites. -4. **Calibrated Selective Abstention**: Peak-to-noise ($Z$-score) and repository symbol overlap gating to abstain from returning noise on counterfactual tasks. -5. **CUD-Pack (Capacity-Sensitive Utility-Density Packing)**: Adaptive knapsack selection engine providing strict zero downstream overflows ($C_{\text{actual}} \le B_{\text{configured}}$) according to RepoNeedle's regex code token model across dual modes: **Compact Mode** ($\le 4\text{k}$ tokens, extracting high-density implementation spans and structural skeletons) and **Complete Context Mode** ($\ge 8\text{k}$ tokens, preserving whole source files). -6. **Change-Impact Ranking Engine (V1.1 Opt-in)**: Seed-relative relationship scoring (evaluating static imports, symbol definitions, call sites, build manifests (`setup.py`, `setup.cfg`, `pyproject.toml`, `cargo.toml`, `package.json`, `go.mod`, `pom.xml`), test-stems, and directory proximity) designed for bug fixing and refactoring tasks. - ---- - -## πŸ“Š Benchmark Validation - -To maintain strict scientific integrity, **Official External Benchmark Results** are reported separately from **Independent Research Evaluations**. Dataset sizes, query counts ($N$), candidate pool properties, and evaluation methodologies are explicitly documented for every measurement. - ---- - -### Part A: Full Official Agent Retrieval Bench (ARB v2) Evaluation ($N=345$ Positive Samples) - -Evaluated directly using the official `agent_retrieval_bench` evaluation package (`agent_retrieval_bench.baseline.sample_metrics`, v0.2.1, commit `9e4d68937f29`) on the complete **Agent Retrieval Bench v2** release ($N=345$ positive retrieval samples across 25 open-source repositories and 218 frozen repository snapshots): - -| System / Ranker | Overall MRR | Recall@5 | Recall@10 | Recall@20 | BCY@4k | BCY@8k | BCY@16k | BCY@32k | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| **RepoNeedle (Full)** | **0.2287** | **0.3011** | **0.4110** | **0.4279** | **0.2436** | **0.3450** | **0.4192** | **0.4279** | -| **Official RepoMap** | 0.2140 | 0.3185 | 0.4786 | **0.6306** | 0.2024 | **0.3788** | **0.5235** | **0.6166** | -| **Official Lexical** | 0.1574 | 0.2292 | 0.3548 | 0.4940 | 0.1488 | 0.2650 | 0.3886 | 0.4882 | -| **Official BM25** | 0.1520 | 0.1899 | 0.3034 | 0.4452 | 0.1220 | 0.2051 | 0.3184 | 0.4365 | -| *Qwen3-Embedding-4B (Ext Ref)* | *0.2379* | *β€”* | *β€”* | *0.6306* | *β€”* | *0.3409* | *β€”* | *β€”* | -| *Qwen3-Embedding-8B (Ext Ref)* | *0.2336* | *β€”* | *β€”* | *0.7029* | *β€”* | *0.3732* | *β€”* | *β€”* | -| *pplx-embed-v1-4b (Ext Ref)* | *0.2267* | *β€”* | *β€”* | *0.6072* | *β€”* | *0.3549* | *β€”* | *β€”* | -| *jina-code-0.5b (Ext Ref)* | *0.1914* | *β€”* | *β€”* | *0.4823* | *β€”* | *0.2783* | *β€”* | *β€”* | - -#### Matched Paired Statistical Tests against Baselines ($N=345$ Identical Queries) - -Evaluated with **10,000 bootstrap resamples** (repository-clustered bootstrap across 25 repositories): - -| Paired Comparison | Mean Diff (Ξ”) | Std Error | Sample 95% CI | Repo-Clustered 95% CI | Paired $t$-test | Wilcoxon $W$ ($p$-value) | Wins / Ties / Losses | Statistically Significant | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| **RepoNeedle vs BM25** | **+0.0767** (+50.5%) | 0.0161 | [+0.0458, +0.1085] | **[+0.0202, +0.1129]** | $t = 4.776$ ($p = 2.65\times 10^{-6}$) | $W = 22384.0$ ($p = 0.0096$) | 148 / 18 / 179 | **YES ($p < 0.01$)** | -| **RepoNeedle vs RepoMap** | **+0.0148** (+6.9%) | 0.0179 | [-0.0198, +0.0497] | **[-0.0466, +0.0619]** | $t = 0.827$ ($p = 0.4089$) | $W = 23821.5$ ($p = 0.3032$) | 122 / 26 / 197 | **NO (Parity in MRR)** | -| **RepoNeedle vs Lexical** | **+0.0713** (+45.3%) | 0.0157 | [+0.0409, +0.1031] | **[+0.0307, +0.1029]** | $t = 4.529$ ($p = 8.17\times 10^{-6}$) | $W = 22521.5$ ($p = 0.1215$) | 133 / 29 / 183 | **YES ($p < 0.001$)** | - -> [!IMPORTANT] -> **Scientific Disclosure on RepoMap Comparison**: RepoNeedle does **NOT** achieve statistically significant overall superiority over RepoMap on MRR ($p = 0.4089$, repository-clustered 95% CI crosses zero: `[-0.0466, +0.0619]`). Furthermore, **RepoMap achieves substantially higher deep recall** (Recall@20: `0.6306` vs `0.4279`, a +47.4% relative advantage for RepoMap) and higher BCY@8k (`0.3788` vs `0.3450`, +9.8% relative advantage). RepoNeedle demonstrates a focused advantage under strict low-token budgets (BCY@4k: `0.2436` vs `0.2024`, +20.4% relative advantage) and specialized regression test discovery (`code2test`). - -#### Per-Task Track Breakdown ($N=345$ Positive Samples) - -| Task Track | Sample Count ($N$) | Metric | Official BM25 | Official Lexical | Official RepoMap | RepoNeedle | Relative Gain vs BM25 | Relative Gain vs RepoMap | -| :--- | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | -| **`code2test`** | $N=106$ | **MRR** | 0.1057 | 0.0663 | 0.1942 | **0.2660** | **+151.7%** | **+37.0%** | -| | | Recall@5 | 0.1305 | 0.0676 | 0.2645 | **0.3918** | +200.2% | +48.1% | -| | | Recall@20 | 0.3066 | 0.2469 | 0.5572 | **0.5931** | +93.4% | +6.4% | -| | | **BCY@8k** | 0.1541 | 0.0928 | 0.3462 | **0.4814** | **+212.4%** | **+39.1%** | -| **`comment2context`** | $N=80$ | **MRR** | **0.1974** | 0.1530 | 0.1532 | 0.1708 | **-13.5%** | +11.5% | -| | | Recall@5 | **0.2146** | 0.1667 | 0.1750 | 0.1646 | -23.3% | -5.9% | -| | | Recall@20 | **0.5292** | 0.4979 | 0.4688 | 0.2083 | -60.6% | -55.6% | -| | | **BCY@8k** | 0.2604 | 0.2167 | **0.2792** | 0.1896 | -27.2% | -32.1% | -| **`trace2code`** | $N=101$ | **MRR** | 0.1638 | 0.2075 | **0.2723** | 0.2337 | +42.7% | **-14.2%** | -| | | Recall@5 | 0.2228 | 0.3432 | **0.4604** | 0.2855 | +28.1% | -38.0% | -| | | Recall@20 | 0.4934 | 0.6964 | **0.8465** | 0.4092 | -17.1% | -51.7% | -| | | **BCY@8k** | 0.1931 | 0.3795 | **0.5066** | 0.2954 | +53.0% | -41.7% | -| **`edit2ripple`** | $N=58$ | **MRR** | 0.1536 | **0.2428** | 0.2323 | 0.2320 | +51.0% | -0.1% | -| | | Recall@5 | 0.2069 | **0.4124** | 0.3678 | 0.3506 | +69.5% | -4.7% | -| | | Recall@20 | 0.4986 | 0.5876 | **0.6121** | 0.4612 | -7.5% | -24.7% | -| | | **BCY@8k** | 0.2428 | **0.4468** | 0.3534 | 0.3966 | +63.3% | +12.2% | - -> [!IMPORTANT] -> **Methodological Disclosure on `code2test` (+151.7%) and Workflow Routing**: -> - **Task-Conditioned Evaluation**: In the official ARB evaluation above, benchmark task labels (`code2test`, `trace2code`, etc.) were supplied directly to the `WorkflowRouter` (`task_type = sample.get("task_type")`), dynamically activating task-specific fusion weights and test-graph linkages. -> - **Not Default AUTO-Mode Performance**: This measures **task-conditioned retrieval capacity**, not unassisted automatic classification. In unassisted production deployment (`TaskType.AUTO`), the engine relies on query text regex heuristics. If a natural-language query lacks explicit test keywords, automatic intent classification may default to `comment2context`, reducing the observed gain toward baseline BM25 performance (`0.1057`). -> - **Candidate Set Integrity**: The task conditioning dynamically modulates ranking weights and structural graph boosts; candidate file pools themselves are never filtered by oracle labels. - ---- - -### Part B: Official ARB Selective Retrieval Evaluation ($N=82$ Abstention Samples) - -Evaluated on the complete official Agent Retrieval Bench Selective Retrieval split (`v2_selective_mixed`, $N=82$ total queries across 71 repository snapshots): - -| Evaluation Track | Sample Count ($N$) | Correctly Abstained | False Positives (Returned Noise) | Abstention Accuracy | -| :--- | :---: | :---: | :---: | :---: | -| **Natural No-Gold Issues** | 50 | 0 | 50 (100.0%) | **0.0%** | -| **Counterfactual Wrong-Repo** | 32 | 0 | 32 (100.0%) | **0.0%** | -| **Combined Selective Benchmark** | **82** | **0** | **82 (100.0%)** | **0.0%** | - -> [!CAUTION] -> **Heuristic Abstention Failure Analysis**: In production benchmark conditions, RepoNeedle's heuristic confidence gating ($Z$-score + repository symbol overlap) achieved **0% abstention accuracy (0/82)**. -> - *Root Cause*: Heuristic confidence scores ranged between `0.64` and `0.86`, far exceeding the default threshold (`0.12` / `0.38`). Natural GitHub issue descriptions filed against a repository inherently contain valid repository vocabulary. Counterfactual queries similarly contain ubiquitous software engineering terminology (`error`, `parse`, `client`, `handler`). -> - *Recommendation*: Heuristic score thresholds are insufficient for reliable abstention in large codebases. Selective retrieval requires a dedicated, trained neural discriminator rather than surface lexical statistics. - ---- - -### Part C: Component Ablations on Full ARB v2 ($N=345$) - -Ablation evaluation conducted on the complete $N=345$ positive ARB v2 benchmark: - -| Configuration | Overall MRR | Drop vs Full | Recall@5 | Recall@20 | Description | -| :--- | :---: | :---: | :---: | :---: | :--- | -| **Full RepoNeedle** | **0.2287** | **0.00%** | **0.3011** | 0.4279 | All components active (Router + Graph + BM25 + Skeletons) | -| *w/o WorkflowRouter* | 0.1642 | **-28.20%** | β€” | β€” | Static uniform weights across all task categories | -| *w/o RepoGraph* | 0.2287 | 0.00% | β€” | β€” | Graph centrality and test-to-target links zeroed | -| **Official RepoMap Baseline** | 0.2140 | -6.45% | 0.3185 | **0.6306** | Standalone PageRank graph centrality ranker | -| **Official BM25 Baseline** | 0.1520 | -33.53% | 0.1899 | 0.4452 | Standalone field BM25 ranker | - ---- - -### Part D: Multi-Scale Token Payload Reduction & Measured Runtime - -#### 1. Actual Content Payload Reduction Across Multiple Languages & File Sizes - -Measured using exact BPE code subword regex (`re.compile(r"[A-Za-z0-9_]+|[^\s\w]|\s+")`, calibrated to OpenAI cl100k / LLaMA / Qwen tokenizers): - -| Corpus File & Language | Lines | Full BPE Tokens | Skeleton BPE Tokens | Actual BPE Token Savings | Character Savings | Architectural Symbols Preserved | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | -| **Python (Short)** | 50L | 280 | 280 | 0.0% *(fits budget)* | 0.0% | **YES** (Complete) | -| **Python (Medium Adapter)** | 250L | 2,060 | 305 | **85.2% reduction** | 85.8% reduction | **YES** (Class, methods, docstrings) | -| **Python (Long Session Mgr)** | 700L | 4,061 | 404 | **90.1% reduction** | 90.0% reduction | **YES** (Class, methods, TTL logic) | -| **Rust (Async RingBuffer)** | 200L | 2,312 | 222 | **90.4% reduction** | 91.1% reduction | **YES** (Struct, impl, fn signatures) | -| **TypeScript (EventDispatcher)** | 200L | 1,920 | 455 | **76.3% reduction** | 74.6% reduction | **YES** (Interface, class, methods) | -| **Go (HTTP Trie Router)** | 200L | 1,225 | 578 | **52.8% reduction** | 44.2% reduction | **YES** (Types, struct, methods) | - -#### 2. Measured Benchmark Latency Distribution (Commodity CPU, Windows 11 AMD64) - -Empirically measured across the full Agent Retrieval Bench evaluation suite: - -| Component / Ranker | Mean Latency | Median ($p_{50}$) | 95th Percentile ($p_{95}$) | 99th Percentile ($p_{99}$) | Method Notes | -| :--- | :---: | :---: | :---: | :---: | :--- | -| **RepoNeedle Query Latency** | `317.7 ms` | `142.4 ms` | `827.2 ms` | `997.6 ms` | Multi-signal scoring + knapsack packing | -| **Official BM25 Query Latency** | `994.4 ms` | `1,158.3 ms` | `1,246.0 ms` | `1,305.7 ms` | Full inverted index posting scan | -| **Official RepoMap Query Latency**| `157.5 ms` | `192.1 ms` | `239.8 ms` | `250.1 ms` | Graph PageRank vector lookup | -| **Official Lexical Query Latency**| `884.1 ms` | `1,005.2 ms` | `1,150.3 ms` | `1,156.6 ms` | BM25 + subword character bigram matching | -| **RepoNeedle Index Build** | `8,622.9 ms` | `10,471.2 ms` | `11,484.8 ms` | `11,551.5 ms` | One-time AST + dependency graph per repo | - --- - -### Part E: System Architecture & Deployment Status - -- **CPU Native Architecture**: RepoNeedle runs entirely in Python/NumPy/SciPy/NetworkX without requiring CUDA GPUs or heavy deep learning runtime dependencies. -- **Release Status**: **V1.1.0 source integrated on main; formal release publication pending.** Features default `legacy` ranker with backwards-compatible contract preservation, and opt-in `change_impact` ranking engine for seed-relative relationship scoring. - +title: RepoNeedle Playground +emoji: πŸͺ‘ +colorFrom: blue +colorTo: indigo +sdk: static +pinned: false +license: mit --- -### Part F: Independent R6 Change-Impact Benchmark ($N=180$ Queries across 18 Repositories) - -To evaluate retrieval generalizability beyond the original 25 ARB repositories, an independent benchmark cohort was constructed across **18 unseen open-source repositories** comprising 180 retrieval queries. +# RepoNeedle Playground (V1.1.0) -Statistical inference was conducted using **repository-clustered statistical models** across independent repository clusters: +This is the static browser interface for the RepoNeedle V1.1.0 public demonstration. It calls a separately hosted FastAPI service; the static Space does not execute the Python retrieval engine itself. -| Partition | Queries ($N$) | Repositories | Baseline MRR | Champion MRR | Delta (Ξ”) | Repository-Clustered Inference (95% CI) | Statistical Significance | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| **Training** | $N=90$ | 9 repos | 0.6213 | 0.6546 | +0.0333 | β€” | Training fit | -| **Validation** | $N=40$ | 4 repos | 0.6472 | 0.6708 | +0.0236 | $p = 0.5073$, 95% CI [0.0000, +0.0472] | Not Statistically Significant | -| **Held-Out** | $N=50$ | 5 repos | 0.5215 | 0.5467 | +0.0252 | $p = 0.5025$, 95% CI [0.0000, +0.0629] | Not Statistically Significant | -| **Pooled Cohort** | $N=180$ | 18 repos | 0.5994 | **0.6282** | **+0.0288** | $p = 0.0078$, 95% CI [+0.0082, +0.0524] | **Statistically Significant ($p < 0.01$)** | - -> [!NOTE] -> **Methodological Context & Scope of R6 Results**: -> - **Pre-Filtered Candidate Pools**: Queries were evaluated against pre-filtered candidate pools averaging approximately **53.79 files per query**. Within these candidate pools, Candidate-Pool Recall@20 is 100%. -> - **Whole-Repository Generalization**: Whole-repository Recall@20 of 60–75% represents a projected estimate under full repository indexing, not an empirically measured whole-codebase scan. -> - **Statistical Power & Default Ranker**: While the pooled 18-repository cohort achieves statistically significant improvement under repository-clustered inference ($p = 0.0078$), the smaller isolated validation ($N=40$, $p = 0.5073$) and held-out ($N=50$, $p = 0.5025$) subsets are statistically underpowered. For this reason, the Change-Impact engine is provided as an **opt-in capability** rather than overriding the default `legacy` ranker. -> - **Aspirational Research Target**: The long-term research objective of $\text{MRR} \ge 0.80$ is an aspirational research milestone, not an achieved production metric. - ---- +**Live backend:** `https://reponeedle.onrender.com` +**Source repository:** https://github.com/krishika08/RepoNeedle +**Dataset:** https://huggingface.co/datasets/RepoNeedle/RepoNeedle-Bench -## πŸš€ Quickstart & Installation +## Demonstration scope -RepoNeedle is self-contained and requires standard Python dependencies (`numpy`, `scipy`, `networkx`, `pydantic`, `pathspec`, `rich`, and `tomli` on Python < 3.11). +The public Playground uses small embedded demonstration snapshots for `pallets/flask` and `psf/requests`. It does not clone arbitrary GitHub repositories, accept repository uploads, or reproduce the complete ARB v2 evaluation. Benchmark results described in the source repository were obtained in separate documented evaluations; this web demo is not itself a benchmark reproduction. -### 1. Installation +The browser sends retrieval requests to the public API. Queries are processed in memory and are not persisted by the demo API. The API is unauthenticated and intended for demonstration use, not production workloads. The free backend may sleep after inactivity, causing a cold-start delay. -```bash -# Clone the verified repository -git clone https://github.com/krishika08/RepoNeedle.git -cd RepoNeedle - -# Install package in editable mode -pip install -e . - -# Or install with development and test dependencies -pip install -e ".[dev]" -``` - -> [!TIP] -> On Python 3.10, `tomli>=2.0.1` is automatically installed via conditional dependency. On Python 3.11 and 3.12, native standard library `tomllib` is used with zero extra dependencies. - ---- - -### 2. Python API - -#### Standard Retrieval (Default Legacy Ranker): -```python -from reponeedle import RepoNeedleEngine, BudgetConfig - -# 1. Initialize engine on a local codebase (uses default 'legacy' ranker) -engine = RepoNeedleEngine.from_directory( - repo_dir="./my-project", - budget_config=BudgetConfig(max_tokens=4000, allow_skeletons=True, mode="auto"), -) - -# 2. Retrieve context for a task (auto-resolves to compact mode for <= 4k) -result = engine.retrieve( - query="Add support for blueprint url_prefix inheritance in route registration", - budget_tokens=4000, -) - -if result.abstained: - print(f"Abstained: {result.explanation}") -else: - print(f"Retrieved {len(result.selected_files)} files ({result.total_tokens} / {result.budget_tokens} tokens):") - for f in result.selected_files: - print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})") - - # Access the exact fenced-markdown serialized prompt ready for LLM consumption: - llm_prompt_context = result.rendered_context -``` - -#### Opt-In: Seed-Based Change-Impact Retrieval: -```python -from reponeedle import RepoNeedleEngine, Query - -# Initialize engine with Change-Impact ranking -engine = RepoNeedleEngine.from_directory( - repo_dir="./my-project", - ranker="change_impact", -) - -# Populate anchor_files with the seed path to activate relationship scoring -query = Query( - text="Fix authentication session expiration", - anchor_files=["auth/session.py"], - budget_tokens=4000, -) - -result = engine.retrieve(query) - -for f in result.selected_files: - print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})") - if f.explanation: - print(f" Attribution: {f.explanation}") -``` - -> [!NOTE] -> **Behavior of Plain-String (Seedless) Change-Impact Queries**: -> The Change-Impact ranking engine reads seed file paths from `query.anchor_files`. When `engine.retrieve()` is called with a plain string query (or without `anchor_files`): -> - **Lexical and semantic scoring remain active**: Candidate files continue to be retrieved and scored using combined BM25 lexical matching and subword semantic retrieval. -> - **Seed suppression is inactive without `anchor_files`**: Seed File Suppression (SFS) is not triggered since there are no designated seed files to down-weight. -> - **Seed-relative relationship boosts are inactive without `anchor_files`**: Relationship scoring across dependencies (static imports, symbols, calls, build manifests, test-stems, and directory proximity) is skipped. -> - **Test Intent Prior adjustments remain available according to detected query intent**: Test files receive boosts or penalties (`tip_multiplier_test` / `tip_multiplier_non_test`) based on detected test intent in the query text. -> -> To activate seed-relative relationship boosts and seed suppression, pass seed paths via `Query(text=..., anchor_files=['path/to/seed.py'])`. - ---- - -### 3. Command Line Interface (CLI) - -```bash -# Default 'legacy' retrieval under a 4000 token limit -reponeedle "Fix AttributeError in SecureCookieSessionInterface cookie saving" --repo ./flask --budget 4000 - -# Opt-in Change-Impact retrieval (applies Test Intent Prior scoring) -reponeedle "fix token expiration in auth/session.py" --repo ./my-project --budget 4000 --ranker change_impact - -# Task-conditioned retrieval for finding regression tests -reponeedle "test suite for HTTPBasicAuth" --repo ./requests --task code2test --budget 2000 - -# Serialize packaged markdown directly to a file -reponeedle "blueprints and route registration" --repo ./flask --budget 4000 --output-file prompt.md -``` - -> [!NOTE] -> **CLI Seed-File Limitation**: The current CLI accepts query text as a positional string and does not currently expose a seed-file argument. Standard CLI queries run with `--ranker change_impact` execute as seedless queries: lexical and semantic scoring remain active, and Test Intent Prior adjustments remain available according to detected query intent, while seed suppression and seed-relative relationship boosts are inactive without `anchor_files`. Full seed-based relationship scoring is available via the Python API (`Query(anchor_files=[...])`). CLI seed-file support is planned as a separate future enhancement. - ---- - -## 🧠 CUD-Pack Architecture (Capacity-Sensitive Utility-Density Packing) - -RepoNeedle integrates the **CUD-Pack** context optimization engine, replacing heuristic packing with capacity-elastic utility-density optimization: - -### 1. Dual Operational Modes -- **Compact Mode ($\le 4\text{k}$ tokens, $\alpha = 0.65$)**: Prioritizes focal implementation spans (`FOCUS_SPAN`) and structural skeletons (`SKELETON`). Prevents single distractor files from consuming the budget and displacing multi-target dependencies. -- **Complete Context Mode ($\ge 8\text{k}$ tokens, $\alpha = 0.25$)**: Prioritizes whole source files (`FULL`), selectively utilizing structural outlines for oversized files ($> 500$ tokens) when capacity is tight. - -### 2. Verified Token Budget Invariants -- **Deterministic Budget Enforcement**: Evaluated across tested nominal budgets (30, 50, 100, 500, 2000, 4000 tokens), CUD-Pack strictly enforces $C_{\text{actual}} \le B_{\text{configured}}$ with zero downstream token overflows according to RepoNeedle's deterministic regex token-counting model (`regex_code_tokenizer_v1`). -- **Overhead Accounting**: Pre-computes file header fences (`# File: {path}\n```{ext}\n...\n```\n\n`) deterministically before admitting candidates into the knapsack. -- **Representation Fallback Hierarchy**: Monotonically degrades candidate representations (`FULL` $\to$ `FOCUS_SPAN` $\to$ `SKELETON`) when full files exceed available token capacity. - -> [!WARNING] -> **Tokenizer Boundary Disclosure**: Budget non-exceedance is enforced strictly according to RepoNeedle's internal regex token model (`regex_code_tokenizer_v1`). This does not imply exact token-count equivalence with arbitrary third-party LLM tokenizers (e.g. OpenAI tiktoken cl100k, LLaMA sentencepiece, or Qwen) or guarantee downstream token behavior for every external provider. - ---- - -## πŸ”¬ Reproducing Benchmark Results - -Run the complete deterministic test suite and benchmarks: - -```bash -# 1. Run automated test suite (271 passed tests locally, 245 in clean CI checkouts) -pytest -q -rs - -# 2. Run component ablation study with 95% Bootstrap CI -python benchmarks/run_ablations.py - -# 3. Verify actual BPE token payload reduction and latency -python scripts/verify_token_savings_and_latency.py - -# 4. Build distribution packages -python -m build --no-isolation -``` - ---- - -## πŸ“¦ Hugging Face Release Artifacts - -RepoNeedle distributes its validated assets primarily via Hugging Face: - -1. **RepoNeedle-Bench Dataset**: [`hf_artifacts/dataset/`](hf_artifacts/dataset/README.md) - - Permissively licensed JSONL datasets (`train.jsonl`, `test.jsonl`), schema, and provenance documentation. -2. **RepoNeedle Playground Space**: [`hf_artifacts/space/`](hf_artifacts/space/README.md) - - Interactive Gradio demonstration allowing side-by-side comparison of RepoNeedle against classical BM25 and RepoMap PageRank. -3. **Automated Publishing Script**: [`scripts/prepare_hf_release.py`](scripts/prepare_hf_release.py) - - Verified local release package ready to publish with `python scripts/prepare_hf_release.py --token `. - ---- - -## πŸ›‘οΈ License & Data Provenance - -- **Source Code**: [MIT License](LICENSE) -- **RepoNeedle-Bench**: [MIT License](hf_artifacts/dataset/README.md) -- **Data Provenance**: Sourced exclusively from permissively licensed open-source projects (BSD-3-Clause, Apache-2.0, MIT). **Zero CC BY-NC or copyleft share-alike data is used.** Full manifest available in [`data_provenance/PROVENANCE.md`](data_provenance/PROVENANCE.md). -- **Upstream Notices**: Agent Retrieval Bench is Copyright (c) 2026 Agent Retrieval Bench contributors (MIT License). All upstream open-source code snapshots retain their original licenses and copyright notices. - ---- +## Reproducing the Space -## πŸ“š Citation +Upload `README.md` and `index.html` from this directory to a Hugging Face Space with the **Static** SDK. The interface is preconfigured with the public demo API URL; users can change the URL to point to their own deployment. -```bibtex -@misc{reponeedle2026, - author = {RepoNeedle Contributors}, - title = {RepoNeedle: Token-Budget-Aware, Workflow-Adaptive Repository Context Retrieval for AI Coding Agents}, - year = {2026}, - publisher = {GitHub}, - howpublished = {\url{https://github.com/krishika08/RepoNeedle}} -} -``` +See [deployment guide](https://github.com/krishika08/RepoNeedle/blob/main/docs/STATIC_PLAYGROUND_DEPLOYMENT.md) in the GitHub repository for backend deployment and configuration instructions. diff --git a/STATIC_PLAYGROUND_DEPLOYMENT.md b/STATIC_PLAYGROUND_DEPLOYMENT.md new file mode 100644 index 0000000..f46d14a --- /dev/null +++ b/STATIC_PLAYGROUND_DEPLOYMENT.md @@ -0,0 +1,69 @@ +# RepoNeedle V1.1.0 public demo deployment + +This document describes the public demo deployment. It does not change the installation path for the core Python package. + +## Components + +| Component | Location | Purpose | +|---|---|---| +| Core engine | `reponeedle/` | CPU-first retrieval library and CLI | +| Static frontend | `hf_artifacts/static_space/` | Browser UI hosted by Hugging Face Static Spaces | +| Demo API | `api.py` | FastAPI wrapper exposing retrieval for the embedded demo corpora | +| Demo snapshots and bundled UI engine | `hf_artifacts/space/` | Small Flask and Requests snapshots and synchronized RepoNeedle package used by the API | +| Dataset | `hf_artifacts/dataset/` | Lightweight synthetic demonstration benchmark | + +## Deploy the API on Render + +Create a Web Service connected to this repository and use: + +- Runtime: Python 3 +- Branch: `main` +- Root directory: repository root (blank) +- Build command: `pip install -r requirements-api.txt` +- Start command: `uvicorn api:app --host 0.0.0.0 --port $PORT` +- Health check path: `/health` +- Instance: Free (optional; subject to provider limits) + +The API imports the embedded corpora and engine from `hf_artifacts/space/`. Do not remove that directory while deploying this backend. + +Verify `https://YOUR-SERVICE.onrender.com/health`. A healthy response is: + +```json +{"status":"ok","service":"reponeedle-demo-api","version":"1.1.0"} +``` + +## Deploy the frontend on Hugging Face + +Create a public Static Space. Upload the contents of `hf_artifacts/static_space/` to the Space root. The `README.md` front matter must retain `sdk: static`. + +The UI defaults to `https://reponeedle.onrender.com`. Users can change the Backend API URL field and click **Connect API** to use another deployment. + +## API contract + +- `GET /` returns service metadata and a scope disclosure. +- `GET /health` is the health check. +- `POST /api/retrieve` accepts `repo_name`, `ranker`, `task_mode`, `budget_tokens`, `seed_files`, and `query`. +- Supported demo repositories: `pallets/flask`, `psf/requests`. +- Supported rankers: `legacy` (default), `change_impact`. +- The API validates query length, budget limits, repository choice, and seed count using Pydantic. + +## Limitations and security boundaries + +- This is a public, unauthenticated demonstration API. Do not send secrets, proprietary source code, or personal data. +- CORS is open so the public Hugging Face frontend can call the API. This is not an authentication or abuse-prevention control. +- The API only retrieves from the two embedded demonstration snapshots. It does not index arbitrary repositories or expose a general upload endpoint. +- The API does not persist visitor queries. Its in-memory caches reset when the service restarts. +- Free hosting may sleep after inactivity and has limited CPU/RAM. Cold starts and request latency can vary. +- The dataset is a small synthetic demo dataset, not the complete Agent Retrieval Bench v2 corpus. +- Selective abstention is experimental and should not be presented as reliable; see `KNOWN_LIMITATIONS.md`. + +## Local development and tests + +Install the package and development/API test extras: + +```bash +pip install -e ".[dev,api]" +pytest -q +``` + +The API tests use FastAPI's `TestClient` and the public-demo corpus shipped with the repository. diff --git a/index.html b/index.html new file mode 100644 index 0000000..9d181e6 --- /dev/null +++ b/index.html @@ -0,0 +1,366 @@ + + + + + + + + RepoNeedle Playground Β· V1.1.0 + + + +
+
+
+
πŸͺ‘
+
RepoNeedle
Repository context retrieval
+
+
PLAYGROUND Β· V1.1.0
+
+ +
+
+
A context-selection engine for coding agents
+

Find the code that
actually matters.

+

Explore token-budget-aware, workflow-adaptive repository retrieval. Compare RepoNeedle with classical BM25 and RepoMap-style PageRank, inspect selected files, and test the optional Change-Impact ranker.

+
+ +
+ +
+
BACKEND API URL
+ + +
Paste the public URL of the deployed Python API, then connect.
+
+ +
+
+

Configure retrieval

Choose a repository snapshot, ranking mode, workflow, and token budget.

+
+
+ + +
+
+
+ + +
+
+ + +
+
+
+ +
4,000 tokens
+
+
+ + +
Used by Change-Impact for seed-relative dependency scoring. Separate multiple paths with commas.
+
+
+ + +
+
+ + + +
+ +
Connect the API before running a query.
+
+
+ +
+
⌁
Your retrieval results will appear here.Connect the backend, enter a coding task, and run the query.
+
+
+
RETRIEVED CONTEXT
+
LEGACY
+
+
+
Files selectedβ€”
+
Tokens packedβ€”
+
Confidenceβ€”
+
Latencyβ€”
+
+

ENGINE DIAGNOSTICS

+

SELECTED CONTEXT FILES

#File pathTierTokensScore
+

BASELINE COMPARISON

#RepoNeedleBM25RepoMap Β· PageRank
+

TOP FILE PREVIEW

No selected file to preview.
+
+
+
+ +
+ RepoNeedle V1.1.0 Β· MIT License Β· GitHub repository Β· Demo benchmark dataset
+ Free backend hosting may sleep after inactivity; the first request can take about a minute to wake up. +
+
+ + + + diff --git a/test_public_api.py b/test_public_api.py new file mode 100644 index 0000000..ba1cffc --- /dev/null +++ b/test_public_api.py @@ -0,0 +1,105 @@ +"""Integration tests for the public RepoNeedle demonstration API.""" + +import pytest +from fastapi.testclient import TestClient + +from api import app + + +@pytest.fixture(scope="module") +def client(): + with TestClient(app) as test_client: + yield test_client + + +def test_root_discloses_demo_scope(client): + response = client.get("/") + assert response.status_code == 200 + payload = response.json() + assert payload["name"] == "RepoNeedle Demo API" + assert payload["version"] == "1.1.0" + assert "Embedded demonstration snapshots only" in payload["disclosure"] + + +def test_health_endpoint(client): + response = client.get("/health") + assert response.status_code == 200 + assert response.json() == { + "status": "ok", + "service": "reponeedle-demo-api", + "version": "1.1.0", + } + + +def test_legacy_retrieval_contract(client): + response = client.post( + "/api/retrieve", + json={ + "repo_name": "pallets/flask", + "ranker": "legacy", + "task_mode": "auto", + "budget_tokens": 4000, + "query": "Find the implementation of Flask request dispatching and related tests.", + }, + ) + assert response.status_code == 200, response.text + payload = response.json() + assert payload["ranker"] == "legacy" + assert payload["task_type"] in {"comment2context", "code2test", "edit2ripple", "trace2code"} + assert payload["tokens_used"] <= payload["token_budget"] + assert isinstance(payload["selected_files"], list) + assert isinstance(payload["comparison"], list) + assert payload["preview"] is None or "content" in payload["preview"] + + +def test_change_impact_auto_infers_task_from_seed(client): + response = client.post( + "/api/retrieve", + json={ + "repo_name": "pallets/flask", + "ranker": "change_impact", + "task_mode": "auto", + "budget_tokens": 4000, + "seed_files": ["src/flask/sessions.py"], + "query": "Fix cookie attribute handling during session modification in SecureCookieSessionInterface.", + }, + ) + assert response.status_code == 200, response.text + payload = response.json() + assert payload["ranker"] == "change_impact" + assert payload["task_type"] == "edit2ripple" + assert "seed-relative dependency boosts" in payload["ranker_note"] + assert payload["tokens_used"] <= payload["token_budget"] + selected_paths = [item["path"] for item in payload["selected_files"]] + assert selected_paths + # Seed File Suppression should keep the seed from occupying the first result. + assert selected_paths[0] != "src/flask/sessions.py" + + +def test_invalid_repository_is_rejected(client): + response = client.post( + "/api/retrieve", + json={"repo_name": "unknown/repository", "query": "find request handling"}, + ) + assert response.status_code == 422 + + +def test_out_of_range_budget_is_rejected(client): + response = client.post( + "/api/retrieve", + json={"repo_name": "pallets/flask", "budget_tokens": 100, "query": "find request handling"}, + ) + assert response.status_code == 422 + + +def test_cors_preflight_is_available(client): + response = client.options( + "/api/retrieve", + headers={ + "Origin": "https://krishikax-reponeedle-playground.hf.space", + "Access-Control-Request-Method": "POST", + "Access-Control-Request-Headers": "content-type", + }, + ) + assert response.status_code == 200 + assert response.headers.get("access-control-allow-origin") == "*" From 0694b07cccbd249c12473fb4c2c4b63e970017c4 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:43:24 +0530 Subject: [PATCH 02/16] Rename README.md to hf_artifacts/static_space/README.md --- README.md => hf_artifacts/static_space/README.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename README.md => hf_artifacts/static_space/README.md (100%) diff --git a/README.md b/hf_artifacts/static_space/README.md similarity index 100% rename from README.md rename to hf_artifacts/static_space/README.md From 808197cc1ab8753562064b4ba2ca9486b7909b9b Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:44:56 +0530 Subject: [PATCH 03/16] Rename index.html to hf_artifacts/static_space/index.html --- index.html => hf_artifacts/static_space/index.html | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename index.html => hf_artifacts/static_space/index.html (100%) diff --git a/index.html b/hf_artifacts/static_space/index.html similarity index 100% rename from index.html rename to hf_artifacts/static_space/index.html From b5ee379534e880ecacd4115f95944899891d0572 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:45:44 +0530 Subject: [PATCH 04/16] Rename STATIC_PLAYGROUND_DEPLOYMENT.md to docs/STATIC_PLAYGROUND_DEPLOYMENT.md --- .../STATIC_PLAYGROUND_DEPLOYMENT.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename STATIC_PLAYGROUND_DEPLOYMENT.md => docs/STATIC_PLAYGROUND_DEPLOYMENT.md (100%) diff --git a/STATIC_PLAYGROUND_DEPLOYMENT.md b/docs/STATIC_PLAYGROUND_DEPLOYMENT.md similarity index 100% rename from STATIC_PLAYGROUND_DEPLOYMENT.md rename to docs/STATIC_PLAYGROUND_DEPLOYMENT.md From cf01082bebe08e80d6c1b9c4844319fa29f5326c Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:46:41 +0530 Subject: [PATCH 05/16] Rename test_public_api.py to tests/tests/test_public_api.py --- test_public_api.py => tests/tests/test_public_api.py | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename test_public_api.py => tests/tests/test_public_api.py (100%) diff --git a/test_public_api.py b/tests/tests/test_public_api.py similarity index 100% rename from test_public_api.py rename to tests/tests/test_public_api.py From db9c5be223a0357cbe7e1a3cd8d0593a685809f9 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:49:31 +0530 Subject: [PATCH 06/16] Rename tests/tests/test_public_api.py to tests/test_public_api.py --- tests/{tests => }/test_public_api.py | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename tests/{tests => }/test_public_api.py (100%) diff --git a/tests/tests/test_public_api.py b/tests/test_public_api.py similarity index 100% rename from tests/tests/test_public_api.py rename to tests/test_public_api.py From 925fbca342d087996c71d352277a1f4f64fa881b Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 02:55:54 +0530 Subject: [PATCH 07/16] Initialize README with project details and setup Add initial README with project overview and installation instructions. --- README.md | 365 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..3853084 --- /dev/null +++ b/README.md @@ -0,0 +1,365 @@ +# πŸͺ‘ RepoNeedle + +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/) +[![CI](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml) +[![Hugging Face Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-RepoNeedle--Bench-blue)](https://huggingface.co/datasets/RepoNeedle/RepoNeedle-Bench) +[![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Space-RepoNeedle--Playground-indigo)](https://huggingface.co/spaces/RepoNeedle/RepoNeedle-Playground) + +**RepoNeedle** is an open-source, token-budget-aware repository context retrieval system designed to help AI coding agents discover the most critical files for a coding task under strict context-token constraints. + +--- + +## ⚑ The Upstream Context Problem in AI Coding + +Autonomous coding agents (e.g., Claude Code, Aider, OpenHands, Cline) face a severe upstream bottleneck: **Context Acquisition**. + +When an agent tackles an issue, bug report, or test failure, passing the entire repository into the prompt is cost-prohibitive or exceeds context windows. Existing approaches face persistent failure modes: +- **Dense Embedding Models**: Rank files without considering file size (tokens), repository topology (call/import graphs), or task intent. They frequently retrieve large, irrelevant files that exhaust the context window. +- **RepoMap / PageRank Centrality**: Heavily prioritizes architectural hubs, but is completely blind to test files (which are leaf nodes with zero in-degree in import graphs) and cannot route specific error frames. +- **Context Bloat & Token Waste**: Studies like ContextBench demonstrate that coding agents pull massive amounts of irrelevant noise into context, degrading reasoning quality via "lost-in-the-middle" effects. +- **Selective Hallucination**: When a query cannot be answered within the repository (e.g. out-of-scope or counterfactual tasks), standard retrievers still force top-$k$ hallucinated files into context instead of abstaining. + +RepoNeedle solves this upstream bottleneck by combining: +1. **Multi-Signal Hybrid Scoring**: Field-weighted BM25 + Subword Semantic Vectors + AST Repository Graph Centrality. +2. **Workflow-Adaptive Routing**: Automatic classification of query intent (`trace2code`, `code2test`, `edit2ripple`, `comment2context`, `selective`) with dynamic weight tuning. +3. **Inverted Test-to-Target Mapping**: Dependency graph inversion and naming convention heuristics to locate regression test suites. +4. **Calibrated Selective Abstention**: Peak-to-noise ($Z$-score) and repository symbol overlap gating to abstain from returning noise on counterfactual tasks. +5. **CUD-Pack (Capacity-Sensitive Utility-Density Packing)**: Adaptive knapsack selection engine providing strict zero downstream overflows ($C_{\text{actual}} \le B_{\text{configured}}$) according to RepoNeedle's regex code token model across dual modes: **Compact Mode** ($\le 4\text{k}$ tokens, extracting high-density implementation spans and structural skeletons) and **Complete Context Mode** ($\ge 8\text{k}$ tokens, preserving whole source files). +6. **Change-Impact Ranking Engine (V1.1 Opt-in)**: Seed-relative relationship scoring (evaluating static imports, symbol definitions, call sites, build manifests (`setup.py`, `setup.cfg`, `pyproject.toml`, `cargo.toml`, `package.json`, `go.mod`, `pom.xml`), test-stems, and directory proximity) designed for bug fixing and refactoring tasks. + +--- + +## πŸ“Š Benchmark Validation + +To maintain strict scientific integrity, **Official External Benchmark Results** are reported separately from **Independent Research Evaluations**. Dataset sizes, query counts ($N$), candidate pool properties, and evaluation methodologies are explicitly documented for every measurement. + +--- + +### Part A: Full Official Agent Retrieval Bench (ARB v2) Evaluation ($N=345$ Positive Samples) + +Evaluated directly using the official `agent_retrieval_bench` evaluation package (`agent_retrieval_bench.baseline.sample_metrics`, v0.2.1, commit `9e4d68937f29`) on the complete **Agent Retrieval Bench v2** release ($N=345$ positive retrieval samples across 25 open-source repositories and 218 frozen repository snapshots): + +| System / Ranker | Overall MRR | Recall@5 | Recall@10 | Recall@20 | BCY@4k | BCY@8k | BCY@16k | BCY@32k | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| **RepoNeedle (Full)** | **0.2287** | **0.3011** | **0.4110** | **0.4279** | **0.2436** | **0.3450** | **0.4192** | **0.4279** | +| **Official RepoMap** | 0.2140 | 0.3185 | 0.4786 | **0.6306** | 0.2024 | **0.3788** | **0.5235** | **0.6166** | +| **Official Lexical** | 0.1574 | 0.2292 | 0.3548 | 0.4940 | 0.1488 | 0.2650 | 0.3886 | 0.4882 | +| **Official BM25** | 0.1520 | 0.1899 | 0.3034 | 0.4452 | 0.1220 | 0.2051 | 0.3184 | 0.4365 | +| *Qwen3-Embedding-4B (Ext Ref)* | *0.2379* | *β€”* | *β€”* | *0.6306* | *β€”* | *0.3409* | *β€”* | *β€”* | +| *Qwen3-Embedding-8B (Ext Ref)* | *0.2336* | *β€”* | *β€”* | *0.7029* | *β€”* | *0.3732* | *β€”* | *β€”* | +| *pplx-embed-v1-4b (Ext Ref)* | *0.2267* | *β€”* | *β€”* | *0.6072* | *β€”* | *0.3549* | *β€”* | *β€”* | +| *jina-code-0.5b (Ext Ref)* | *0.1914* | *β€”* | *β€”* | *0.4823* | *β€”* | *0.2783* | *β€”* | *β€”* | + +#### Matched Paired Statistical Tests against Baselines ($N=345$ Identical Queries) + +Evaluated with **10,000 bootstrap resamples** (repository-clustered bootstrap across 25 repositories): + +| Paired Comparison | Mean Diff (Ξ”) | Std Error | Sample 95% CI | Repo-Clustered 95% CI | Paired $t$-test | Wilcoxon $W$ ($p$-value) | Wins / Ties / Losses | Statistically Significant | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| **RepoNeedle vs BM25** | **+0.0767** (+50.5%) | 0.0161 | [+0.0458, +0.1085] | **[+0.0202, +0.1129]** | $t = 4.776$ ($p = 2.65\times 10^{-6}$) | $W = 22384.0$ ($p = 0.0096$) | 148 / 18 / 179 | **YES ($p < 0.01$)** | +| **RepoNeedle vs RepoMap** | **+0.0148** (+6.9%) | 0.0179 | [-0.0198, +0.0497] | **[-0.0466, +0.0619]** | $t = 0.827$ ($p = 0.4089$) | $W = 23821.5$ ($p = 0.3032$) | 122 / 26 / 197 | **NO (Parity in MRR)** | +| **RepoNeedle vs Lexical** | **+0.0713** (+45.3%) | 0.0157 | [+0.0409, +0.1031] | **[+0.0307, +0.1029]** | $t = 4.529$ ($p = 8.17\times 10^{-6}$) | $W = 22521.5$ ($p = 0.1215$) | 133 / 29 / 183 | **YES ($p < 0.001$)** | + +> [!IMPORTANT] +> **Scientific Disclosure on RepoMap Comparison**: RepoNeedle does **NOT** achieve statistically significant overall superiority over RepoMap on MRR ($p = 0.4089$, repository-clustered 95% CI crosses zero: `[-0.0466, +0.0619]`). Furthermore, **RepoMap achieves substantially higher deep recall** (Recall@20: `0.6306` vs `0.4279`, a +47.4% relative advantage for RepoMap) and higher BCY@8k (`0.3788` vs `0.3450`, +9.8% relative advantage). RepoNeedle demonstrates a focused advantage under strict low-token budgets (BCY@4k: `0.2436` vs `0.2024`, +20.4% relative advantage) and specialized regression test discovery (`code2test`). + +#### Per-Task Track Breakdown ($N=345$ Positive Samples) + +| Task Track | Sample Count ($N$) | Metric | Official BM25 | Official Lexical | Official RepoMap | RepoNeedle | Relative Gain vs BM25 | Relative Gain vs RepoMap | +| :--- | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | +| **`code2test`** | $N=106$ | **MRR** | 0.1057 | 0.0663 | 0.1942 | **0.2660** | **+151.7%** | **+37.0%** | +| | | Recall@5 | 0.1305 | 0.0676 | 0.2645 | **0.3918** | +200.2% | +48.1% | +| | | Recall@20 | 0.3066 | 0.2469 | 0.5572 | **0.5931** | +93.4% | +6.4% | +| | | **BCY@8k** | 0.1541 | 0.0928 | 0.3462 | **0.4814** | **+212.4%** | **+39.1%** | +| **`comment2context`** | $N=80$ | **MRR** | **0.1974** | 0.1530 | 0.1532 | 0.1708 | **-13.5%** | +11.5% | +| | | Recall@5 | **0.2146** | 0.1667 | 0.1750 | 0.1646 | -23.3% | -5.9% | +| | | Recall@20 | **0.5292** | 0.4979 | 0.4688 | 0.2083 | -60.6% | -55.6% | +| | | **BCY@8k** | 0.2604 | 0.2167 | **0.2792** | 0.1896 | -27.2% | -32.1% | +| **`trace2code`** | $N=101$ | **MRR** | 0.1638 | 0.2075 | **0.2723** | 0.2337 | +42.7% | **-14.2%** | +| | | Recall@5 | 0.2228 | 0.3432 | **0.4604** | 0.2855 | +28.1% | -38.0% | +| | | Recall@20 | 0.4934 | 0.6964 | **0.8465** | 0.4092 | -17.1% | -51.7% | +| | | **BCY@8k** | 0.1931 | 0.3795 | **0.5066** | 0.2954 | +53.0% | -41.7% | +| **`edit2ripple`** | $N=58$ | **MRR** | 0.1536 | **0.2428** | 0.2323 | 0.2320 | +51.0% | -0.1% | +| | | Recall@5 | 0.2069 | **0.4124** | 0.3678 | 0.3506 | +69.5% | -4.7% | +| | | Recall@20 | 0.4986 | 0.5876 | **0.6121** | 0.4612 | -7.5% | -24.7% | +| | | **BCY@8k** | 0.2428 | **0.4468** | 0.3534 | 0.3966 | +63.3% | +12.2% | + +> [!IMPORTANT] +> **Methodological Disclosure on `code2test` (+151.7%) and Workflow Routing**: +> - **Task-Conditioned Evaluation**: In the official ARB evaluation above, benchmark task labels (`code2test`, `trace2code`, etc.) were supplied directly to the `WorkflowRouter` (`task_type = sample.get("task_type")`), dynamically activating task-specific fusion weights and test-graph linkages. +> - **Not Default AUTO-Mode Performance**: This measures **task-conditioned retrieval capacity**, not unassisted automatic classification. In unassisted production deployment (`TaskType.AUTO`), the engine relies on query text regex heuristics. If a natural-language query lacks explicit test keywords, automatic intent classification may default to `comment2context`, reducing the observed gain toward baseline BM25 performance (`0.1057`). +> - **Candidate Set Integrity**: The task conditioning dynamically modulates ranking weights and structural graph boosts; candidate file pools themselves are never filtered by oracle labels. + +--- + +### Part B: Official ARB Selective Retrieval Evaluation ($N=82$ Abstention Samples) + +Evaluated on the complete official Agent Retrieval Bench Selective Retrieval split (`v2_selective_mixed`, $N=82$ total queries across 71 repository snapshots): + +| Evaluation Track | Sample Count ($N$) | Correctly Abstained | False Positives (Returned Noise) | Abstention Accuracy | +| :--- | :---: | :---: | :---: | :---: | +| **Natural No-Gold Issues** | 50 | 0 | 50 (100.0%) | **0.0%** | +| **Counterfactual Wrong-Repo** | 32 | 0 | 32 (100.0%) | **0.0%** | +| **Combined Selective Benchmark** | **82** | **0** | **82 (100.0%)** | **0.0%** | + +> [!CAUTION] +> **Heuristic Abstention Failure Analysis**: In production benchmark conditions, RepoNeedle's heuristic confidence gating ($Z$-score + repository symbol overlap) achieved **0% abstention accuracy (0/82)**. +> - *Root Cause*: Heuristic confidence scores ranged between `0.64` and `0.86`, far exceeding the default threshold (`0.12` / `0.38`). Natural GitHub issue descriptions filed against a repository inherently contain valid repository vocabulary. Counterfactual queries similarly contain ubiquitous software engineering terminology (`error`, `parse`, `client`, `handler`). +> - *Recommendation*: Heuristic score thresholds are insufficient for reliable abstention in large codebases. Selective retrieval requires a dedicated, trained neural discriminator rather than surface lexical statistics. + +--- + +### Part C: Component Ablations on Full ARB v2 ($N=345$) + +Ablation evaluation conducted on the complete $N=345$ positive ARB v2 benchmark: + +| Configuration | Overall MRR | Drop vs Full | Recall@5 | Recall@20 | Description | +| :--- | :---: | :---: | :---: | :---: | :--- | +| **Full RepoNeedle** | **0.2287** | **0.00%** | **0.3011** | 0.4279 | All components active (Router + Graph + BM25 + Skeletons) | +| *w/o WorkflowRouter* | 0.1642 | **-28.20%** | β€” | β€” | Static uniform weights across all task categories | +| *w/o RepoGraph* | 0.2287 | 0.00% | β€” | β€” | Graph centrality and test-to-target links zeroed | +| **Official RepoMap Baseline** | 0.2140 | -6.45% | 0.3185 | **0.6306** | Standalone PageRank graph centrality ranker | +| **Official BM25 Baseline** | 0.1520 | -33.53% | 0.1899 | 0.4452 | Standalone field BM25 ranker | + +--- + +### Part D: Multi-Scale Token Payload Reduction & Measured Runtime + +#### 1. Actual Content Payload Reduction Across Multiple Languages & File Sizes + +Measured using exact BPE code subword regex (`re.compile(r"[A-Za-z0-9_]+|[^\s\w]|\s+")`, calibrated to OpenAI cl100k / LLaMA / Qwen tokenizers): + +| Corpus File & Language | Lines | Full BPE Tokens | Skeleton BPE Tokens | Actual BPE Token Savings | Character Savings | Architectural Symbols Preserved | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | +| **Python (Short)** | 50L | 280 | 280 | 0.0% *(fits budget)* | 0.0% | **YES** (Complete) | +| **Python (Medium Adapter)** | 250L | 2,060 | 305 | **85.2% reduction** | 85.8% reduction | **YES** (Class, methods, docstrings) | +| **Python (Long Session Mgr)** | 700L | 4,061 | 404 | **90.1% reduction** | 90.0% reduction | **YES** (Class, methods, TTL logic) | +| **Rust (Async RingBuffer)** | 200L | 2,312 | 222 | **90.4% reduction** | 91.1% reduction | **YES** (Struct, impl, fn signatures) | +| **TypeScript (EventDispatcher)** | 200L | 1,920 | 455 | **76.3% reduction** | 74.6% reduction | **YES** (Interface, class, methods) | +| **Go (HTTP Trie Router)** | 200L | 1,225 | 578 | **52.8% reduction** | 44.2% reduction | **YES** (Types, struct, methods) | + +#### 2. Measured Benchmark Latency Distribution (Commodity CPU, Windows 11 AMD64) + +Empirically measured across the full Agent Retrieval Bench evaluation suite: + +| Component / Ranker | Mean Latency | Median ($p_{50}$) | 95th Percentile ($p_{95}$) | 99th Percentile ($p_{99}$) | Method Notes | +| :--- | :---: | :---: | :---: | :---: | :--- | +| **RepoNeedle Query Latency** | `317.7 ms` | `142.4 ms` | `827.2 ms` | `997.6 ms` | Multi-signal scoring + knapsack packing | +| **Official BM25 Query Latency** | `994.4 ms` | `1,158.3 ms` | `1,246.0 ms` | `1,305.7 ms` | Full inverted index posting scan | +| **Official RepoMap Query Latency**| `157.5 ms` | `192.1 ms` | `239.8 ms` | `250.1 ms` | Graph PageRank vector lookup | +| **Official Lexical Query Latency**| `884.1 ms` | `1,005.2 ms` | `1,150.3 ms` | `1,156.6 ms` | BM25 + subword character bigram matching | +| **RepoNeedle Index Build** | `8,622.9 ms` | `10,471.2 ms` | `11,484.8 ms` | `11,551.5 ms` | One-time AST + dependency graph per repo | + +--- + +### Part E: System Architecture & Deployment Status + +- **CPU Native Architecture**: RepoNeedle runs entirely in Python/NumPy/SciPy/NetworkX without requiring CUDA GPUs or heavy deep learning runtime dependencies. +- **Release Status**: **V1.1.0 source integrated on main; formal release publication pending.** Features default `legacy` ranker with backwards-compatible contract preservation, and opt-in `change_impact` ranking engine for seed-relative relationship scoring. + +--- + +### Part F: Independent R6 Change-Impact Benchmark ($N=180$ Queries across 18 Repositories) + +To evaluate retrieval generalizability beyond the original 25 ARB repositories, an independent benchmark cohort was constructed across **18 unseen open-source repositories** comprising 180 retrieval queries. + +Statistical inference was conducted using **repository-clustered statistical models** across independent repository clusters: + +| Partition | Queries ($N$) | Repositories | Baseline MRR | Champion MRR | Delta (Ξ”) | Repository-Clustered Inference (95% CI) | Statistical Significance | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| **Training** | $N=90$ | 9 repos | 0.6213 | 0.6546 | +0.0333 | β€” | Training fit | +| **Validation** | $N=40$ | 4 repos | 0.6472 | 0.6708 | +0.0236 | $p = 0.5073$, 95% CI [0.0000, +0.0472] | Not Statistically Significant | +| **Held-Out** | $N=50$ | 5 repos | 0.5215 | 0.5467 | +0.0252 | $p = 0.5025$, 95% CI [0.0000, +0.0629] | Not Statistically Significant | +| **Pooled Cohort** | $N=180$ | 18 repos | 0.5994 | **0.6282** | **+0.0288** | $p = 0.0078$, 95% CI [+0.0082, +0.0524] | **Statistically Significant ($p < 0.01$)** | + +> [!NOTE] +> **Methodological Context & Scope of R6 Results**: +> - **Pre-Filtered Candidate Pools**: Queries were evaluated against pre-filtered candidate pools averaging approximately **53.79 files per query**. Within these candidate pools, Candidate-Pool Recall@20 is 100%. +> - **Whole-Repository Generalization**: Whole-repository Recall@20 of 60–75% represents a projected estimate under full repository indexing, not an empirically measured whole-codebase scan. +> - **Statistical Power & Default Ranker**: While the pooled 18-repository cohort achieves statistically significant improvement under repository-clustered inference ($p = 0.0078$), the smaller isolated validation ($N=40$, $p = 0.5073$) and held-out ($N=50$, $p = 0.5025$) subsets are statistically underpowered. For this reason, the Change-Impact engine is provided as an **opt-in capability** rather than overriding the default `legacy` ranker. +> - **Aspirational Research Target**: The long-term research objective of $\text{MRR} \ge 0.80$ is an aspirational research milestone, not an achieved production metric. + +--- + +## πŸš€ Quickstart & Installation + +RepoNeedle is self-contained and requires standard Python dependencies (`numpy`, `scipy`, `networkx`, `pydantic`, `pathspec`, `rich`, and `tomli` on Python < 3.11). + +### 1. Installation + +```bash +# Clone the verified repository +git clone https://github.com/krishika08/RepoNeedle.git +cd RepoNeedle + +# Install package in editable mode +pip install -e . + +# Or install with development and test dependencies +pip install -e ".[dev]" +``` + +> [!TIP] +> On Python 3.10, `tomli>=2.0.1` is automatically installed via conditional dependency. On Python 3.11 and 3.12, native standard library `tomllib` is used with zero extra dependencies. + +--- + +### 2. Python API + +#### Standard Retrieval (Default Legacy Ranker): +```python +from reponeedle import RepoNeedleEngine, BudgetConfig + +# 1. Initialize engine on a local codebase (uses default 'legacy' ranker) +engine = RepoNeedleEngine.from_directory( + repo_dir="./my-project", + budget_config=BudgetConfig(max_tokens=4000, allow_skeletons=True, mode="auto"), +) + +# 2. Retrieve context for a task (auto-resolves to compact mode for <= 4k) +result = engine.retrieve( + query="Add support for blueprint url_prefix inheritance in route registration", + budget_tokens=4000, +) + +if result.abstained: + print(f"Abstained: {result.explanation}") +else: + print(f"Retrieved {len(result.selected_files)} files ({result.total_tokens} / {result.budget_tokens} tokens):") + for f in result.selected_files: + print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})") + + # Access the exact fenced-markdown serialized prompt ready for LLM consumption: + llm_prompt_context = result.rendered_context +``` + +#### Opt-In: Seed-Based Change-Impact Retrieval: +```python +from reponeedle import RepoNeedleEngine, Query + +# Initialize engine with Change-Impact ranking +engine = RepoNeedleEngine.from_directory( + repo_dir="./my-project", + ranker="change_impact", +) + +# Populate anchor_files with the seed path to activate relationship scoring +query = Query( + text="Fix authentication session expiration", + anchor_files=["auth/session.py"], + budget_tokens=4000, +) + +result = engine.retrieve(query) + +for f in result.selected_files: + print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})") + if f.explanation: + print(f" Attribution: {f.explanation}") +``` + +> [!NOTE] +> **Behavior of Plain-String (Seedless) Change-Impact Queries**: +> The Change-Impact ranking engine reads seed file paths from `query.anchor_files`. When `engine.retrieve()` is called with a plain string query (or without `anchor_files`): +> - **Lexical and semantic scoring remain active**: Candidate files continue to be retrieved and scored using combined BM25 lexical matching and subword semantic retrieval. +> - **Seed suppression is inactive without `anchor_files`**: Seed File Suppression (SFS) is not triggered since there are no designated seed files to down-weight. +> - **Seed-relative relationship boosts are inactive without `anchor_files`**: Relationship scoring across dependencies (static imports, symbols, calls, build manifests, test-stems, and directory proximity) is skipped. +> - **Test Intent Prior adjustments remain available according to detected query intent**: Test files receive boosts or penalties (`tip_multiplier_test` / `tip_multiplier_non_test`) based on detected test intent in the query text. +> +> To activate seed-relative relationship boosts and seed suppression, pass seed paths via `Query(text=..., anchor_files=['path/to/seed.py'])`. + +--- + +### 3. Command Line Interface (CLI) + +```bash +# Default 'legacy' retrieval under a 4000 token limit +reponeedle "Fix AttributeError in SecureCookieSessionInterface cookie saving" --repo ./flask --budget 4000 + +# Opt-in Change-Impact retrieval (applies Test Intent Prior scoring) +reponeedle "fix token expiration in auth/session.py" --repo ./my-project --budget 4000 --ranker change_impact + +# Task-conditioned retrieval for finding regression tests +reponeedle "test suite for HTTPBasicAuth" --repo ./requests --task code2test --budget 2000 + +# Serialize packaged markdown directly to a file +reponeedle "blueprints and route registration" --repo ./flask --budget 4000 --output-file prompt.md +``` + +> [!NOTE] +> **CLI Seed-File Limitation**: The current CLI accepts query text as a positional string and does not currently expose a seed-file argument. Standard CLI queries run with `--ranker change_impact` execute as seedless queries: lexical and semantic scoring remain active, and Test Intent Prior adjustments remain available according to detected query intent, while seed suppression and seed-relative relationship boosts are inactive without `anchor_files`. Full seed-based relationship scoring is available via the Python API (`Query(anchor_files=[...])`). CLI seed-file support is planned as a separate future enhancement. + +--- + +## 🧠 CUD-Pack Architecture (Capacity-Sensitive Utility-Density Packing) + +RepoNeedle integrates the **CUD-Pack** context optimization engine, replacing heuristic packing with capacity-elastic utility-density optimization: + +### 1. Dual Operational Modes +- **Compact Mode ($\le 4\text{k}$ tokens, $\alpha = 0.65$)**: Prioritizes focal implementation spans (`FOCUS_SPAN`) and structural skeletons (`SKELETON`). Prevents single distractor files from consuming the budget and displacing multi-target dependencies. +- **Complete Context Mode ($\ge 8\text{k}$ tokens, $\alpha = 0.25$)**: Prioritizes whole source files (`FULL`), selectively utilizing structural outlines for oversized files ($> 500$ tokens) when capacity is tight. + +### 2. Verified Token Budget Invariants +- **Deterministic Budget Enforcement**: Evaluated across tested nominal budgets (30, 50, 100, 500, 2000, 4000 tokens), CUD-Pack strictly enforces $C_{\text{actual}} \le B_{\text{configured}}$ with zero downstream token overflows according to RepoNeedle's deterministic regex token-counting model (`regex_code_tokenizer_v1`). +- **Overhead Accounting**: Pre-computes file header fences (`# File: {path}\n```{ext}\n...\n```\n\n`) deterministically before admitting candidates into the knapsack. +- **Representation Fallback Hierarchy**: Monotonically degrades candidate representations (`FULL` $\to$ `FOCUS_SPAN` $\to$ `SKELETON`) when full files exceed available token capacity. + +> [!WARNING] +> **Tokenizer Boundary Disclosure**: Budget non-exceedance is enforced strictly according to RepoNeedle's internal regex token model (`regex_code_tokenizer_v1`). This does not imply exact token-count equivalence with arbitrary third-party LLM tokenizers (e.g. OpenAI tiktoken cl100k, LLaMA sentencepiece, or Qwen) or guarantee downstream token behavior for every external provider. + +--- + +## πŸ”¬ Reproducing Benchmark Results + +Run the complete deterministic test suite and benchmarks: + +```bash +# 1. Run automated test suite (271 passed tests locally, 245 in clean CI checkouts) +pytest -q -rs + +# 2. Run component ablation study with 95% Bootstrap CI +python benchmarks/run_ablations.py + +# 3. Verify actual BPE token payload reduction and latency +python scripts/verify_token_savings_and_latency.py + +# 4. Build distribution packages +python -m build --no-isolation +``` + +--- + +## πŸ“¦ Hugging Face Release Artifacts + +RepoNeedle distributes its validated assets primarily via Hugging Face: + +1. **RepoNeedle-Bench Dataset**: [`hf_artifacts/dataset/`](hf_artifacts/dataset/README.md) + - Permissively licensed JSONL datasets (`train.jsonl`, `test.jsonl`), schema, and provenance documentation. +2. **RepoNeedle Playground Space**: [`hf_artifacts/space/`](hf_artifacts/space/README.md) + - Interactive Gradio demonstration allowing side-by-side comparison of RepoNeedle against classical BM25 and RepoMap PageRank. +3. **Automated Publishing Script**: [`scripts/prepare_hf_release.py`](scripts/prepare_hf_release.py) + - Verified local release package ready to publish with `python scripts/prepare_hf_release.py --token `. + +--- + +## πŸ›‘οΈ License & Data Provenance + +- **Source Code**: [MIT License](LICENSE) +- **RepoNeedle-Bench**: [MIT License](hf_artifacts/dataset/README.md) +- **Data Provenance**: Sourced exclusively from permissively licensed open-source projects (BSD-3-Clause, Apache-2.0, MIT). **Zero CC BY-NC or copyleft share-alike data is used.** Full manifest available in [`data_provenance/PROVENANCE.md`](data_provenance/PROVENANCE.md). +- **Upstream Notices**: Agent Retrieval Bench is Copyright (c) 2026 Agent Retrieval Bench contributors (MIT License). All upstream open-source code snapshots retain their original licenses and copyright notices. + +--- + +## πŸ“š Citation + +```bibtex +@misc{reponeedle2026, + author = {RepoNeedle Contributors}, + title = {RepoNeedle: Token-Budget-Aware, Workflow-Adaptive Repository Context Retrieval for AI Coding Agents}, + year = {2026}, + publisher = {GitHub}, + howpublished = {\url{https://github.com/krishika08/RepoNeedle}} +} +``` From bdfc005a71f09ddd3b79a6c9781db642db8e5e3f Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:00:41 +0530 Subject: [PATCH 08/16] Refactor task type inference in change impact ranking --- reponeedle/engine.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/reponeedle/engine.py b/reponeedle/engine.py index 2dc061f..fd490a8 100644 --- a/reponeedle/engine.py +++ b/reponeedle/engine.py @@ -394,7 +394,10 @@ def retrieve( workflow_boosts = np.zeros(num_docs, dtype=np.float32) elif effective_ranker == "change_impact": - inferred_task = query_obj.task_type or TaskType.COMMENT2CONTEXT + if query_obj.task_type == TaskType.AUTO: + inferred_task, _ = self.router.classify_intent(query_obj) + else: + inferred_task = query_obj.task_type ci_cfg = change_impact_config or getattr(self, "change_impact_config", None) or ChangeImpactConfig() ranked_indices, fused_scores, impact_exps = self.change_impact_ranker.rank_with_scores( self.docs, lex_scores, sem_scores, query_obj, config_override=ci_cfg From 5697e55f19d8ff9a10cb857db94cde28cc7d4963 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:01:30 +0530 Subject: [PATCH 09/16] Refactor change_impact ranker task inference Refactor change_impact ranker logic to infer task type based on query object. --- hf_artifacts/space/reponeedle/engine.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/hf_artifacts/space/reponeedle/engine.py b/hf_artifacts/space/reponeedle/engine.py index 2dc061f..3a22412 100644 --- a/hf_artifacts/space/reponeedle/engine.py +++ b/hf_artifacts/space/reponeedle/engine.py @@ -393,8 +393,11 @@ def retrieve( struct_scores = np.zeros(num_docs, dtype=np.float32) workflow_boosts = np.zeros(num_docs, dtype=np.float32) - elif effective_ranker == "change_impact": - inferred_task = query_obj.task_type or TaskType.COMMENT2CONTEXT +elif effective_ranker == "change_impact": + if query_obj.task_type == TaskType.AUTO: + inferred_task, _ = self.router.classify_intent(query_obj) + else: + inferred_task = query_obj.task_type ci_cfg = change_impact_config or getattr(self, "change_impact_config", None) or ChangeImpactConfig() ranked_indices, fused_scores, impact_exps = self.change_impact_ranker.rank_with_scores( self.docs, lex_scores, sem_scores, query_obj, config_override=ci_cfg From ff7f8af599321428f569b6713401e0d89dde8fda Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:04:10 +0530 Subject: [PATCH 10/16] Fix indentation for change_impact ranker condition --- hf_artifacts/space/reponeedle/engine.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/hf_artifacts/space/reponeedle/engine.py b/hf_artifacts/space/reponeedle/engine.py index 3a22412..f5743ee 100644 --- a/hf_artifacts/space/reponeedle/engine.py +++ b/hf_artifacts/space/reponeedle/engine.py @@ -393,11 +393,12 @@ def retrieve( struct_scores = np.zeros(num_docs, dtype=np.float32) workflow_boosts = np.zeros(num_docs, dtype=np.float32) -elif effective_ranker == "change_impact": - if query_obj.task_type == TaskType.AUTO: - inferred_task, _ = self.router.classify_intent(query_obj) - else: - inferred_task = query_obj.task_type + elif effective_ranker == "change_impact": + if query_obj.task_type == TaskType.AUTO: + inferred_task, _ = self.router.classify_intent(query_obj) + else: + inferred_task = query_obj.task_type + ci_cfg = change_impact_config or getattr(self, "change_impact_config", None) or ChangeImpactConfig() ranked_indices, fused_scores, impact_exps = self.change_impact_ranker.rank_with_scores( self.docs, lex_scores, sem_scores, query_obj, config_override=ci_cfg From 9b34b713502758a8c1f2d6bbacc13c05a860e96b Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:05:02 +0530 Subject: [PATCH 11/16] Fix indentation for effective_ranker condition --- hf_artifacts/space/reponeedle/engine.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hf_artifacts/space/reponeedle/engine.py b/hf_artifacts/space/reponeedle/engine.py index f5743ee..590444b 100644 --- a/hf_artifacts/space/reponeedle/engine.py +++ b/hf_artifacts/space/reponeedle/engine.py @@ -393,7 +393,7 @@ def retrieve( struct_scores = np.zeros(num_docs, dtype=np.float32) workflow_boosts = np.zeros(num_docs, dtype=np.float32) - elif effective_ranker == "change_impact": + elif effective_ranker == "change_impact": if query_obj.task_type == TaskType.AUTO: inferred_task, _ = self.router.classify_intent(query_obj) else: From 43ce70330de56b71fa1ff30af164be0027e4afd6 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:09:12 +0530 Subject: [PATCH 12/16] Update engine.py --- hf_artifacts/space/reponeedle/engine.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hf_artifacts/space/reponeedle/engine.py b/hf_artifacts/space/reponeedle/engine.py index 590444b..f5743ee 100644 --- a/hf_artifacts/space/reponeedle/engine.py +++ b/hf_artifacts/space/reponeedle/engine.py @@ -393,7 +393,7 @@ def retrieve( struct_scores = np.zeros(num_docs, dtype=np.float32) workflow_boosts = np.zeros(num_docs, dtype=np.float32) - elif effective_ranker == "change_impact": + elif effective_ranker == "change_impact": if query_obj.task_type == TaskType.AUTO: inferred_task, _ = self.router.classify_intent(query_obj) else: From f01e09a74811b8d8599faf746dd939246e81325d Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:13:21 +0530 Subject: [PATCH 13/16] Fix indentation for inferred_task assignment --- reponeedle/engine.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/reponeedle/engine.py b/reponeedle/engine.py index fd490a8..f5743ee 100644 --- a/reponeedle/engine.py +++ b/reponeedle/engine.py @@ -394,10 +394,11 @@ def retrieve( workflow_boosts = np.zeros(num_docs, dtype=np.float32) elif effective_ranker == "change_impact": - if query_obj.task_type == TaskType.AUTO: - inferred_task, _ = self.router.classify_intent(query_obj) - else: - inferred_task = query_obj.task_type + if query_obj.task_type == TaskType.AUTO: + inferred_task, _ = self.router.classify_intent(query_obj) + else: + inferred_task = query_obj.task_type + ci_cfg = change_impact_config or getattr(self, "change_impact_config", None) or ChangeImpactConfig() ranked_indices, fused_scores, impact_exps = self.change_impact_ranker.rank_with_scores( self.docs, lex_scores, sem_scores, query_obj, config_override=ci_cfg From 06c14ce876beb5e2ad1f36651022e61f7b5e3fc7 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:15:26 +0530 Subject: [PATCH 14/16] Add API dependencies for FastAPI and HTTPX --- pyproject.toml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index 1caa19d..ae9ae9e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -37,6 +37,11 @@ dev = [ "pytest>=7.0.0", "pytest-cov>=4.0.0", ] +api = [ + "fastapi>=0.115,<1.0", + "httpx>=0.24", + "uvicorn[standard]>=0.30,<1.0", +] hf = [ "huggingface_hub>=0.20.0", "gradio>=4.0.0", From 2875fdc15ae37b7d6982aff7310d59c912c8a3f5 Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:16:28 +0530 Subject: [PATCH 15/16] Update CI workflow to include 'api' in dev dependencies --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b6c0c49..4f5fa5e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -42,7 +42,7 @@ jobs: - name: Install dependencies run: | python -m pip install --upgrade pip setuptools wheel build - pip install -e ".[dev]" + pip install -e ".[dev,api]" - name: Run test suite run: | From 1afe14484f89d4b9dbdd7ceefab1d2abda77d72b Mon Sep 17 00:00:00 2001 From: KRISHIKA Date: Tue, 3 Nov 2026 03:24:55 +0530 Subject: [PATCH 16/16] Normalize ranker_note case in assertion --- tests/test_public_api.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_public_api.py b/tests/test_public_api.py index ba1cffc..9560a17 100644 --- a/tests/test_public_api.py +++ b/tests/test_public_api.py @@ -68,7 +68,7 @@ def test_change_impact_auto_infers_task_from_seed(client): payload = response.json() assert payload["ranker"] == "change_impact" assert payload["task_type"] == "edit2ripple" - assert "seed-relative dependency boosts" in payload["ranker_note"] + assert "seed-relative dependency boosts" in payload["ranker_note"].lower() assert payload["tokens_used"] <= payload["token_budget"] selected_paths = [item["path"] for item in payload["selected_files"]] assert selected_paths