Skip to content

TESTING.md

Latest commit

 

History

History
77 lines (60 loc) · 3.63 KB

TESTING.md

File metadata and controls

77 lines (60 loc) · 3.63 KB
Raw

Testing

Running the suite

pip install -r requirements.txt
pytest tests/ -v

17 tests, no network access or running Ollama instance required — see "Mocking the embedder" below.

What's covered

File Covers
tests/test_parser.py Tree-sitter extraction for Python/JS/Go, the anonymous-node regression, unsupported-language and SQL fallbacks
tests/test_chunker.py Import-block generation, fixed-window fallback, oversized-chunk splitting
tests/test_database.py Repo status lifecycle, chunk insert/fetch by vector index, metadata search, importer lookup, conversation history ordering
tests/test_indexing_and_retrieval.py Full pipeline: index a sample repo → verify SQLite/FAISS linkage, hybrid retrieval, import graph resolution, re-indexing idempotency

Mocking the embedder

tests/conftest.py provides a mock_embedder fixture: a deterministic, hash-based fake embedding model. This lets the test suite exercise every other real component (tree-sitter parsing, chunking, SQLite, FAISS, the retriever's merge/filter logic, graph building, the FastAPI routes) without downloading the actual sentence-transformers model — useful for CI environments without internet access, and for keeping the suite fast (~6 seconds for all 17 tests).

The mock embedder means vector-search relevance itself isn't meaningfully tested (random vectors aren't semantically similar to anything) — this is intentional; retrieval-quality tests that need real semantics (e.g. "does this actually retrieve the login function for a paraphrased question") should be run manually with the real model, see below.

Testing with the real embedding model

python3 -c "
from backend.embeddings.embedder import get_embedder
e = get_embedder()  # downloads bge-small-en-v1.5 on first run (~130MB)
print(e.embed_query('test').shape)
"

Then run the indexing pipeline against a real repo and try a few paraphrased questions through /chat or the Streamlit UI to sanity-check retrieval quality.

Testing with a live Ollama instance

ollama serve &
ollama pull llama3
python run_api.py &
curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"repo_id": "<your-indexed-repo-id>", "question": "Explain this project."}'

Testing the API layer directly

fastapi.testclient.TestClient runs the full FastAPI app in-process (no real network socket), which is how the manual integration verification during development exercised every endpoint (/health, /repos/upload, /repos/{id}/status, /repos/{id}/stats, /search, /repos/{id}/graph/mermaid, /repos/{id}/summary, /repos, and /chat with a real unreachable-Ollama error path to verify graceful degradation). This pattern is easy to extend into a formal tests/test_api.py if you want route-level regression tests in addition to the service-level tests already in the suite.

Adding a new language

  1. Add its extension(s) to EXTENSION_LANGUAGE_MAP in backend/parser/language_config.py.
  2. Add its tree_sitter_languages grammar name to _TS_GRAMMAR_NAME.
  3. Add a LanguageProfile with its function/class/method/import node type names (inspect a grammar's node types with tree_sitter_languages.get_parser("<lang>").parse(b"...").root_node and print .sexp() on a sample file to find the right type names).
  4. Add a test case to tests/test_parser.py mirroring the existing Python/JS/Go examples.