Skip to content

perf+fix: optimize indexing/search pipeline and harden interface/runtime#12

Open
DevvGwardo wants to merge 9 commits into
MisterGuy420:mainfrom
DevvGwardo:optimize/full-stack-functions-updates-20260302
Open

perf+fix: optimize indexing/search pipeline and harden interface/runtime#12
DevvGwardo wants to merge 9 commits into
MisterGuy420:mainfrom
DevvGwardo:optimize/full-stack-functions-updates-20260302

Conversation

@DevvGwardo

Copy link
Copy Markdown
Contributor

Summary

This PR delivers a broad optimization and reliability pass across Peppy core functions, interface/runtime wiring, and tests.

Core optimizations

  • Cache freshness validation (peppy/cache.py)
    • Added manifest-based validation (path, size, mtime) to prevent stale cached indices from being reused after source changes.
  • Indexer throughput + search prep (peppy/indexer.py)
    • Added auto worker sizing (max_workers=None => adaptive based on CPU).
    • Added precomputed lookup maps (symbol_lookup, symbol_type_lookup) at index time for faster search queries.
  • Search fast paths + pattern semantics (peppy/searcher.py)
    • Added fast-path symbol lookups using precomputed index maps.
    • Normalized file pattern matching to support both relative-to-root and absolute path matching.

Correctness / robustness fixes

  • Parser decode safety (peppy/parsers.py)
    • Decode source with errors="replace" in tree-sitter extraction path to avoid failures on imperfect encoding input.
  • Regex safety in high-level interface (peppy/interface.py)
    • Escaped user-provided symbol names in find_definition() and find_usages() to prevent regex metacharacter bugs.
  • Server async loop quality (peppy/server.py)
    • Switched to asyncio.get_running_loop() for executor dispatch.
  • Packaging entrypoint fix (pyproject.toml)
    • Console script now points to sync wrapper (peppy.server:run) instead of async function.

Test improvements

  • Unit parser test fix (tests/unit/test_parsers.py)
    • Added missing assert for class name check.
  • Performance test setup correction (tests/performance/test_scalability.py)
    • Ensured indexer/searcher share the same cache instance where intended.

Impact

  • Better correctness after codebase changes (fewer stale-index surprises).
  • Faster repeated symbol queries on medium/large repos.
  • More robust behavior with odd file encodings and regex-sensitive names.
  • Cleaner runtime behavior and packaging ergonomics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant