VitaLITy 2.0 – REST API

Backend API for VitaLITy, built with Flask, Zilliz Cloud (vector DB), and LangChain. It provides paper retrieval (by ID, similarity, abstract), 2D UMAP endpoints, and LLM-powered chat, summarization, and literature review.

---

Requirements

• Python 3.10+
• Zilliz Cloud (required)
• Azure OpenAI, Supabase (full app only)

---

Setup

1. Environment

python3 -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install --upgrade pip

If you only need the public paper REST API (get papers from the vitality dataset):

pip install -e .

If you want the full VitaLITy app (to chat with your papers):

pip install -e ".[full,rerank]"

If you are developing the VitaLITy app:

pip install -e ".[full,rerank,dev]"

2. Environment variables

Create a .env file in the project root:

cp .env.example .env

Then edit .env and fill in your own values:

# Azure OpenAI (LLM)
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/
AZURE_OPENAI_API_VERSION=2025-04-01-preview
AZURE_OPENAI_API_KEY=your-api-key
AZURE_OPENAI_AVAILABLE_MODELS={"gpt-5-mini":"gpt-5-mini","gpt-5.6-luna":"gpt-5.6-luna","gpt-5.6-terra":"gpt-5.6-terra"}
AZURE_OPENAI_DEFAULT_MODEL=gpt-5.6-luna

# Azure OpenAI Embeddings (optional – used when embedding type "ada" is selected)
AZURE_OPENAI_EMBED_DEPLOYMENT=text-embedding-3-small
AZURE_OPENAI_EMBED_API_VERSION=2024-02-01

# Zilliz Cloud (required)
ZILLIZ_URI=https://your-cluster.api.region.zillizcloud.com
ZILLIZ_TOKEN=your-zilliz-api-key

# Supabase (required for authenticated chat persistence)
# Use the project root URL, without /rest/v1. This is a server-only secret.
SUPABASE_URL=https://your-project-ref.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your-service-role-or-secret-key

# Optional
PORT=3000

# Full profile Chat Agent admission (ignored by papers profile)
# AGENT_MAX_CONCURRENT=2
# AGENT_MAX_PENDING=8

3. (Optional) Pre-warm the local cache

python script/export_zilliz_static_data.py

This is optional: if the cache is missing or outdated, the API downloads fresh data from Zilliz on startup. Run this command only to pre-warm the local metadata, UMAP, and fingerprint files.

---

Run

Development:

python main.py

Server runs at http://localhost:3000 (or the port in PORT).

Production:

uvicorn main:app --host 127.0.0.1 --port 8000 --workers 1

---

API overview

Paper retrieval

Endpoint	Method	Description
/getPapers	POST	Get papers by IDs or full payload (with filters)
/getSimilarPapers	POST	Similar papers for one or more seed paper IDs (bulk vector search + RRF)
/getPaperCitations	POST	References and cited-by papers from OpenAlex for one DOI
/getUmapPoints	GET	2D UMAP coordinates for visualization
/getMetaData	GET	Metadata for UI filters

Public MCP

Both the papers and full profiles expose the same public, read-only MCP server at /mcp. It uses Streamable HTTP and calls the same application services as the REST paper endpoints.

Tool	Description
search_papers_bool	Case-insensitive exact phrase or Boolean search, with limit/offset pagination
search_papers_bm25	Paginated lexical BM25 search with metadata filters
search_papers_semantic	Paginated dense semantic search with metadata filters
filter_papers	Paginated metadata-only paper filtering
find_similar_papers	Paginated similar papers from up to 10 seed IDs, with filters
get_paper	Fetch one corpus paper by stable ID
get_paper_citations	OpenAlex references and cited-by papers for a DOI

Production host and browser origins for /mcp, Socket.IO, and related allowlists are hard-coded in config.py (VITALITY_PUBLIC_HOST). Localhost dev origins are included automatically.

LLM

Endpoint	Method	Description
/chat/v2	POST	Chat agent (typed SSE)
/summarize	POST	Summarize selected papers
/literatureReview	POST	Generate a literature review

/getSimilarPapers uses the collection's configured paper embedding and combines the per-seed result lists with reciprocal rank fusion (RRF).

Example requests

Similar papers by paper list:

POST /getSimilarPapers
{
  "seed_ids": ["paper-id-1", "paper-id-2"],
  "limit": 25,
  "min_year": 2020,
  "source": ["CHI"]
}

Chat:

POST /chat/v2
{
  "client_request_id": "…",
  "chat_id": "…",
  "text": "What are the main themes in these papers?"
}

---

Project structure

├── main.py              # ASGI app entry (uvicorn main:app)
├── config.py            # Paths, Zilliz and search settings
├── logger_config.py     # Logging (including optional Google Cloud)
├── prompt.py            # LLM prompts
├── load_to_zilliz.py    # Load JSON into Zilliz collections
├── pyproject.toml       # Dependency source of truth (papers / full / rerank / dev)
├── requirements.txt     # Temporary shim → -e .[full,rerank]
├── supabase/
│   └── migrations/       # Versioned Supabase database schema and RLS policies
├── data/               # Local cache (meta_data.json, umap_data.json; fetched from Zilliz)
├── service/              # Core logic
│   ├── bootstrap.py
│   ├── citations.py
│   ├── embed.py
│   ├── fulltext.py
│   ├── lib.py
│   ├── memory_manager.py
│   ├── metadata_normalizer.py
│   ├── paper_qa.py
│   ├── search.py
│   ├── static_cache.py
│   └── zilliz.py
├── agents/
│   ├── agent_v1_legacy/
│   │   ├── agent_tools.py
│   │   ├── grounded_writer.py
│   │   ├── intent_classifier.py
│   │   ├── query_rewriter.py
│   │   ├── rag_core.py
│   │   ├── runner.py
│   │   ├── session_state.py
│   │   └── summary_routes.py
│   └── agent_v2/
│       ├── logging.py
│       ├── models.py
│       ├── reranker.py
│       ├── router.py
│       ├── runner.py
│       └── search_executor.py
├── model/
│   ├── const.py         # e.g. EMBED (specter, ada)
│   └── paper.py         # Paper request and response schemas

---

Credits

VitaLITy was created by Arpit Narechania, Alireza Karduni, Ryan Wesslen, and Emily Wall.

---

Citation

@article{narechania2021vitality,
  title={vitaLITy: Promoting Serendipitous Discovery of Academic Literature with Transformers \& Visual Analytics},
  author={Narechania, Arpit and Karduni, Alireza and Wesslen, Ryan and Wall, Emily},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2022},
  doi={10.1109/TVCG.2021.3114820},
  publisher={IEEE}
}

---

License

MIT License.

---

Contact

For questions or issues, open a GitHub issue or contact Arpit Narechania.