An open-source agentic harness for scientific literature analysis. ScienceAI coordinates frontier models to extract data, replicate findings, and perform systematic reviews with full provenance tracking.
pip install scienceai-llmThree frontier models (GPT-5.2, Claude Opus 4.5, Gemini 3 Pro) independently reproduced the primary findings of a published meta-analysis โ Scolaro et al. (2014), smoking and fracture healing โ using ScienceAI as the harness. Given 19 source papers and no access to the original forest plots or study assignments, all three models recovered the primary finding that smoking approximately doubles fracture nonunion risk (OR 2.15โ2.37 vs. published OR 2.32).
This was presented as a podium talk and poster session at the Johns Hopkins Bloomberg School of Public Health AI Research Day (March 2026), where it received the BSPH AI Research Day Staff Award alongside Jeff Leek's keynote.
๐ Project Poster & Abstract (Google Drive)
| Standard LLM Chatbot | ScienceAI |
|---|---|
| Single conversation context | Multi-agent system with specialized analyst agents |
| Manual upload of each document excerpt | Automatic processing of hundreds of PDFs |
| Limited by context window (~200K tokens) | Processes entire paper collections regardless of size |
| Requires you to extract data manually | Automated data extraction with structured schemas |
| One-off responses | Persistent analysis with downloadable results |
| No systematic validation | Built-in validation and provenance tracking |
| Generic responses | Evidence-based answers with source citations |
ScienceAI employs a Principal Investigator (PI) that:
- Breaks down your research question into manageable sub-tasks
- Creates specialized Analyst Agents for each sub-task
- Coordinates parallel data extraction across your entire paper collection
- Synthesizes findings from multiple analysts
- Provides comprehensive, evidence-backed answers
This means you can ask: "Extract healing times, sample sizes, and intervention types from all these papers" and ScienceAI will automatically create the right analysts, define extraction schemas, process all papers, and return structured CSV dataโsomething impossible with a standard chatbot.
- ๐ Automated Paper Processing: Upload PDFs and let ScienceAI extract text, figures, tables, and metadata automatically
- ๐ค AI-Driven Multi-Agent Analysis: The PI delegates tasks to specialized Analyst Agents that work autonomously
- ๐ Structured Data Extraction: Define data schemas and extract information systematically across all papers
- ๐ฌ Interactive Research Discussion: Ask complex research questions and receive evidence-backed answers
- ๐ Provenance Tracking: Every extracted data point includes source quotes and derivation explanations
- ๐ Export & Visualization: Download extracted data as CSV, export papers with metadata, view analysis results in an interactive interface
- ๐ Dark Mode: Fully supported dark mode for comfortably working in low-light environments, including specialized styling for data viewers.
- ๐พ Project Management: Save and resume research projects with full checkpoint support
Requirements: Python 3.11+ and an API key for at least one supported provider (OpenAI, Anthropic, or Google)
pip install scienceai-llmscienceaiThis starts a local web server. Open your browser to:
http://localhost:4242
You will be prompted to enter an API key for your chosen provider. Enter your project name and click "Start" to create a new project or load an existing one.
Tip: You can switch between OpenAI, Anthropic (Claude), and Google (Gemini) models using the "LLM Provider" card in the main menu. See Configuration for setup details.
Papers Panel (Left Side): This is your literature library showing all uploaded PDFs with:
- Search Bar at the top to filter papers by title, author, or keywords
- Automatically Detected Metadata: Author, Date, Title, Journal
- Paper IDs: Each paper gets a unique identifier
- Analyst Tracking: Shows which analysts have processed each paper
- Add Papers Button: Upload additional PDFs to your project at any time
You can upload PDFs individually or as a zip folder during project creation, or add more later via the "Add Papers" button.
Science Discussion Panel (Right Side): This is where you interact with the Principal Investigator (PI). The PI:
- Understands complex research questions
- Plans multi-step analysis strategies
- Creates and manages Analyst Agents to accomplish your goals
- Presents synthesized findings with evidence
Key Features:
- Message Status: Messages show "Processed" (waiting for your input) or "Pending" (PI is working)
- "Show work..." Links: Click to see detailed tool calls and PI reasoning (see below)
- Timestamps: Track when each interaction occurred
- Brain Indicator ๐ง : A floating emoji shows real-time context (memory) usage. It turns yellow
โ ๏ธ or red ๐ด as the model's memory fills up.
Messages from the PI include a "Show work..." link. This transparency feature lets you see exactly what the PI is doing behind the scenes.
Click "Show work..." to reveal:
- Tool Calls: Every function the PI called (e.g.,
read_paper_chunks,create_analyst,search_database) - Arguments: The exact parameters passed to each tool
- Outputs: Results returned from each operation
- Reasoning: The PI's step-by-step decision-making process
This is invaluable for:
- Understanding how ScienceAI processes your requests
- Debugging unexpected results
- Learning how to phrase better questions
- Trust through complete transparency
Click "Hide work..." to collapse the details again.
Example Questions to Ask:
- "Extract sample sizes, intervention types, and outcomes from all studies"
- "Which papers found significant effects for [specific intervention]?"
- "Create a summary table comparing study methodologies"
- "What are the outcome measures used across these papers?"
If you wish to start fresh while keeping your uploaded papers, use the Reset Conversation button (or the undo arrow icon in the chat interface). This will:
- Clear the chat history
- Reset the Principal Investigator's memory
- Fix any potential database locks
- Keep your uploaded papers and extracted data collections
Analysis Panel (Bottom Section): When you request data extraction or specific analyses, the PI creates specialized Analyst Agents. This panel shows:
- Analyst Categories: Different types of analysts (e.g., "Study Categorization & Eligibility Analyst", "Nonunion and Union Status Analyst")
- Data Collections: Each analyst creates structured data collections with names like "NonunionSmokingData2"
- Load Button: Click to view the extracted data in a table format
- Download Button: Export data as CSV for analysis in Excel, R, or Python
Each analyst autonomously:
- Defines an extraction schema based on your request
- Processes all relevant papers
- Validates extracted data for accuracy
- Provides results with source citations
Data Tables: Click "Load" on any data evidence_files to see the extracted data in a structured table format. Each row represents data from a paper, with columns showing:
- Standard Fields: Data you requested (e.g., smoking status, healing time, sample size)
- Provenance Metadata: Automatically added by ScienceAI
_source_quote: The exact text from the paper supporting this data_derivation: Explanation of how calculated/inferred values were determined_source_location: Where in the paper this data was found
Key Features:
- Sortable Columns: Click headers to sort
- Download CSV: Click the download button to export for further analysis
- Source Verification: Every data point links back to the original paper text
In the Analysis Panel, each data collection offers multiple view formats:
JSON Data Eye Icon (๐๏ธ): Click the eye icon next to "JSON Data" to open an interactive JSON viewer featuring:
- Syntax Highlighting: Easy-to-read colored formatting
- Collapsible Sections: Expand/collapse nested objects and arrays
- Copy Button: Copy the entire JSON to clipboard
- Raw Format: See the exact data structure as stored
CSV Data Eye Icon (๐๏ธ): Click the eye icon next to "CSV Data" to open a spreadsheet-style viewer with:
- Grid Layout: See your data in familiar rows and columns
- Quick Preview: View data without downloading
- Inspect Format: Check CSV structure before exporting
These viewers help you:
- Verify data quality before export
- Debug extraction issues by inspecting raw values
- Choose the best format (JSON vs CSV) for your workflow
- Inspect data structure and field types
Click the Close button or press Esc to dismiss the viewer.
Export Button (๐ฆ): Located in the bottom control panel, this opens the Export Papers menu where you can:
Select Papers to Export:
- All: Export every paper in your project
- User Defined Tag: Filter by custom tags you've applied
Customize Filenames with detected metadata:
- Choose which fields to include: DOI, Date, First Author, Title, Journal, Tags
- Set the order of fields in the filename
- Choose separator (underscore, dash, space)
- Preview:
2023_Smith_ImplantFailureRates_JBJS.pdf
Bottom Control Panel Buttons:
- ๐พ Checkpoints: Download auto-generated checkpoint saves that allow you to resume your project at the last saved state or share it with others
- ๐ฆ Export: Export papers with custom filenames
- ๐ Extracted Data: Combines ALL extracted data into a single CSV file that you can use for analysis and verification of extracted data quality (column names may be very long, so you may want to rename them)
- โ Close: Return to project selection screen
Upload 100+ papers, ask the PI to categorize them by intervention type, extract study characteristics, and generate summary tablesโall automatically.
Request extraction of effect sizes, sample sizes, and study parameters. ScienceAI handles the schema definition, extraction, validation, and CSV export.
Ask "What methodologies are under-represented?" and let analysts scan all papers to identify patterns and gaps.
"Summarize all findings related to [X]" triggers analysts to extract relevant sections, synthesize findings, and provide citations.
ScienceAI can also be used as a Python library to integrate its capabilities into your own scripts and applications.
from scienceai.client import ScienceAI
# Initialize the client (starts backend automatically)
client = ScienceAI(project_name="MyResearchProject")You can upload papers programmatically and trigger preprocessing.
# Upload papers and wait for preprocessing to complete
client.upload_papers(["/path/to/paper1.pdf", "/path/to/paper2.pdf"])
# Or upload without immediate preprocessing
client.upload_papers(["/path/to/paper3.pdf"], trigger_preprocess=False)
# Manually trigger preprocessing later
client.preprocess()Interact with the Principal Investigator to ask questions or request analyses.
# Send a message and wait for the response (blocking)
response = client.chat("Summarize the findings of the uploaded papers.")
print(response)
# Non-blocking chat
client.chat_background("Extract sample sizes from all papers.")
# Poll for status
while True:
result = client.poll()
if result:
print("Response received:", result)
break
print("Working...")
time.sleep(1)
# Get full history
history = client.history()Your main interfaceโa conversational AI that:
- Understands research objectives
- Plans analysis strategies
- Creates and manages Analyst Agents
- Synthesizes multi-agent findings
- Communicates results clearly
Specialized workers created on-demand:
- Each has a focused research goal
- Autonomously defines data schemas
- Extracts, validates, and exports data
- Provides evidence-backed conclusions
- Flexible Schemas: Support for numbers, dates, text blocks, categorical data, and more
- Derivation Support: Extract calculated or inferred values with explanations
- Automatic Provenance: Every data point links to source location and quotes
- Validation: Built-in error checking and re-extraction on failure
- Persistent project storage
- Efficient paper and metadata management
- Data collection tracking
- Checkpoint and export functionality
ScienceAI supports multiple LLM providers with flexible authentication options:
- OpenAI (GPT-4, GPT-5, o4-mini): Default provider
- Anthropic (Claude Sonnet/Opus 4.5): Via direct API or Google Vertex AI
- Google (Gemini 3 Pro): Via API key or Vertex AI service account
OpenAI (Required for Default Setup)
# Method 1: Interactive setup
scienceai --setup-keys
# Method 2: Direct key setting
scienceai --set-key openai YOUR_OPENAI_API_KEY
# Method 3: Environment variable
export OPENAI_API_KEY="sk-..."Anthropic Claude (Optional)
# Direct API (recommended for personal use)
scienceai --set-key anthropic YOUR_ANTHROPIC_API_KEY
# Or via environment variable
export ANTHROPIC_API_KEY="sk-ant-..."Google Gemini (Optional)
# Standard API key (simple setup)
scienceai --set-key google YOUR_GOOGLE_API_KEY
# Or via environment variable
export GOOGLE_API_KEY="..."
# or
export GEMINI_API_KEY="..."For production deployments or enterprise use, you can use a GCP service account for both Gemini and Claude on Vertex AI:
Setup:
scienceai --gcp-service-account /path/to/service-account.jsonThis will:
- Validate your service account file
- Extract the project ID automatically
- Prompt you interactively:
โ Valid service account file for project: my-project-123 This service account can be used for: 1. Google Gemini (native GCP models) 2. Claude on Vertex AI (Anthropic partner models) Use this service account for Claude on Vertex AI? (y/n): - Ask for your preferred Vertex AI region:
Common Vertex AI regions: - us-east5 (US East) - us-central1 (US Central) - europe-west1 (Europe West) Enter Vertex AI region (default: us-east5): - Save the configuration
Remove GCP Configuration:
scienceai --remove-gcp-configThis command allows you to selectively remove Gemini and/or Claude Vertex configurations, reverting to API key authentication.
Priority Order:
- If both GCP service account AND API key are configured for a provider:
- GCP Service Account takes priority (recommended for production)
- API Key is used as fallback
This design allows smooth transitions between development (API key) and production (service account) environments.
Switch between providers via the LLM Provider card in the menu UI. Select:
- OpenAI (GPT models)
- Claude (Anthropic direct API)
- Claude on Vertex (via GCP - if configured)
- Gemini (Google models)
Unavailable providers (missing API keys) are grayed out.
Test all configured API keys:
scienceai --validate-keysOutput:
Validating configured API keys...
โ openai: Valid (gpt-5.2 accessible)
โ anthropic: Valid (claude-sonnet-4-5 accessible)
โ google: Invalid (API key expired)
โ Some keys failed validation
# API Key Management
scienceai --setup-keys # Interactive key setup
scienceai --set-key PROVIDER KEY # Set a specific key
scienceai --validate-keys # Validate all keys
# GCP Service Account
scienceai --gcp-service-account PATH # Configure service account
scienceai --remove-gcp-config # Remove service account config
# Provider Selection
scienceai --provider anthropic # Start with specific provider
# Server Options
scienceai --port 8080 # Custom port (default: 4242)
scienceai --skip-validation # Skip startup key validation
# Logging
scienceai -v # Verbose (INFO level)
scienceai --debug # Debug logging
scienceai --log-level WARNING # Specific log levelAPI keys and GCP configuration are stored in:
~/Documents/ScienceAI/scienceai-keys.json
Example structure:
{
"openai": "sk-...",
"anthropic": "sk-ant-...",
"google": "AIza...",
"google_gcp": {
"service_account_path": "/path/to/sa.json",
"project_id": "my-project-123",
"region": "us-east5"
},
"anthropic_vertex": {
"service_account_path": "/path/to/sa.json",
"project_id": "my-project-123",
"region": "us-east5"
}
}The Principal Investigator (src/scienceai/principal_investigator.py) is the central orchestrator of the system. It uses an LLM-driven reasoning loop to:
- Plan Research: Decomposes user queries into sub-tasks.
- Delegate: Spawns Analyst Agents using
delegate_research()to handle specific data extraction or analysis tasks. - Execute Code: Uses
run_python_code()to perform statistical analysis, generate plots, or manipulate data using Python (pandas, matplotlib, etc.). - Synthesize: Aggregates results from multiple analysts using
reflect_on_delegations()to provide a cohesive answer. - Transparency: All PI actions are recorded and visible via the "Show work..." feature in the UI, exposing tool calls, arguments, and internal reasoning.
Analyst Agents (src/scienceai/analyst.py) are specialized, autonomous workers created by the PI. Each analyst has a specific goal (e.g., "Extract patient demographics") and follows this workflow:
- Paper Selection: Identifies relevant papers using
get_all_papers()or filters by criteria. - Schema Generation: Automatically generates a JSON schema for data extraction based on its goal.
- Concurrent Extraction: Runs
extract_data()across all selected papers in parallel. - Validation: Uses
reflect_on_evidence()to verify that extracted data is supported by the source text. - Data Collection: Saves structured results into a named collection (e.g.,
DemographicsData) which becomes available to the PI and the user.
The Data Extraction Engine (src/scienceai/data_extractor.py) is the core NLP component responsible for turning unstructured PDF text into structured data.
- Supported Types:
number,date,text_block,categorical,boolean,array,object. - Provenance Injection: Automatically adds metadata to every extracted field:
_source_quote: The verbatim text from the paper supporting the data._source_location: Page number and context._derivation: Logic used to calculate values (e.g., "Calculated as 15 males + 12 females").
- Reflection & Validation: The
reflect_on_data_extraction()function acts as a critic, comparing the extracted JSON against the paper's text to catch hallucinations or errors before saving.
Managed by DatabaseManager (src/scienceai/database_manager.py), the system uses a file-based storage approach for portability and simplicity.
- Paper Ingestion: PDFs are hashed (
sha256) to prevent duplicates. Text, tables, and figures are extracted and stored. - Storage Format: Uses
dictdatabaseto store project state, chat history, and data collections as JSON files. - Checkpoints: The system supports full project checkpointing. The
save_database()function creates a zip archive of the project directory, allowing users to backup, share, or resume their work at any time. - Export: Data can be exported as CSVs, and papers can be renamed/exported based on their metadata.
We welcome contributions! Here's how:
- Report Bugs: Open an issue on GitHub with reproduction steps
- Feature Requests: Suggest new capabilities or improvements
- Pull Requests: Fork, develop, and submit PRs for review
See LICENSE file for details.
Papers not processing? Check that PDFs are valid and not password-protected.
API errors? Verify your API key or Service Account is valid and has available credits.
Analyst not completing? Check the chat panel for error messagesโthe PI will explain any issues.
Cannot download data? Ensure analysts have completed their data collections before exporting.
"Context Limit Reached" Warning? This means the conversation has exceeded the LLM's memory. ScienceAI will automatically compress older messages to free up space. You can also use the Reset Conversation feature to clear the history while keeping your uploaded papers.
Ready to transform your literature review workflow? Install ScienceAI and start asking research questions!
pip install scienceai-llm
scienceai







