A RAG Based Intelligent Research Assistant for Academic Document Analysis, Automated Question Answering and Literature Review Using Hybrid Retrieval
An agentic, open source research assistant that lets users to upload their own academic PDF corpus and interact with it through grounded question answering and structured literature review generation - running entirely on free tools with no GPU required.
- Project Overview
- System Architecture
- PDF Ingestion and Classification
- Dual Path Metadata Extraction
- Document Type Acceptance Criteria
- Section Aware Chunking
- Embedding Model
- Dual Indexing - FAISS and BM25
- Hybrid Retrieval with Reciprocal Rank Fusion
- CrossEncoder Reranking
- Agentic Q&A Mode - Mode 1
- Literature Review Mode - Mode 2
- LLM API Used
- Evaluation Metrics
- Model Response Comparison
- Deployment Tools and Stack
- Project Structure
- Installation and Setup
- How to Run
- Environment Variables
- Screenshots
- References
This project builds an end to end Agentic Retrieval Augmented Generation pipeline deployed as a Flask web application. It accepts a user uploaded collection of academic PDFs, classifies and processes them through a dual path extraction pipeline, indexes them using both dense and sparse retrieval methods, and supports two modes of scholarly interaction.
Mode 1 - Agentic Q&A: A LangGraph directed state graph that retrieves, self critiques, and refines answers before generating a response.
Mode 2 - Literature Review: Automatically generates a structured five section academic review of any single uploaded paper with an explicit hallucination safeguard on future directions.
The system uses the Groq API for LLM inference, runs on CPU only hardware, and requires no paid subscriptions beyond the free Groq tier.
All pipeline components - embedding model, FAISS index, BM25 index, chunk corpus, and metadata registry are loaded into process memory at server startup and held for the lifetime of the server, eliminating cold start latency per request.
Every uploaded PDF is passed through detect_pdf_type() which samples up to 6 pages and computes four signals per page:
Noise ratio — proportion of non-standard OCR artefact characters:
Broken word ratio — proportion of tokens of length ≤ 2:
Short line ratio — proportion of lines with character count between 0 and 8:
Full image flag — true if any embedded image covers more than 50% of page area.
Document level classification:
| Type | Rule |
|---|---|
| FULLY SCANNED | scanned pages ≥ 70% |
| HYBRID | scanned > 25% and digital > 25% |
| SCANNED + OCR | OCR pages ≥ 30% |
| DIGITAL BORN | digital pages ≥ 60% |
SHA-256 hashing is applied to every file before processing for content level deduplication.
- Heading extraction via
ocr_extract_headings()using four compiled regex patterns targeting Roman numeral headings, decimal headings, and known all-caps section names - OCR corruption corrections:
H.→II.,n.→II.,m.→III. - Multi line heading merge strategy for split headings
- Table name extraction via
ocr_extract_table_names()with five regex patterns across body text and vision extracted content - Image page detection via
detect_image_pages()— pages rendered to PNG at 2x resolution and submitted tometa-llama/llama-4-scout-17b-16e-instructvia Groq vision API - Title and authors: Groq LLM first, regex fallback via
ocr_extract_title_authors()
- LlamaIndex
SimpleDirectoryReaderfor primary text extraction - LlamaIndex output preferred over PyMuPDF if:
- Heading extraction via Groq LLM - first 8,000 characters submitted with pipe delimited response format
- Same LLM first, regex fallback strategy for title and authors
Both paths share call_llm() backed by llama-3.1-8b-instant on Groq with exponential backoff retry up to 5 attempts.
| Type | Accepted |
|---|---|
| DIGITAL BORN | Accepted |
| SCANNED + OCR | Accepted |
| FULLY SCANNED | Rejected |
| HYBRID | Rejected |
Rejected files are deleted from disk, their hash evicted from seen_hashes, and a rejection record returned to the frontend with a red badge. This gate prevents unextractable or noisy text from propagating into the knowledge base.
build_chunks() segments each paper's text by heading occurrence through a linear scan, creating a sections dictionary mapping each heading to its body text. When no headings are available the full document is treated as a single Full Text section.
Each section is chunked using LlamaIndex SentenceSplitter: chunk_size = 512 tokens chunk_overlap = 64 tokens
The 64 token overlap ensures semantically coherent sentences spanning chunk boundaries are represented in both adjacent chunks.
Word based fallback if LlamaIndex unavailable:
Chunks under 20 tokens are discarded. Vision extracted image page content is appended as separate source_type = "visual" chunks, ensuring scanned tables remain searchable.
Model: all-MiniLM-L6-v2 (SentenceTransformers)
- Produces 384 dimensional dense embedding vectors
- Encodes in batches of 32
- All embeddings L2 normalised before indexing:
Chosen because it is lightweight, runs on CPU only, and produces strong semantic similarity representations without requiring GPU hardware — directly aligned with the project constraint of 8GB RAM.
faiss.IndexFlatIP - exact inner product similarity. Since all vectors are L2 normalised, inner product equals cosine similarity:
Index persisted to chunks_store/faiss_index.bin.
BM25Okapi over lowercased, whitespace tokenised chunk texts:
With
BM25 captures exact lexical matching that dense retrieval misses - critical for humanities texts where theorist names and specialised terminology must be matched precisely.
Both indices are queried with an expanded candidate pool of k' = 4k. The two ranked lists are merged using Reciprocal Rank Fusion:
Where
Across all 10 test queries, RRF fusion consistently outperformed both FAISS alone and BM25 alone.
Model: cross-encoder/ms-marco-MiniLM-L-6-v2
Unlike the bi-encoder used for FAISS, the CrossEncoder performs joint encoding of each query-chunk pair:
Producing a scalar relevance score per candidate. All candidates re-sorted by descending
Reranking is bypassed when |R| ≤ k. No fine tuning required - pre trained weights used directly, fitting within the 8GB RAM constraint.
Post rerank scores were substantially higher and showed greater variation than pre-rerank RRF scores across all test queries, confirming the CrossEncoder's ability to separate strongly relevant chunks from borderline ones.
| Framework | Reason not chosen |
|---|---|
| LangChain standard chains | Linear - no conditional branching or self critique loop |
| LlamaIndex query engine | No native state graph or retry logic |
| AutoGen / CrewAI | Multi agent - overkill for a single retrieval loop |
| Manual while loop | Unstructured, hard to extend or debug |
LangGraph was chosen because it allows the agentic loop to be expressed as a typed directed state graph with conditional edges, making the retrieve → critic → refine → generate cycle explicit, structured, and easy to extend.
QAState TypedDict fields:
question, paper_id, current_query, retrieved_chunks, sufficient, attempt, max_retries, answer, bibliography, sources_used
Retrieve node: calls HybridRetriever.retrieve() with top_k=5
Self critic node: submits first 300 characters of each chunk to llama-3.1-8b-instant with binary YES/NO sufficiency prompt (max_tokens=5)
Refine node: submits original question and insufficiency signal to Groq, returns reformulated query (max_tokens=60)
Generate node: constructs structured context block from first 400 characters per chunk, generates academic answer using only provided sources (max_tokens=800)
Maximum retrieval executions bounded at 3:
Five fixed section queries issued to HybridRetriever.retrieve() with top_k=4, scoped to the target paper:
| Section | Query |
|---|---|
| Background | background context history theory |
| Arguments | main arguments claims theoretical framework |
| Evidence | evidence examples cases data findings |
| Limitations | limitations problems criticism weaknesses |
| Future Directions | future research directions unresolved questions gaps |
Before generating future directions, a verification LLM call is issued requesting only explicitly stated future research statements. If the response equals the sentinel NONE_FOUND, the section is replaced with:
"The paper does not explicitly state future research directions or gaps."
This prevents the model from fabricating research agendas from its own training data - one of the most common failure modes of automated summarisation systems.
A fixed 3 second inter-call delay is enforced between all LLM generation calls to respect Groq API rate limits.
Provider: Groq
Model: llama-3.1-8b-instant
Vision model: meta-llama/llama-4-scout-17b-16e-instruct (image page extraction only)
Groq was chosen over OpenAI and Anthropic APIs because:
- Free tier available with generous token limits
- Extremely fast inference optimised for LLaMA models
- No GPU required locally - all heavy computation offloaded to Groq
llama-3.1-8b-instantis capable enough for academic Q&A at minimal latency
call_llm() implements exponential backoff retry up to 5 attempts with parsed wait times from rate limit error responses.
Measures how grounded the generated answer is in retrieved source chunks:
Score close to 1.0 means every sentence in the answer has a closely matching passage in the retrieved sources.
Measures semantic alignment between question and answer:
Both metrics computed using all-MiniLM-L6-v2 embeddings and sklearn cosine_similarity — fully self contained with no additional API calls.
| Metric | Average Score |
|---|---|
| Faithfulness | 0.6802 |
| Answer Relevancy | 0.7275 |
| Query Latency | 11.25 seconds |
Score bands: High ≥ 0.75 / Moderate 0.60–0.74 / Low < 0.60
| Paper | Question | Claude (Reference Answer) | Final Model Response |
|---|---|---|---|
| Addressing Gender Inequality in Tech Workplaces: Challenges and Opportunities (Kuteesa, Dagunduro, Ajuwon & Chikwe, 2024) | What percentage of computing jobs do women hold according to the 2022 NCWIT report cited in this paper? | Women hold only 26% of computing jobs according to the 2022 National Center for Women & Information Technology (NCWIT) report. | According to the 2022 NCWIT report cited in the paper, women hold 26% of computing jobs. |
| Addressing Gender Inequality in Tech Workplaces: Challenges and Opportunities (Kuteesa et al., 2024) | What percentage of Silicon Valley executive roles do women occupy, as reported by the Kapor Center? | Women comprise just 11% of Silicon Valley's executive roles. | According to the provided sources, women occupy 11% of Silicon Valley's executive roles, as reported by the Kapor Center. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux, Sandoval & Kraft, 2023) | What are the three types of technology the JP RWEE policy brief focuses on for rural women? | (1) Productive technologies related to income-generating activities, (2) Labour and time-saving household technologies, and (3) ICT devices and platforms that provide access to information and facilitate communication. | The JP RWEE policy brief focuses on three types of technology for rural women: 1. Productive technologies related to income-generating activities. 2. Labour and time-saving household technologies. 3. ICT devices and platforms that provide access to information and facilitate communication. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | How much more time do women and girls globally spend on unpaid care and domestic work compared to men? | Women and girls spend on average about three times as many hours on unpaid care and domestic work as men, a figure that has risen since COVID-19. | Unfortunately, the provided sources do not provide a direct answer. However, Source 5 mentions reducing rural women's time poverty through labour-saving technologies, and Source 4 mentions reducing the time burden of collecting water. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | What was the estimated urban–rural gap in Internet access for women in 20 Global South countries, according to a 2018 African After Access survey? | There was an estimated 125% gap in Internet access between women in urban and rural areas across the 20 surveyed Global South countries. | Unfortunately, the provided sources do not contain information about the estimated urban–rural gap in Internet access for women in 20 Global South countries, according to a 2018 African After Access survey. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | How many programme participants in Nepal received access to irrigation through JP RWEE's shallow tube well installations, and how many hectares were irrigated? | 1,310 rural women's households received irrigation access through 131 shallow tube wells, irrigating 393 hectares of land. | Unfortunately, the provided sources do not contain sufficient information to answer the question. We can infer that the JP RWEE programme in Nepal did provide access to irrigation through shallow tube well installations, but the exact numbers are not specified. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | Which factor did male students rate significantly higher than female students in the importance section? | Interest in technology (I8). | Based on the provided sources, male students rated factor P8 (I am interested in technologies and their applications) significantly higher than female students. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | How many undergraduate students participated in the quantitative survey conducted for this study? | 186 undergraduate students in Germany participated. | According to Source 2, the study conducted a quantitative survey of undergraduate students in Germany using an online questionnaire, in which 186 people took part. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | What statistical tests were used to analyse gender differences in the survey data? | Non-parametric tests: primarily the Mann–Whitney U test and the Kruskal–Wallis test, were used due to the lack of normal distribution, supplemented by parametric T-tests for corroboration. | It is not possible to determine the specific statistical tests used to analyze the data. (Evaluation Faithfulness: 0.4753; Answer Relevancy: 0.7921) |
| Paper | Question | Claude (Reference Answer) | Final Model Response |
|---|---|---|---|
| Addressing Gender Inequality in Tech Workplaces: Challenges and Opportunities (Kuteesa, Dagunduro, Ajuwon & Chikwe, 2024) | What percentage of computing jobs do women hold according to the 2022 NCWIT report cited in this paper? | Women hold only 26% of computing jobs according to the 2022 National Center for Women & Information Technology (NCWIT) report. | According to the 2022 NCWIT report cited in the paper, women hold 26% of computing jobs. |
| Addressing Gender Inequality in Tech Workplaces: Challenges and Opportunities (Kuteesa et al., 2024) | What percentage of Silicon Valley executive roles do women occupy, as reported by the Kapor Center? | Women comprise just 11% of Silicon Valley's executive roles. | According to the provided sources, women occupy 11% of Silicon Valley's executive roles, as reported by the Kapor Center. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux, Sandoval & Kraft, 2023) | What are the three types of technology the JP RWEE policy brief focuses on for rural women? | (1) Productive technologies related to income-generating activities, (2) Labour and time-saving household technologies, and (3) ICT devices and platforms that provide access to information and facilitate communication. | The JP RWEE policy brief focuses on three types of technology for rural women: 1. Productive technologies related to income-generating activities. 2. Labour and time-saving household technologies. 3. ICT devices and platforms that provide access to information and facilitate communication. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | How much more time do women and girls globally spend on unpaid care and domestic work compared to men? | Women and girls spend on average about three times as many hours on unpaid care and domestic work as men, a figure that has risen since COVID-19. | Unfortunately, the provided sources do not provide a direct answer. However, Source 5 mentions reducing rural women's time poverty through labour-saving technologies, and Source 4 mentions reducing the time burden of collecting water. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | What was the estimated urban–rural gap in Internet access for women in 20 Global South countries, according to a 2018 African After Access survey? | There was an estimated 125% gap in Internet access between women in urban and rural areas across the 20 surveyed Global South countries. | Unfortunately, the provided sources do not contain information about the estimated urban–rural gap in Internet access for women in 20 Global South countries, according to a 2018 African After Access survey. |
| Technology and Innovation for Rural Women's Economic Empowerment (Mayoux et al., 2023) | How many programme participants in Nepal received access to irrigation through JP RWEE's shallow tube well installations, and how many hectares were irrigated? | 1,310 rural women's households received irrigation access through 131 shallow tube wells, irrigating 393 hectares of land. | Unfortunately, the provided sources do not contain sufficient information to answer the question. We can infer that the JP RWEE programme in Nepal did provide access to irrigation through shallow tube well installations, but the exact numbers are not specified. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | Which factor did male students rate significantly higher than female students in the importance section? | Interest in technology (I8). | Based on the provided sources, male students rated factor P8 (I am interested in technologies and their applications) significantly higher than female students. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | How many undergraduate students participated in the quantitative survey conducted for this study? | 186 undergraduate students in Germany participated. | According to Source 2, the study conducted a quantitative survey of undergraduate students in Germany using an online questionnaire, in which 186 people took part. |
| Why Do Few Women Choose IT-Related Programmes? (Schlegel & Geiger, 2025) | What statistical tests were used to analyse gender differences in the survey data? | Non-parametric tests: primarily the Mann–Whitney U test and the Kruskal–Wallis test, were used due to the lack of normal distribution, supplemented by parametric T-tests for corroboration. | It is not possible to determine the specific statistical tests used to analyze the data. (Evaluation Faithfulness: 0.4753; Answer Relevancy: 0.7921) |
Key findings:
- Final Model correctly retrieved the NCWIT 26% figure and Kapor Center 11% Silicon Valley executive roles figure where Model 1 failed on the latter
- Final Model correctly attributed the Kelly (2008) digital divide framework where Model 1 incorrectly cited Attewell (2001)
- Final Model retrieved Sturman's Foucauldian governmentality framing where Model 1 returned a null response
- Answer relevancy of 0.8380 confirms the system reliably produces responses that address the question asked
| Layer | Tool |
|---|---|
| Backend | Flask (Python) |
| Production server | Gunicorn |
| LLM inference | Groq API (llama-3.1-8b-instant) |
| Embedding | SentenceTransformers all-MiniLM-L6-v2 |
| Dense index | FAISS IndexFlatIP |
| Sparse index | BM25Okapi (rank-bm25) |
| Reranker | CrossEncoder ms-marco-MiniLM-L-6-v2 |
| PDF parsing | PyMuPDF (fitz), LlamaIndex, pdfplumber |
| Vision extraction | Groq vision API (llama-4-scout-17b-16e-instruct) |
| Agentic loop | LangGraph StateGraph |
| Frontend | Vanilla JS, DM typeface, Marked.js v9.1.6 |
| Environment | python-dotenv |
| Hardware | CPU only, 8GB RAM |
git clone https://github.com/Rafa2208/RAG-Research-Assistant.git
cd RAG-Research-Assistant
python -m venv venv
venv\Scripts\activate # Windows
pip install -r requirements.txtCreate a .env file in the root folder:
GROQ_API_KEY=your_groq_api_key_here
python app.pyOpen your browser at http://localhost:5000
Workflow:
- Enter your Groq API key and click Connect
- Upload your academic PDFs
- Select Q&A or Literature Review mode
- Select scope - all papers or single paper
- Ask questions or generate a literature review
| Variable | Description |
|---|---|
GROQ_API_KEY |
Your Groq API key from console.groq.com |
The key can also be entered at runtime through the web interface on Page 1 without needing a .env file.
- Lewis et al. (2020), Retrieval Augmented Generation for Knowledge Intensive NLP Tasks
- Gao et al. (2023), RAG for Large Language Models: A Survey
- Karpukhin et al. (2020), Dense Passage Retrieval
- Es et al. (2023), RAGAS: Automated Evaluation of RAG
- Yao et al. (2022), ReAct: Synergizing Reasoning and Acting
- Asai et al. (2023), Self RAG
- Jiang et al. (2023), FLARE: Active Retrieval Augmented Generation
- Shao et al. (2023), ITER RETGEN
- Singh et al. (2025), Agentic RAG Survey
- Robertson and Zaragoza (2009), BM25 and Beyond
- Yu et al. (2024), RankRAG
- Jeong et al. (2024), Adaptive RAG
- Ali et al. (2024), Automated Literature Review using NLP and RAG
- Agarwal et al. (2024), LitLLM