Turns unstructured product reviews into the ops, marketing, and product decisions they imply — category-level sentiment, aspect intelligence, per-category topics, and a confidence-gated /analyze API.
NLP capstone · built by Srinidhi Jagannathan & Tuhina Priya
Teams sit on thousands of reviews but act on gut feel. The signal that matters — which aspects drive negative sentiment, which products are at risk, what each category actually talks about — is buried in free text. SmartReview turns that text into a ranked, monitorable decision layer instead of a word cloud.
| User | Job-to-be-done | What SmartReview gives them |
|---|---|---|
| Ops / CX lead | "Where do I invest to move the satisfaction needle?" | Aspect intelligence — service & shipping are the weakest aspects, i.e. the highest-ROI fixes |
| Product manager | "Which products are at risk right now?" | A watch-list ranked by negative-review share (top risk: Wireless Headphones, 25% negative) |
| Marketing | "What should category copy foreground?" | Per-category topics + strongest aspects (battery, design) to mirror customer vocabulary |
| Platform / eng | "Can I trust and route model output?" | /analyze API with calibrated probabilities for confidence-gated routing |
An 8-stage pipeline, framed by the decision each stage supports:
| Stage | Output → decision it enables |
|---|---|
| 1. EDA | Category / rating / sentiment baselines |
| 2. Preprocessing | Negation-aware, feature-rich text for everything downstream |
| 3. Sentiment classification | 0.856 5-fold CV accuracy (0.737 macro-F1) → trustworthy positive-share metric |
| 4. Aspect-based sentiment | Which attributes drive sentiment → where ops should invest |
| 5. Entity / brand extraction | Brand-level sentiment (gated to n ≥ 3) → fair comparisons |
| 6. Topic modeling | Per-category themes → marketing copy that mirrors customer language |
| 7. Similarity & recommendation | In-category "customers also viewed" recs |
| 8. Insights & production | Ranked actions, monitoring plan, /analyze API |
| Decision | What I shipped | Tempting alternative | Why |
|---|---|---|---|
| Sentiment features | Char + word n-grams (0.856 CV) | Word-only | Ablation showed char n-grams drive the gain — they absorb typos/morphology common in reviews |
| ABSA approach | Heuristic lexicon + WordNet + negation windows, labeled "directional" | Supervised ABSA | No annotated aspect ground truth in 200 reviews — honest directional signal beats a fake accuracy number |
| API output | Calibrated probabilities on /analyze |
Raw argmax | Lets downstream route low-confidence reviews to a human |
| Recommendations | In-category by default; diversity behind a flag | Cross-category discovery | Relevance first; diversity is opt-in and explicitly unmeasured |
- Sentiment model: 0.856 CV accuracy / 0.737 macro-F1 (trust the CV interval, not the 40-review hold-out point estimate).
- Recommended actions, ranked by expected impact (full detail in
business_summary.txt):- Proactively review the highest-negative product (Wireless Headphones, 25%).
- Reduce friction on the weakest aspects —
service,shipping. - Expand the aspect lexicon to category-specific terms (current coverage 75.5%).
- A/B the
/analyzeAPI as a "customers also viewed" widget.
- Production monitoring: aspect-coverage baseline 75.5%; alert if the rolling 7-day rate drops >15pp.
- P0 — Collect ~300 human-annotated
(review, aspect, sentiment)triples to convert ABSA from heuristic to measured. - P1 — Replace the Word2Vec bonus (under-fit on 200 reviews) with sentence-transformers or a larger external review corpus.
- P2 — Measure recommendation diversity against real engagement, then decide whether to default it on.
- Only the sentiment classifier has ground-truth labels and measurable accuracy. ABSA and topic outputs are heuristic/unsupervised — directional signal, not measured accuracy; topic names should be reviewed before being shown to stakeholders.
- Small dataset (200 reviews → 40-review hold-out): each misclassification swings hold-out accuracy ~2.5pp.
Python · scikit-learn · NLTK · gensim · Hugging Face Transformers · pandas/NumPy · Matplotlib · FastAPI
git clone https://github.com/sjagannathan17/smartreview-nlp.git
cd smartreview-nlp
pip install -r requirements.txt
jupyter notebook SmartReview_Capstone.ipynb
# API: uvicorn smartreview_api:app --reloadThe small labeled splits ship in dataset/; the full 200K-review dataset-large.csv (scaling appendix only) is excluded for size — available on request.
Joint NLP capstone by Srinidhi Jagannathan and Tuhina Priya.