Understanding customer sentiment at scale is critical for product teams to prioritize improvements and allocate resources effectively. This project builds an NLP pipeline to automatically classify Microsoft product reviews as positive, neutral, or negative, and identifies the key themes driving sentiment across 8 Microsoft products.
- Source: Simulated Microsoft product review dataset
- Size: 400 reviews across 8 Microsoft products
- Products: Teams, Office 365, Azure, Surface, Windows, Copilot, Xbox, GitHub
- Features: Review text, sentiment label, product, rating, user type
- Classes: Positive, Neutral, Negative
- Generated realistic product review dataset with product-specific sentiment distributions reflecting real world feedback patterns
- Text preprocessing: lowercasing, punctuation removal, stopword removal, lemmatization
- Feature engineering using TF-IDF with unigrams and bigrams
- Trained Logistic Regression classifier with balanced class weights
- Evaluated using precision, recall, and F1 score per sentiment class
- Built product sentiment scorecard to surface actionable business insights
| Metric | Score |
|---|---|
| Overall Accuracy | 72% |
| Negative F1 | 0.59 |
| Neutral F1 | 0.82 |
| Positive F1 | 0.73 |
| Baseline (random) | 33% |
The model performs significantly above the random baseline. Neutral sentiment achieved the highest F1 score of 0.82. Negative sentiment is hardest to classify precisely due to overlap with neutral language.
| Product | Positive % | Negative % | Sentiment Score |
|---|---|---|---|
| GitHub | 58% | 12% | +0.46 |
| Office 365 | 56% | 20% | +0.36 |
| Xbox | 56% | 22% | +0.34 |
| Copilot | 50% | 20% | +0.30 |
| Surface | 50% | 26% | +0.24 |
| Windows | 48% | 28% | +0.20 |
| Microsoft Teams | 48% | 32% | +0.16 |
| Azure | 40% | 36% | +0.04 |
- Azure has the lowest sentiment score driven by pricing transparency and documentation complaints
- Microsoft Teams has the second highest negative rate driven by performance and reliability issues
- GitHub and Office 365 are the strongest performers reflecting strong developer experience and ecosystem integration
- Top negative themes: crashes, pricing confusion, forced updates, slow support response times
- Top positive themes: seamless integration, AI capabilities, time savings, build quality
- Borderline neutral reviews are the hardest to classify, even for humans, highlighting the need for more training data in the neutral class
| Review | Predicted | Confidence |
|---|---|---|
| "Copilot has completely changed how I work, love it" | POSITIVE | 60.53% |
| "Teams keeps freezing and updates keep breaking things" | NEGATIVE | 80.48% |
| "Azure pricing is okay but documentation could be better" | POSITIVE | 38.87% |
The third review highlights a known model limitation: borderline neutral reviews with mixed signals are frequently misclassified. This is a known challenge in three-class sentiment analysis.
- Python 3.9
- pandas, numpy
- nltk (text preprocessing)
- scikit-learn (TF-IDF, Logistic Regression)
- matplotlib, seaborn
- wordcloud
nlp-sentiment-analysis/
├── notebooks/
│ ├── 01_sentiment_analysis.ipynb # Full analysis
│ ├── eda_analysis.png # EDA visualizations
│ ├── wordclouds.png # Word clouds by sentiment
│ ├── confusion_matrix.png # Model evaluation
│ ├── top_words.png # Most predictive words
│ └── product_sentiment.png # Product scorecard
├── .gitignore
├── README.md
└── requirements.txt
# Clone the repo
git clone https://github.com/philipatosam/nlp-sentiment-analysis.git
cd nlp-sentiment-analysis
# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Launch Jupyter
jupyter notebook- Expand dataset with real customer support ticket data
- Implement BERT or RoBERTa for improved accuracy on neutral class
- Build topic modeling pipeline using LDA to extract granular themes
- Deploy as real-time API to auto-classify incoming support tickets
- Build a live monitoring dashboard to track sentiment trends monthly