PricePulse — AI-Powered Price Intelligence

📂 GitHub Repository · 🎥 Demo Video

Turn price tracking into confident buying decisions. PricePulse continuously compares electronics prices across Flipkart and Croma, stores a historical record, detects unusual movements, forecasts likely prices, and presents the answer in a live dashboard: buy now, wait, or set an alert.

PricePulse is an end-to-end price-intelligence platform for consumers who do not want to manually revisit product pages every day. It combines resilient browser-based data collection, a clean API, time-series forecasting, anomaly detection, sentiment analysis, and an interactive Streamlit experience in one practical project.

---

Table of Contents

• Why PricePulse?
• How It Works
• Daily Intelligence Pipeline
• Features
• At-a-Glance Intelligence View
• Technology Stack
• Project Structure
• Quick Start
• API Highlights
• Optional Email Alerts
• Design Principles
• Known Limitations & Design Decisions
• Future Improvements
• License

---

Why PricePulse?

• Compare before you buy — find the cheaper available option between Flipkart and Croma.
• Track price history, not just today's deal — every scrape builds a durable time series in SQLite.
• Use AI to time a purchase — Prophet forecasts, drop probabilities, and buy recommendations support the decision.
• Catch suspicious movements — Isolation Forest flags price anomalies and MRP checks surface questionable discounts.
• Stay informed automatically — watchlists and email digests notify users when their target price is reached.
• Explore live or historical data — search any product on demand, or inspect tracked products through the dashboard and REST API.

How It Works

flowchart LR
    A[Flipkart] --> S[Selenium + BeautifulSoup scrapers]
    B[Croma] --> S
    S --> C[Clean, validate & deduplicate]
    C --> D[(SQLite price history)]
    D --> E[Comparison engine]
    D --> F[Prophet forecasting]
    D --> G[Isolation Forest anomalies]
    D --> H[Watchlist & email alerts]
    E --> I[FastAPI]
    F --> I
    G --> I
    H --> I
    I --> J[Streamlit dashboard]

Loading

Daily Intelligence Pipeline

sequenceDiagram
    participant Scheduler as APScheduler
    participant Stores as Flipkart & Croma
    participant Pipeline as Cleaning pipeline
    participant DB as SQLite
    participant API as FastAPI / Dashboard

    Scheduler->>Stores: Search tracked products daily
    Stores-->>Pipeline: Product names, prices, MRP, ratings & URLs
    Pipeline->>Pipeline: Validate, normalize and remove duplicates
    Pipeline->>DB: Save products and price history
    DB->>API: Latest prices, trends and ML inputs
    API-->>API: Compare, forecast, detect anomalies, check alerts
    API-->>Scheduler: Send target-price email digest when triggered

Loading

Features

Capability	What it delivers
Live two-store search	Searches Flipkart and Croma on demand and returns relevant product results, prices, MRP, discounts, ratings, reviews, and direct product links.
Scheduled tracking	APScheduler runs the tracked-product scraper daily at 9:00 AM IST and persists each observation.
Smart comparison	Finds the lowest observed price per platform and highlights the cheaper store and potential saving.
Price trends	Displays historical price movement, minimum/maximum/average price, and anomaly markers.
AI forecasting	Facebook Prophet predicts future prices with confidence ranges, weekly seasonality, and estimated drop probability.
Buy recommendation	Converts forecast signals into an actionable buy/wait recommendation with a confidence level.
Anomaly and MRP checks	Isolation Forest identifies unusual price changes; MRP validation helps spot inflated reference prices.
Sentiment analysis	VADER scores review sentiment and presents positive, neutral, and negative trends.
Watchlist alerts	Users can set a target price; the scheduler checks it after every scrape and sends a consolidated email alert.
Exportable reports	Download product price-history reports as CSV or PDF.
Interactive dashboard	A Streamlit dashboard brings live search, comparisons, trends, forecasts, anomalies, sentiment, watchlists, and alerts together.

At-a-Glance Intelligence View

pie title PricePulse decision signals
    "Live retailer prices" : 30
    "Historical price trend" : 25
    "Forecast & seasonality" : 20
    "Anomaly / MRP checks" : 15
    "Watchlist alerts" : 10

Loading

The dashboard combines these signals so a price is not viewed in isolation: a product can be cheap today, but its trend, forecast, and retailer comparison determine whether it is actually a good time to buy.

Technology Stack

Layer	Technologies
Data collection	Python, Selenium, Chrome WebDriver, Beautiful Soup
Scheduling	APScheduler with CronTrigger
Data processing	Pandas, NumPy, regular expressions
Database	SQLite, SQLAlchemy
Machine learning	Facebook Prophet, scikit-learn Isolation Forest
Sentiment analysis	VADER Sentiment
Backend API	FastAPI, Pydantic, Uvicorn
Dashboard & charts	Streamlit, Plotly
Reporting & alerts	fpdf2, CSV export, SMTP/Gmail, python-dotenv

Project Structure

PricePulse/
├── scraper/       # Flipkart/Croma Selenium scrapers and daily scheduler
├── processing/    # Data cleaning, comparison, and report generation
├── database/      # SQLAlchemy models, persistence, and watchlist operations
├── ml/            # Forecasting, anomaly detection, recommendations, sentiment
├── api/           # FastAPI app, schemas, and REST routes
├── dashboard/     # Streamlit user interface
├── alerts/        # Email notification service
└── requirements.txt

Quick Start

1. Create an environment and install dependencies

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Initialise the database

python -m database.init_db

3. Start the API

uvicorn api.main:app --reload --port 8000

Open the interactive API documentation at http://localhost:8000/docs.

4. Start the dashboard

In another terminal, with the environment activated:

streamlit run dashboard/app.py

5. Start daily tracking (optional)

python -m scraper.scheduler

The scheduler runs once when it starts, then every day at 9:00 AM IST. Edit PRODUCTS_TO_TRACK in scraper/scheduler.py to choose the products to monitor.

API Highlights

Endpoint	Purpose
GET /api/search/?query=...	Live Flipkart and Croma search with an instant price comparison
GET /api/prices/	Latest tracked products and prices
GET /api/prices/{product_id}	Historical price records for one product
GET /api/prices/{product_id}/mrp-check	Flags whether the listed MRP appears inflated against real observed prices
GET /api/compare/	Cross-platform tracked-product comparison
GET /api/predict/{product_id}	Forecast, confidence bands, and price-drop probability
GET /api/predict/{product_id}/recommendation	Buy-now / wait / neutral recommendation with reasoning
GET /api/predict/{product_id}/seasonality	Weekly seasonality pattern — cheapest and priciest days
GET /api/compare/anomalies	Detected abnormal price movements
GET /api/compare/sentiment	Review sentiment scoring and trend
POST /api/watchlist/	Create a target-price watchlist item
GET /api/watchlist/{email}	View a user's active watchlist
POST /api/watchlist/check-now	Manually trigger threshold checks and digest email
GET /api/reports/{product_id}/pdf	Download a product report as PDF
GET /api/reports/{product_id}/csv	Download raw price history as CSV

Full interactive documentation with request/response schemas is auto-generated at /docs when the API is running.

Optional Email Alerts

To enable Gmail notifications, configure these values in a local .env file:

EMAIL_ADDRESS=you@example.com
EMAIL_PASSWORD=your_gmail_app_password
# Optional: defaults to data/prices.db
DB_PATH=data/prices.db

Use a Gmail App Password, not your normal account password. Generate one under Google Account → Security → 2-Step Verification → App Passwords.

Design Principles

• Modular by design: scraping, processing, storage, ML, API, dashboard, reporting, and alerts are separated into focused layers.
• Extensible sources: every scraper produces the same normalized product structure, making new retailer integrations straightforward.
• Decision-focused UX: the product is designed to answer "where is it cheaper, and should I buy now?" rather than merely displaying raw data.
• API-first architecture: the Streamlit UI consumes FastAPI endpoints, so other clients could use the same intelligence layer.
• Swappable persistence layer: the database is accessed exclusively through SQLAlchemy's ORM, so switching from SQLite to PostgreSQL requires changing a single connection string, not application code.

Known Limitations & Design Decisions

Building against live, unofficial e-commerce sites surfaces real engineering constraints. These are documented deliberately rather than hidden:

• Rate limiting on live search. Flipkart applies bot detection to rapid repeated requests. The scraper detects empty result pages, applies a cooldown, and retries once; the dashboard also enforces a client-side cooldown between searches to avoid triggering this in the first place.
• Category-dependent HTML structure. Flipkart serves different card layouts for different product categories (e.g. headphones vs. phones). The scraper supports multiple known layouts with fallback selectors rather than assuming one fixed structure.
• Platform inventory mismatch. Not every product exists on every platform. A relevance filter checks that scraped results genuinely match the search query before they're shown, rather than displaying loosely related items as if they were direct matches.
• Forecasting needs history. Prophet requires a minimum amount of historical data to produce a meaningful forecast. Products with fewer than seven days of price history fall back to a clearly labeled synthetic demonstration forecast rather than an unreliable real prediction.
• SQLite for development simplicity. SQLite was chosen to keep local setup dependency-free. The SQLAlchemy ORM layer means migrating to PostgreSQL for concurrent/production use is a configuration change, not a rewrite.

Future Improvements

• Containerize the full stack with Docker and Docker Compose for one-command setup.
• Migrate from SQLite to PostgreSQL for concurrent write support at scale.
• Add automated tests (pytest) covering scrapers, cleaning logic, and API routes.
• Introduce proxy rotation for more resilient large-scale scraping.
• Expand retailer coverage beyond Flipkart and Croma.
• Add CI/CD via GitHub Actions for automated testing and deployment on push.

License

This project is licensed under the MIT License.

---

Built with Python to make price tracking more transparent, actionable, and intelligent.