AgroGuard

Hybrid crop-disease diagnosis and weather-risk alerting for farmers in Maharashtra, India.

AgroGuard lets a farmer photograph a diseased leaf, get an AI diagnosis with a suggested action plan, see a live 15-city disease-risk map driven by real weather, ask an agronomy chatbot for follow-up advice, and — when disease is detected — automatically warn other registered farmers within 15 km.

Overview

AgroGuard is a full-stack web application that combines computer-vision leaf diagnosis with weather-driven risk forecasting and an LLM chatbot to help farmers act early on crop disease.

The backend is a FastAPI service that loads five TensorFlow/Keras image classifiers in-process (rice/general, potato, corn, wheat, and a tomato/pepper "AR" model) plus a scikit-learn weather-risk model. Live weather comes from the OpenWeather API for 15 Maharashtra cities. Diagnoses are persisted to PostgreSQL, and each new detection triggers proximity-based notifications to nearby users. The chatbot is powered by Groq (llama-3.1-8b-instant).

The frontend is a React 19 + Vite single-page app with camera-based image capture, a Leaflet risk map, a detection-history view, and a floating chat widget. It ships with trained model weights ready to serve.

Key Features

Photo-based disease diagnosis — upload/capture a leaf image, choose the crop (rice, potato, corn, wheat), and receive a disease label, confidence score, a Low/Medium/High severity band, a plain-language explanation, and a list of immediate actions.
Per-crop model routing — the detection endpoint dispatches to the correct classifier based on the crop form field.
Live weather-risk model — a scikit-learn classifier scores disease risk from current/forecast weather features (temperature, humidity, wind, pressure, visibility) pulled from OpenWeather.
15-city Maharashtra risk map — a Leaflet map colors each city Low/Medium/ High and can overlay the logged-in user's own farm location.
Hybrid diagnosis — an endpoint that fuses live weather risk with an optional image diagnosis into a single localized report.
Proximity alerts — when a farmer logs a detection, every other registered user within 15 km (great-circle / haversine distance) gets an in-app notification.
Detection history — per-user list of past detections with computed severity and a per-detection delete.
Agronomy chatbot — a Groq-backed assistant scoped to Maharashtra crops (rice/potato/corn/wheat) for follow-up questions.
JWT authentication — signup/login with bcrypt-hashed passwords and bearer tokens.

How It Works

Image → diagnosis → store → notify

flowchart TD
    A[Farmer captures leaf image + selects crop] --> B[POST /detections/ with image, lat, lon, crop]
    B --> C{Validate image type<br/>jpeg / png}
    C -- invalid --> C1[400 error]
    C -- valid --> D{Route by crop}
    D -- potato --> E1[Potato classifier]
    D -- corn --> E2[Corn classifier]
    D -- wheat --> E3[Wheat classifier]
    D -- rice / other --> E4[Rice/general classifier]
    E1 & E2 & E3 & E4 --> F[disease label + confidence]
    F --> G[compute severity<br/>Low / Medium / High]
    G --> H[(Store Detection in PostgreSQL)]
    H --> I[Update user's last-known location]
    I --> J[Query all other users with location]
    J --> K{haversine distance <= 15 km?}
    K -- yes --> L[(Insert Notification)]
    K -- no --> M[skip]
    L --> N[Fetch live weather risk for nearest city]
    M --> N
    N --> O[Return diagnosis + actions + environmental risk]

Weather-risk flow

flowchart LR
    A[lat/lon or city] --> B[Find nearest of 15 Maharashtra cities]
    B --> C[OpenWeather current/forecast API]
    C --> D[Build feature row:<br/>temp, humidity, wind, pressure, visibility,<br/>temp x humidity interaction]
    D --> E[scaler.transform]
    E --> F[sklearn model.predict / predict_proba]
    F --> G[Map class to crop-specific disease name]
    G --> H[Bucket probability into Low / Medium / High risk]

Chatbot flow

flowchart LR
    A[User message] --> B[POST /chatbot/message<br/>JWT required]
    B --> C[Groq chat completion<br/>llama-3.1-8b-instant]
    C --> D[System prompt: AgroGuard agronomy officer<br/>Maharashtra rice/potato/corn/wheat]
    D --> E[Reply + conversation_id]

Architecture

flowchart TB
    subgraph Client["Frontend - React 19 + Vite"]
        UI[Camera capture / History / Chat widget]
        MAP[Leaflet risk map]
    end

    subgraph API["Backend - FastAPI"]
        AUTH[Auth: signup / login / JWT]
        DET[Detection + hybrid routes]
        RISK[Risk-map route]
        CHAT[Chatbot route]
        NOTIF[Notifications route]
        HIST[History routes]
    end

    subgraph Models["In-process models"]
        TF[5 TensorFlow/Keras<br/>image classifiers]
        SK[scikit-learn<br/>weather-risk model + scaler]
    end

    DB[(PostgreSQL)]
    OW[[OpenWeather API]]
    GROQ[[Groq LLM API]]

    UI -->|fetch / FormData| DET
    UI --> AUTH
    UI --> NOTIF
    UI --> HIST
    UI --> CHAT
    MAP -->|GET /risk-map| RISK

    DET --> TF
    DET --> SK
    RISK --> SK
    RISK --> OW
    SK --> OW
    DET --> DB
    AUTH --> DB
    NOTIF --> DB
    HIST --> DB
    CHAT --> GROQ

Tech Stack

Layer	Technology	Why
Backend framework	FastAPI + Uvicorn	Async, typed request models, auto OpenAPI/Swagger docs
Image models	TensorFlow / Keras	Loads the pretrained `.keras` CNN classifiers in-process
Weather-risk model	scikit-learn (+ joblib)	Lightweight tabular classifier over weather features
Numerical / imaging	NumPy, pandas, Pillow	Feature vectors, tabular framing, image decode/resize
Weather data	OpenWeather API (via `requests`)	Live current + 5-day forecast per city
Chatbot	Groq (`llama-3.1-8b-instant`)	Fast hosted LLM for agronomy Q&A
Database / ORM	PostgreSQL + SQLAlchemy	Relational store for users, detections, notifications
Auth	python-jose (JWT) + passlib/bcrypt	Bearer-token auth, hashed passwords
Frontend	React 19 + Vite	Fast SPA dev + build
Map	Leaflet + react-leaflet	City-level risk visualization
Styling / UI	Tailwind CSS v4, framer-motion, lucide-react	Utility styling, animation, icons
Routing	react-router-dom v7	SPA routing + protected routes

Project Structure

agroguard/
├── .gitattributes                 # *.keras tracked via Git LFS
├── backend/
│   ├── requirements.txt           # base Python dependencies (see Setup)
│   ├── model_summary.txt
│   └── app/
│       ├── main.py                # FastAPI app, CORS, auth routes, router wiring
│       ├── config.py              # JWT config from env, bcrypt context
│       ├── deps.py                # get_db dependency
│       ├── api/
│       │   └── chatbot.py         # /chatbot/message route
│       ├── core/
│       │   └── security.py        # get_current_user
│       ├── models/                # SQLAlchemy models: user, detection, notification
│       ├── schemas/               # Pydantic request/response schemas
│       ├── routes/                # detection, risk_map, corn, wheat, ar,
│       │                          #   history, notifications, prediction_history, auth
│       ├── services/
│       │   ├── chatbot_service.py     # Groq chat completion
│       │   └── chatbot_context.py
│       ├── ml/
│       │   ├── image_classifier/  # rice/general model + labels + preprocess
│       │   ├── potato_disease/    # potato model + classes
│       │   ├── corn_disease/      # corn model + classes
│       │   ├── wheat_disease/     # wheat model (lazy-loaded) + classes
│       │   ├── ar_model/          # tomato/pepper model + labels
│       │   └── weather_risk/      # sklearn model + scaler (.pkl) + predictor
│       └── utils/
│           ├── distance.py        # haversine
│           ├── constants.py       # OpenWeather key + city configuration
│           ├── history_logger.py  # JSON prediction log + severity + bias check
│           └── prediction_history.json
└── frontend/
    ├── package.json
    ├── vite.config.js
    ├── index.html
    └── src/
        ├── App.jsx                # routes + global chat widget
        ├── pages/                 # Login, Register, Dashboard, MapView, AnalyzeCrop
        ├── components/            # dashboard/, chatbot/, notificaitons/, layout/
        ├── services/              # diagnosis, chatbot, history, notification API clients
        ├── hooks/                 # useNotifications
        └── api/auth.js            # login / signup / profile

Model weight files (*.keras) and the weather-risk .pkl assets live inside the ml/ subfolders. The .keras files are stored via Git LFS and must be pulled after cloning (see Setup).

Core Implementation

In-process model loading

Each classifier module loads its Keras model from a co-located weight file. Most load eagerly at import time (tf.keras.models.load_model(...)), while the potato and wheat models are lazy-loaded on first request so the server can start before those weights are present:

image_classifier/weights/model.keras (rice/general) — eager
corn_disease/corn_disease_v2_robust.keras — eager
ar_model/model.keras (tomato/pepper) — eager
potato_disease/model.keras — lazy (get_model())
wheat_disease/wheat_model.keras — lazy (get_wheat_model(), with a clear FileNotFoundError if the weight file is not yet present)

The weather-risk model (disease_model_optimized.pkl) and its scaler (weather_scaler_optimized.pkl) are loaded once via joblib.load.

Per-crop routing

POST /detections/ reads the crop form field, normalizes it, and dispatches: potato → predict_potato, corn → predict_corn_disease, wheat → predict_wheat, and everything else falls back to the rice/general classifier. Confidence is clamped to [0, 1] and mapped to severity via compute_severity (>= 0.8 High, >= 0.5 Medium, else Low). Some predictors apply small heuristics — e.g. potato has a "sensitivity boost" toward Late Blight and per-crop low-confidence thresholds fold weak predictions into "Uncertain".

Weather-risk model

weather_risk/predictor.py builds a single feature row (temperature, humidity, wind speed, wind bearing, visibility, pressure, and a temp x humidity interaction term), scales it, and runs model.predict / model.predict_proba. The predicted class is mapped to a crop-specific disease name (e.g. potato → Late/Early Blight, rice → Rice Blast / Sheath Blight) and the max probability is bucketed into Low/Medium/High risk. For coordinates, it finds the nearest of the 15 supported Maharashtra cities, resolves its OpenWeather query name, and reports the nearest-station distance.

15 km proximity notify

After storing a detection, POST /detections/ updates the reporting user's last-known location, then loads every other user that has coordinates and computes the haversine distance. Any user within RADIUS_KM = 15 gets a Notification row containing the disease, crop, rounded distance, and reporter name. (Note: a code comment references "5km" but the effective radius is 15 km.)

Chatbot

chatbot_service.generate_reply calls Groq's chat completions with llama-3.1-8b-instant, a fixed AgroGuard agronomy system prompt scoped to Maharashtra crops, temperature=0.6, and max_tokens=450. The route requires a valid JWT and returns { reply, conversation_id }.

AI / ML Components

Image classifiers (TensorFlow/Keras)

Model	Crop(s)	Classes	Input	Output
Rice / general	Rice (default fallback)	BrownSpot, Healthy, Hispa, LeafBlast	224x224, ResNet50 `preprocess_input`	class + confidence
Potato	Potato	Late Blight, Early Blight, Healthy	300x300, `/255` scaling	disease + confidence
Corn	Corn	corn_healthy, corn_leaf_blight, corn_rust	224x224, ResNet50 `preprocess_input`	class + display_name + confidence
Wheat	Wheat	Black Point, Fusarium Foot Rot, Healthy, Leaf Blight, Wheat Blast	224x224, built-in Rescaling layer	disease + confidence
AR	Tomato + Pepper Bell	15 tomato/pepper classes (bacterial spot, blights, leaf mold, viruses, healthy, ...)	224x224, `/255` scaling	disease + severity + actions + explanation

All image classifiers output softmax-style class probabilities; the app takes the argmax as the label and the max probability as confidence. Low-confidence predictions may be relabeled "Uncertain".

Weather-risk model (scikit-learn)

A tabular classifier (disease_model_optimized.pkl) with a companion scaler, driven by live OpenWeather data. It predicts a disease-risk class per crop and a Low/Medium/High risk band from the class probability.

Chatbot LLM (Groq)

llama-3.1-8b-instant via Groq, dedicated to conversational agronomy advice. The chatbot is a focused assistant scoped to Maharashtra crops.

Pretrained weights (Git LFS)

The .keras model files are tracked with Git LFS (see .gitattributes). Run git lfs pull after cloning to download the weights so the models load (a plain git clone fetches only LFS pointers). The weather-risk .pkl assets are committed directly.

API Endpoints

Base URL in the frontend clients: http://127.0.0.1:8000.

Method	Path	Auth	Purpose
POST	`/signup`	No	Register a user (name, email, password, lat, lon)
POST	`/login`	No	OAuth2 password login → JWT bearer token
GET	`/me`	Yes	Current user profile
GET	`/protected`	Yes	Auth smoke-test endpoint
POST	`/detections/`	Yes	Diagnose image → store + notify nearby users
DELETE	`/detections/{id}`	Yes	Delete one of the user's detections
GET	`/detections/`	Yes	List user's detections (optional `?severity=`)
GET	`/risk-map`	No	15-city risk map (+ optional `lat`/`lon` user point)
POST	`/risk-map/hybrid-diagnosis`	Yes	Fuse live weather risk with optional image diagnosis
GET	`/notifications/`	Yes	List the user's notifications
POST	`/notifications/{id}/read`	Yes	Mark one notification read
POST	`/notifications/read-all`	Yes	Mark all notifications read
POST	`/chatbot/message`	Yes	Ask the Groq agronomy chatbot
POST	`/ml/corn-disease`	No	Corn classifier only (raw)
POST	`/predict/wheat`	No	Wheat classifier only (raw)
POST	`/predict/ar`	No	Tomato/pepper AR classifier (raw)
GET	`/history`	No	Recent ML predictions from the JSON log (`?limit=`)

Interactive docs are available at http://127.0.0.1:8000/docs when the backend is running.

Data Model

erDiagram
    USERS ||--o{ DETECTIONS : makes
    USERS ||--o{ NOTIFICATIONS : receives

    USERS {
        int id PK
        string name
        string email UK
        string hashed_password
        float latitude
        float longitude
    }

    DETECTIONS {
        int id PK
        int user_id FK
        string crop
        string disease
        float confidence
        string severity
        float latitude
        float longitude
        datetime created_at
    }

    NOTIFICATIONS {
        int id PK
        int user_id FK
        string title
        string message
        json data
        bool is_read
        datetime created_at
    }

Tables are auto-created on startup via Base.metadata.create_all(...) — there are no migrations. The notifications.data JSON column carries the disease, crop, distance, and reporting farmer's name.

Setup & Installation

Prerequisites

Python 3.10+
Node.js 18+ (for the Vite frontend)
PostgreSQL (a reachable database)
Git LFS — required to pull the .keras model weights
API keys: Groq (chatbot) and OpenWeather (weather risk — see the note in Environment Variables)

1. Clone and pull LFS weights

git clone <repo-url> agroguard
cd agroguard
git lfs install
git lfs pull        # REQUIRED - downloads the actual *.keras model weights

2. Backend dependencies

cd backend
python -m venv .venv
# Windows:  .venv\Scripts\activate
# macOS/Linux:  source .venv/bin/activate

pip install -r requirements.txt

Install the ML, imaging, and chatbot runtime dependencies:

pip install tensorflow scikit-learn joblib pillow pandas numpy requests groq email-validator

These provide the model runtime (TensorFlow/scikit-learn/joblib), image and tabular handling (Pillow/pandas/NumPy), the OpenWeather client (requests), the Groq chatbot client, and email validation.

3. Frontend dependencies

cd ../frontend
npm install

4. Database

Create a PostgreSQL database and point DATABASE_URL at it (see below). Tables are created automatically on first backend startup — no manual migration step.

5. Environment

Create a .env file in backend/ (loaded via python-dotenv) with the variables in the next section.

Environment Variables

Backend variables (read from backend/.env):

Variable	Required	Purpose
`DATABASE_URL`	Yes	SQLAlchemy PostgreSQL connection string; app raises on startup if unset
`SECRET_KEY`	Yes	JWT signing secret; app raises on startup if unset
`ALGORITHM`	No	JWT algorithm (default `HS256`)
`ACCESS_TOKEN_EXPIRE_MINUTES`	No	Token lifetime in minutes (default `60`)
`GROQ_API_KEY`	Yes (for chatbot)	Groq API key used by the chatbot service

OpenWeather key: the OpenWeather API key used by the weather-risk calls is configured in backend/app/utils/constants.py (as DEFAULT_API_KEY).

The frontend targets the backend API base URL (http://127.0.0.1:8000) set in the service/api modules.

Running the App

Backend

cd backend
uvicorn app.main:app --reload --port 8000

Swagger UI: http://127.0.0.1:8000/docs

Frontend

cd frontend
npm run dev

Vite serves the SPA (default http://localhost:5173), which talks to the backend at http://127.0.0.1:8000. Sign up, log in, then capture a crop image from the dashboard to run a diagnosis.

Security & Reliability

Password hashing — bcrypt via passlib; plaintext passwords are never stored.
JWT auth — bearer tokens signed with SECRET_KEY; protected routes resolve the current user from the token.
Input validation — the detection endpoint rejects non-JPEG/PNG uploads and malformed images before inference.
DB connection health — SQLAlchemy engine uses pool_pre_ping=True.
Graceful weather fallback — weather-risk fetch failures are caught so a diagnosis still returns even if OpenWeather is unreachable.
CORS — configured with an open origin policy (allow_origins=["*"]) for local development.

Name		Name	Last commit message	Last commit date
backend		backend
frontend		frontend
.gitattributes		.gitattributes
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation