🌱 PlantDoctor AI

PlantDoctor AI is an AI-powered plant disease detection and diagnosis system that combines Deep Learning and Generative AI.

The system uses a CNN to identify plant diseases from leaf images and a Retrieval-Augmented Generation (RAG) pipeline to provide information about symptoms, causes, prevention, and management.

The backend is built using FastAPI and exposes REST APIs for plant analysis and RAG-based question answering.

---

📌 Problem Statement

Plant diseases can significantly affect crop yield and quality. Early identification of diseases can help farmers take timely preventive and corrective actions.

PlantDoctor AI aims to automate the initial identification of plant diseases by analyzing a plant leaf image using a Convolutional Neural Network (CNN).

The trained model classifies leaf images into 15 plant disease/health categories.

---

🎯 Objectives

• Integrate the model into a FastAPI backend
• Build a Retrieval-Augmented Generation (RAG) pipeline
• Generate embeddings and store them using FAISS
• Retrieve relevant plant disease information from a knowledge base
• Use Gemini for context-based response generation
• Generate structured responses using Pydantic
• Expose CNN and RAG functionality through FastAPI REST APIs

---

📂 Dataset

The project uses the PlantVillage dataset.

The dataset is organized into class-specific directories, allowing class labels to be inferred directly from folder names.

The model uses 15 classes:

Pepper__bell___Bacterial_spot
Pepper__bell___healthy
Potato___Early_blight
Potato___Late_blight
Potato___healthy
Tomato_Bacterial_spot
Tomato_Early_blight
Tomato_Late_blight
Tomato_Leaf_Mold
Tomato_Septoria_leaf_spot
Tomato_Spider_mites_Two_spotted_spider_mite
Tomato__Target_Spot
Tomato__Tomato_YellowLeaf__Curl_Virus
Tomato__Tomato_mosaic_virus
Tomato_healthy

---

🔄 Machine Learning Pipeline

PlantVillage Dataset
        ↓
Dataset Exploration
        ↓
Image Preprocessing
        ↓
Train / Validation / Test Split
        ↓
Baseline CNN
        ↓
Improved CNN
        ↓
Model Training
        ↓
Model Evaluation
        ↓
Model Export
        ↓
FastAPI Backend
        ↓
CNN + RAG Integration
        ↓
REST APIs

---

🤖 Generative AI & RAG

PlantDoctor AI uses Retrieval-Augmented Generation (RAG) to provide detailed information about detected plant diseases.

The RAG pipeline uses a plant disease knowledge base containing information about symptoms, causes, prevention, and management.

RAG Pipeline

Plant Disease Knowledge Base
        ↓
TextLoader
        ↓
RecursiveCharacterTextSplitter
        ↓
Gemini Embeddings
        ↓
FAISS Vector Store
        ↓
Retriever
        ↓
Relevant Documents
        ↓
Gemini
        ↓
Pydantic Structured Response

---

🧹 Data Preprocessing

The PlantVillage dataset is organized into separate directories for each plant disease and healthy class. The class labels are inferred automatically from the directory structure.

Image Preprocessing

Images are loaded using TensorFlow's image_dataset_from_directory utility.

The following preprocessing configuration is used:

• Image size: 224 × 224
• Color channels: RGB
• Batch size: 32
• Label mode: categorical
• Dataset shuffling: Enabled
• Random seed: 42

Dataset Split

The dataset is divided into training, validation, and testing sets.

Dataset	Percentage
Training	70%
Validation	20%
Testing	10%

A 30% portion of the dataset is first selected as a temporary dataset. This temporary dataset is then divided into validation and test sets, resulting in an approximate final split of 70% / 20% / 10%.

Input Format

Each image is resized to:

224 × 224 × 3

---

🧠 Model Development

Baseline CNN

A baseline Convolutional Neural Network (CNN) was first developed to establish an initial performance benchmark for the plant disease classification task.

The baseline model was trained on the training dataset and evaluated using the validation and test datasets.

---

Improved CNN

An improved CNN architecture was developed to achieve better classification performance.

The final model consists of five convolutional blocks. Each block contains a convolutional layer followed by Batch Normalization and MaxPooling.

Architecture

Input: 224 × 224 × 3
        ↓
Conv2D - 32 filters
        ↓
Batch Normalization
        ↓
MaxPooling2D
        ↓
Conv2D - 64 filters
        ↓
Batch Normalization
        ↓
MaxPooling2D
        ↓
Conv2D - 128 filters
        ↓
Batch Normalization
        ↓
MaxPooling2D
        ↓
Conv2D - 256 filters
        ↓
Batch Normalization
        ↓
MaxPooling2D
        ↓
Conv2D - 512 filters
        ↓
Batch Normalization
        ↓
MaxPooling2D
        ↓
Flatten
        ↓
Dense - 512
        ↓
Dropout
        ↓
Dense - 256
        ↓
Dropout
        ↓
Dense - 15
        ↓
Softmax

---

📊 Model Evaluation

The final improved CNN was evaluated on the test dataset to measure its performance on previously unseen images.

Test Accuracy

The final model achieved 96.17% Test Accuracy on the test dataset.

Test Accuracy: 0.961706280708313

---

💾 Model Export

After training and evaluation, the best-performing CNN model was exported in Keras format for use during inference.

The trained model is stored at:

model/improved_cnn.keras
model/class_names.json

---

🚀 FastAPI Backend

The trained CNN model and RAG pipeline are integrated into a FastAPI backend to provide plant disease prediction and AI-powered disease analysis through REST APIs.

API Endpoints

• POST /predict — CNN disease prediction
• POST /ask — RAG-based question answering
• POST /analyze — Combined CNN + RAG analysis

---

🔌 API Workflow

The PlantDoctor AI analysis workflow combines CNN-based disease prediction with RAG-based information retrieval and Generative AI.

User
  │
  │ Upload Leaf Image
  ▼
FastAPI Server
  │
  │ POST /analyze
  ▼
Image Preprocessing
  │
  │ Resize → 224 × 224
  │ Convert → RGB
  ▼
CNN Model
  │
  ▼
Disease + Confidence
  │
  ▼
RAG Retriever
  │
  ▼
FAISS Vector Store
  │
  ▼
Relevant Context
  │
  ▼
Gemini
  │
  ▼
Pydantic Structured Output
  │
  ├── Symptoms
  ├── Cause
  ├── Prevention
  └── Management
  │
  ▼
JSON Response
  │
  ▼
User

---

🧪 Running Locally

1. Clone the repository and navigate to the project directory.
2. Create and activate a Python virtual environment.
3. Install dependencies from requirements.txt.
4. Configure the required Gemini API key in .env.
5. Run the FastAPI server using Uvicorn.
6. Open http://127.0.0.1:8000/docs.
7. Test the APIs using Swagger UI:
   • POST /predict → CNN disease prediction
   • POST /ask → RAG-based question answering
   • POST /analyze → Complete CNN + RAG analysis

---

📖 API Documentation

FastAPI provides interactive Swagger documentation at:

http://127.0.0.1:8000/docs

GET /

Checks whether the API is running.

POST /predict

Performs CNN-based plant disease prediction from an uploaded leaf image.

POST /ask

Uses the RAG pipeline with FAISS and Gemini to answer plant disease-related questions.

POST /analyze

Combines CNN prediction with RAG to return disease information including symptoms, cause, prevention, and management.

---

📁 Project Structure

PlantDoctor-AI/
│
├── app/
│   └── main.py
│
├── knowledge/
│   └── plant_diseases.txt
│
├── model/
│   ├── improved_cnn.keras
│   └── class_names.json
│
├── notebooks/
│   ├── data-exploration.ipynb
│   ├── baseline-cnn.ipynb
│   └── improved-cnn.ipynb
│
├── rag/
│   ├── generator.py
│   ├── ingest.py
│   └── retriever.py
│
├── frontend/
│   └── index.html
│
├── requirements.txt
├── .gitignore
├── .gitattributes
└── README.md

---

🛠️ Technologies Used

• Python — Core programming language
• TensorFlow & Keras — Deep Learning and CNN model
• NumPy & Pandas — Data processing
• Matplotlib — Data visualization and model evaluation
• LangChain — RAG pipeline and document processing
• Google Gemini — Embeddings and Generative AI
• FAISS — Vector storage and similarity search
• Pydantic — Structured AI output
• FastAPI — REST API backend
• Uvicorn — ASGI server
• Pillow — Image processing
• Git & GitHub — Version control
• Git LFS — Storage of the trained .keras model

---