🎮 Reinforcement Learning Maze Solver

A Reinforcement Learning–based Path Planning Application comparing Q-Learning and Deep Q-Learning (DQN) in randomized grid mazes.

As maze size increases, tabular Q-learning struggles due to exploding state space and slow convergence, while Deep Q-learning scales better, learning faster and finding optimal paths more effectively.

---

🚀 Overview

This project implements a configurable maze environment and two RL agents:

• Q-Learning Agent (Tabular)
• Deep Q-Learning Agent (Neural Network)

The goal is to teach an agent to navigate from the start (0,0) to the goal while avoiding traps.

The maze is randomly generated with:

• Empty cells
• Trap cells (-1)
• A goal cell (+1)

We train both RL methods and compare success rate, learning efficiency, and path optimality.

---

🧱 Maze Environment

Defined in maze.py.

Features

• Random trap placement
• Start at (0,0)
• Goal at bottom-right
• States represented as flattened grid
• Rewards:
  • +1 for reaching goal
  • -1 for trap
  • -0.01 penalty for step

---

🤖 RL Models

1️⃣ Q-Learning (Tabular)

A dictionary-based agent:

• Stores Q(s,a) for each state–action pair
• Good for small state spaces
• Fails when maze grows large
• Uses:
  • ε-greedy exploration
  • Learning rate α
  • Discount γ

Implemented in: models.py → QLearningAgent

---

2️⃣ Deep Q-Learning (DQN)

A PyTorch neural network approximates Q(s,a):

• Input: flattened grid state
• Hidden layers: 256 → 128 → 64 → 32
• Output: Q-values for 4 actions
• Replay memory for stable learning
• Learns faster and generalizes better

Implemented in: models.py → DQLAgent

---

🎨 Visualization

Both Q-learning and DQN support: ✔ Real-time maze visualization with pygame
✔ Animated agent movement
✔ Trap/goal highlighting

You can disable visualization for faster training.

---

📂 File Structure

📁 RL-maze-project/
│── README.md
│── maze.py               # Maze environment
│── models.py             # QLearningAgent + DQLAgent
│── test.py               # Training launcher (Q-learning + DQN)

---

▶️ How to Run

Install dependencies

pip install numpy pygame torch

Run the training script

python test.py

This script automatically:

• Creates maze
• Trains Deep Q-learning
• Trains Q-learning
• Prints success rate + average steps

---

📊 Example Output

Deep Q Learning Results:
Maze: 5  Traps: 2  Episodes: 100
Success Rate: 0.92
Average Steps: 14.7

Q Learning Results:
Maze: 5  Traps: 2  Episodes: 100
Success Rate: 0.34
Average Steps: 52.1

DQN learns faster with smoother convergence.

---

🧠 Why DQN Performs Better

• Tabular Q-table scales as O(n²) for n×n maze
• DQN generalizes across unseen states
• Replay memory stabilizes learning
• Neural network learns patterns (e.g., trap distribution)

As maze grows:

• Q-learning → fails (state explosion)
• DQN → still converges

---

🌟 Future Improvements

• Double DQN
• Dueling DQN
• Prioritized Experience Replay
• Dynamic maze generation per episode
• Add stochastic wind / moving traps

---