SnakeAgent

A deep reinforcement learning project focused on training and evaluating autonomous agents to play the classic Snake game.

Features

• Implements state-of-the-art reinforcement learning algorithms, specifically Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C).
• Custom Convolutional Neural Network (CNN) feature extractor built with PyTorch to process visual game states.
• Modular evaluation pipeline comparing model performances (Scores, Steps, Rewards) with automated Matplotlib graphing.
• Comprehensive training pipelines including TensorBoard logging, periodic model checkpointing, and evaluation callbacks.
• Built on top of the robust Stable-Baselines3 framework and OpenAI Gym.

Architecture Overview

The system relies on a custom OpenAI Gym environment (SnakeEnv-v0) simulating the Snake game. Visual game states are processed by a custom CNN (CustomCNN), which downsamples spatial features through multiple convolutional layers into a flattened vector. This processed state representation is then fed into Stable-Baselines3 policy networks (DQN or PPO) to predict optimal discrete actions.

Tech Stack

• Reinforcement Learning Framework: Stable-Baselines3
• Deep Learning Framework: PyTorch, Torchvision
• Environment: OpenAI Gym (v0.21.0), Pygame
• Data Processing & Visualization: NumPy, Matplotlib

Setup and Installation

1. Clone the repository:

   git clone [https://github.com/Bhavikupadhyay/SnakeAgent.git](https://github.com/Bhavikupadhyay/SnakeAgent.git)
   cd SnakeAgent

2. Install the required dependencies:

   pip install -e .

   (Note: Code execution environment logs indicate Python 3.7+ is recommended.)

Usage Examples

Training a Model

Training scripts are separated by algorithm. To start training a PPO model, run:

python models/PPO/train.py

Evaluating Trained Models

To test a trained model and observe its behavior:

python models/PPO/test.py

To evaluate all algorithms (A2C, DQN, PPO) side-by-side and generate performance graphs:

python tests/results.py

Project Structure

models/ - Algorithm-specific training and testing scripts (DQN, PPO) alongside model artifacts and logs.

tests/ - Scripts for environment requirement checks, random policy baselines, and cross-algorithm evaluation.

utils/ - Shared architectural utilities including the custom CNN feature extractor, custom evaluation functions, and TensorBoard callbacks.

setup.py - Package definition and dependency management.