🤖 Minecraft AI Bot

An intelligent Minecraft bot powered by fine-tuned Phi-3 LLM with natural language understanding, autonomous task execution, and advanced crafting capabilities. Control your bot through natural conversation in-game or via AI-assisted commands.

✨ Features

🎯 Natural Language Understanding

Conversational AI: Chat naturally with your bot ("how are you?", "thanks!", etc.)
Context Awareness: Bot understands "me", "I", "my" references to identify players
Multi-Step Reasoning: Handles complex requests like "mine logs and give them to me"
Smart Intent Detection: Automatically determines what needs to be done

🛠️ Autonomous Capabilities

Intelligent Mining: Finds and mines specific blocks with pathfinding
Advanced Crafting: Crafts items by auto-gathering missing ingredients for complex recipes.
Combat System: Can attack specified mobs with autonomous weapon selection on command (e.g., "kill that zombie"). Full autonomous combat coming soon.
Automated Farming: Harvests and replants crops autonomously
Autonomous Smelting System: Bot can now smelt items end-to-end using Mineflayer’s Furnace API. Automatically detects smeltable items (iron ore → ingots, sand → glass), gathers fuel, finds or crafts a furnace, places it, smelts asynchronously, and integrates outputs into the crafting chain.

⚡ Advanced Crafting & Smelting Chain

Recursive Crafting + Smelting: Bot automatically gathers raw materials, smelts required resources, and crafts intermediate components before producing the final output.
Smelting Integration: If a recipe requires processed items (e.g., iron_ingot), the bot: (1) mines raw items, (2) finds or crafts a furnace, (3) collects fuel, (4) smelts items, and (5) proceeds with crafting.
Fuel Efficiency Management: Dynamically selects the most efficient available fuel (coal > charcoal > logs > planks) and avoids unnecessary fuel usage.

🚀 AI-Powered Architecture

Fine-tuned Phi-3: Quantized GGUF model optimized for Minecraft tasks
GPU Acceleration: Runs on Google Colab's free T4 GPU
Multi-Step Planning: Breaks complex tasks into executable action sequences
Adaptive Learning: Improves through natural language interactions

💡 Why PhiCraft Is Unique

Combines Mineflayer with a fine-tuned Phi-3 LLM
Understands natural language (not commands only)
Performs multi-step reasoning to break tasks into actions
Supports contextual conversation ("follow me", "give me logs")
Uses GGUF quantized models for high-speed inference

🎥 Demo

Player: how are you?
Bot: I'm doing great! Ready to help you mine, build, or fight!

Player: get me some planks
Bot: ⛓️ Executing 3 steps...
[1/3] ⛏️ Mining 3x logs...
[2/3] 🔨 Crafting 12x planks...
[3/3] 🎁 Giving planks to Player...
Bot: ✅ All steps completed!

🪵 Mining

🏃 Follow & Navigation

⚔️ Targeted Combat (Kill specified mob)

🏗️ Architecture

┌─────────────────┐
│  Minecraft      │
│  Game Server    │
└────────┬────────┘
         │
         │ Mineflayer Protocol
         │
┌────────▼────────┐      HTTP/REST      ┌──────────────┐
│   Node.js Bot   │◄──────────────── ──►│ Express API  │
│   (Actions &    │                     │  (port 3000) │
│    Pathfinding) │                     └──────────────┘
└────────┬────────┘
         │
         │ HTTP POST /parse
         │
┌────────▼────────────────┐
│   ngrok Tunnel          │
│   (Public HTTPS)        │
└────────┬────────────────┘
         │
         │ HTTPS
         │
┌────────▼────────────────┐
│  Google Colab           │
│  ┌──────────────────┐   │
│  │ Phi-3 GGUF Model │   │
│  │ (T4 GPU)         │   │
│  │ Q4_K Quantized   │   │
│  └──────────────────┘   │
│  Flask Server (5000)    │
└─────────────────────────┘

Components

Node.js Bot: Minecraft bot using Mineflayer for game interactions
Express API: REST endpoints for external control and monitoring
Colab AI Server: GPU-accelerated Phi-3 inference via ngrok tunnel
Recipe System: Pre-loaded crafting recipes for autonomous operation

📦 Prerequisites

Required

Node.js 16.x or higher
npm 8.x or higher
Python 3.10 or higher (for local AI, optional)
Minecraft Server (Java Edition 1.20.1 or compatible)
Google Account (for Colab GPU access)
ngrok Account (free tier sufficient)

🚀 Installation

1. Clone Repository

git clone https://github.com/Vedant-Git-dev/PhiCraft.git
cd PhiCraft

2. Install Node.js Dependencies

cd node_bot
npm install mineflayer mineflayer-pathfinder mineflayer-armor-manager mineflayer-collectblock mineflayer-pvp vec3

3. Install Python Dependencies

cd ../python_ai
pip install python-dotenv requests

4. Setup Google Colab AI Server

Create a Google Colab notebook
Get ngrok auth token from ngrok dashboard
Upload your Phi-3 GGUF model to Colab or Google Drive
Run the server.py file from notebooks/server.py
Configure model path and ngrok token
Run the notebook and copy the generated ngrok URL

⚙️ Configuration

Node.js Bot Configuration

Create node_bot/.env:

# Minecraft Server
MC_HOST=localhost
MC_PORT=25565
MC_VERSION=1.20.1
BOT_USERNAME=PhiAssistant # Or anything you like

# Command Settings
COMMAND_PREFIX=!

# AI Server (from Colab)
COLAB_SERVER_URL=https://xxxx-xxx-xxx.ngrok.io

# API Server
NODE_PORT=3000

# Debug (optional)
DEBUG=false

Colab Server Configuration

In your Colab notebook:

# Model Configuration
MODEL_PATH = '/content/phi3_model.gguf'  # Your GGUF model path
NGROK_TOKEN = 'your_ngrok_token_here'    # From ngrok dashboard

# Inference Settings
n_ctx = 4096              # Context window
n_gpu_layers = 35         # GPU layers (-1 for all)
temperature = 0.7         # Sampling temperature
top_p = 0.95             # Nucleus sampling

🎮 Usage

Starting the Bot

Terminal 1: Start Colab AI Server

Open your Colab notebook
Run all cells
Copy the ngrok URL from output
Update COLAB_SERVER_URL in node_bot/.env

Terminal 2: Start Node.js Bot

cd node_bot
npm start

Expected output:

✅ Bot spawned successfully!
⛓️ Crafting chain system initialized
📚 Loaded 423 recipe types from database
🤖 AI Bot online! I can understand natural language!

Terminal 3: Join Minecraft

Start your Minecraft server
Join the server
Bot will auto-join as configured username

In-Game Commands

The bot responds to three command styles:

1. Prefix Commands

!mine 10 diamonds
!craft wooden_pickaxe
!fight zombies
!follow Steve
!status

2. Mentions

@PhiAssistant mine diamonds
@PhiAssistant follow me

3. Natural Language

how are you?
get me some wood
I need a diamond pickaxe
mine logs and give them to me

📖 Commands

Mining

!mine <count> <block>     # Mine specific blocks
mine 10 diamonds          # Natural language
get me some iron ore      # Smart intent

Crafting

!craft <item> [count]     # Craft items (auto-gathers materials)
craft me a pickaxe        # Natural language
I need 10 torches         # Smart reasoning

Combat

!fight [mob] [radius]     # Attack nearby mobs
fight zombies             # Target specific mob
attack all mobs           # Attack all hostiles

Farming

!harvest [crop] [radius]  # Harvest and replant
harvest wheat             # Natural language
farm the crops            # Smart intent

Navigation

!goto <x> <y> <z>        # Navigate to coordinates
!follow [player]         # Follow a player
follow me                # Quick follow
come here                # Natural command

Inventory

!give [item] [count]     # Give items to player
give me diamonds         # Natural language
!status                  # Show bot status

Utility

!help                    # Show command list
!stop                    # Stop current action

Smelting

!smelt <item> [count]        # Smelt any smeltable item
smelt iron ore               # Natural language smelting
make me iron ingots          # Full pipeline: mine → fuel → furnace → smelt

🎯 Advanced Features

Multi-Step Command Execution

The bot can break down complex requests into multiple steps:

Player: I need a stone pickaxe
Bot: ⛓️ Executing 4 steps...
[1/4] ⛏️ Mining oak_log...
[2/4] 🔨 Crafting sticks...
[3/4] ⛏️ Mining cobblestone...
[4/4] 🔨 Crafting stone_pickaxe...
Bot: ✅ All steps completed!

Smart Material Gathering

Bot automatically gathers all required materials:

Player: craft diamond pickaxe
→ Bot checks: need diamonds + sticks
  → Need sticks? Craft from planks
    → Need planks? Craft from logs
      → Need logs? Mine oak trees
→ All materials gathered
→ Craft pickaxe
→ Success!

Context Awareness

Bot understands player references:

Player: give me diamonds
→ Bot: "me" = PlayerName
→ Gives diamonds to PlayerName

Player: follow me
→ Bot: "me" = PlayerName
→ Follows PlayerName

🔧 Troubleshooting

Bot Not Responding

Issue: Bot joins but ignores commands

Solutions:

Check Colab is running: !test
Verify COLAB_SERVER_URL in .env
Test connection: curl https://your-url.ngrok.io/health
Check bot logs for error messages

Crafting Fails

Issue: "No recipe found" or "Cannot craft"

Solutions:

Check recipe database loaded: Look for "Loaded X recipes" in logs
Verify item name: Use exact Minecraft item names
Check inventory space: Bot needs empty slots

Slow Response

Issue: Commands take 10+ seconds

Solutions:

First command is always slow (model warmup) - normal
Check Colab GPU active: Run !nvidia-smi in Colab
Reduce context length in Colab config

Connection Lost

Issue: "Cannot reach Colab server"

Solutions:

Colab notebooks timeout after 30min inactivity
Restart Colab cell to get new ngrok URL
Update .env with new URL
Restart bot: npm start

📊 Performance

Metric	Value
Command Processing	2-5 seconds
Mining Speed	~0.5 blocks/second
Crafting Chain	5-30 seconds (depends on complexity)
AI Inference	2-3 seconds (Colab T4)
Memory Usage	~150MB (Node.js)
GPU Memory	~4-6GB (Colab)

🤝 Contributing

Contributions are welcome! Please follow these steps:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Setup

# Install dependencies
npm install

# Run in development mode
npm run dev

# Run tests
npm test

# Lint code
npm run lint

🙏 Acknowledgments

Mineflayer - Minecraft bot framework
Microsoft Phi-3 - Base LLM model
llama.cpp - GGUF inference engine
PrismarineJS - Minecraft protocol libraries
ngrok - Secure tunneling service
Google Colab - Free GPU compute

📞 Support

Issues: GitHub Issues
Discussions: GitHub Discussions

🗺️ Roadmap

📈 Project Stats

Lines of Code: ~5,000+
Response Time: <5s average
Supported Commands: 50+
Recipe Database: 400+ recipes
Uptime: 99.9% (Colab Pro)

Made with ❤️ by Vedant

If you find this project helpful, please consider giving it a ⭐️!

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
demo		demo
node_bot		node_bot
notebooks		notebooks
python_ai		python_ai
.gitignore		.gitignore
README.md		README.md

Folders and files

Latest commit

History

Repository files navigation

🤖 Minecraft AI Bot

✨ Features

🎯 Natural Language Understanding

🛠️ Autonomous Capabilities

⚡ Advanced Crafting & Smelting Chain

🚀 AI-Powered Architecture

💡 Why PhiCraft Is Unique

📋 Table of Contents

🎥 Demo

🪵 Mining

🏃 Follow & Navigation

⚔️ Targeted Combat (Kill specified mob)

🏗️ Architecture

Components

📦 Prerequisites

Required

Recommended

🚀 Installation

1. Clone Repository

2. Install Node.js Dependencies

3. Install Python Dependencies

4. Setup Google Colab AI Server

⚙️ Configuration

Node.js Bot Configuration

Colab Server Configuration

🎮 Usage

Starting the Bot

Terminal 1: Start Colab AI Server

Terminal 2: Start Node.js Bot

Terminal 3: Join Minecraft

In-Game Commands

1. Prefix Commands

2. Mentions

3. Natural Language

📖 Commands

Mining

Crafting

Combat

Farming

Navigation

Inventory

Utility

Smelting

🎯 Advanced Features

Multi-Step Command Execution

Smart Material Gathering

Context Awareness

🔧 Troubleshooting

Bot Not Responding

Crafting Fails

Slow Response

Connection Lost

📊 Performance

🤝 Contributing

Development Setup

🙏 Acknowledgments

📞 Support

🗺️ Roadmap

📈 Project Stats

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages