snake

Snake

First test game. Chosen for simplicity + the properties we need.

---

Why Snake

• Discrete grid: No continuous state to discretize
• Clear events: Move, eat, grow, die — small, unambiguous event vocabulary
• Conditional rules: Self-collision only matters when length > 1. Wall-collision is always death. Simple conditional logic for the model to learn.
• Emergent play styles: Wall-huggers, center-players, food-chasers, cautious coilers — distinct strategies exist even in a simple game
• Single entity focus: One player entity that changes over time (grows). Simpler than multi-entity games but not trivial.
• Variable episode length: Good games last longer. Episode length itself correlates with skill/strategy.

---

Game Specification

Grid

• Size: 10x10 (configurable)
• Walls on all borders
• Interior is open

Entities

• Snake: Head position + body segment positions. Moves one cell per tick in current direction.
• Food: Single position. Respawns at random empty cell when collected.

Actions

{UP, DOWN, LEFT, RIGHT}

No "do nothing" — snake always moves. Cannot reverse direction (instant 180 = death in classic rules, or just disallowed).

Rules

head enters wall cell        → DEATH
head enters own body cell    → DEATH  (only possible when length > 1)
head enters food cell        → GROW (body extends by 1), SCORE +1, food respawns
otherwise                    → MOVE (tail retracts, head advances)

State

snake_head:     (x, y)
snake_body:     [(x, y), ...]    # ordered from head to tail
snake_dir:      {U, D, L, R}
food_pos:       (x, y)
score:          int
alive:          bool

---

Event Vocabulary

Atomic events

INPUT_U, INPUT_D, INPUT_L, INPUT_R     # player action
MOVE                                    # snake moved without collision
EAT                                     # head entered food cell
GROW                                    # body extended (always follows EAT)
FOOD_SPAWN                             # new food appeared
DIE_WALL                               # head hit wall
DIE_SELF                               # head hit own body

With positions

MOVE player (x, y)                     # new head position
EAT food@(x, y)                        # food position collected
FOOD_SPAWN (x, y)                      # new food position
DIE_WALL @(x, y)                       # where the collision happened
DIE_SELF @(x, y)                       # where self-collision happened

Snapshot content

HEAD (x, y)
BODY [(x1,y1), (x2,y2), ...]          # or just length if body tracking is too verbose
DIR {U|D|L|R}
FOOD (x, y)
SCORE n
LEN n

Body is tricky — it can be long. Options:

• Full body positions (verbose, complete)
• Head + direction + length (compact, body is deterministic from move history)
• Head + tail + length (compromise)

Start with head + length + direction. Full body can be reconstructed from move history if needed.

---

Expected Play Styles

Even with simple agents:

• Wall hugger: Follows walls, avoids center. Safe but slow food collection.
• Direct chaser: Beelines toward food. Fast scoring, high death risk at longer lengths.
• Coiler: Stays in a compact area, moves in tight patterns. Very safe but slow.
• Space filler: Systematically covers the grid. Efficient but rigid.

These should emerge as distinct patterns in the event sequences without any labels.

---

Agents for Data Generation

Random agent

Pick a random legal action each tick. Won't produce strategies but will explore the event space (and die a lot).

Greedy agent

Always move toward food (Manhattan distance). Simple heuristic, produces "direct chaser" traces.

Wall-following agent

Prefer moves that keep a wall adjacent. Produces "wall hugger" traces.

Hamiltonian agent

Follow a fixed Hamiltonian cycle through the grid. Never dies (if the cycle exists), produces "space filler" traces. Boring but perfect — useful baseline.

Mixed agent

Randomly switch between strategies every N ticks. Produces stance-shifting data.

---

Open Questions

• Grid size: 10x10 is standard but might produce too-long body sequences. 8x8? 6x6 for initial experiments?
• Food count: Single food is classic. Multiple food items add complexity (which to target?) but also richer strategy.
• Speed: Does the snake speed up as it grows? Classic Snake does. Adds a temporal dynamic.
• Body in tokens: How to represent the growing body efficiently? This is a Snake-specific tokenization challenge.

---

Proof of Concept Results

Snake was the first game tested. The proof-of-concept trained a 31K-parameter transformer on 200 episodes from mixed agents.

Training Configuration

Parameter	Value
Model	2-layer, 32-dim, 4-head transformer
Context window	64 tokens
Vocabulary	74 tokens
Training data	200 episodes (40% Random, 40% Greedy, 20% WallFollower)
Training steps	5000

Results

Metric	Result
Loss	4.47 → 0.25 (random baseline: ln(74) ≈ 4.3)
Physical validity	95% — moves are adjacent cells, positions in bounds
Rule validity	100% — EAT→GROW+FOOD_SPAWN, DIE→EOS
Structural validity	45%*

*Structural validity is low because the model often hits the 64-token context limit mid-sequence without generating EOS. The test expects complete BOS→EOS episodes — the model generates valid gameplay that simply runs longer than the context window allows.

What the Model Learned

Sampled sequences read like real Snake games. The model learned:

• Movement: One cell per tick in the direction of input
• Eating: Food collection triggers growth and food respawn
• Death: Wall collision and self-collision end the episode
• Physics: Positions stay within bounds, no teleportation

Sample Token Sequence

BOS SNAP PLAYER X5 Y5 DIR_R LEN1 FOOD X8 Y6 SCORE V0
  TICK INPUT_L MOVE X4 Y5
  TICK INPUT_D MOVE X4 Y6
  TICK INPUT_D MOVE X4 Y7
  TICK INPUT_R MOVE X5 Y7
  TICK INPUT_R MOVE X6 Y7
  ...