diff --git a/.claude/skills/check-vacuum/SKILL.md b/.claude/skills/check-vacuum/SKILL.md new file mode 100644 index 0000000..dcb5685 --- /dev/null +++ b/.claude/skills/check-vacuum/SKILL.md @@ -0,0 +1,15 @@ +--- +name: check-vacuum +description: Check the Lab 5 notebook and use focused specialists to review the environment, requirements, and code features. +disable-model-invocation: true +--- + +# Check Vacuum + +1. Work from the CS440-Lab5 repository root. Run `python scripts/check_project.py`. +2. Report errors and unfinished TODOs. A passing check does not prove that Q-learning works. +3. Invoke `environment-specialist` with `context/environment.md` and only the environment cell from `q_learning.ipynb`. Ask it to check movement, rewards, reset, and terminal handling. +4. Invoke `requirements-specialist` with `context/requirements.md` and the check summary. Ask it to identify missing Lab 5 deliverables. +5. Invoke `configuration-reviewer` with `.claude/settings.json`, this skill, and the three specialist definitions. Supply the checker output and only relevant learning-function excerpts if reviewing Q-learning. +6. Combine the specialists' short findings. Use actual subagent calls and report any invocation failure. +7. Return: checks run, hyperparameters, findings, and remaining work. Do not change files unless requested or invent training results. diff --git a/.claude/skills/compare-hyperparameters/SKILL.md b/.claude/skills/compare-hyperparameters/SKILL.md new file mode 100644 index 0000000..7a40fbe --- /dev/null +++ b/.claude/skills/compare-hyperparameters/SKILL.md @@ -0,0 +1,18 @@ +--- +name: compare-hyperparameters +description: Sweep alpha, epsilon, and gamma over several seeds and summarize the trade-offs, to support the hyperparameter explanation the lab asks for. +disable-model-invocation: true +argument-hint: "[--alpha ...] [--epsilon ...] [--gamma ...] [--seeds N]" +--- + +# Compare Hyperparameters + +1. Work from the CS440-Lab5 repository root. Run `python scripts/sweep_hyperparameters.py $ARGUMENTS`. With no arguments it sweeps alpha and epsilon around the notebook's current values over 5 seeds. Training seeds and evaluation seeds are disjoint from the notebook's own seeds. +2. Add `--out sweep_results.json` if the user wants the table saved. Otherwise do not write files. The script never changes the notebook or `results.json`. +3. Summarize the table: success rate (mean and range across seeds), average evaluation reward and steps, and late-training traps for each setting. +4. Explain the trade-offs only as far as the numbers show them: + - alpha: how fast values move versus how noisy they are. + - epsilon: exploration of rarely visited cells versus wasted or risky steps while training. + - gamma: how strongly the +10 goal outweighs the per-step cost. +5. Say when differences fall within the seed-to-seed range and cannot be told apart. Do not call any setting "best" or "optimal" on that basis alone. The sweep covers a small grid and a small number of seeds. +6. Return: the settings tried, the table, what the evidence supports, and a suggested wording for the README's hyperparameter explanation. Do not edit the README or the notebook parameters unless asked, and do not invent results. diff --git a/.claude/skills/verify-results/SKILL.md b/.claude/skills/verify-results/SKILL.md new file mode 100644 index 0000000..27e7b47 --- /dev/null +++ b/.claude/skills/verify-results/SKILL.md @@ -0,0 +1,14 @@ +--- +name: verify-results +description: Sanity-check results.json for consistency and compare the learned policy with a model-based optimum, then review the claims made about it. +disable-model-invocation: true +--- + +# Verify Results + +1. Work from the CS440-Lab5 repository root. If `results.json` is missing, say so and stop; do not create or invent results. Tell the user to run the notebook's training and recording cells. +2. Run `python scripts/verify_results.py`. It checks that outcome counts add up, that reward matches steps, and that evaluation is not better than the 6-step best case. It also compares each learned action with a value-iteration solution. That solution is used only for verification and is never given to the learner. +3. Report any `INCONSISTENT` lines first. Policy differences from optimal are findings, not failures. Note which cells differ and by how much, and that rarely visited cells are expected to be less accurate. +4. Read the Results section of `README.md` and check each number and claim against `results.json` and the script output. Flag any claim the output does not support, such as "optimal", "best", or an explanation of why a cell's action was chosen. +5. Note that one seed shows only that a configuration worked in that run, and that the policy is evaluated from the start cell only. +6. Return: the recorded timestamp, consistency result, cells that differ from optimal, README claims that need correcting, and what remains unverified. Do not edit files unless asked, and do not rerun training. diff --git a/intent/intent.md b/intent/intent.md new file mode 100644 index 0000000..553e007 --- /dev/null +++ b/intent/intent.md @@ -0,0 +1,32 @@ +# Vacuum Robot Q-Learning Project + +## Objective +Implement Q-learning for a vacuum robot in a 4x4 grid environment to navigate from the start position (0,0) to the reward location (3,3). + +## Environment Specifications +- **Grid Size**: 4x4 cells +- **Start Position**: (0,0) +- **Reward Location**: (3,3) - provides positive reward upon reaching +- **Obstacle**: Located at (1,1) - robot cannot move into this cell +- **Trap**: Located at (1,3) - incurs penalty if entered + +## Robot Actions +Available operations: +1. **Left** - Move one cell left +2. **Right** - Move one cell right +3. **Up** - Move one cell up +4. **Down** - Move one cell down +5. **Suck** - Clean the current cell (not used initially as environment is clean) +6. **NoOp** - Stay in place (incurs cost, not beneficial) + +## Constraints & Assumptions +- Environment starts clean, so Suck action is not needed initially +- NoOp action incurs a cost without progressing toward the goal, so it should be avoided +- Robot cannot move out of the environment or into obstacle cells +- Goal: Minimize cumulative cost to reach (3,3) using Q-learning + +## Implementation Requirements +- Implement Q-table initialization and updates +- Handle state transitions respecting obstacles and traps +- Apply appropriate action selection (e.g., epsilon-greedy) +- Track visited states for learning progress \ No newline at end of file