CAM Parameter Identification using Reinforcement Learning

This project implements a reinforcement learning framework for identifying Computer-Aided Manufacturing (CAM) parameters, focusing on machining operations through integration with hyperMILL.

Project Structure

├── src/
│   ├── environments/        # Custom RL environments implementation
│   ├── hypermill_api/     # hyperMILL API integration
├── experiments/           # Experiment results
├── scripts/              # Training and evaluation scripts
│   ├── train/            # Training scripts
│   ├── experiment_registry.py  # Experiment tracking system
│   ├── list_experiments.py     # Utility to list and filter experiments
│   ├── update_experiment.py    # Utility to update experiment metadata
├── data/                 # hyperMILL Files
└── tests/               # Unit and integration tests

Key Components

Environment (CAM_Parameter_Identification_Env)

State Space

• Tools: Matrix of tool properties (index, type, diameter, cutting length, cutting edges)
• Job ID: Current job identifier
• Job Type: Binary classification (0: roughing, 1: finishing)
• Maximum Cutting Depth: Scalar value [0, 51] mm
• Last Tool ID: Previously used tool identifier

Action Space

Normalized continuous space [-1, 1] for:

1. Tool Number Selection: Mapped to discrete tool indices
2. Feedrate XY: [100, 5000] mm/min
3. Stepfactor (Horizontal Engagement): [0.1, 0.5]
4. Vertical Step Depth: [1, 51.0] mm

Reward Function

Combines normalized machining time and accumulated work:

• Time component: Normalized to [-1, 0]
• Work component: Force-based cost, normalized to [-1, 0]
• Weighted combination based on configurable time_weight parameter

Training Implementation

Algorithm: PPO (Proximal Policy Optimization)

Current Configuration:

• n_steps: 32 (steps per update)
• batch_size: 8 (minibatch size)
• Policy: MultiInputPolicy
• Monitoring: TensorBoard integration

This is just the current configuration but is subject for experimentation.

Experiment Registry System

The project includes a comprehensive experiment tracking system that maintains a central registry of all training runs:

Key Features

• Centralized Tracking: All experiment metadata stored in a single JSON registry
• Automatic Parameter Capture: Records all model hyperparameters, including defaults
• Status Tracking: Monitors experiment completion status (completed, failed, in progress)
• Resource Linking: Maintains links to all experiment artifacts (models, logs, etc.)
• Markdown Summaries: Generates readable documentation of all experiments

Registry Components

• experiment_registry.py: Core registry class for storing and retrieving experiment data
• list_experiments.py: Command-line utility for viewing and filtering experiments
• update_experiment.py: Tool for adding observations and updating experiment status

Technical Features

Force Calculation

• Implements Altintas Force Model
• Considers both cutting and edge coefficients
• Calculates tangential, feed, and radial forces

Job Analysis

• Uses cutting file to get toolpath lengths, machining time, etc
• Calculates accumulated work using the force model

Validation Constraints

• Tool cutting edge length validation
• Radius compatibility checks

Setup

1. Open hyperMILL Python Terminal
2. Create and activate virtual environment:

python -m virtualenv D:\gm_Gruss\cam_rl_parameter_identification\.venv
D:\gm_Gruss\cam_rl_parameter_identification\.venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

Usage

1. Open hyperMILL Python Terminal
2. Run bash D:\gm_Gruss\cam_rl_parameter_identification\.venv\Scripts\activate
3. Inside the Python Panel run the test_env script before starting a training

Training

# Edit experiment details in scripts/train/train.py first:
# - experiment_name
# - experiment_description
# - reward_function_description
# - experiment_changes
# - model hyperparameters

Then run the training sript inside hyperMILL.

This will:

• Create a timestamped experiment directory
• Initialize the CAM environment and agent
• Start training with specified parameters
• Save models and logs in the experiment directory
• Register the experiment in the central registry

Experiment Tracking

Viewing Experiments

# List all experiments in a compact table
python scripts/list_experiments.py --compact

# Filter experiments by keyword
python scripts/list_experiments.py --filter "reward"

# Filter by status
python scripts/list_experiments.py --status completed

# Generate markdown summary
python scripts/list_experiments.py --regenerate

Updating Experiments

After analyzing results, add observations:

# List available experiments
python scripts/update_experiment.py --list-all

# Add observations
python scripts/update_experiment.py "cam_training_20230517_151234" --observations "Achieved 15% reduction in machining time"

# Update environment information
python scripts/update_experiment.py "cam_training_20230517_151234" --environment "time_weight=0.7"

# Mark as completed/failed
python scripts/update_experiment.py "cam_training_20230517_151234" --status completed

Monitoring

Monitor training progress using TensorBoard:

tensorboard --logdir experiments/

The experiment summary is available at experiments/experiment_summary.md.