Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ In the relay's terminal, you should see roughly this sequence:

```
[RELAY] listening on socket: "baton.sock"
✓ Successfully created named pipe listener
[OK] Successfully created named pipe listener
[...] TCP - Successfully connected to iMotions server.
[...] TX - packet len=50 text="E;1;PilotDataSync;;;;;AltitudeSync;1250.5;1250.5\r\n"
```
Expand Down
36 changes: 17 additions & 19 deletions inference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,15 @@ No other manual package installation is required.
The scripts are meant to be run in this order. Every command below is written to be run **from
the project root** (`PilotDataSynchronization/`), with the virtual environment activated.

| Step | Script | Purpose | Input | Output |
|---|---|---|---|---|
| 1 | `inference/Data/data_logger.py` | Collects live telemetry from the relay over TCP and appends it to a raw CSV | Telemetry socket stream | `inference/Data/raw_flight_data.csv` |
| 1b | `inference/generate_balanced_data.py` | (Optional, no hardware needed) Generates synthetic, class-balanced flight data for testing the pipeline | none | `inference/Data/synthetic_flight_data.csv` |
| 2 | `inference/label_generator.py` | Applies rule-based thresholds (altitude, speed, vertical speed, roll, g-force, heading change) to assign one of 13 event labels per row | `inference/Data/raw_flight_data.csv` | `inference/Data/labeled_flight_data.csv` |
| 3 | `inference/validate_labels.py` | Sanity-checks the labeled dataset: required columns, valid label set, data ranges, spot-checks, and label distribution | `inference/Data/labeled_flight_data.csv` | Console report only (exit code 0/1) |
| 4 | `inference/prepare_data.py` | (Optional) Cleans, median-fills, standardizes, and splits the labeled data into train/test CSVs — independent of the training step below | `inference/Data/labeled_flight_data.csv` | `inference/dataset/*.csv`, `label_mapping.json`, `scaler_params.json` |
| 5 | `inference/train_model.py` | Trains a Random Forest classifier (80/20 stratified split) and reports accuracy/precision/recall | `inference/Data/labeled_flight_data.csv` | `inference/Models/bestModel.pkl`, `inference/Models/finalModel.pkl`, `inference/dataset/test.csv` |
| 6 | `inference/test_model.py` | Loads a trained model, runs inference, and (if ground-truth labels are present) evaluates it | Trained model + held-out test data (see below) | `inference/predictions_output.csv`, `inference/evaluation_metrics.json` |
| Step | Script | Purpose | Input | Output |
| ---- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| 1 | `inference/Data/data_logger.py` | Collects live telemetry from the relay over TCP and appends it to a raw CSV | Telemetry socket stream | `inference/Data/raw_flight_data.csv` |
| 1b | `inference/generate_balanced_data.py` | (Optional, no hardware needed) Generates synthetic, class-balanced flight data for testing the pipeline | none | `inference/Data/synthetic_flight_data.csv` |
| 2 | `inference/label_generator.py` | Applies rule-based thresholds (altitude, speed, vertical speed, roll, g-force, heading change) to assign one of 13 event labels per row | `inference/Data/raw_flight_data.csv` | `inference/Data/labeled_flight_data.csv` |
| 3 | `inference/validate_labels.py` | Sanity-checks the labeled dataset: required columns, valid label set, data ranges, spot-checks, and label distribution | `inference/Data/labeled_flight_data.csv` | Console report only (exit code 0/1) |
| 4 | `inference/prepare_data.py` | (Optional) Cleans, median-fills, standardizes, and splits the labeled data into train/test CSVs — independent of the training step below | `inference/Data/labeled_flight_data.csv` | `inference/dataset/*.csv`, `label_mapping.json`, `scaler_params.json` |
| 5 | `inference/train_model.py` | Trains a Random Forest classifier (80/20 stratified split) and reports accuracy/precision/recall | `inference/Data/labeled_flight_data.csv` | `inference/Models/bestModel.pkl`, `inference/Models/finalModel.pkl`, `inference/dataset/test.csv` |
| 6 | `inference/test_model.py` | Loads a trained model, runs inference, and (if ground-truth labels are present) evaluates it | Trained model + held-out test data (see below) | `inference/predictions_output.csv`, `inference/evaluation_metrics.json` |

### Step 4 and step 5/6 are independent

Expand Down Expand Up @@ -105,25 +105,30 @@ label_generator.py`); the scripts resolve their own input/output paths relative
The system labels data with 13 event types:

**Flight phases**

- `TAXI` — Ground operations (altitude < 50ft, speed < 30 knots)
- `TAKEOFF` — Transition from ground to air (low altitude, climbing, 50-100 knots)
- `CRUISE` — Steady flight at altitude (altitude > 3000ft, stable vertical speed)
- `APPROACH` — Descending for landing (500-3000ft, descending)
- `LANDING` — Final approach and touchdown (altitude < 500ft, descending)

**Maneuver events**

- `TURN_LEFT` — Left turn (roll < -5° or heading change < -3°/s)
- `TURN_RIGHT` — Right turn (roll > 5° or heading change > 3°/s)

**Speed events**

- `HIGH_SPEED` — Velocity > 200 knots
- `LOW_SPEED` — Velocity < 60 knots (while airborne)

**Altitude events**

- `HIGH_ALTITUDE` — Altitude > 10,000 feet
- `LOW_ALTITUDE` — Altitude < 1,000 feet (while airborne)

**Special events**

- `HIGH_G_FORCE` — G-force > 1.5g
- `NORMAL_FLIGHT` — Default steady flight (none of the above conditions)

Expand All @@ -140,6 +145,7 @@ class distribution, splits into train/test sets, saves the held-out test split t
set, prints accuracy/precision/recall, and saves the trained model.

Model parameters:

- `n_estimators`: 100 trees
- `max_depth`: None (unlimited depth)
- `min_samples_split`: 2
Expand All @@ -153,7 +159,7 @@ Model parameters:
`inference/Models/finalModel.pkl`. It looks for test data in this order:
`inference/dataset/test.csv` (the held-out split `train_model.py` saves — the model never trained
on these rows), then `inference/Data/labeled_flight_data.csv` (fallback; this is the full dataset
the model *did* train on, so metrics from this fallback measure memorization, not generalization),
the model _did_ train on, so metrics from this fallback measure memorization, not generalization),
then `inference/labeled_flight_data.csv`. Run `train_model.py` before `test_model.py` so the
held-out split exists and the first candidate is used.

Expand All @@ -163,14 +169,6 @@ held-out split exists and the first candidate is used.
1b** — `generate_balanced_data.py` writes synthetic data to `inference/Data/synthetic_flight_data.csv`
rather than overwriting the real collected `raw_flight_data.csv`. Copy or rename it to
`raw_flight_data.csv` before running `label_generator.py`.
- **`UnicodeEncodeError: 'charmap' codec can't encode character '✓'` on Windows** — several
scripts print a ✓ character, and the default Windows console codepage (cp1252) can't encode it.
Set `PYTHONUTF8=1` before running (e.g. `set PYTHONUTF8=1` in cmd.exe,
`$env:PYTHONUTF8=1` in PowerShell, `export PYTHONUTF8=1` in bash), or run `chcp 65001` first.
- **`ValueError: Missing required columns` in `label_generator.py`, `validate_labels.py`, or
`prepare_data.py`** — the input CSV is missing one of `altitude`, `velocity`, `vertical_speed`,
`heading`, `roll`, `g_force` (and `pitch`/`yaw`/`event_label` for later steps). Check the header
row of your CSV against `CSV_FIELDS` in `Data/data_logger.py`.
- **`validate_labels.py` reports "Negative velocity values found" or missing label categories** —
this is expected with small or synthetic datasets that don't exercise every flight phase (e.g.
`TAKEOFF`, `APPROACH`, `LANDING`, `LOW_ALTITUDE` require specific altitude/speed/vertical-speed
Expand All @@ -180,7 +178,7 @@ held-out split exists and the first candidate is used.
produce `inference/Models/bestModel.pkl`.
- **`inference/.venv` not picked up / `uv run` uses the wrong Python** — make sure you're running
`uv` commands from inside `inference/` (where `pyproject.toml` lives), or pass `--project
inference` from the repository root.
inference` from the repository root.

## Files

Expand Down
8 changes: 4 additions & 4 deletions inference/label_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ def _validate_labels(self, df: pd.DataFrame) -> None:
if unexpected_labels:
logger.warning(f"Found unexpected labels: {unexpected_labels}")

logger.info("✓ All rows have valid labels")
logger.info("[OK] All rows have valid labels")

def _print_label_statistics(self, df: pd.DataFrame) -> None:
"""
Expand Down Expand Up @@ -304,8 +304,8 @@ def main():
"""Main execution function."""
# Define paths relative to inference folder
inference_dir = Path(__file__).parent
input_file = inference_dir / 'Data' / 'raw_flight_data.csv' # ← input file
output_file = inference_dir / 'Data' / 'labeled_flight_data.csv' # ← output file
input_file = inference_dir / 'Data' / 'raw_flight_data.csv' # input file
output_file = inference_dir / 'Data' / 'labeled_flight_data.csv' # output file

logger.info("="*60)
logger.info("Flight Event Labeling System")
Expand All @@ -329,7 +329,7 @@ def main():
logger.error(f"Labeling failed: {e}")
raise

logger.info("\n✓ Labeling process completed successfully")
logger.info("\n[OK] Labeling process completed successfully")


if __name__ == '__main__':
Expand Down
34 changes: 17 additions & 17 deletions inference/test_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def load_model(self, model_path: Path):
raise FileNotFoundError(f"Model file not found: {model_path}")

self.model = joblib.load(model_path)
logger.info(f"✓ Model loaded successfully: {type(self.model).__name__}")
logger.info(f"[OK] Model loaded successfully: {type(self.model).__name__}")

# Display model parameters
if hasattr(self.model, 'n_estimators'):
Expand All @@ -82,7 +82,7 @@ def load_test_data(self, data_path: Path) -> pd.DataFrame:
raise FileNotFoundError(f"Test data file not found: {data_path}")

df = pd.read_csv(data_path)
logger.info(f"✓ Loaded {len(df)} test samples")
logger.info(f"[OK] Loaded {len(df)} test samples")

# Verify required feature columns exist
missing_cols = [col for col in self.feature_columns if col not in df.columns]
Expand All @@ -92,11 +92,11 @@ def load_test_data(self, data_path: Path) -> pd.DataFrame:
# Check if labels exist (for evaluation)
has_labels = self.target_column in df.columns
if has_labels:
logger.info(f"✓ Test data contains ground truth labels")
logger.info(f"[OK] Test data contains ground truth labels")
logger.info("\nActual label distribution:")
print(df[self.target_column].value_counts().to_string())
else:
logger.warning("⚠ Test data does not contain labels (evaluation will be skipped)")
logger.warning("[WARN] Test data does not contain labels (evaluation will be skipped)")

return df

Expand All @@ -123,7 +123,7 @@ def run_inference(self, df: pd.DataFrame) -> np.ndarray:
logger.info(f"\nGenerating predictions for {len(X_test)} samples...")
predictions = self.model.predict(X_test)

logger.info("✓ Inference completed")
logger.info("[OK] Inference completed")
logger.info(f"\nPredicted label distribution:")
unique, counts = np.unique(predictions, return_counts=True)
for label, count in zip(unique, counts):
Expand Down Expand Up @@ -159,19 +159,19 @@ def evaluate_predictions(self, y_true: pd.Series, y_pred: np.ndarray) -> dict:
}

# Display evaluation metrics
logger.info("\n📊 Evaluation Metrics:")
logger.info("\n[METRICS] Evaluation Metrics:")
logger.info("="*60)
logger.info(f" Accuracy: {accuracy:.4f} ({accuracy*100:.2f}%)")
logger.info(f" Precision: {precision:.4f} ({precision*100:.2f}%)")
logger.info(f" Recall: {recall:.4f} ({recall*100:.2f}%)")
logger.info("="*60)

# Display detailed classification report
logger.info("\n📋 Detailed Classification Report:")
logger.info("\n[REPORT] Detailed Classification Report:")
print("\n" + classification_report(y_true, y_pred, zero_division=0))

# Display confusion matrix
logger.info("🔢 Confusion Matrix:")
logger.info("[MATRIX] Confusion Matrix:")
cm = confusion_matrix(y_true, y_pred)
classes = sorted(y_true.unique())

Expand All @@ -192,7 +192,7 @@ def save_predictions(self, df: pd.DataFrame, predictions: np.ndarray, output_pat
predictions: Predicted labels
output_path: Path to save predictions
"""
logger.info(f"\n💾 Saving predictions to: {output_path}")
logger.info(f"\n[SAVE] Saving predictions to: {output_path}")

# Create output DataFrame with original data and predictions
output_df = df.copy()
Expand All @@ -208,7 +208,7 @@ def save_predictions(self, df: pd.DataFrame, predictions: np.ndarray, output_pat
output_path.parent.mkdir(parents=True, exist_ok=True)
output_df.to_csv(output_path, index=False)

logger.info(f"✓ Saved {len(output_df)} predictions")
logger.info(f"[OK] Saved {len(output_df)} predictions")
logger.info(f" Columns: {', '.join(output_df.columns.tolist())}")

def save_metrics(self, metrics: dict, output_path: Path):
Expand All @@ -221,14 +221,14 @@ def save_metrics(self, metrics: dict, output_path: Path):
"""
import json

logger.info(f"\n💾 Saving metrics to: {output_path}")
logger.info(f"\n[SAVE] Saving metrics to: {output_path}")

output_path.parent.mkdir(parents=True, exist_ok=True)

with output_path.open('w') as f:
json.dump(metrics, f, indent=2)

logger.info("✓ Metrics saved successfully")
logger.info("[OK] Metrics saved successfully")


def main():
Expand Down Expand Up @@ -303,30 +303,30 @@ def main():
# Save metrics
tester.save_metrics(metrics, output_metrics)
else:
logger.info("\n⚠ Skipping evaluation (no ground truth labels available)")
logger.info("\n[WARN] Skipping evaluation (no ground truth labels available)")

# Save predictions
tester.save_predictions(test_data, predictions, output_predictions)

# Final summary
logger.info("\n" + "="*60)
logger.info("✅ Testing Complete!")
logger.info("[OK] Testing Complete!")
logger.info("="*60)
logger.info(f"\nModel used: {model_path}")
logger.info(f"Test data: {data_path_used}")
logger.info(f"Predictions saved: {output_predictions}")

if metrics:
logger.info(f"Metrics saved: {output_metrics}")
logger.info(f"\n📊 Final Results:")
logger.info(f"\n[METRICS] Final Results:")
logger.info(f" Accuracy: {metrics['accuracy']:.4f}")
logger.info(f" Precision: {metrics['precision']:.4f}")
logger.info(f" Recall: {metrics['recall']:.4f}")

logger.info("\n✓ All outputs saved in inference/ folder")
logger.info("\n[OK] All outputs saved in inference/ folder")

except Exception as e:
logger.error(f"\n❌ Testing failed: {e}")
logger.error(f"\n[ERROR] Testing failed: {e}")
raise


Expand Down
6 changes: 3 additions & 3 deletions inference/train_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ def train_model(self, X_train: pd.DataFrame, y_train: pd.Series):
logger.info(f"\nTraining on {len(X_train)} samples...")
self.model.fit(X_train, y_train)

logger.info("✓ Model training completed")
logger.info("[OK] Model training completed")

# Display feature importance
self._display_feature_importance()
Expand Down Expand Up @@ -239,10 +239,10 @@ def save_model(self, models_dir: Path):
logger.info("\nSaving trained models:")

joblib.dump(self.model, best_model_path)
logger.info(f" ✓ Saved: {best_model_path}")
logger.info(f" [OK] Saved: {best_model_path}")

joblib.dump(self.model, final_model_path)
logger.info(f" ✓ Saved: {final_model_path}")
logger.info(f" [OK] Saved: {final_model_path}")


def main():
Expand Down
4 changes: 2 additions & 2 deletions relay/src/state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -379,11 +379,11 @@ impl State {
return Ok(());
}
Ok(l) => {
println!("✓ Successfully created named pipe listener");
println!("[OK] Successfully created named pipe listener");
l
}
Err(e) => {
eprintln!("✗ Failed to create listener: {} (kind: {:?})", e, e.kind());
eprintln!("[ERROR] Failed to create listener: {} (kind: {:?})", e, e.kind());
return Err(anyhow!("Failed to create listener: {}", e));
}
};
Expand Down
2 changes: 1 addition & 1 deletion relay/src/view.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ fn spawn_error_message(state: &State) -> Option<UIElement> {
.error_message
.as_ref()
.map(|err| {
container(text(format!("⚠️ {}", err)))
container(text(format!("[WARN] {}", err)))
.padding(10)
.width(Length::Fill)
.style(container::rounded_box)
Expand Down
Loading