ML Model Explainability & Monitoring Platform

A production-quality, single-file ML model governance and explainability dashboard built with Plotly.js. This portfolio project demonstrates advanced skills in ML model monitoring, interpretability, fairness testing, and business analytics.

📊 Overview

This interactive dashboard provides comprehensive ML model governance tools for data analysts and ML engineers, covering:

• Model Performance Monitoring – Track accuracy, precision, recall, F1 across production deployments
• Explainability (XAI) – SHAP-style beeswarm plots, feature importance, partial dependence plots
• Data Drift Detection – Population Stability Index (PSI), KL divergence, distribution shifts
• Fairness & Bias Analysis – Demographic parity, equal opportunity, disparate impact metrics
• What-If Analysis – Interactive prediction explorer with local explanations
• Model Comparison – Side-by-side version analysis and radar charts

✨ Key Features

1. Model Registry

• Switch between 3 production models (Customer Churn, Loan Default, Demand Forecasting)
• Real-time drift status (green/amber/red indicators)
• Key metrics at a glance: Accuracy, F1, AUC, predictions served

2. Performance Over Time

• 6-month performance trends with Plotly multi-line charts
• Annotated events: model retraining, feature engineering, pipeline issues
• Visual highlighting of degradation zones

3. Feature Importance & Explainability

Global Explanability:

• Permutation Feature Importance – Bar chart showing feature contribution to predictions
• SHAP Beeswarm Plot – Scatter plot per feature showing:
  • X-axis: SHAP value (impact on model output)
  • Y-axis: Feature
  • Color: Feature value (blue=low, red=high)
  • Each dot = one prediction, revealing feature distribution impact

Local Explainability:

• Partial Dependence Plots (PDP) – How model output changes as one feature varies
• Individual Conditional Expectation (ICE) – Gray lines showing individual prediction trajectories
• Waterfall Charts – Feature-by-feature contribution breakdown for specific predictions

4. Drift Detection

PSI (Population Stability Index):

• Quantifies distribution shift between training and production data
• Thresholds: Green (<0.1), Amber (0.1-0.25), Red (>0.25)
• Time-series heatmap: PSI values across features and time periods

Distribution Comparison:

• Overlaid KDE histograms: Reference (training) vs. Current (production)
• KL Divergence implementation in JavaScript
• Interactive feature selector

5. Prediction Distribution Monitor

• Histogram: Baseline vs. current prediction probability distributions
• Calibration Curve – Predicted probability vs. actual outcome rate
• Identifies model miscalibration in production

6. Evaluation Metrics

• Confusion Matrix – Interactive heatmap (TN, FP, FN, TP)
• ROC Curve – AUC metric, visual threshold sensitivity
• Precision-Recall Curve – Trade-off analysis
• Threshold Slider – Real-time metric updates (sensitivity, specificity, F1)

7. Fairness & Bias Analysis

Metrics across demographic groups (Age, Gender, Region):

• Demographic Parity – Positive prediction rate by group
• Equal Opportunity – True Positive Rate (TPR) by group
• Disparate Impact Ratio – Legal threshold monitoring (0.8-1.25)
• Summary table with fairness metrics

8. What-If Analysis / Prediction Explorer

Interactive form with sliders for all input features:

• Real-time Prediction Output – Current model score + confidence
• Sensitivity Analysis – How each feature change affects prediction
• Waterfall Chart – SHAP-style local explanations
• "Find Flip Point" – Suggests feature changes to alter prediction

9. Model Comparison

• Side-by-side metrics: Accuracy, F1, AUC, Latency, Training Time, Model Size
• Radar Chart – Multi-metric spider diagram comparing versions
• Version history trend: accuracy & F1 across model releases

🛠️ Technical Implementation

Architecture

• Single HTML File – No build tools, npm, or external dependencies needed
• Plotly.js from CDN – All visualizations via Plotly
• Vanilla JavaScript – Pure JS algorithms, no framework dependencies
• Responsive Design – Mobile-friendly CSS Grid layout
• Dark Theme – Enterprise design with Tailwind-inspired colors

ML Algorithms (JavaScript)

All implemented in vanilla JavaScript:

// PSI Calculation
function calculatePSI(referenceData, currentData, feature, bins = 10)
  - Bins both distributions
  - Calculates proportion ratio
  - Returns sum of KL divergence

// Feature Permutation Importance
function calculateFeatureImportance(data, features)
  - Shuffles each feature
  - Measures impact on baseline accuracy
  - Returns importance scores

// SHAP-style Values
function calculateSHAPValues(data, features, numSamples = 200)
  - Feature contribution decomposition
  - Normalized feature value encoding
  - Returns per-sample SHAP values

// Logistic Regression (What-If)
  - Coefficient-based prediction: z = w0 + Σ(wi * xi)
  - Sigmoid transform: 1 / (1 + e^-z)
  - Feature contributions via partial derivatives

Data Generation

Seeded Random Generator – Reproducible synthetic data:

• 5,000 training samples per model
• 10 features: age, income, tenure, usage_frequency, support_tickets, etc.
• Gaussian distributions with realistic correlation structure
• Production data with 6 months of synthetic drift
• Fairness-relevant demographics with realistic distributions

📈 Explainability Methods

1. SHAP Beeswarm Plot (Standout Feature)

Inspired by SHAP library's beeswarm plot:

• Each dot = one prediction
• X-axis = SHAP value (positive = increases churn prediction, negative = decreases)
• Y-axis = feature
• Color = feature value (blue=low, red=high)
• Reveals both magnitude and direction of feature impact
• Shows when feature importance varies by feature value

2. Partial Dependence Plots (PDP)

Shows average model behavior:

• X-axis = feature value range
• Y-axis = predicted probability
• Blue line = average effect (PDP)
• Gray lines = individual prediction trajectories (ICE)

3. Permutation Feature Importance

Measures impact on accuracy:

• Shuffle each feature
• Measure accuracy degradation
• Importance = baseline accuracy - shuffled accuracy
• Shows which features matter most to the model

4. Local Explanations (What-If)

SHAP-style waterfall chart:

• Base score + feature contributions
• Shows how each feature pushed prediction higher/lower
• Suggests changes to flip prediction

📊 Drift Detection Methods

1. Population Stability Index (PSI)

PSI = Σ (Current% - Reference%) * ln(Current% / Reference%)

• Thresholds: <0.1 (stable), 0.1-0.25 (warning), >0.25 (alert)
• Per-feature + per-time-period tracking

2. KL Divergence

Implemented in PSI calculation:

• Measures difference between distributions
• Foundation for drift detection

3. Distribution Comparison

• Histogram + KDE overlays
• Reference (training) vs. Current (production)
• Interactive feature selector

4. Drift Heatmap

Time × Feature matrix showing PSI evolution:

• Identifies when/which features drift
• Color intensity = drift severity

🎯 Fairness Metrics

Demographic Parity

• Positive prediction rate should be equal across groups
• Formula: P(ŷ=1 | group A) ≈ P(ŷ=1 | group B)

Equal Opportunity

• True Positive Rate (TPR) should be equal across groups
• Focuses on false negatives (missing positive cases)

Disparate Impact Ratio

• Ratio of selection rates: minority group / majority group
• Legal threshold: 0.8 to 1.25 (80% rule)
• Outside range indicates potential discrimination

🚀 How to Use

Opening the Dashboard

1. Download index.html to your machine
2. Open in any modern browser (Chrome, Firefox, Safari, Edge)
3. No server required – everything runs client-side

Exploring Features

Performance Tab:

• View model metrics for each production model
• Click model cards to switch between Churn/Loan/Demand models
• Hover over performance chart to see exact metrics

Explainability Tab:

• Scroll through feature importance bar chart
• Examine SHAP beeswarm plot to understand feature impacts
• Use dropdown to explore PDP/ICE for different features

Drift Detection Tab:

• View heatmap showing which features drifted over time
• Check PSI bar chart against thresholds
• Use dropdown to compare distributions for specific features

Fairness & Bias Tab:

• Select demographic group (Age/Gender/Region)
• Compare prediction rates across protected groups
• Check disparate impact ratios

What-If Analysis Tab:

• Adjust sliders to explore different customer profiles
• Watch prediction update in real-time
• View sensitivity analysis to see feature importance
• Click "What changes would flip prediction?" for actionable insights

Model Comparison Tab:

• Select two model versions to compare
• View metric deltas with color coding (green=better)
• Examine radar chart for multi-metric comparison
• Scroll to see version history trends

💾 Data Structure

Training Data (Internal)

{
  age: 35,
  income: 60,
  tenure: 5,
  usage_frequency: 25,
  support_tickets: 3,
  payment_days_overdue: 0,
  account_balance: 5000,
  monthly_charges: 65,
  contract_type: "Month-to-month",
  internet_service: "Fiber optic",
  churn_probability: 0.35,  // synthetic label
  churn: 0                    // binary outcome
}

Production Data

• Same features as training
• Applied drift transformations (Gaussian noise)
• Different distribution properties (enables drift detection demo)

🎨 Design System

Color Palette

• Dark Background: #0f172a (slate-950)
• Card Background: #1e293b (slate-800)
• Primary: #3b82f6 (blue-500)
• Secondary: #10b981 (emerald-500)
• Warning: #f59e0b (amber-500)
• Danger: #ef4444 (red-500)

Typography

• Font Family: System sans-serif (macOS/Windows/Linux compatible)
• Responsive sizing: 1.5rem headers → 0.85rem labels
• Mobile-optimized: Tablet (768px) and mobile breakpoints

📱 Responsive Design

• Desktop (1200px+): 3-column grid layouts
• Tablet (768-1199px): 2-column grids
• Mobile (<768px): Single column, stacked tabs

Charts automatically resize on window resize using Plotly's responsive mode.

🔍 Browser Compatibility

• Chrome/Edge 90+
• Firefox 88+
• Safari 14+
• All Chromium-based browsers
• Mobile browsers (iOS Safari, Chrome Android)

📚 Key Takeaways for Portfolio

This dashboard demonstrates:

1. ML Governance Skills

   • Model registry management
   • Performance monitoring
   • Drift detection

2. Explainability (XAI)

   • SHAP-style visualizations
   • Feature importance methods
   • Local vs. global explanations

3. Data Analysis

   • Statistical distributions
   • Fairness metrics
   • Hypothesis testing (threshold analysis)

4. Software Engineering

   • Single-file architecture
   • Algorithm implementation in JS
   • Responsive UI design
   • Data visualization

5. Business Understanding

   • What-if analysis for business decisions
   • Fairness implications
   • Model trade-offs

📄 License

MIT License – Free to use, modify, and distribute.

👤 Author

Mayank Joshi MSc Business Analytics GitHub: mayankjoshiii

---

🎓 Learning Resources Used

• SHAP: Model-agnostic feature importance
• PDP/ICE: Partial Dependence Plots from scikit-learn
• PSI: Population Stability Index (credit risk monitoring)
• Fairness Metrics: AI Fairness 360 (IBM)
• Plotly.js: Interactive visualization library
• Logistic Regression: Classification baseline

🤝 Contributing

This is a portfolio project. For suggestions or improvements, please open an issue on GitHub.

---

Last Updated: March 2026 Status: Production-ready for portfolio demonstration