An Explainable Machine Learning System for Financial Fraud Detection using Ensemble Learning, Model Comparison, and Explainable AI (SHAP).
FraudShield AI is an end-to-end machine learning framework that detects fraudulent financial transactions using multiple publicly available datasets. The system integrates data preprocessing, feature engineering, model training, evaluation, explainability, and an interactive Streamlit dashboard into a unified pipeline.
- Overview
- Problem Statement
- Objectives
- Key Features
- Technologies Used
- System Architecture
- Workflow
- Project Structure
- Datasets
- Machine Learning Models
- Model Performance
- Explainable AI
- Dashboard
- Installation
- Usage
- Results
- Future Improvements
- License
- Author
- Acknowledgements
FraudShield AI is designed to detect fraudulent financial transactions using supervised machine learning techniques. The project supports multiple fraud datasets, performs automated preprocessing, compares multiple machine learning models, selects the best-performing model, and explains predictions using SHAP (SHapley Additive Explanations).
Unlike traditional fraud detection systems that rely solely on rule-based methods, FraudShield AI combines modern ensemble learning algorithms with Explainable AI to improve both predictive performance and model transparency.
Financial fraud causes billions of dollars in losses annually. Detecting fraudulent transactions is difficult because fraudulent events represent only a tiny fraction of all transactions, resulting in highly imbalanced datasets.
Traditional rule-based systems struggle to adapt to evolving fraud patterns and often generate a high number of false positives.
FraudShield AI addresses these challenges by:
- Handling class imbalance using SMOTE
- Comparing multiple machine learning algorithms
- Selecting the optimal model automatically
- Providing interpretable predictions through Explainable AI (SHAP)
- Detect fraudulent financial transactions with high accuracy.
- Build a reusable fraud detection framework.
- Support multiple public fraud datasets.
- Perform automated preprocessing and feature engineering.
- Compare multiple machine learning algorithms.
- Select the best-performing model automatically.
- Explain predictions using SHAP.
- Provide an interactive Streamlit dashboard for visualization.
- Multi-dataset support
- Automated preprocessing pipeline
- Missing value handling
- Feature engineering
- Class balancing using SMOTE
- Five machine learning algorithms
- Automatic model comparison
- Model leaderboard
- Explainable AI using SHAP
- Fraud prediction
- Interactive Streamlit dashboard
- Dataset explorer
- Analytics dashboard
- Performance visualization
- Model evaluation reports
- Python 3
- SHAP
- Pandas
- NumPy
- Matplotlib
- Plotly
- Streamlit
- Joblib
Financial Datasets
│
▼
Data Loading Module
│
▼
Data Preprocessing Module
│
▼
Feature Engineering Module
│
▼
Machine Learning Models
│
┌─────--─────┬─────--────┬─────--─────┬───-───────┬
▼ ▼ ▼ ▼ ▼
Logistic DecisionTree RandomForest XGBoost LightGBM
Regression
│
▼
Model Evaluation & Comparison
│
▼
Best Model Selection
│
▼
Explainable AI (SHAP)
│
▼
Interactive Dashboard
Datasets
│
▼
Data Loading
│
▼
EDA
│
▼
Preprocessing
│
▼
Feature Engineering
│
▼
SMOTE
│
▼
(only on)Train
│
▼
Test split
│
▼
Model Training
│
▼
Model Evaluation
│
▼
Best Model Selection
│
▼
Explainable AI
│
▼
Dashboard
FraudShield-AI
│
├── dashboard/
│ ├── app.py
│ └── pages/
│
├── datasets/
│
├── docs/
│
├── images/
│
├── models/
│
├── notebooks/
│
├── outputs/
│ ├── reports/
│ ├── model_leaderboard.csv
│ ├── confusion_matrix.png
│ ├── roc_curve.png
│ └── precision_recall.png
│
├── src/
│ ├── analysis/
│ ├── dashboard/
│ ├── data/
│ ├── evaluation/
│ ├── explainability/
│ ├── features/
│ ├── models/
│ └── utils/
│
├── tests/
│
├── README.md
├── requirements.txt
├── LICENSE
└── main.py
FraudShield AI is designed to support multiple fraud detection datasets to evaluate machine learning models under different fraud scenarios.
| Dataset | Type | Class Distribution | Purpose |
|---|---|---|---|
| Credit Card Fraud Detection 2023 | Synthetic | 50% Fraud / 50% Legitimate | Primary benchmark dataset used for model training and evaluation. |
| IEEE-CIS Fraud Detection | Real-world E-commerce | Highly Imbalanced | Supported for realistic fraud detection experiments involving online transactions. |
| PaySim | Synthetic Mobile Money Transactions | Imbalanced | Supported for mobile payment fraud detection and research. |
The datasets used in this project have different characteristics and difficulty levels.
-
Credit Card Fraud Detection 2023 is a balanced synthetic dataset containing an equal number of fraudulent and legitimate transactions. Because of its balanced distribution and engineered feature space, machine learning models can achieve very high predictive performance.
-
IEEE-CIS Fraud Detection is a real-world e-commerce fraud dataset with highly imbalanced classes, missing values, and complex feature relationships. It represents a significantly more challenging fraud detection problem.
-
PaySim is a synthetic simulation of mobile money transactions designed to emulate financial fraud scenarios while preserving realistic transaction behavior.
Current Benchmark
The performance metrics, leaderboard, and evaluation results presented in this repository are generated using the Credit Card Fraud Detection 2023 dataset.
The project architecture is designed to support IEEE-CIS and PaySim, allowing the same machine learning pipeline to be applied to datasets with different characteristics and fraud distributions.
The following models are implemented and evaluated.
- Logistic Regression
- Decision Tree
- Random Forest
- XGBoost
- LightGBM
The best-performing model is automatically selected based on evaluation metrics.
Five supervised machine learning models were trained and evaluated using a unified preprocessing pipeline on the Credit Card Fraud Detection 2023 dataset. Model performance was measured using Accuracy, Precision, Recall, F1-Score, ROC-AUC, and Training Time.
| Rank | Model | Accuracy | Precision | Recall | F1 Score | ROC-AUC | Training Time |
|---|---|---|---|---|---|---|---|
| 🥇 | Random Forest | 0.999807 | 0.999930 | 0.999683 | 0.999807 | 0.999999 | 65.52 s |
| 🥈 | XGBoost | 0.999754 | 0.999859 | 0.999648 | 0.999754 | 0.999964 | 1.48 s |
| 🥉 | LightGBM | 0.999727 | 0.999842 | 0.999613 | 0.999727 | 0.999960 | 1.27 s |
| 4 | Decision Tree | 0.999639 | 0.999648 | 0.999631 | 0.999639 | 0.999639 | 11.00 s |
| 5 | Logistic Regression | 0.998294 | 0.998996 | 0.997591 | 0.998293 | 0.999807 | 0.61 s |
Evaluation Context
The reported performance metrics were obtained using the Credit Card Fraud Detection 2023 dataset. This dataset is a synthetic, class-balanced benchmark containing approximately 50% fraudulent and 50% legitimate (non-fraudulent) transactions. The balanced class distribution and highly informative feature representation make this dataset considerably easier to classify than real-world fraud datasets. Consequently, machine learning models can achieve substantially higher performance metrics on this benchmark than on highly imbalanced, real-world datasets such as IEEE-CIS.
🏆 Random Forest achieved the highest overall performance among all evaluated models.
Highlights
- Accuracy: 99.98%
- Precision: 99.99%
- Recall: 99.97%
- F1 Score: 99.98%
- ROC-AUC: 99.9999%
Although XGBoost and LightGBM achieved comparable predictive performance with significantly shorter training times, Random Forest demonstrated the strongest balance across all evaluation metrics and was therefore selected as the final deployment model.
FraudShield AI incorporates SHAP (SHapley Additive Explanations) to improve model transparency.
The Explainable AI module provides:
- Feature importance
- SHAP summary plots
- Local prediction explanations
- Global model interpretation
This enables users to understand why the model classified a transaction as fraudulent.
The Streamlit dashboard includes:
- Home
- Model Comparison
- Fraud Prediction
- Model Leaderboard
- Explainable AI
- Feature Importance
- Analytics
- Dataset Explorer
- About
Clone the repository
git clone https://github.com/YOUR_USERNAME/FraudShield-AI.git
cd FraudShield-AIInstall dependencies
pip install -r requirements.txtRun the application
streamlit run dashboard/app.py- Load a supported fraud dataset.
- Perform preprocessing.
- Train all machine learning models.
- Compare model performance.
- Select the best model.
- Predict fraudulent transactions.
- Explore SHAP explanations.
- Analyze results using the dashboard.
The proposed FraudShield AI framework successfully:
- Trained and evaluated five supervised machine learning models.
- Automatically selected the highest-performing model.
- Achieved 99.98% Accuracy on the evaluation dataset.
- Achieved 99.99% ROC-AUC.
- Reduced the impact of class imbalance using SMOTE.
- Generated SHAP explanations for model transparency.
- Produced automated evaluation reports and a model leaderboard.
- Delivered an interactive Streamlit dashboard for fraud analysis.
Possible future enhancements include:
- Deep Learning models
- Graph Neural Networks
- Real-time fraud detection
- REST API integration
- Docker deployment
- Kubernetes deployment
- Cloud deployment (AWS/Azure/GCP)
- MLOps pipeline
- Continuous model monitoring
- Drift detection
- Incremental learning
- Real-time transaction streaming
This project is licensed under the MIT License.
This project utilizes publicly available datasets and open-source libraries.
Special thanks to:
- Scikit-Learn
- XGBoost
- LightGBM
- SHAP
- Streamlit
- Pandas
- NumPy
- Plotly
- Matplotlib
- IEEE-CIS Fraud Detection Dataset
- PaySim Dataset
- Credit Card Fraud Detection Dataset