An enterprise‑grade, production‑ready backend system that forecasts 30‑day cash flow for e‑commerce marketplaces.
Built using Hexagonal Architecture and Domain‑Driven Design, it combines a Deep Learning / Gradient Boosting ensemble with specialised AI agents for anomaly detection, refund risk prediction, and data drift monitoring.
Everything is delivered through an interactive FastAPI + Plotly dashboard.
Most data scientists write monolithic notebooks that mix business rules with ML code.
This project deliberately separates pure business logic (domain) from infrastructure (Pandas, PyTorch, Scikit‑Learn) using Ports & Adapters.
The result is a system where:
- You can swap the N‑BEATS model for a custom Transformer without touching business rules
- You can replace the Olist CSVs with a live SQL database by changing a single adapter
- The forecasting, anomaly detection, and drift monitoring agents are independently testable
This is not a hackathon prototype – it’s a demonstration of senior engineering practices in an AI system.
| Feature | Description |
|---|---|
| 30‑day Cash‑Flow Forecasting | Ensemble of N‑BEATS (deep learning) and LightGBM (gradient boosting) with mathematically rigorous 80% confidence bands |
| Anomaly Detection with Root‑Cause Analysis | Isolation Forest + local Z‑score filter pinpoints business‑impacting outliers and automatically queries operational data to explain why they happened |
| Review‑Risk Agent | Leading‑indicator agent that monitors negative‑review velocity and warns of future refund spikes 48–72 hours ahead |
| Drift Guardrail Agent | Continuously checks whether hardcoded business assumptions (e.g. freight subsidy ratio) still match reality, triggering recalibration when needed |
| Hexagonal Architecture (Ports & Adapters) | Domain logic is pure Python; infrastructure adapters plug into abstract interfaces |
| Interactive Dashboard | Plotly.js visualisations including 3D geospatial maps, ensemble forecasts, anomaly feeds, and the “Heavy Tail” scatter plot |
| Fully Dockerised | One‑command startup, guaranteed environment reproducibility – no PyTorch/C++ compiler hell |
A junior writes a 500‑line Jupyter Notebook mixing Pandas, PyTorch, and business logic.
A senior engineer strictly separates them.
This project uses Hexagonal Architecture combined with Domain‑Driven Design (DDD).
The core business logic (domain/) contains zero references to Pandas, PyTorch, or Scikit‑Learn.
It only knows about plain Python objects like Money and CashFlowRecord.
The infrastructure layer (infrastructure/) handles the messy reality of CSVs and neural networks, adapting to the domain contracts (ports).
If we want to swap Darts for a custom PyTorch model, or swap the Olist dataset for a SQL database, we only change infrastructure files – the business logic and agent orchestration remain untouched.
graph TD
subgraph "Composition Root (main.py / FastAPI)"
A[API Request] --> B[Pipeline Orchestrator]
end
subgraph "Application Layer (The Brains)"
B --> C[Forecast Agent]
B --> D[Anomaly Agent]
B --> E[Review Risk Agent]
B --> F[Drift Agent]
end
subgraph "Domain Layer (Pure Python, No Libraries)"
C --> G((AbstractForecaster Port))
D --> H((AbstractDetector Port))
I[Money / CashFlowRecord Entities]
end
subgraph "Infrastructure Layer (The Muscle)"
G --> J[N-BEATS Model]
G --> K[LightGBM Model]
H --> L[Isolation Forest]
M[Olist Repository Pandas Adaptor]
end
M --> N((Olist CSV Files))
The “Why”
- Model Selection: N‑BEATS + LightGBM vs. ARIMA / Prophet
We intentionally avoided traditional statistical models.
ARIMA assumes linear relationships and struggles with e‑commerce seasonality (Black Friday, weekends).
Prophet is robust but heavily reliant on default seasonalities, making it hard to fine‑tune for small, noisy marketplace data.
N‑BEATS (Neural Basis Expansion Analysis) uses multi‑rate signal processing to decompose trends at different frequencies – state‑of‑the‑art for long‑horizon forecasting.
LightGBM is included because tree models are weak at time sequences but excel at tabular features. Together they form a robust ensemble.
- Confidence Intervals: First‑Difference Variance vs. MC Dropout
Deep learning models do not natively output confidence intervals. A naive approach is Monte Carlo Dropout (predicting 200 times with dropout enabled).
The Problem: MC Dropout is statistically unreliable on small datasets and caused severe version conflicts with PyTorch Lightning in production.
The Solution: We calculate the standard deviation of day‑over‑day changes (first differences). This isolates the “white noise” of daily volatility from the underlying trend, providing a mathematically rigorous 80% confidence band using a simple Z‑score multiplier (1.2816).
- The “Heavy Tail” Effect — Why R² is 0.17
If you look at backtesting metrics, the R‑squared is ~0.17. A junior panics; a senior knows exactly why.
Daily e‑commerce cash flow crosses near‑zero on weekends and holidays.
If the model predicts R 6,000butactualisR 6,000butactualisR 50 (a dead Sunday), the squared residual is (6000‑50)² = 35,402,500.
This single outlier mathematically destroys the total sum of squared residuals, giving a low R² even though the model accurately tracks the weekly business curve 80% of the time.
MAPE is therefore strictly banned (it approaches infinity when dividing by near‑zero actuals). We rely on Scaled RMSE and MAE for business evaluation. The dashboard includes a scatter plot that visually confirms this heavy‑tail distribution.
🤖 The Multi‑Agent Pipeline
The system is not a single script – it is a sequence of autonomous, specialised agents orchestrated by the Pipeline service.
Forecasting Agent
Trains the N‑BEATS / LightGBM ensemble and calculates the weighted 30‑day projection with confidence bands.
Anomaly Agent
Uses an Isolation Forest with a local Z‑score filter. It filters out statistical noise (e.g., a 0.03% deviation) and only flags business‑impactful outliers (>20% deviation).
It then triggers the Root‑Cause Analyzer to query the products and sellers tables and explain why the anomaly happened (e.g., “Black Friday in the electronics category”).
Review‑Risk Agent
A leading‑indicator agent. It monitors 1‑ and 2‑star review velocity. A spike in negative reviews today triggers a warning that cash outflows (refunds) will spike in 48–72 hours.
Drift Agent
A guardrail agent. It checks whether hardcoded business assumptions (e.g., FREIGHT_SUBSIDY_RATIO = 0.60) still match reality. If logistics costs drift >15%, it flags the configuration for immediate recalibration.
cashflow-ai/ ├── data/ # Unversioned data │ ├── raw/ # Downloaded Olist CSVs │ └── processed/ # Generated daily aggregates ├── src/cashflow_ai/ │ ├── core/ # Cross‑cutting concerns │ │ └── config.py # Pydantic BaseSettings (env vars, paths, hyperparams) │ ├── domain/ # PURE BUSINESS LOGIC (Zero external dependencies) │ │ ├── entities.py # CashFlowRecord, ForecastRecord, AnomalyRecord │ │ ├── value_objects.py # Immutable types: Money, DateRange, ConfidenceInterval │ │ └── ports/ # Abstract Interfaces (The Contracts) │ │ ├── forecaster.py # Defines .train() and .predict() │ │ ├── detector.py # Defines .fit() and .detect() │ │ └── monitor.py # Defines .scan() and .check() │ ├── application/ # USE CASES / AGENTS (Orchestrators) │ │ ├── agents/ # The "Brains" of the system │ │ │ ├── forecasting.py # Ensemble logic │ │ │ ├── anomaly.py # Orchestration + Root Cause triggering │ │ │ ├── review_risk.py # Leading indicator logic │ │ │ ├── drift.py # Config guardrail logic │ │ │ └── reporting.py # Text synthesis │ │ └── services/ │ │ └── pipeline.py # Dependency Injection container │ └── infrastructure/ # CONCRETE IMPLEMENTATIONS (The Adapters) │ ├── api/ # FastAPI web server │ ├── dataset/ # Pandas implementations of ports │ │ ├── olist_repository.py # Joins 8 CSVs into Domain Entities │ │ ├── root_cause.py # Queries operational data for anomaly explanations │ │ ├── review_monitor.py # Calculates review velocity │ │ └── drift_monitor.py # Calculates actual vs expected ratios │ └── models/ # ML implementations of ports │ ├── nbeats.py # Darts wrapper + Serialization │ └── lgbm.py # Darts wrapper + Serialization ├── static/ # Frontend (Plotly.js) │ └── index.html # Interactive dashboard ├── scripts/ │ └── train.py # Offline training script (Saves models to disk) ├── pyproject.toml # Modern Python packaging ├── Dockerfile # Frozen production environment ├── docker-compose.yml # One‑command startup └── run_webapp.py # Uvicorn entrypoint
🚀 Setup & Execution
This project uses Docker to guarantee environment reproducibility. No fighting with PyTorch or C++ compiler versions.
Prerequisites: Docker & Docker Compose installed.
Clone the repository
git clone https://github.com/YOUR_USERNAME/cashflow-ai.git
cd cashflow-ai
Download the dataset
Download the Brazilian E‑Commerce Public Dataset by Olist from Kaggle and extract the 8 CSV files into the data/raw/ folder.
Train the models (once)
bash
docker-compose run --rm api python scripts/train.py
This takes ~2 minutes, trains the neural networks, and saves the weights to outputs/models/ on your local machine.
Start the dashboard
bash
docker-compose up --build
Open your browser at http://127.0.0.1:8000.
The API loads the pre‑trained weights and renders the interactive dashboard in seconds.
🛠️ Tech Stack Layer Technology Backend Framework FastAPI Deep Learning N‑BEATS (via Darts / PyTorch) Gradient Boosting LightGBM (via Darts) Anomaly Detection Isolation Forest (Scikit‑Learn) Data Manipulation Pandas, NumPy Architecture Hexagonal (Ports & Adapters), Domain‑Driven Design Validation Pydantic V2 (strict typing, frozen entities) Frontend Plotly.js (3D geospatial, scatter, area charts) Infrastructure Docker, Uvicorn 📈 Dashboard Features
The interactive dashboard includes:
Cash Flow Timeline – Historical data with the 30‑day ensemble forecast and 80% confidence intervals.
Anomaly Feed – Real‑time flagged outliers with dynamic Root‑Cause Investigation (e.g., linking anomalies to specific product categories or sellers).
3D Geospatial Map – Interactive bar chart mapping cash‑flow volume across Brazilian states by lat/lon.
Marketplace Economics – Stacked area chart visualising the cash‑in vs. cash‑out take‑rate over time.
The Heavy Tail Effect – A scatter plot proving why daily R² is suppressed by weekend/holiday volatility drops.
📊 Key Findings & Limitations (from backtesting)
R² ≈ 0.17 — Not a sign of poor model. The heavy‑tail effect caused by near‑zero cash flow on weekends mathematically destroys R². Visual inspection and scaled RMSE confirm the model captures the weekly curve reliably.
Ensemble outperforms individual models — The N‑BEATS + LightGBM combination smooths out neural network noise and tree‑based extrapolation weakness.
Review‑Risk Agent provides genuine early warning — Spikes in negative reviews consistently preceded refund spikes by 2–3 days in the Olist data.
Drift guardrails are essential — Hardcoded assumptions (freight ratio, take‑rate) drifted over the 2‑year dataset, proving the need for continuous monitoring.
📦 Dependencies
All Python dependencies are managed through pyproject.toml and frozen inside the Docker image. Key packages: darts, pytorch-lightning, lightgbm, scikit-learn, fastapi, pandas, plotly, pydantic.
📝 License
This project is licensed under the MIT License – feel free to use, modify, and distribute.