Real-Time Fraud Detection with dbt, Kafka & Snowflake

Overview

This project implements real-time fraud detection using:

• Kafka (Streaming transaction events)
• Snowflake (Data warehouse for storing transactions)
• dbt (Data transformations, anomaly detection, and aggregations)
• Jupyter Notebook (For querying and analyzing results)

Read the Full Article on Medium: Link to Medium

Setup & Installation

1. Clone the repository

git clone https://github.com/usefusefi/dbt-fraud-detection.git
cd dbt-fraud-detection

2. Install dbt & dependencies

pip install dbt-core dbt-snowflake kafka-python pandas

3. Configure dbt (Update profiles.yml)

default:
  outputs:
    dev:
      type: snowflake
      account: your_snowflake_account
      user: your_user
      password: your_password
      role: your_role
      warehouse: your_warehouse
      schema: fraud_detection
      threads: 4
  target: dev

4. Run dbt transformations

dbt run

5. Start Kafka producer (Streaming transactions)

python scripts/kafka_fraud_producer.py

6. Analyze results in Snowflake

Run this SQL in Snowflake:

SELECT * FROM fact_fraud_detection ORDER BY fraud_transactions DESC;