Crypto-PerpetualFutures

Fully bidirectional, delta-neutral carry strategy on crypto perpetual futures. Harvests funding rate dislocations in both directions across 15 liquid pairs using a z-score signal that identifies anomalies relative to each asset's own funding history.

---

Get Started

1. Create and activate the conda environment

conda create -n qts-perp python=3.11 -y
conda activate qts-perp
pip install ccxt pandas pyarrow requests tqdm

Important: always use the qts-perp environment. Reading parquet files written by a different Python/PyArrow version will fail with a histogram mismatch error.

2. Select the interpreter in VSCode

Cmd+Shift+P → Python: Select Interpreter → pick qts-perp

For notebooks: Cmd+Shift+P → Notebook: Select Notebook Kernel → pick qts-perp

---

Data Pipeline

Run the steps below in order. All scripts are idempotent — re-running resumes from the last saved timestamp.

Step 1 — Fetch Binance data (requires VPN)

Binance blocks US-based IPs (HTTP 451). Connect your VPN to a non-US server first.

# Test pull — BTC + ETH, last year only
python src/fetch_binance.py --symbols BTC ETH --start 2024-01-01

# Full pull — all 15 pairs, Jan 2020 → today (~30 min)
python src/fetch_binance.py

Saves to data/raw/binance/ (one Parquet file per symbol per series):

• funding_rates/ — 8h funding rates
• ohlcv_perp/ — 8h mark-price OHLCV
• ohlcv_spot/ — 8h spot index OHLCV

Step 2 — Fetch Open Interest from Bybit (VPN may be needed)

Binance's OI endpoint is hard-capped at ~30 days of history. Bybit provides free OI going back to Dec 2020.

Note: Bybit also blocks US IPs via CloudFront. If you get a 403, switch your VPN to a different server.

# Full pull — all 15 pairs (~30–45 min)
python src/fetch_bybit_oi.py

# Subset
python src/fetch_bybit_oi.py --symbols BTC ETH SOL

Saves 1h OI to data/raw/binance/open_interest/ (same directory as Binance files).

Step 3 — Fetch macro data

FRED (3m T-bill) — runs anywhere:

python src/fetch_macro.py --fred-only

VIX + SPY — requires Bloomberg Terminal (run on university computer):

# Install on the Bloomberg computer first:
pip install pdblp pandas requests

python src/fetch_macro.py --bloomberg-only

Then copy vix_daily.csv and spy_daily.csv from the Bloomberg computer's src/macro_output/ folder to data/raw/macro/ on your main machine.

Saves to data/raw/macro/:

• fred_rfr.csv — 3m T-bill annualised % + daily decimal
• vix_daily.csv — VIX daily close
• spy_daily.csv — SPY daily OHLCV

Step 4 — Build master panel

python src/build_panel.py

Merges all raw data into data/processed/master_panel.parquet. Takes ~10 seconds.

---

Master Panel

data/processed/master_panel.parquet

Property	Value
Shape	(101,115 × 11)
Index	MultiIndex(datetime [UTC, 8h], symbol)
Timestamps	6,741 standard settlement times (00:00, 08:00, 16:00 UTC)
Symbols	15 (see universe below)

Columns:

Column	Description
funding_rate	Funding rate at each 8h settlement
perp_open/high/low/close/volume	Mark-price 8h OHLCV
spot_close	Spot index 8h close
open_interest	OI in base currency (Bybit, 1h → aligned to 8h)
vix_close	VIX daily close (forward-filled to 8h grid)
spy_close	SPY daily close (forward-filled to 8h grid)
rfr_daily_decimal	3m T-bill daily rate (forward-filled, 100% complete)

Coverage:

Column	Non-null	Notes
funding_rate	87.5%	ARB/POL only from 2023
perp_close	87.4%	Same reason
open_interest	73.5%	Bybit OI starts Dec 2020
vix_close	97.7%	Null only on US market holidays
rfr_daily_decimal	100%	FRED covers every calendar day

How to load:

import pandas as pd

panel = pd.read_parquet("data/processed/master_panel.parquet")

# Single symbol time series
btc = panel.xs("BTCUSDT", level="symbol").dropna(subset=["funding_rate"])

# Funding rate matrix — shape (6741, 15)
funding = panel["funding_rate"].unstack("symbol")

# Cross-section at a specific timestamp
panel.loc[pd.Timestamp("2024-01-01 00:00", tz="UTC")]

---

Universe (15 symbols)

Symbol	Notes
BTCUSDT	Full history from Jan 2020
ETHUSDT	Full history from Jan 2020
SOLUSDT
BNBUSDT
XRPUSDT
DOGEUSDT
AVAXUSDT
POLUSDT	MATIC rebranded → POL on Binance/Bybit in Sept 2023; data from ~Sept 2024 on Binance perps
LINKUSDT
ADAUSDT
DOTUSDT
ATOMUSDT
LTCUSDT
UNIUSDT
ARBUSDT	Listed ~Sept 2023

---

Known Data Quirks

• Binance funding timestamps occasionally have sub-millisecond offsets (e.g. 00:00:00.002). build_panel.py rounds all timestamps to the nearest second before aligning.
• Off-grid funding events (~1,600 Binance settlements at non-standard hours for newly listed symbols) are dropped — the panel keeps only standard 00:00/08:00/16:00 UTC timestamps.
• Binance OI history is hard-capped at ~30 days regardless of timeframe. Use fetch_bybit_oi.py instead.
• VPN required for both Binance (HTTP 451) and Bybit (CloudFront 403) from US IPs.

---

Project Structure

Crypto-PerpetualFutures/
│
├── data/
│   ├── raw/
│   │   ├── binance/
│   │   │   ├── funding_rates/       # per-symbol parquet, 8h
│   │   │   ├── ohlcv_perp/          # mark-price 8h OHLCV
│   │   │   ├── ohlcv_spot/          # spot index 8h OHLCV
│   │   │   └── open_interest/       # Bybit 1h OI (stored here for convenience)
│   │   └── macro/
│   │       ├── vix_daily.csv        # Bloomberg Terminal
│   │       ├── spy_daily.csv        # Bloomberg Terminal
│   │       └── fred_rfr.csv         # FRED public API
│   │
│   └── processed/
│       └── master_panel.parquet     # merged panel, ready for notebook
│
├── src/
│   ├── fetch_binance.py     # funding rates, perp/spot OHLCV (Step 1)
│   ├── fetch_bybit_oi.py    # open interest from Bybit (Step 2)
│   ├── fetch_macro.py       # Bloomberg + FRED macro data (Step 3)
│   ├── build_panel.py       # merges everything → master_panel (Step 4)
│   └── validate.py          # data quality checks (optional)
│
├── strategy.py              # importable signal + backtest logic
└── notebook.ipynb           # research notebook