Skip to content

performance.md

Latest commit

 

History

History
138 lines (96 loc) · 3.7 KB

performance.md

File metadata and controls

138 lines (96 loc) · 3.7 KB
Raw

Performance Tuning Guide

Overview

This guide explains how to keep the system fast for large portfolios (50+ assets) and slow environments.

Measured benchmarks

Measured with python scripts/benchmark.py on a 4-asset portfolio (v10.6.5):

Benchmark Symbols Elapsed Peak memory
Batch query 4 0.091 s 1.45 MB
Batch scoring 4 0.116 s 1.43 MB
Lazy loading (5J50.DE, 345 rows) 1 0.042 s 0.39 MB

The synthetic 100-asset tests in tests/test_performance.py assert batch scoring completes in under 60 seconds and peak memory stays under 500 MB.

Batch scoring

For portfolios with 50+ assets, use parallel batch scoring:

from quant.analytics.scoring import batch_score_assets_parallel

results = batch_score_assets_parallel(
    symbols=symbols,
    price_data=prices,
    fundamentals=fundamentals,
    news=news,
    tiers=tiers,
    max_workers=4,
)

Recommendations:

  • Under 20 assets: the thread path is used automatically (lower overhead).
  • 20 to 50 assets: max_workers=4.
  • 50+ assets: max_workers=8 or the CPU core count.

batch_score_assets_optimized(symbols, tiers) loads all market history in a single query, then scores in parallel.

Database optimization

Query batching

Instead of N per-symbol queries, use one batch query:

from quant.data.database import batch_query_portfolio_data

data = batch_query_portfolio_data(symbols)
prices = data["prices"]

Lazy loading

For very long histories, stream rows in chunks:

from quant.data.database import load_prices_lazy

for chunk in load_prices_lazy("AAPL", chunk_size=5000):
    process(chunk)

Caching

Cache expensive, JSON-serializable calculations on disk:

from quant.analytics.cache import disk_cache

@disk_cache(max_age_days=1)
def expensive_calculation(x: int) -> int:
    return x * x

Cache management is internal; quant doctor reports the cache location and freshness. The hidden quant cache-stats / quant clear-cache commands remain for maintenance.

Reliability

  • quant.utils.retry.retry_with_backoff retries transient external failures with exponential backoff.
  • quant.analytics.scoring.score_asset_with_fallbacks degrades gracefully when data is missing, marking data_quality as PARTIAL or FALLBACK.
  • quant.portfolio.tier_manager.load_tiers_safe never crashes on a missing, empty, or corrupted tiers file.

Monitoring

Use the read-only diagnosis to spot performance issues:

quant doctor

Look for WARNING on "Data freshness" (slow updates) or "Signal cache" (slow scoring).

Benchmarking

Run the performance suite:

pytest tests/test_performance.py -v

It reports batch scoring time for 50 and 100 assets, batch query time, and memory usage.

Honest performance math (v10.7.0)

The old "+2.5 percent since 13 Sep" was a naive value change and lied whenever money entered. From v10.7.0 the Overview return uses Modified Dietz over the period between two dates, using the flows table:

R = (V_end - V_start - F) / (V_start + sum(w_i * F_i))
w_i = (days_in_period - days_from_start_to_flow) / days_in_period

Deposits do not count as profit. Flow signs: buy = +amount (money entered invested), sell = -amount (money left invested), dividend = -amount (left invested, landed in cash). The Overview sentence reads: "Since 13 Sep: +20.84 EUR. Of that: you added 0.00 EUR, market moved +20.84 EUR. Return: +2.4 percent."

Per-asset performance is simply the price change over the period, labeled "price change" (honest for a held asset). The value series the chart reads is portfolio_value_history(date, invested_eur), written by the daily job from shares times close; it never includes cash.