Skip to content

Stop the losses of the first portfolio: re-evaluate at today's price, trust Jev less - #20

Merged
botta0oss merged 1 commit into
mainfrom
claude/lucid-lovelace-h1c8tm
Sep 25, 2026
Merged

botta0oss merged 1 commit into
mainfrom
claude/lucid-lovelace-h1c8tm

Conversation

@botta0oss

Copy link
Copy Markdown
Owner

Why

The first real simulated portfolio lost $11.86 of $50 (−24 %) in about 30 hours over 13 bets. $10.99 of the loss came from 7 crypto price markets. The analysis of the exported workbook found structural causes, not bad luck; this PR fixes them. The new section "What went wrong in the first portfolio" in docs/strategy.md summarises it.

Changes

  1. Re-evaluation at today's price (portfolio.evaluate_prediction, plans.signal_at)
    • Bug fixed: the evaluation reused the stored blend, which had been pooled with the price of the forecast. Against a price that had moved, this invented edges. Real case: a manual bet on a 19-hour-old forecast showed +$22 expected on $1.89.
    • The blend is now recomputed at the current price with today's calibration and weights, and so is the signal.
    • Before buying, a new forecast is required when:
      • the forecast is older than FORECAST_MAX_AGE_HOURS (6); or
      • the price moved more than FORECAST_MAX_PRICE_MOVE since the forecast (0.5 in log-odds, so moves near the extremes count more).
    • The plan then says «A new forecast is needed».
  2. Minimum hours to the end per preset: 72, 24 or 12 hours (too_close).
    • Two losing bets were bought 24 minutes before the end.
    • The check uses the real hours: days_to_end rounds up to one day.
  3. Price markets excluded: backend/markets/kinds.py recognises markets decided by an asset's price («Bitcoin above $84,000 on…», «ETH reach $2,800», «gold hit $3,000», «Up or Down»).
    • With EXCLUDE_PRICE_MARKETS=true they get no bets (the plan shows «Avoid»).
    • Automatic forecasts, «Assess all» and alerts skip them, so no Jev call is spent there.
    • A forecast asked by hand still runs.
  4. Less trust in Jev:
    • MODEL_WEIGHT_MAX goes from 0.5 to 0.25.
    • Jev's weight shrinks when it is more than MODEL_DISAGREEMENT_LOGIT (2) log-odds from the price.
    • Every losing bet came from a gap of about 50 points between Jev and the price, and the biggest gaps are the likeliest Jev errors.
    • The reduction lives inside pool(), so forecasts, buy/sell levels, backtests and the plan agree.
    • forecast_of now starts from the base weight, so the reduction is not applied twice.
  5. Uncertainty correction: the factor is narrowed (down to 0.75) only if the blend beats the market price on the same resolved markets, by two standard errors of the paired Brier gain. It was at 0.75 only because the blend was consistent with itself.
  6. Minimum prudent return per bet: 10 %, 6 % or 3 % by preset (roi_too_low). Annualised returns of short bets always passed the old time check.

The UI shows the new reasons, the reduced weight in «Why this signal» and «How it works», the new parameters in Settings, and the preset limits. Docs (EN and IT) and .env.example are updated.

Tests

  • New tests/test_trust.py, using the real losing cases:
    • 66 % against 5.5 %, 67 % against 11.5 % and 72 % against 7.5 % now give no signal;
    • the 19-hour-old forecast and a big price move are refused;
    • a market 25 minutes from the end, and a price market, get no bet and no automatic forecast;
    • the uncertainty is not narrowed without skill against the market;
    • the minimum return per bet;
    • the price-market recogniser.
  • The existing tests keep the earlier forecast parameters through tests/conftest.py, since they test the mechanics, not the values.
  • test_strategy.py now includes the minimum return in its hand-rebuilt check.
  • Full suite: 208 passed.
  • Playwright (IT/EN): no JS errors or missing translations on opportunities, market detail, how it works, settings and portfolio.
  • API on the demo data: the price market shows «Avoid», stale forecasts show «A new forecast is needed».

After merging

  • Run a backtest (now working) and apply the suggested calibration.
  • Existing forecasts older than 6 hours need a new one before any buy.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JLWZrfy12imc6dQRsEtFjj


Generated by Claude Code

… trust Jev less

The first real portfolio lost 24% in a day and a half, almost all on crypto
price markets. Six changes:

1. The blend is recomputed at the current price (calibrated Jev pooled with
   the price of now) instead of reusing the stored one, pooled with the price
   of the forecast. Forecasts older than FORECAST_MAX_AGE_HOURS or whose price
   moved more than FORECAST_MAX_PRICE_MOVE (log-odds) need a new one before
   buying. The signal is recomputed at the current price too.
2. Minimum hours to the end per preset (72/24/12): no bets when the price
   already knows the outcome. Uses the real hours, not days rounded up to 1.
3. Markets decided by an asset's price are recognised (markets/kinds.py) and
   excluded (EXCLUDE_PRICE_MARKETS): no bets, and automatic forecasts, «Assess
   all» and alerts skip them.
4. MODEL_WEIGHT_MAX 0.5 -> 0.25, and Jev's weight shrinks beyond
   MODEL_DISAGREEMENT_LOGIT (2) log-odds from the price, inside pool() so
   forecasts, buy/sell levels and backtests agree.
5. The uncertainty is narrowed only when the blend beats the market price on
   resolved markets (paired Brier gain, two standard errors).
6. Minimum prudent return per bet (10/6/3%), since annualised returns of
   short bets always passed.

Existing tests keep the earlier forecast parameters; tests/test_trust.py
covers the new behaviour with the real losing cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JLWZrfy12imc6dQRsEtFjj
@botta0oss
botta0oss merged commit 2edfffc into main Sep 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants