diff --git a/README.md b/README.md index b0643d5..1a65596 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ A production-grade trading bot that models financial markets as curved physical space and finds optimal trades using Feynman's path integral — the same mathematics that governs how particles move through quantum fields. -Every trade decision is authorized by a cryptographic governance protocol, signed with Ed25519, and stored in an immutable Merkle-chained audit trail. +Every trade decision is authorized by a cryptographic governance protocol and stored in a SHA-256 runtime evidence hash chain. Optional evidence bundles support Merkle roots and Ed25519 signatures where that path is used. --- @@ -23,7 +23,8 @@ Instead of simple indicators (RSI, MACD), this system: | Guide | Description | |-------|-------------| -| [Architecture Guide](docs/ARCHITECTURE.md) | System design — physics core, 20-stage pipeline, manifold geometry | +| [Architecture Guide](docs/ARCHITECTURE.md) | System design — physics core, canonical pipeline, manifold geometry | +| [Rootfile Hamiltonian Canon](docs/ROOTFILE_HAMILTONIAN_CANON.md) | One-to-one map from the lawful-collapse canon to the current rootfile runtime | | [API Reference](docs/API.md) | Complete class and method reference for all modules | | [Tutorials](docs/TUTORIAL.md) | Step-by-step guides: paper trading, dashboard, Telegram bot, brokers | | [Examples](docs/examples/README.md) | Working Python code snippets | @@ -38,7 +39,7 @@ Instead of simple indicators (RSI, MACD), this system: RAW MARKET DATA (MT5 / Deriv / TradingView) ↓ ┌─────────────────────────────────────────┐ -│ 20-STAGE PIPELINE │ +│ CANONICAL PIPELINE │ │ 1. Data Ingestion │ │ 2. State Construction (microstructure) │ │ 3. ICT Extraction (OB, FVG, BOS) │ @@ -56,7 +57,7 @@ RAW MARKET DATA (MT5 / Deriv / TradingView) │ 15. Scheduler Collapse ← CIRCUIT BREAKER│ │ 16. Execution (paper / live) │ │ 17. Reconciliation ← PnL DIVERGENCE │ -│ 18. Evidence Emission (Ed25519+Merkle) │ +│ 18. Evidence Emission (SHA-256 chain) │ │ 19. Weight Update (PPO + backward law) │ │ 20. Completed │ └─────────────────────────────────────────┘ @@ -75,7 +76,7 @@ The repository now includes a non-breaking rootfile overlay that makes the archi | `core.orchestration` | Select admissible paths and mint execution authority | `trading.kernel.scheduler`, `trading.kernel.apex_engine`, constraints | | `core.authority` | Canonical `ExecutionToken` facade and token validation | TAEP scheduler tokens plus trading scheduler token compatibility | | `core.execution` | Shadow/live/broker execution boundaries | `trading.shadow`, `apps.telegram.trading_live`, broker adapters | -| `tachyonic_chain` | Evidence and Merkle audit-chain exports | `trading.evidence.evidence_chain` | +| `tachyonic_chain` | Runtime SHA-256 audit-chain exports; optional bundle adapters | `tachyonic_chain.audit_log`, `trading.evidence.evidence_chain` | | `backend_api` | Telegram, dashboard, and read/control surfaces | `apps.telegram`, `trading.dashboard` | The law of motion is: data prepares state, simulation proposes, orchestration authorizes, execution acts, and evidence records. Shadow and live execution boundaries now validate scheduler-issued authority through `core.authority.validate_token(...)`; proposal modules remain token-free so analysis stays cheap and safe. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 95485af..d42dfa1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -6,6 +6,8 @@ ApexQuantumICT treats financial markets as a curved Riemannian manifold and sele The rootfile-first overlay makes that contract explicit without breaking the existing engine. Current modules stay in place, while canonical namespaces describe the architecture in first-order layers: state preparation, proposal generation, authorization, execution, evidence, validation, and API observation. +For the one-to-one mathematical canon, see [Rootfile Hamiltonian Canon](ROOTFILE_HAMILTONIAN_CANON.md). It maps the thirteen lawful-collapse laws to the current pipeline, broker settlement, evidence, and ML feedback paths. + --- ## Rootfile-First Overlay @@ -295,7 +297,9 @@ O1-O18 remain the legacy ICT/SMC operator contract. O19-O25 are analytics-only o --- -## 20-Stage Canonical Pipeline +## Canonical Pipeline + +The active decision loop has nineteen stage handlers plus completion/failure bookkeeping. Older docs may call this a 20-stage pipeline because `COMPLETED` is represented as a terminal stage result. ``` RAW MARKET DATA (MT5 / Deriv / TradingView) @@ -337,7 +341,7 @@ ROOTFILE AUDIT CHAIN (SHA-256 JSONL HASH CHAIN) | 3. ICT_EXTRACTION | Identify order blocks, FVGs, BOS/CHOCH, liquidity zones, session | | 4. GEOMETRY_COMPUTATION | Compute ϕ(p,t) → metric g_ij → Γⁱⱼₖ → curvature K → regime | | 5. TRAJECTORY_GENERATION | RK4 integration → N candidate paths with geodesic-seeded initial conditions | -| 6. RAMANUJAN_COMPRESSION | Cluster paths into behavioral families; reduce redundancy | +| 6. RAMANUJAN_COMPRESSION | Deterministically cluster paths by liquidity/time/entry/risk/topology signatures | | 7. ADMISSIBILITY_FILTERING | Π_total gate: discard paths violating constraints | | 8. ACTION_EVALUATION | Compute S[γ] = w_L·S_L + w_T·S_T + w_E·S_E + w_R·S_R (+ optional S_HFT) for each path | | 9. PATH_INTEGRAL | Weight each path: P ∝ exp(−S/ℏ); calibrate ℏ for ESS≈0.5 | @@ -345,7 +349,7 @@ ROOTFILE AUDIT CHAIN (SHA-256 JSONL HASH CHAIN) | 11. PATH_SELECTION | γ* = argmax weight = argmin action | | 12. PROPOSAL_GENERATION | Extract (direction, entry, stop, target, size, predicted_pnl) from γ* | | 13. ADMISSIBILITY_CHECK | Final risk gate: check_all_limits() from risk manager | -| 14. ENTROPY_GATE | ΔS < threshold: reject if trajectory variance too high | +| 14. ENTROPY_GATE | Measure posterior path uncertainty and information gain; reject if uncertainty is too high | | 15. SCHEDULER_COLLAPSE | Issue ExecutionToken or REFUSE; **circuit breaker** wraps this call | | 16. EXECUTION | Submit to shadow/paper/live broker via ExecutionToken | | 17. RECONCILIATION | PnL divergence: \|predicted − realized\| / max(\|predicted\|, 1) | @@ -353,6 +357,23 @@ ROOTFILE AUDIT CHAIN (SHA-256 JSONL HASH CHAIN) | 19. WEIGHT_UPDATE | PPO reward + backward law: w_new ← Π_simplex(w_old + η·J) | | 20. COMPLETED | Update state, log metrics, prepare for next cycle | +### Lawful-Collapse Stage Map + +```text +raw state -> geometry -> paths -> action -> projectors -> entropy +-> scheduler -> execution -> reconciliation -> evidence -> ML feedback +``` + +| Runtime group | Canon coverage | Primary implementation | +|---------------|----------------|------------------------| +| Raw state | H1 state space | `PipelineContext`, data ingestion, state construction | +| Geometry | H2-H4 geometry, connection, curvature | `trading/geometry/*` | +| Paths | H5-H6 path space and compression | `trading/path_integral/*`, Ramanujan compression stage | +| Action/projectors | H7-H8 scoring and admissibility | `trading/action/*`, `trading/risk/risk_manager.py` | +| Entropy/authority | H9-H10 delta-S and scheduler collapse | entropy gate, `trading/kernel/scheduler.py` | +| Collapse/reconciliation | H11-H12 broker execution and reality check | execution/reconciliation stages, MT5 and Deriv settlement CLIs | +| Evidence/learning | H13 evidence plus ML feedback | `tachyonic_chain/audit_log.py`, PPO pending state, refusal-risk rebuild | + --- ## Production Hardening (T3-A) diff --git a/docs/ROOTFILE_HAMILTONIAN_CANON.md b/docs/ROOTFILE_HAMILTONIAN_CANON.md new file mode 100644 index 0000000..3bcd63f --- /dev/null +++ b/docs/ROOTFILE_HAMILTONIAN_CANON.md @@ -0,0 +1,105 @@ +# Rootfile Hamiltonian Canon + +## Summary + +The rootfile trading runtime is best described as lawful collapse of possible +market futures into auditable state. + +The original canon uses the phrase "nine Hamiltonians", but the working system +maps more accurately to thirteen operational laws. Those thirteen laws are the +canonical architecture layer for this repository. They sit above the active +pipeline stages, broker settlement modules, evidence chain, and ML feedback +path. + +The practical runtime flow is: + +```text +raw state -> geometry -> paths -> action -> projectors -> entropy +-> scheduler -> execution -> reconciliation -> evidence -> ML feedback +``` + +## One-To-One Rootfile Map + +| Canon law | Existing rootfile home | Current implementation meaning | +|---|---|---| +| H1 State space | `PipelineContext`, data ingestion, state construction | MT5, Deriv, and replay inputs become market state, OHLCV, microstructure, symbol, source, and stage context. | +| H2 Geometry | `trading/geometry/liquidity_field.py`, `metric.py` | ICT and microstructure features become liquidity field `phi` and the conformal metric. | +| H3 Connection | `trading/geometry/connection.py` | Christoffel symbols bend candidate trajectories through liquidity gradients. | +| H4 Curvature | `trading/geometry/curvature.py` | Gaussian curvature classifies basin, flat, saddle, and transition regimes. | +| H5 Path space | `trading/path_integral/trajectory_generator.py` | Candidate futures are generated as trajectories through market state space. | +| H6 Ramanujan compression | Pipeline Ramanujan compression stage | Trajectories are grouped into deterministic behavior families using liquidity, time, entry, risk, and topology signatures. | +| H7 Action | Pipeline action evaluation and `trading/action/*` | Paths receive weighted liquidity, time, entry, risk, and curvature-aware costs. | +| H8 Admissibility | Pipeline admissibility stages and `trading/risk/risk_manager.py` | Illegal paths and unsafe proposals are refused before execution. | +| H9 Entropy / delta S | Pipeline entropy gate and scheduler `delta_s` input | The gate measures prior path uncertainty, posterior action-weight uncertainty, and information gain before scheduler collapse. | +| H10 Scheduler authority | `trading/kernel/scheduler.py` and scheduler-collapse stage | The scheduler is the sole authority that may issue an execution token. | +| H11 Collapse / execution | Pipeline execution stage, broker modules, demo runner | Authorized proposals become paper fills, MT5 demo orders, or Deriv demo contracts. | +| H12 Reconciliation | Pipeline reconciliation plus MT5 and Deriv settlement modules | Intended vs actual execution is checked; realized closed-trade PnL feeds settlement and learning. | +| H13 Evidence | `tachyonic_chain/audit_log.py`, `trading/evidence/evidence_chain.py`, audit CLIs | Runtime evidence is a SHA-256 JSONL hash chain; the separate evidence bundle path supports Merkle roots and Ed25519 signatures where used. | + +## Active Pipeline Alignment + +The active `PipelineOrchestrator` runs nineteen decision stage handlers plus +completion or failure bookkeeping. Older summaries call this a 20-stage pipeline +because `COMPLETED` is counted as a terminal stage result. + +| Runtime stage group | Canon coverage | +|---|---| +| Data ingestion and state construction | H1 | +| ICT extraction, liquidity field, metric, connection, curvature | H2, H3, H4 | +| Trajectory generation and path-family compression | H5, H6 | +| Action evaluation, path integral, interference, path selection | H7 | +| Proposal generation and admissibility checks | H8 | +| Entropy gate | H9 | +| Scheduler collapse | H10 | +| Execution | H11 | +| Reconciliation | H12 | +| Evidence emission and weight update | H13 plus ML feedback | + +Broker settlement is deliberately outside the immediate tick-loop collapse +because MT5 positions and Deriv contracts close asynchronously. Settlement is +the bridge from broker reality back into learning: + +| Broker path | Settlement path | Learning rule | +|---|---|---| +| MT5 demo order | `scripts.trading.settle_demo_trades` | Use realized position history, not floating open PnL. | +| Deriv demo contract | `scripts.trading.settle_deriv_contracts` | Use closed contract profit/loss, not proposal-time payout estimates. | + +## Evidence And Anchoring + +There are two evidence layers, and they should not be conflated: + +| Layer | Implementation | What it proves | +|---|---|---| +| Runtime execution evidence | `tachyonic_chain/audit_log.py` | Each JSONL record stores `previous_hash`, canonical payload data, and `record_hash` using SHA-256. | +| Optional evidence bundles | `trading/evidence/evidence_chain.py` | Bundle-level Merkle roots and Ed25519 signatures where that path is used. | + +GitHub commits then publicly anchor the code and curated reports. That is not a +public blockchain transaction; it is public source-control anchoring layered on +top of the local evidence hash chain. + +## What We Have Now + +The merged rootfile system has proven: + +- MT5 demo execution, settlement, evidence, and offline ML rebuild from a closed + take-profit trade. +- Deriv demo execution, settlement, evidence, and offline ML rebuild from a + closed one-contract canary. +- Scheduler-token enforcement before broker execution. +- Durable Deriv PPO pending-state persistence before the canary runner stops. +- V1 Ramanujan path-family signatures in the active pipeline. +- Measured entropy and information-gain reporting into scheduler collapse. +- Post-outcome falsification scoring for closed MT5 and Deriv demo settlements. + +The next proof target is a second guarded Deriv demo contract where settlement +returns PPO feedback as `updated` or `transition_stored` instead of +`missing_pending_state`. + +## Known Weak Spots + +- Ramanujan compression now has deterministic v1 signatures. Future work can + deepen the signatures with richer ICT lineage and empirical family scoring. +- The entropy gate now measures prior/posterior uncertainty. Future work should + calibrate the threshold empirically against the 300-shadow/forward proof set. +- Runtime evidence is SHA-256 hash chained; public-chain notarization would be a + separate explicit feature. diff --git a/tests/rootfile/test_demo_trade_settlement.py b/tests/rootfile/test_demo_trade_settlement.py index 0f25561..1bab485 100644 --- a/tests/rootfile/test_demo_trade_settlement.py +++ b/tests/rootfile/test_demo_trade_settlement.py @@ -122,6 +122,8 @@ def test_settle_demo_trades_writes_ledger_evidence_and_ml_artifacts(tmp_path: Pa assert report.scanned == 1 assert report.closed == 1 assert report.records[0].ppo_feedback_status == "missing_pending_state" + assert report.records[0].falsification_status == "correct_authorization" + assert report.records[0].falsification_hash is not None assert ledger_path.exists() assert evidence_path.exists() assert dataset_path.exists() @@ -129,6 +131,8 @@ def test_settle_demo_trades_writes_ledger_evidence_and_ml_artifacts(tmp_path: Pa rows = read_refusal_risk_dataset(dataset_path) assert rows[0]["realized_pnl"] == 0.9 assert rows[0]["close_reason"] == "tp" + assert rows[1]["event_type"] == "falsification_score" + assert rows[1]["outcome"] == "correct_authorization" assert json.loads(model_path.read_text(encoding="utf-8"))["runtime_integration"] == "offline_only" diff --git a/tests/rootfile/test_deriv_contract_settlement.py b/tests/rootfile/test_deriv_contract_settlement.py index ec92e1d..0bba74c 100644 --- a/tests/rootfile/test_deriv_contract_settlement.py +++ b/tests/rootfile/test_deriv_contract_settlement.py @@ -221,12 +221,16 @@ def test_settle_deriv_contracts_writes_ledger_evidence_and_ml_artifacts(tmp_path assert report.closed == 1 assert report.records[0].status == "closed" assert report.records[0].ppo_feedback_status == "missing_pending_state" + assert report.records[0].falsification_status == "correct_authorization" + assert report.records[0].falsification_hash is not None assert ledger_path.exists() assert verify_execution_evidence_chain(evidence_path).valid rows = read_refusal_risk_dataset(dataset_path) - assert len(rows) == 1 + assert len(rows) == 2 assert rows[0]["broker"] == "deriv" assert rows[0]["realized_pnl"] == 0.95 + assert rows[1]["event_type"] == "falsification_score" + assert rows[1]["outcome"] == "correct_authorization" assert json.loads(model_path.read_text(encoding="utf-8"))["runtime_integration"] == "offline_only" diff --git a/tests/rootfile/test_hamiltonian_missing_pieces.py b/tests/rootfile/test_hamiltonian_missing_pieces.py new file mode 100644 index 0000000..aa07003 --- /dev/null +++ b/tests/rootfile/test_hamiltonian_missing_pieces.py @@ -0,0 +1,153 @@ +"""Focused tests for the implemented Hamiltonian canon gaps.""" + +from __future__ import annotations + +from pathlib import Path + +from tachyonic_chain.audit_log import verify_execution_evidence_chain +from trading.feedback.falsification import append_falsification_evidence, score_decision +from trading.kernel.scheduler import CollapseDecision +from trading.pipeline.orchestrator import PipelineContext, PipelineOrchestrator + + +class _RiskManager: + kill_switch_active = False + + def trigger_kill_switch(self, reason): + self.kill_switch_active = True + self.reason = reason + + +class _PassthroughBreaker: + def call(self, fn, **kwargs): + return True, fn(**kwargs) + + +def _context_with_weights(weights): + context = PipelineContext(symbol="EURUSD", timestamp=1.0, source="test") + context.admissible_paths = [ + {"id": f"traj_{index}", "energy": 0.1, "weight": weight} + for index, weight in enumerate(weights) + ] + return context + + +def test_measured_delta_s_sharpens_when_path_distribution_concentrates(): + orchestrator = PipelineOrchestrator( + risk_manager=_RiskManager(), + use_microstructure=False, + use_weight_learning=False, + ) + + flat = _context_with_weights([1.0, 1.0, 1.0, 1.0]) + sharp = _context_with_weights([1000.0, 1.0, 1.0, 1.0]) + + flat_result = orchestrator._stage_entropy_gate(flat) + sharp_result = orchestrator._stage_entropy_gate(sharp) + + assert flat_result["prior_entropy"] > 0.999 + assert flat_result["delta_s"] > sharp_result["delta_s"] + assert sharp_result["information_gain"] > flat_result["information_gain"] + assert sharp.action_scores["delta_s"] == sharp_result["delta_s"] + assert sharp.action_scores["information_gain"] == sharp_result["information_gain"] + + +def test_scheduler_receives_measured_delta_s_from_context(): + class RecorderScheduler: + config = {"max_entropy": 0.5} + + def __init__(self): + self.seen_delta_s = None + + def authorize_collapse(self, **kwargs): + self.seen_delta_s = kwargs["delta_s"] + return CollapseDecision.REFUSED, None + + scheduler = RecorderScheduler() + orchestrator = PipelineOrchestrator( + scheduler=scheduler, + risk_manager=_RiskManager(), + use_microstructure=False, + use_weight_learning=False, + ) + orchestrator.collapse_breaker = _PassthroughBreaker() + + context = PipelineContext(symbol="EURUSD", timestamp=1.0, source="test") + context.risk_check_passed = True + context.proposal = {"symbol": "EURUSD", "size": 0.01} + context.admissible_paths = [{"id": "traj_a", "energy": 0.1, "action": 0.2}] + context.action_scores["delta_s"] = 0.217 + context.action_scores["information_gain"] = 0.783 + + result = orchestrator._stage_scheduler_collapse(context) + + assert result["decision"] == CollapseDecision.REFUSED.name + assert scheduler.seen_delta_s == 0.217 + + +def test_ramanujan_signatures_are_deterministic_behavior_families(): + orchestrator = PipelineOrchestrator( + risk_manager=_RiskManager(), + use_microstructure=False, + use_weight_learning=False, + ) + context = PipelineContext(symbol="EURUSD", timestamp=1.0, source="test") + context.ict_geometry = { + "fvg_zones": [{"low": 1.10, "high": 1.11}], + "liquidity_pools": [{"price": 1.12}], + "sweeps": [{"side": "buy"}], + "session": "NY AM", + } + context.trajectories = [ + {"id": "up", "path": [(0, 1.1000), (1, 1.1004), (2, 1.1008)]}, + {"id": "down", "path": [(0, 1.1000), (1, 1.0996), (2, 1.0992)]}, + {"id": "turn", "path": [(0, 1.1000), (1, 1.1004), (2, 1.0998)]}, + ] + + first = orchestrator._stage_ramanujan_compression(context) + second = orchestrator._stage_ramanujan_compression(context) + + assert first == second + assert context.path_signatures["up"]["topology"] == "monotonic_bullish" + assert context.path_signatures["down"]["entry"] == "bearish_entry" + assert context.path_signatures["turn"]["topology"] == "oscillating" + assert all("liquidity=fvg1_pool1_sweep1" in key for key in context.path_families) + + +def test_falsification_classifies_refusals_and_authorizations(): + avoided = score_decision("REFUSED", would_have_pnl=-1.0, symbol="EURUSD") + missed = score_decision("REFUSED", would_have_pnl=1.5, symbol="EURUSD") + authorized = score_decision("AUTHORIZED", realized_pnl=0.25, symbol="EURUSD") + + assert avoided.classification == "avoided_loss" + assert avoided.correct is True + assert missed.classification == "missed_opportunity" + assert missed.correct is False + assert authorized.classification == "correct_authorization" + assert authorized.correct is True + + +def test_falsification_evidence_preserves_hash_chain(tmp_path: Path): + evidence_path = tmp_path / "execution_evidence.jsonl" + + first = score_decision( + "REFUSED", + would_have_pnl=-0.5, + symbol="EURUSD", + broker="mt5", + reference_id="refusal_a", + ) + second = score_decision( + "AUTHORIZED", + realized_pnl=-1.0, + symbol="EURUSD", + broker="deriv", + reference_id="contract_a", + ) + + append_falsification_evidence(first, evidence_path) + append_falsification_evidence(second, evidence_path) + + assert first.evidence_hash is not None + assert second.evidence_hash is not None + assert verify_execution_evidence_chain(evidence_path).valid diff --git a/trading/feedback/__init__.py b/trading/feedback/__init__.py index fc794a2..47a4fd3 100644 --- a/trading/feedback/__init__.py +++ b/trading/feedback/__init__.py @@ -17,17 +17,25 @@ settle_deriv_contract, settle_deriv_contracts, ) +from .falsification import ( + FalsificationScore, + append_falsification_evidence, + score_decision, +) __all__ = [ "DerivContractTrade", "DerivSettlementRecord", "DerivSettlementRunReport", "DemoTrade", + "FalsificationScore", "SettlementRecord", "SettlementRunReport", + "append_falsification_evidence", "classify_close_reason", "load_deriv_contract_trades", "load_demo_trades", + "score_decision", "settle_demo_trades", "settle_deriv_contract", "settle_deriv_contracts", diff --git a/trading/feedback/demo_settlement.py b/trading/feedback/demo_settlement.py index 7b9ba49..e23ecbb 100644 --- a/trading/feedback/demo_settlement.py +++ b/trading/feedback/demo_settlement.py @@ -13,6 +13,7 @@ from typing import Any, Dict, Iterable, List, Optional, Sequence, Set from tachyonic_chain.audit_log import append_execution_evidence +from trading.feedback.falsification import append_falsification_evidence, score_decision logger = logging.getLogger(__name__) @@ -61,6 +62,8 @@ class SettlementRecord: close_reason: Optional[str] = None closed_at: Optional[float] = None ppo_feedback_status: str = "not_attempted" + falsification_status: str = "not_scored" + falsification_hash: Optional[str] = None evidence_hash: Optional[str] = None @@ -384,6 +387,23 @@ def settle_demo_trades( ppo_pending_path, ) record.evidence_hash = append_settlement_evidence(record, evidence_log_path) + falsification = score_decision( + "AUTHORIZED", + realized_pnl=record.realized_pnl, + predicted_pnl=record.predicted_pnl, + symbol=record.symbol, + broker="mt5", + reference_id=record.ticket, + metadata={ + "close_reason": record.close_reason, + "source": record.source, + }, + ) + record.falsification_status = falsification.classification + record.falsification_hash = append_falsification_evidence( + falsification, + evidence_log_path, + ) append_settlement_record(record, ledger_path) existing.add(record.ticket) new_closed += 1 diff --git a/trading/feedback/deriv_settlement.py b/trading/feedback/deriv_settlement.py index 72822a0..c01a4e4 100644 --- a/trading/feedback/deriv_settlement.py +++ b/trading/feedback/deriv_settlement.py @@ -12,6 +12,7 @@ from tachyonic_chain.audit_log import append_execution_evidence from trading.feedback.demo_settlement import _build_ml_artifacts +from trading.feedback.falsification import append_falsification_evidence, score_decision logger = logging.getLogger(__name__) @@ -55,6 +56,8 @@ class DerivSettlementRecord: close_reason: Optional[str] = None closed_at: Optional[float] = None ppo_feedback_status: str = "not_attempted" + falsification_status: str = "not_scored" + falsification_hash: Optional[str] = None evidence_hash: Optional[str] = None @@ -455,6 +458,24 @@ def settle_deriv_contracts( ppo_pending_path, ) record.evidence_hash = append_deriv_settlement_evidence(record, evidence_log_path) + falsification = score_decision( + "AUTHORIZED", + realized_pnl=record.realized_pnl, + predicted_pnl=record.predicted_pnl, + symbol=record.symbol, + broker="deriv", + reference_id=record.contract_id, + metadata={ + "close_reason": record.close_reason, + "contract_type": record.contract_type, + "source": record.source, + }, + ) + record.falsification_status = falsification.classification + record.falsification_hash = append_falsification_evidence( + falsification, + evidence_log_path, + ) append_deriv_settlement_record(record, ledger_path) existing.add(record.contract_id) new_closed += 1 diff --git a/trading/feedback/falsification.py b/trading/feedback/falsification.py new file mode 100644 index 0000000..27896c7 --- /dev/null +++ b/trading/feedback/falsification.py @@ -0,0 +1,134 @@ +"""Post-outcome falsification scoring for authorized and refused decisions.""" + +from __future__ import annotations + +import time +from dataclasses import asdict, dataclass, field +from pathlib import Path +from typing import Any, Dict, Optional + +from tachyonic_chain.audit_log import append_execution_evidence + + +AUTHORIZED_DECISIONS = {"authorized", "executed", "filled", "success"} +REFUSED_DECISIONS = {"refused", "blocked", "failed", "rejected"} + + +@dataclass +class FalsificationScore: + """Classification of a decision after forward outcome is known.""" + + decision: str + classification: str + correct: Optional[bool] + symbol: Optional[str] = None + broker: Optional[str] = None + reference_id: Optional[str] = None + realized_pnl: Optional[float] = None + would_have_pnl: Optional[float] = None + predicted_pnl: Optional[float] = None + reason: str = "" + metadata: Dict[str, Any] = field(default_factory=dict) + evidence_hash: Optional[str] = None + + def to_payload(self) -> Dict[str, Any]: + payload = asdict(self) + payload.pop("evidence_hash", None) + payload["pnl_prediction"] = self.predicted_pnl + return payload + + +def _maybe_float(value: Any) -> Optional[float]: + if value is None or value == "": + return None + try: + return float(value) + except (TypeError, ValueError): + return None + + +def score_decision( + decision: str, + *, + realized_pnl: Any = None, + would_have_pnl: Any = None, + predicted_pnl: Any = None, + symbol: Optional[str] = None, + broker: Optional[str] = None, + reference_id: Optional[str] = None, + metadata: Optional[Dict[str, Any]] = None, +) -> FalsificationScore: + """Score an authorized or refused decision after forward outcome data exists.""" + normalized = str(decision or "").strip().lower() + realized = _maybe_float(realized_pnl) + hypothetical = _maybe_float(would_have_pnl) + predicted = _maybe_float(predicted_pnl) + + if normalized in AUTHORIZED_DECISIONS: + if realized is None: + classification = "pending_authorization_outcome" + correct = None + reason = "authorized decision has no realized outcome yet" + elif realized >= 0: + classification = "correct_authorization" + correct = True + reason = "authorized decision closed non-negative" + else: + classification = "bad_authorization" + correct = False + reason = "authorized decision closed negative" + elif normalized in REFUSED_DECISIONS: + if hypothetical is None: + classification = "unscored_refusal" + correct = None + reason = "refused decision has no forward outcome estimate" + elif hypothetical < 0: + classification = "avoided_loss" + correct = True + reason = "refusal avoided a negative forward outcome" + elif hypothetical > 0: + classification = "missed_opportunity" + correct = False + reason = "refusal skipped a positive forward outcome" + else: + classification = "correct_refusal" + correct = True + reason = "refusal avoided a neutral forward outcome" + else: + classification = "unscored_decision" + correct = None + reason = "decision type is not recognized for falsification scoring" + + return FalsificationScore( + decision=str(decision), + classification=classification, + correct=correct, + symbol=symbol, + broker=broker, + reference_id=str(reference_id) if reference_id is not None else None, + realized_pnl=realized, + would_have_pnl=hypothetical, + predicted_pnl=predicted, + reason=reason, + metadata=metadata or {}, + ) + + +def append_falsification_evidence( + score: FalsificationScore, + evidence_log_path: str | Path, +) -> str: + """Append falsification scoring evidence without touching broker state.""" + reference = score.reference_id or f"{int(time.time())}" + record_hash = append_execution_evidence( + event_type="falsification_score", + execution_id=f"falsification_{reference}", + operation="forward_outcome_falsification", + symbol=score.symbol, + outcome=score.classification, + token_status="post_outcome_verified" if score.correct is not None else "outcome_missing", + payload=score.to_payload(), + log_path=evidence_log_path, + ) + score.evidence_hash = record_hash + return record_hash diff --git a/trading/pipeline/orchestrator.py b/trading/pipeline/orchestrator.py index 746591d..79e383a 100644 --- a/trading/pipeline/orchestrator.py +++ b/trading/pipeline/orchestrator.py @@ -88,6 +88,8 @@ class PipelineContext: ict_geometry: Dict = field(default_factory=dict) geometry_data: Dict = field(default_factory=dict) # Riemannian geometry trajectories: List[Dict] = field(default_factory=list) + path_families: Dict[str, List[str]] = field(default_factory=dict) + path_signatures: Dict[str, Dict[str, str]] = field(default_factory=dict) admissible_paths: List[Dict] = field(default_factory=list) action_scores: Dict = field(default_factory=dict) selected_path: Optional[Dict] = None @@ -629,15 +631,131 @@ def _stage_trajectory_generation(self, context: PipelineContext) -> Dict: } def _stage_ramanujan_compression(self, context: PipelineContext) -> Dict: - """Stage 5: Compress paths into families""" - # Group trajectories by behavior type - families = { - 'sweep_continuation': context.trajectories[:2], - 'reversal': context.trajectories[2:4], - 'consolidation': context.trajectories[4:], + """Stage 5: Compress paths into deterministic behavior families.""" + families: Dict[str, List[str]] = {} + signatures: Dict[str, Dict[str, str]] = {} + + for index, trajectory in enumerate(context.trajectories): + trajectory_id = str(trajectory.get('id') or f'traj_{index}') + trajectory['id'] = trajectory_id + + signature = self._path_signature(trajectory, context) + family_key = "|".join( + f"{name}={signature[name]}" + for name in ("liquidity", "time", "entry", "risk", "topology") + ) + trajectory['signature'] = signature + trajectory['family'] = family_key + signatures[trajectory_id] = signature + families.setdefault(family_key, []).append(trajectory_id) + + context.path_families = families + context.path_signatures = signatures + + return { + 'families': sorted(families), + 'family_count': len(families), + 'signatures': signatures, + } + + @staticmethod + def _trajectory_points(path: Any) -> List[Tuple[float, float]]: + """Normalize trajectory path points into (time, price) pairs.""" + points: List[Tuple[float, float]] = [] + if not isinstance(path, list): + return points + + for index, point in enumerate(path): + try: + if isinstance(point, dict): + timestamp = float(point.get('timestamp', point.get('t', index))) + price = float(point.get('price', point.get('p'))) + else: + timestamp = float(point[0]) + price = float(point[1]) + except (TypeError, ValueError, IndexError, KeyError): + continue + points.append((timestamp, price)) + return points + + @staticmethod + def _count_items(value: Any) -> int: + if isinstance(value, (list, tuple, set)): + return len(value) + if isinstance(value, dict): + return len(value) + return 1 if value else 0 + + def _path_signature(self, trajectory: Dict, context: PipelineContext) -> Dict[str, str]: + """Build the v1 Ramanujan path-family signature.""" + points = self._trajectory_points(trajectory.get('path', [])) + prices = [price for _, price in points] + start = prices[0] if prices else 0.0 + end = prices[-1] if prices else start + delta = end - start + tolerance = max(abs(start) * 1e-5, 1e-9) + + if abs(delta) <= tolerance: + direction = "flat" + elif delta > 0: + direction = "bullish" + else: + direction = "bearish" + + price_deltas = [ + prices[i + 1] - prices[i] + for i in range(len(prices) - 1) + if abs(prices[i + 1] - prices[i]) > tolerance + ] + signs = [1 if item > 0 else -1 for item in price_deltas] + turns = sum(1 for i in range(len(signs) - 1) if signs[i] != signs[i + 1]) + if not price_deltas or direction == "flat": + topology = "flat" + elif turns == 0: + topology = f"monotonic_{direction}" + else: + topology = "oscillating" + + max_excursion = max((abs(price - start) for price in prices), default=0.0) + if max_excursion <= tolerance * 2: + risk = "low" + elif max_excursion <= tolerance * 8: + risk = "medium" + else: + risk = "high" + + ict = context.ict_geometry or {} + fvg_count = self._count_items( + ict.get('fvg_zones') + or ict.get('fair_value_gaps') + or ict.get('fvg') + ) + pool_count = self._count_items( + ict.get('liquidity_pools') + or ict.get('liquidity_zones') + or ict.get('pools') + ) + sweep_count = self._count_items(ict.get('sweeps') or ict.get('sweep')) + liquidity = ( + f"fvg{min(fvg_count, 3)}" + f"_pool{min(pool_count, 3)}" + f"_sweep{min(sweep_count, 3)}" + ) + + session = ( + ict.get('session') + or context.market_state.get('session') + or context.raw_data.get('session') + or "unknown" + ) + + return { + "liquidity": liquidity, + "time": str(session).lower().replace(" ", "_"), + "entry": f"{direction}_entry", + "risk": risk, + "topology": topology, } - - return {'families': list(families.keys())} def _stage_admissibility_filtering(self, context: PipelineContext) -> Dict: """Stage 6: Π_total - Filter illegal paths""" @@ -957,20 +1075,104 @@ def _stage_admissibility_check(self, context: PipelineContext) -> Dict: return {'admissible': True, 'risk_ok': True, 'risk_level': risk_check.level.value} def _stage_entropy_gate(self, context: PipelineContext) -> Dict: - """Stage 13: ΔS check - information gain threshold""" - # Mock entropy calculation - delta_s = 0.3 # Would compute from path variance - - threshold = 0.5 - passed = delta_s < threshold + """Stage 13: measured uncertainty gate for scheduler collapse.""" + metrics = self._measure_path_uncertainty(context) + delta_s = metrics['posterior_entropy'] + information_gain = metrics['information_gain'] + + context.action_scores['delta_s'] = delta_s + context.action_scores['information_gain'] = information_gain + context.action_scores['prior_entropy'] = metrics['prior_entropy'] + context.action_scores['posterior_entropy'] = metrics['posterior_entropy'] + + threshold = float(getattr(self.scheduler, 'config', {}).get('max_entropy', 0.5)) + passed = delta_s <= threshold self._audit_gate( "stage13_entropy", "passed" if passed else "failed", delta_s=f"{delta_s:.4f}", + information_gain=f"{information_gain:.4f}", + posterior_entropy=f"{metrics['posterior_entropy']:.4f}", + prior_entropy=f"{metrics['prior_entropy']:.4f}", threshold=f"{threshold:.4f}", ) - return {'delta_s': delta_s, 'passed': passed} + return {**metrics, 'delta_s': delta_s, 'threshold': threshold, 'passed': passed} + + @staticmethod + def _normalized_entropy(scores: List[float]) -> float: + clean_scores: List[float] = [] + for score in scores: + try: + value = float(score) + except (TypeError, ValueError): + continue + if np.isfinite(value) and value > 0.0: + clean_scores.append(value) + + clean = np.array(clean_scores, dtype=float) + if clean.size <= 1: + return 0.0 + + total = float(clean.sum()) + if total <= 0.0: + return 1.0 + + probs = clean / total + entropy = -float(np.sum(probs * np.log(probs + 1e-12))) + return max(0.0, min(1.0, entropy / float(np.log(clean.size)))) + + def _posterior_scores(self, context: PipelineContext) -> List[float]: + weights: List[float] = [] + for path in context.admissible_paths: + try: + weight = float(path.get('weight', 0.0)) + except (TypeError, ValueError): + continue + if np.isfinite(weight) and weight > 0.0: + weights.append(weight) + if weights: + return weights + + actions = [] + for path in context.admissible_paths: + try: + action = float(path.get('action')) + except (TypeError, ValueError): + path_id = path.get('id') + result = context.action_scores.get(path_id, {}) + try: + action = float(result.get('total_action')) + except (TypeError, ValueError, AttributeError): + continue + if np.isfinite(action): + actions.append(action) + + if actions: + values = np.array(actions, dtype=float) + shifted = values - float(values.min()) + scale = max(float(values.std()), float(getattr(context, '_epsilon', 0.015)), 1e-6) + return [float(np.exp(-item / scale)) for item in shifted] + + fallback_scores = [] + for path in context.admissible_paths: + try: + energy = abs(float(path.get('energy', 0.0))) + except (TypeError, ValueError): + energy = 0.0 + fallback_scores.append(1.0 / (1.0 + energy)) + return fallback_scores + + def _measure_path_uncertainty(self, context: PipelineContext) -> Dict[str, float]: + path_count = len(context.admissible_paths) + prior_entropy = self._normalized_entropy([1.0] * path_count) + posterior_entropy = self._normalized_entropy(self._posterior_scores(context)) + information_gain = max(0.0, prior_entropy - posterior_entropy) + return { + "prior_entropy": prior_entropy, + "posterior_entropy": posterior_entropy, + "information_gain": information_gain, + } def _stage_scheduler_collapse(self, context: PipelineContext) -> Dict: """Stage 15: Scheduler authorization (Λ) — requires prior risk gate passage.""" @@ -1030,6 +1232,8 @@ def _stage_scheduler_collapse(self, context: PipelineContext) -> Dict: "stage15_scheduler", "passed" if decision == CollapseDecision.AUTHORIZED else "refused", decision=decision.name, + delta_s=f"{delta_s:.4f}", + information_gain=f"{context.action_scores.get('information_gain', 0.0):.4f}", symbol=context.symbol, token=token.token_id if token else None, ) @@ -1202,7 +1406,7 @@ def _stage_reconciliation(self, context: PipelineContext) -> Dict: ) self.scheduler.update_action_weights( pnl=-pnl_divergence * 10, - delta_s=0.3, + delta_s=float(context.action_scores.get('delta_s', 0.3)), status='mismatch', contrib={'L': 25, 'T': 25, 'E': 25, 'R': 25}, constraints_passed=False, @@ -1283,7 +1487,7 @@ def _stage_weight_update(self, context: PipelineContext) -> Dict: # Update weights — use actual gate results, not hardcoded True result = self.scheduler.update_action_weights( pnl=pnl, - delta_s=0.3, + delta_s=float(context.action_scores.get('delta_s', 0.3)), status=context.reconciliation_status, contrib=contrib, constraints_passed=getattr(context, 'risk_check_passed', True),