Hybrid Post-Quantum Trust Establishment for Secure Messaging

A proof-of-concept synchronous trust-establishment protocol designed for Signal-style secure messaging architectures. Implemented entirely in Rust, this project enforces mandatory out-of-band user verification using Short Authentication Strings (SAS). It establishes forward-secret key material via a Hybrid Key Encapsulation Mechanism (KEM) to mitigate Man-in-the-Middle (MitM) attacks without relying on trusted relay servers.

🚀 Core Engineering & Security Features

This implementation prioritizes memory safety, side-channel attack (SCA) mitigation, and strict control flow:

• Memory Zeroization: Sensitive structures and intermediate Finite State Machine (FSM) states are explicitly wiped from memory immediately after use via the Zeroize and ZeroizeOnDrop traits.
• Constant-Time Execution: Decapsulation and MAC validation use the subtle crate to ensure constant-time operations, neutralizing timing attacks.
• Implicit Rejection: The Hybrid KEM mitigates validation leaks by deterministically deriving a "fake" key using an internal random seed and an HKDF if the MAC is invalid.
• Compile-Time State Safety: The protocol's FSM is enforced using Rust's typestate pattern, encapsulating protocol steps within a generic Session<S> structure to prevent arbitrary state modifications.
• Domain Separation: Hash-reuse attacks are thwarted by integrating constant byte strings into SHA-256 hash updates based on the protocol step.

🔐 Cryptographic Architecture

The protocol outputs two mutually authenticated 32-byte shared secrets ($SK_{ec}$ and $SK_{seka}$) alongside a Diffie-Hellman ratchet public key to initialize standard messaging ratchets.

• Hybrid KEM Combiner: Implements the XOR-then-MAC (XtM) secure combiner.
• Post-Quantum KEM: Utilizes ML-KEM for IND-CCA security against quantum adversaries with classical access.
• Classical KEM: Utilizes Elliptic-Curve DHKEM.
• Authentication & Integrity: Employs Ed25519 for EUF-CMA digital signatures and Poly1305 (via XOR adaptation) for the deterministic MAC.
• Commitment Scheme: Uses a hash-then-open instantiation with SHA-256 for secure SAS verification.

⚙️ Protocol Control Flow

• The FSM is managed by a custom Protocol Controller that maintains the ActiveSession enum and exposes a central handle_message() method.
• State transitions are strictly validated at compile time and runtime to guarantee a secure control flow.

🛠️ Integration & Build

This module implements standardized traits from the kem crate (RustCrypto), such as TryDecapsulate and Encapsulate, acting as a drop-in replacement for standard cryptographic APIs.

📖 Threat Model

The architecture was designed following the OWASP threat modeling methodology. It addresses:

• Theoretical vulnerabilities: Mitigates Dolev-Yao active adversaries and passive quantum adversaries (harvest now, decrypt later).
• Practical threats: Defends against RAM-based threats and execution monitoring via hardware compromise.