---README.md**

# Adaptive Runtime Quantization for Edge Devices

[![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://python.org)
[![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-red.svg)](https://pytorch.org)
[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)

## Overview

A lightweight framework that **dynamically selects FP32 / INT8 / INT4 precision**
per inference call based on input complexity — without any offline calibration.

Unlike HAWQ, BRECQ, or SmoothQuant (which are static/offline),
this method adapts precision **at runtime** using a tiny scorer network.

---

## Key Contributions

| Feature | Detail |
|---|---|
| InputComplexityScorer | <1% parameter overhead |
| Batch-level routing | minimal latency overhead |
| No calibration dataset needed | fully dynamic |
| Target deployment | MIUI / HyperOS / Snapdragon edge |

---

## Architecture

Input Tokens │ ▼ ┌─────────────────────┐ │ InputComplexityScorer│ ← tiny network, scores 0.0~1.0 └─────────────────────┘ │ ├── score > 0.7 → FP32 (complex input) ├── score > 0.3 → INT8 (medium input) └── score ≤ 0.3 → INT4 (simple input) │ ▼ AdaptiveQuantizedLinear (all transformer layers)

---

## Benchmark Results

| Mode | Latency | vs FP32 |
|---|---|---|
| FP32 | baseline | 1.00x |
| INT8 | faster | ~1.1x |
| INT4 | faster | ~1.2x |
| **Adaptive** | **dynamic** | **>1.0x** |

---

## Quick Start

```bash
git clone https://github.com/YOUR_USERNAME/adaptive-quant.git
cd adaptive-quant
pip install torch numpy
python adaptive_quant.py

---

Results

Unit Tests : 7/7 passed
  ✅ ComplexityScorer
  ✅ AdaptiveLinear
  ✅ QuantizationError
  ✅ CacheValidity
  ✅ EdgeCases
  ✅ FullForwardPass
  ✅ GradientFlow

---

Why This Matters for Xiaomi

• Snapdragon 8 Gen series supports INT4/INT8 via HTP (Hexagon Tensor Processor)
• HyperOS on-device AI benefits directly from adaptive precision
• No cloud dependency — fully on-device inference

---

File Structure

adaptive-quant/
├── adaptive_quant.py     ← main code
├── config.json           ← model config
├── model_weights.pt      ← pretrained weights (~50MB)
├── README.md             ← this file
└── LICENSE

## License

MIT License — free to use, modify, and distribute.