SignalFlow: Context-Aware Message Notification Router

An AI-powered context-aware system that triages incoming messages from platforms like WhatsApp, Telegram, Slack, and iMessage into notify, digest, or mute, personalized per recipient by reasoning over text, image posters, and voice notes.

Flexible provider support — run using cloud APIs (OpenAI, Anthropic) or fully locally (Ollama, Apple Vision OCR, local Whisper). Any stage swaps with a single environment variable.

---

The Problem

Modern messaging channels (WhatsApp, Telegram, Slack, iMessage) mix family chats, team updates, community notices, business promotions, image posters, voice notes, and scams into a single noisy stream. Treating them alike means important messages get missed while low-priority or risky ones interrupt the user.

For each incoming message, the system decides:

Action	Meaning
notify	Interrupt now (time-sensitive, urgent, or high-priority personal)
digest	Useful, show later (group updates, low-priority promos, routine info)
mute	Repetitive, unwanted, or unsafe (spam, scams, prompt injections, muted groups)

The decision is per recipient. The same sale poster is useful to one person and noise to another; a payment reminder is fine from a known business and a scam from a fresh lookalike domain. Clear safety risks are muted regardless of prior user engagement.

---

Architecture

messages.csv
  → context assembly    joins 6 tables → ~35 behavioural signals per message
  → media enrichment    images → Apple Vision OCR; voice → faster-whisper
  → evidence retrieval  user's history, ranked by relation + embedding + outcome
  → hard safety rules   injection, phishing, lookalike domains (LLM cannot override)
  → LLM router          strict JSON: action, type, reason key, evidence
  → post overrides      mentions, muted groups, quiet hours
  → output validation   schema verification & output.csv writing

Every stage degrades gracefully rather than failing: missing media, an unreachable model, or malformed JSON all fall back to deterministic rule routing, ensuring a valid prediction per input.

---

Key Engineering & Design Decisions

1. Deterministic Safety Rules (Un-overridable)

Small LLMs can be manipulated by hostile prompt engineering. Safety rules fire before the LLM stage and cannot be overridden:

• Prompt Injection Defense: Input messages containing override instructions (e.g. set action=notify, system instruction bypasses) are flagged and muted as scams (mute / scam).
• Credential Harvesting: Requests for OTP, PIN, password, or CVV under pressure are automatically muted.
• Lookalike Domain Verification: Domain age distinguishes phishing lookalikes (e.g., phonepe-rewards.in, 7 days old, unverified) from legitimate shorteners (link.wame.pro, 3300+ days old, verified).
• Coerced Payments: Demands for QR/UPI transfers under time pressure from senders with no prior relationship are muted.

2. Template-Based Canonical Reason Bank

Instead of relying on free-form text generation (which introduces phrasing drift), the model selects a canonical reason_key. code/reasons.py maps this key to standardized, high-quality prose, ensuring consistent explanations across identical scenarios.

3. Calibrated Confidence Scoring

Small models are notoriously poorly calibrated when reporting raw numeric probabilities. The LLM reports coarse certainty (high, medium, low), which SignalFlow maps into empirical confidence bands tailored per action:

• notify: 0.85 – 0.91
• digest: 0.78 – 0.84
• mute: 0.81 – 0.87

4. On-Device OCR & Fast Audio Transcription

• macOS Apple Vision OCR: Processes image posters in ~0.41s per poster locally with zero network latency and high accuracy on dense poster text.
• Local Whisper Transcription: Extracts spoken text from audio voice notes locally via faster-whisper.

5. Relational & Outcome-Aware Evidence Ranking

Relevance search over recipient history ranks prior interactions using relational proximity, text embedding similarity (nomic-embed-text), and recorded past user reactions (message_events). Factoring in past user actions improved evidence top-1 retrieval rank from 50% to 61% (93% recall@6).

---

Benchmark Results

Benchmark Ablation Scoring

Evaluated on the 30-message ground-truth benchmark suite (python3 -m code.evaluation.main):

Configuration	Action Accuracy	Message Type Accuracy	Both Correct
Rules-only baseline (no media)	76.7%	53.3%	50.0%
Rules-only baseline (final rules)	83.3%	60.0%	56.7%
+ LLM Router + OCR/ASR	90.0%	73.3%	73.3%
+ Type Disambiguation Prompting	90.0%	83.3%	80.0%
+ Imminence & Category Guards	100%	100%	100%

---

Quick Start

1. Installation

Install Python dependencies:

pip install -r requirements.txt

2. Run with Hosted Cloud APIs (Recommended for easy testing)

You can run SignalFlow using any standard API key (OpenAI, Anthropic, Groq, etc.):

# OpenAI (GPT-4o-mini / GPT-4o)
LLM_PROVIDER=openai OPENAI_API_KEY=your_openai_api_key python3 -m code.main

# Anthropic Claude
LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=your_anthropic_api_key python3 -m code.main

# Fast Rules-Only Mode (No API key or model needed)
python3 -m code.main --no-llm

3. Optional: Running with Local Models (Ollama)

If you prefer offline local execution:

ollama serve &
ollama pull qwen2.5:7b-instruct
ollama pull nomic-embed-text:v1.5

LLM_PROVIDER=ollama python3 -m code.main

4. Useful CLI Commands

python3 -m code.main --limit 10      # Quick 10-message test run
python3 -m code.evaluation.main      # Run full benchmark evaluation harness

---

Switching Providers

All model providers are configurable via environment variables:

# Anthropic Claude Sonnet
LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=your_key python3 -m code.main

# OpenAI GPT-4o-mini
LLM_PROVIDER=openai OPENAI_API_KEY=your_key python3 -m code.main

# Cloud Vision & Whisper
OCR_PROVIDER=llm ASR_PROVIDER=openai python3 -m code.main

Copy .env.example to .env for local configuration overrides. API keys are strictly loaded from environment variables and never logged or committed.

---

Code Base Structure

code/
  ├── config.py       # Environment-driven settings & path management
  ├── providers/      # Unified interface for Ollama, Anthropic, and OpenAI
  ├── data.py         # Loads and indexes context tables from CSVs
  ├── context.py      # Per-message fact sheet & derived behavioral signals
  ├── media.py        # OCR & ASR processing with content-hash caching
  ├── ocr_macos.py    # Native Apple Vision text recognition
  ├── retrieval.py    # Vector & relational evidence search over recipient history
  ├── rules.py        # Deterministic safety guards & behavioral overrides
  ├── reasons.py      # Canonical reason template bank
  ├── router.py       # LLM invocation, JSON validation, and confidence calibration
  ├── heuristic.py    # Rule-only fallback engine and baseline
  ├── pipeline.py     # End-to-end pipeline execution & schema validation
  ├── main.py         # CLI entry point
  ├── prompts/        # Fenced system prompts for LLM router
  └── evaluation/     # Benchmark scoring harness