Propasafe Hybrid

A sentence-level propaganda detection system that combines a locally fine-tuned BERT classifier with LLM-based technique labeling and explainability. Propasafe-Hybrid identifies propagandistic sentences in news articles, assigns specific technique labels from a closed 17-technique taxonomy, and generates concise rationales — all while reducing API token usage through a cost-aware pre-filtering stage that forwards only high-likelihood sentences to the LLM.

Built on top of Propasafe, extending it with an explainability module while preserving its offline, privacy-conscious architecture.

📄 Paper: Propasafe-Hybrid: A Text-Based Hybrid Propaganda Detection Tool — FLAIRS-39, 2026
🤖 BERT Model: Download from Figshare

Architecture

┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│ Browser Extension│────▶│  FastAPI Backend │────▶│  OpenAI API     │
│ (Content Script) │◀────│  + BERT Model    │◀────│  (Batch Request)│
└─────────────────┘     └──────────────────┘     └─────────────────┘

Hybrid Sieve Pipeline

1. Article Extraction: Raw HTML fetched via requests, DOM pre-cleaned with lxml (removes modals, ads, navbars), then text extracted via trafilatura → readability-lxml → newspaper3k cascade. Text normalized and tokenized via NLTK Punkt (max 400 sentences).
2. Local BERT Sieve: All sentences scored by fine-tuned BERT model (batched inference)
3. Threshold Filter: Only sentences with bert_score >= 0.5 proceed to LLM
4. Batched LLM Analysis: Flagged sentences sent to OpenAI in a single request
5. Technique Labeling: LLM identifies specific propaganda techniques with evidence

Quick Start

Prerequisites

• Docker Desktop
• OpenAI API key
• Fine-tuned BERT model (download here) - place in backend/app/model/best_model.keras

Run Backend

# Set API key
export OPENAI_API_KEY="sk-..."

# Start container
docker compose up --build

Backend runs at http://127.0.0.1:8000

Verify Installation

curl http://127.0.0.1:8000/health
# {"ok": true}

API Endpoints

POST /extract-and-classify-hybrid

Primary endpoint. Streams NDJSON progress events.

curl -X POST http://127.0.0.1:8000/extract-and-classify-hybrid \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/article"}' \
  --no-buffer

Response (NDJSON stream):

{"event":"extraction_done","sentence_count":42,"title":"Article Title"}
{"event":"bert_done","threshold":0.5,"flagged_count":6,"total_count":42}
{"event":"openai_request_started","flagged_count":6}
{"event":"results","results":[...]}

Result object:

{
  "sentence": "Original text",
  "bert_score": 0.723,
  "severity": "mild",
  "techniques": ["Loaded Language"],
  "evidence": {"Loaded Language": "Uses emotionally charged terms"},
  "llm_status": "ok"
}

llm_status values:

• ok — LLM confirmed ≥1 technique
• empty — LLM returned no techniques (sentence still highlighted by BERT)
• not_run — BERT score below threshold, LLM not invoked
• error — API call failed

POST /extract-and-classify

BERT-only endpoint (no LLM). Returns JSON array of scored sentences.

POST /explain-sentence

On-demand detailed explanation for a specific sentence.

curl -X POST http://127.0.0.1:8000/explain-sentence \
  -H "Content-Type: application/json" \
  -d '{"sentence":"...", "techniques":["Loaded Language"]}'

Configuration

Variable	Default	Description
OPENAI_API_KEY	required	OpenAI API key
OPENAI_MODEL	gpt-5.2-2025-12-11	OpenAI model for technique labeling
PROPAGANDA_THRESHOLD	0.5	BERT score threshold for LLM analysis
BERT_BATCH_SIZE	32	Batch size for BERT inference
MAX_SENTENCES	400	Maximum sentences per article
PROPASAFE_VERBOSITY	medium	LLM response detail: low/medium/verbose

Verbosity detail: low ≤20 word evidence, medium ≤25 words, verbose ≤30 words + explanation up to 120 words.

Set in .env file or pass as environment variable:

# Option 1: Add to .env file
echo "PROPASAFE_VERBOSITY=verbose" >> .env

# Option 2: Pass directly
PROPASAFE_VERBOSITY=verbose docker compose up

Browser Extension

Chrome extension for in-page propaganda highlighting.

Installation

1. Open chrome://extensions
2. Enable Developer Mode
3. Load unpacked from extension/ directory

Features

• Real-time sentence highlighting (yellow=mild, red=severe)
• Click highlights for analysis card with:
  • BERT score and severity
  • Detected techniques
  • Evidence snippets
  • On-demand detailed explanations
• Popup with pie chart distribution

Project Structure

propasafe-hybrid/
├── backend/app/
│   ├── main.py           # FastAPI endpoints, BERT inference
│   └── openai_client.py  # LLM batching, structured outputs
├── extension/
│   ├── content-script.js # In-page highlighting
│   ├── popup.html/js     # Extension popup with Chart.js
│   └── vendor/           # mark.js, chart.js
├── extractor/
│   └── app.py            # Article extraction
├── Dockerfile
└── docker-compose.yml

BERT Model

Fine-tuned on propaganda detection dataset. Model file: backend/app/model/best_model.keras

Severity thresholds:

• bert_score >= 0.50: Mild (yellow highlight)
• bert_score >= 0.75: Severe (red highlight)

Supported propaganda techniques (17): loaded language · name calling or labeling · repetition · exaggeration or minimization · doubt · appeal to fear/prejudice · flag-waving · causal oversimplification · slogans · appeal to authority · black-and-white fallacy · thought-terminating cliche · whataboutism · reductio ad hitlerum · red herring · bandwagon · obfuscation/intentional vagueness/confusion

Authors

• Thomas Kimmeth — John Jay College of Criminal Justice, CUNY
• Avijit Roy — John Jay College of Criminal Justice, CUNY · GitHub · Website
• Vivek Sharma — John Jay College of Criminal Justice, CUNY

Citation

If you use this work, please cite:

@article{kimmeth2026propasafe,
  title={Propasafe-Hybrid: A Text-Based Hybrid Propaganda Detection Tool},
  author={Kimmeth, Thomas and Roy, Avijit and Sharma, Vivek},
  journal={The International FLAIRS Conference Proceedings},
  volume={39},
  number={1},
  year={2026},
  month={May},
  doi={10.32473/flairs.39.1.141595},
  url={https://journals.flvc.org/FLAIRS/article/view/141595}
}

License

This project is licensed under the MIT License.