Explainable detection of synthetic singing in complete music mixtures.
SingerShield is a reproducible research project and deployable prototype for distinguishing bonafide singing from deepfake singing. It trains directly on song mixtures from the SingFake benchmark, reports performance on unseen singers and languages, and exposes the trained model through a CLI, REST API, and web interface.
Status: the repository is complete, but it intentionally does not ship a trained checkpoint or copyrighted audio. The SingFake authors do not release checkpoints for copyright reasons. Train
artifacts/final.ptwith the documented command before starting inference.
Speech deepfake detectors do not automatically generalize to singing. Background music masks synthesis artifacts, vocal acoustics differ from normal speech, and a detector can silently learn the identity, language, or codec instead of learning deepfake evidence. SingerShield treats these as measurable research problems rather than presenting a probability as proof.
The default experiment asks:
Can a lightweight audio model detect synthetic singing in complete music mixtures and generalize to unseen singers and languages?
- compact log-mel CNN trained on unseparated music mixtures;
- no generated training examples, destructive preprocessing, or data augmentation;
- non-destructive on-the-fly resampling and windowing;
- automatic validation threshold chosen on the validation split;
- F1, ROC-AUC, average precision, EER, calibration error, and confusion matrix;
- grouped reports for SingFake T01–T04, language, and synthesis source;
- window-level evidence timeline for long recordings;
- command-line interface, FastAPI service, and Gradio interface;
- Docker, GitHub Actions, tests, model card, dataset guide, and responsible-use notes.
flowchart TD
A["Song mixture"] --> B["In-memory windows"]
B --> C["Log-mel frontend"]
C --> D["Compact CNN"]
D --> E["Window probabilities"]
E --> F["Song score + reliability"]
F --> G["CLI / API / Web UI"]
The original audio files are never overwritten. During training, the loader converts a file to mono, resamples it in memory, and selects a fixed-duration window. During inference, overlapping windows produce an evidence timeline and their probabilities are averaged into a song-level score.
SingerShield/
├── configs/ Training configurations
├── data/ Local dataset mount; ignored by Git
├── artifacts/ Checkpoints and reports; ignored by Git
├── docs/ Dataset, model, experiment, and deployment notes
├── src/singershield/ Package source
├── tests/ Unit and inference-contract tests
├── .github/workflows/ci.yml Continuous integration
├── Dockerfile
├── docker-compose.yml
└── pyproject.toml
Python 3.10 or 3.11 is recommended.
git clone https://github.com/your-username/SingerShield.git
cd SingerShield
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install -e ".[all,dev]"PyTorch installation can depend on the CUDA version. If the default wheel is not appropriate, install PyTorch using its official selector first, then repeat the editable installation.
The repository includes workspace settings, recommended extensions, test discovery, reusable
tasks, and launch configurations in .vscode/. After selecting the .venv interpreter:
- use Terminal → Run Task to install dependencies, run tests, validate the manifest, or train;
- use Run and Debug to start quick training, evaluation, the web interface, or the REST API;
- use the beaker icon in the Activity Bar to run or debug individual tests.
SingerShield uses the research-only SingFake dataset. Download the paper version from the official website after accepting its terms. Do not commit or redistribute the audio.
The scanner recognizes a split and label anywhere in the path. A compatible layout is:
data/raw/
├── train/
│ ├── bonafide/
│ └── spoof/
├── validation/
│ ├── bonafide/
│ └── spoof/
├── T01/
├── T02/
├── T03/
└── T04/
Each test folder still needs a bonafide or spoof component unless those labels are provided
through the annotation CSV. Create and validate the manifest:
singershield manifest scan \
--audio-root data/raw \
--output data/manifest.csv
singershield validate \
--manifest data/manifest.csv \
--audio-root data/rawIf the downloaded release includes a CSV with local filenames:
singershield manifest import \
--annotations data/singfake_annotations.csv \
--audio-root data/raw \
--output data/manifest.csvThe importer recognizes common columns such as filename, Bonafide Or Spoof, Set, singer,
language, and model. URL-only rows are reported and skipped; SingerShield does not scrape or
circumvent access controls. See docs/DATASET.md for the manifest contract.
Start with the quick configuration:
singershield train --config configs/quick.yamlOutputs:
artifacts/
├── final.pt
├── history.csv
├── resolved_config.yaml
└── training_summary.json
final.pt contains the model weights, architecture, decision threshold, label mapping, and
validation metrics. It does not contain training audio.
For the longer experiment:
singershield train --config configs/full.yamlsingershield evaluate \
--checkpoint artifacts/final.pt \
--manifest data/manifest.csv \
--audio-root data/raw \
--split test \
--output artifacts/evaluationThe evaluation directory contains predictions.csv, metrics.json, an ROC curve, a confusion
matrix, and a calibration plot. If the manifest includes metadata, metrics.json also contains
separate results for T01–T04, languages, and synthesis sources.
singershield predict example.wav --checkpoint artifacts/final.ptExample response shape (numbers are illustrative, not a claimed model result):
{
"label": "deepfake",
"deepfake_probability": 0.81,
"confidence": 0.81,
"threshold": 0.57,
"reliability": "medium",
"duration_seconds": 30.0,
"windows": []
}singershield serve-web \
--checkpoint artifacts/final.pt \
--host 127.0.0.1 \
--port 7860The interface displays the song-level result, calibrated threshold, reliability label, and a
probability timeline. Public sharing is disabled unless --share is explicitly supplied.
singershield serve-api \
--checkpoint artifacts/final.pt \
--host 0.0.0.0 \
--port 8000curl -X POST http://localhost:8000/predict \
-F "file=@example.wav"Health check: GET /health. Interactive documentation: http://localhost:8000/docs.
Place a trained checkpoint at artifacts/final.pt, then run:
docker compose up --build- API:
http://localhost:8000 - Web UI:
http://localhost:7860
python -m pytest
python -m ruff check src testsThe pure data/configuration tests can also run without ML dependencies:
PYTHONPATH=src python -m unittest discover -s testsA high score is not proof that a named artist used AI. A low score is not proof of authenticity. Expected failure modes include unseen synthesis methods, language shift, codec artifacts, instrumental passages, unusual vocal effects, source separation artifacts, and domain mismatch.
Use the system for research, triage, and human review. Never use it as the sole basis for content removal, payment holds, legal allegations, or reputational claims.
- Record the dataset release and accepted terms.
- Validate that singers do not unintentionally leak across splits.
- Report the random seed and complete YAML configuration.
- Select the decision threshold on validation data only.
- Keep T01–T04 separate in the final report.
- Report calibration and EER in addition to accuracy/F1.
- Publish code and aggregate predictions, not copyrighted audio.
- Fill in docs/MODEL_CARD.md after training.
If you use SingFake, cite the original authors:
Yongyi Zang, You Zhang, Mojtaba Heydari, and Zhiyao Duan. “SingFake: Singing Voice Deepfake Detection.” IEEE ICASSP, 2024.
SingerShield code is released under the MIT License. SingFake and every downloaded recording remain governed by their original terms; the dataset is not covered by this repository’s license.