The Evolution of Lyrics in German Rap

NLP project that uses Latent Dirichlet Allocation (LDA) to discover latent topics in ~7,500 German rap songs and track how the lyrical landscape has evolved over 30 years.

Overview

We build an LDA topic model on lyrics from 77 German rap artists, scraped via the Genius API. The model identifies five topics — Street, Sex & Party, Love & Life, Competition, and Lifestyle — and we use them to:

• Visualize how dominant topics in German rap shifted from the early 90s to 2020
• Weight the timeline by Spotify popularity to capture listener preferences
• Trace the career arc of rapper Sido, showing his transition from gangsta rap to mainstream pop
• Cluster all 77 artists in 2D space based on their lyrical topic distributions (via PCA)

Key Findings

• Love & Life dominated the 90s when rap entered German mainstream culture
• Competition peaked around 2005, correlating with the era of public rap battles (beefs)
• Lifestyle (trap) has grown rapidly since ~2010, driven by artists like Ufo361, RIN and Yung Hurn
• Street rap saw a revival starting in 2009 with Haftbefehl, amplified by Gzuz, Bonez MC and RAF Camora
• Spotify listeners show a strong bias toward Street and Sex & Party topics

Project Structure

├── Project - Main.ipynb        # Full pipeline: scraping, preprocessing, modeling, visualization
├── Project - Experiments.ipynb  # Model selection experiments (LDA vs NMF, coherence scores)
├── all_songs.pkl                # Pre-built dataset (~7,500 songs with metadata and topic labels)
├── rapper.ods                   # Artist selection spreadsheet
├── maske.png                    # Word cloud mask image
└── report/
    ├── report.pdf               # Full academic report (IEEE format)
    └── report.tex               # LaTeX source

Tech Stack

• Topic Modeling: Gensim (LDA, NMF, coherence evaluation)
• NLP: spaCy (German tokenization, lemmatization, stop word removal)
• Data: Genius API (lyrics), Spotify API (popularity scores)
• Visualization: matplotlib, pyLDAvis, WordCloud
• Clustering: scikit-learn (KMeans, TF-IDF)

Usage

The dataset is included as all_songs.pkl, so you can reproduce all results without re-scraping. Open Project - Main.ipynb and run from the cell that loads the pickle file.

To re-scrape lyrics from scratch, you'll need API tokens for Genius and Spotify.

Context

University project for the course TDDE16 (Text Mining) at Linköping University.