Technology Trends Analysis: Stack Overflow Developer Survey

An end-to-end data analytics project examining current and emerging technology trends based on the Stack Overflow Annual Developer Survey, completed as part of the IBM Data Analytics Professional Certificate on Coursera.

Project Overview

This project analyses survey responses from over 65,000 software developers across 185 countries to answer key questions about the state of the technology industry:

• Which programming languages, databases, platforms and frameworks are developers currently using?
• Which technologies are growing in demand — and which are declining?
• How does compensation vary across countries, regions and experience levels?
• What drives developer job satisfaction — and is compensation one of those drivers?
• How do work arrangement preferences (remote / hybrid / in-person) vary by role and region?
• Who are the survey respondents, and what does the developer demographic look like?

Key Findings

• JavaScript, HTML/CSS, Python and SQL form a universally valued technology core across all regions and roles
• Rust and Go are the standout emerging languages — both appear prominently in "want to learn" rankings despite relatively low current usage
• PostgreSQL leads both current usage and future desire among databases, signalling genuine developer preference rather than legacy inertia
• North America — and the USA specifically — leads compensation by a wide margin; the Netherlands stands out among European countries
• Compensation barely predicts job satisfaction (Pearson r ≈ 0.10) — one of the most counterintuitive findings in the dataset
• Hybrid work is the new baseline expectation — preferred across all regions and most developer roles, with fully remote second and in-person last
• Asia shows the highest in-person work rate of any region; South America and North America lead in remote adoption

Project Structure

├── surveyDataAnalysis.ipynb    # Main analysis notebook (all sections)
├── popular-languages.csv       # Scraped language salary data
├── survey_data.sqlite          # SQLite database (created from main dataset)
├── /images                     # Cognos dashboard screenshots
├── dashboard.html              # Individual interactive chart exports
├── dashboard.pdf               # PDF dashboard exports
├── .gitignore
└── README.md

Notebook Structure

Section	Content
1	Introduction & business questions
2	Data collection (CSV, web scraping, SQLite)
3	Data wrangling (cleaning, standardization, feature engineering)
4	Exploratory data analysis
5	Key findings & recommendations
6	Interactive Dash dashboard

Data Sources

• Primary dataset: Stack Overflow Developer Survey (educational version provided by IBM/Coursera)
• Supplementary: Programming language salary data scraped from an IBM Skills Network HTML page
• Note: The dataset is an artificially created educational version. Results should not be used for real-world decision making without validation against the original Stack Overflow Public Dataset

Tools & Technologies

Category	Tools
Data manipulation	Python, Pandas, NumPy
Visualisation	Matplotlib, Seaborn, Plotly
Dashboard	Dash, Plotly
Data collection	Requests, BeautifulSoup
Database	SQLite, sqlite3
BI Tool	IBM Cognos Analytics
Environment	Jupyter Notebook

Dashboard

The project includes an interactive dashboard built with Dash, mirroring the IBM Cognos Analytics dashboard produced during the course. The dashboard has three tabs:

• Current Technology Usage — top languages, databases, platforms (word cloud) and web frameworks
• Future Technology Trends — desired languages, databases, platforms (treemap) and frameworks
• Demographics — age distribution (pie), respondent map, education level (line chart), age × education (stacked bar)

Running the Dashboard

# Install dependencies
pip install pandas matplotlib seaborn plotly dash wordcloud

# Run the notebook — the dashboard cell starts a local server
# Open http://127.0.0.1:8050 in your browser

Installation & Usage

# Clone the repository
git clone https://github.com/vshimano/Technology_Trends_Analysis.git
cd Technology_Trends_Analysis

# Install required libraries
pip install pandas matplotlib seaborn plotly dash wordcloud requests beautifulsoup4

# Launch Jupyter
jupyter notebook

# Open surveyDataAnalysis.ipynb and run all cells

> **Note:** The SQLite database file is not included in the repository due to 
> file size. It is generated automatically when you run the notebook — 
> Section 2 creates it from the main dataset.

Note for Windows users: The notebook uses urllib.request to download the SQLite database file — no additional tools required. The wget command used in the original course environment is not needed.

Cognos Analytics Dashboard

In addition to the Python dashboard, a separate dashboard was built in IBM Cognos Analytics as part of the course requirements. Screenshots of the Cognos dashboard and a PDF file are included below for reference.

Dashboard HTML Download full dashboard PDF

Author

Victoria Shimanovich
PhD in Applied Mathematics | Computer Science Educator | Data Analytics
[ https://www.linkedin.com/in/victoria-shimanovich-a7309764/] | [https://github.com/vshimano]

Acknowledgements

• Dataset and course structure: IBM Data Analytics Professional Certificate on Coursera
• Survey methodology: Stack Overflow Annual Developer Survey