Data Engineer Interview Challenge

Scenario

You are working with EcoOnline, a SaaS company providing workplace safety software. Customers use the platform to distribute safety questionnaires to their employees and track adherence over time.

You have been given questionnaire_responses.csv — a dataset extracted from a production system containing employee questionnaire responses across multiple companies. The data contains realistic quality issues that you would encounter in a real engagement.

Your Task (25 minutes)

Using your preferred AI coding agent (e.g. Claude Code, GitHub Copilot, Cursor), build a Python script that:

1. Loads and cleans the data, handling any quality issues you find
2. Produces a summary report per company showing:
   • Total responses
   • Completion rate
   • Average score (as a percentage)
   • Most common failed question
3. Outputs the report as a new CSV file
4. Anomaly flagging — Identify any company where the completion rate dropped more than 20 percentage points week-over-week. Output these as a separate anomalies.csv with the company, week, and the size of the drop
5. Unit tests — Write at least 3 meaningful tests covering your cleaning and transformation logic (e.g. date parsing, score normalisation, anomaly detection)

Guidelines

• Talk through your thinking as you work — we are assessing how you approach the problem, not just the final output
• You are expected to use an AI coding agent — this is not a memory test
• Make reasonable assumptions where the requirements are vague, but be ready to explain them
• Validate your output before presenting it

Setup

git clone https://github.com/EcoOnline/data-engineer-interview-challenge.git
cd data-engineer-interview-challenge

Python 3.10+ required. Use any libraries you need.