Data-Cleaning App

Version 2.0.1 — final stable release

Data-Cleaning App is a CustomTkinter desktop application for importing CSV files, identifying data-quality problems, previewing selected cleaning rules, applying targeted corrections, and safely exporting cleaned and review-required data.

Important

AI-assisted development disclosure: I defined the project requirements, selected the cleaning and saving workflows, chose interface priorities, tested each iteration, reported defects and edge cases, and directed revisions through the final release. ChatGPT generated the implementation code in response to that direction and feedback. I am publishing this project as an example of requirements development, product direction, iterative testing, quality assurance, and responsible AI-assisted software development—not as code I wrote independently from scratch.

My role

• Selected and refined the project direction for a practical CSV-cleaning desktop application.
• Defined the validation rules, cleaning options, correction workflow, save behavior, and user-interface priorities.
• Tested the application with employee and non-employee CSV files, malformed inputs, missing values, duplicates, and save/backup scenarios.
• Reported visual and functional problems, including table alignment, clipped columns, filename behavior, and review-file handling.
• Directed the Version 2.0.1 visual corrections and validated the final submission-ready workflow on Ubuntu.

Main features

• CSV import with clear validation for empty, malformed, header-only, one-column, blank-header, duplicate-header, and wrong-file-type inputs.
• Automatic analysis of whitespace, missing values, duplicate rows, formatting inconsistencies, and employee-data issues.
• Original Data, Cleaned Data, and Problems Found views with independent search and filtering.
• Preview-first cleaning that leaves the source file unchanged until the user explicitly saves.
• Formatting rules for whitespace, departments, phone numbers, dates, salaries, and email addresses.
• Type-aware missing-value strategies for text, numeric, and date columns.
• Formatting and missing-value scope for all compatible columns or one selected column.
• Suggested corrections, manual cell corrections, and explicit Leave for Review behavior.
• Safe Save New Copy and Replace Original File workflows with timestamped names and backups.
• Companion review files containing unresolved records.
• Content-aware table sizing, aligned headings and values, and horizontal scrolling for wide datasets.

Main workflow

1. Open a CSV file.
2. Review the automatic analysis and detected problems.
3. Search or filter the Problems Found view.
4. Select automatic cleaning rules and optional column scope.
5. Accept suggestions, enter manual corrections, or leave selected issues for review.
6. Preview the cleaned data without changing the source file.
7. Review changes and unresolved issues.
8. Save a timestamped new copy or replace the original after creating a backup.
9. Optionally export unresolved rows to a companion review file.

Screenshots

Original data and cleaning controls

Problems Found and selected-problem correction

Cleaned-data preview

Remaining issues requiring review

Version 2.0.1 improvements

Version 2.0.1 is a focused visual-correction release. It adds:

• A wider Original Row # column so the heading remains visible.
• Wider email and phone columns based on displayed values.
• Content-aware column sizing using representative rows from the current CSV.
• Matching alignment for every heading and the data beneath it.
• Left alignment for text, centered compact values, and right alignment for salary.
• Fixed column widths with horizontal scrolling for stable review layouts.

Type-aware missing-value rules

Text columns

• Leave blank.
• Replace with a custom value, defaulting to N/A.

Numeric columns

• Leave blank.
• Replace with 0.
• Replace with the column mean.
• Replace with the column median.

Salary replacements retain two decimal places. Age and performance-score replacements are rounded to whole numbers.

Date columns

• Leave blank.
• Replace with a valid custom date.

Custom dates are validated and standardized to YYYY-MM-DD before being applied.

Safe saving behavior

Save New Copy

The suggested filename uses a timestamp:

employee_data_raw_cleaned_20260715_163500.csv

The related review file uses the same base name:

employee_data_raw_cleaned_20260715_163500_review_required.csv

Replace Original File

The source is backed up before replacement:

employee_data_raw_backup_20260715_163500.csv

A timestamped review file is also created when unresolved records remain.

Installation

Requirements

• Python 3.10 or newer
• Tkinter
• CustomTkinter
• pandas
• PyTest for the automated test suite

On Ubuntu, install Tkinter if necessary:

sudo apt update
sudo apt install python3-tk

Run the application

git clone https://github.com/YOUR-USERNAME/data-cleaning-app.git
cd data-cleaning-app
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python main.py

Included sample data

• sample_data/employee_data_raw.csv is the coursework dataset used for the original assignment and electronic portfolio demonstration.
• sample_data/video_game_catalog_raw.csv is a synthetic non-employee dataset that demonstrates the application's ability to process a different CSV structure.

Generated cleaned copies, backups, review files, virtual environments, caches, and local IDE settings are intentionally excluded from the repository.

Testing

Run the complete suite with:

python -m pytest -q

Version 2.0.1 includes 23 automated tests covering validation, analysis, cleaning rules, type-aware missing values, manual corrections, save behavior, backups, and the complete load-analyze-clean-save-reload workflow.

The cleaned portfolio copy passed all tests:

23 passed

See TESTING.md for detailed automated coverage and the manual GUI checklist.

Project structure

data-cleaning-app/
├── README.md
├── TESTING.md
├── requirements.txt
├── pytest.ini
├── main.py
├── data_cleaning_app/
│   ├── __init__.py
│   ├── constants.py
│   ├── models.py
│   ├── services.py
│   └── gui.py
├── sample_data/
│   ├── employee_data_raw.csv
│   └── video_game_catalog_raw.csv
├── screenshots/
└── tests/

Data and privacy

The application processes CSV files locally and does not require network access. The source file remains unchanged while the user analyzes data and previews cleaning. Replacement occurs only after the user explicitly chooses Replace Original File, and a timestamped backup is created first.

Before publishing work derived from other datasets, verify that the data is fictional, public, anonymized, or otherwise appropriate to share.

Current limitations

• Excel files are not supported in this version.
• The visible preview is limited to 500 matching rows, although the entire file is analyzed, cleaned, reviewed, and saved.
• Processing occurs on the GUI thread, so extremely large CSV files may temporarily make the interface unresponsive.
• Employee-specific checks depend on recognized column names such as email, phone, department, hire_date, salary, age, and performance_score.
• Column-type inference is conservative and may classify unusual custom datasets imperfectly.
• Mean or median replacement can be mathematically valid without being the best business choice; the preview should be reviewed before saving.
• Direct spreadsheet-style editing is not included; manual corrections use the selected-problem panel.
• Reset Cleaning discards the current preview rather than providing a step-by-step undo history.
• The GUI was designed primarily for Ubuntu, and small visual differences may occur on other operating systems.