Please visit my portfolio to learn more about this project.
This project focused on data analysis and visualization using Tableau and Python, incorporating data scraped from the website PropertyFinder.ae. The objective was to analyze and gain insights from a dataset containing information about properties, agents, companies, amenities, and language. The dataset included various attributes such as property type, price, area, agent details, company information, and amenity names.
We performed several tasks throughout the project and implemented various visualization techniques to better explore and understand the Real Estate Market of UAE. To achieve that, we collected the data from scraping the PropertyFinder.ae website and then carefully cleaned the data. After that we implemented some additional segmentation of the data to help with our analysis.
Step-by-step description of the key tasks and accomplishments are as follows:
Data Scraping:
- Developed a web scraping script to extract property data from PropertyFinder.ae.
- Gathered property details such as property type, price, area, bedrooms, bathrooms, and location.
Data Cleaning and Segmentation:
- Imported the scraped data into Python and loaded it into a pandas DataFrame.
- Conducted data cleaning, including handling missing values, removing duplicates, and formatting columns.
- Performed data transformations and created derived columns to enhance the analysis.
Data Visualization in Tableau:
- Created interactive dashboards in Tableau to provide a comprehensive view of the data.
- Developed multiple visualizations, including bar charts, line charts, scatter plots, maps, and heat maps, to showcase different aspects of the data.
- Implemented filters, tooltips, and context filters to enable interactivity and allow users to dynamically explore and analyze the data.
- Designed dashboards with multiple sheets and coordinated actions to provide a cohesive and intuitive user experience.
Analysis and Insights:
- Explored the relationship between property prices and various factors such as area, bedrooms, and property type using various plots.
- Analyzed the distribution of property types listed by agents using stacked bar charts and treemaps.
- Visualized the price range distribution across different property types and emirates using several plots.
Interesting Findings:
- There is a general trend toward the availability of expensive properties. We observed a compelling trend indicating a higher abundance of expensive properties. As property values decrease, their availability also declines.
- Apartments are the predominant property type, followed by villa and townhouse.
- Jumeirah Village Circle, Downtown Dubai and Dubai Marina are the best place for finding a property within lowest price range.
Please note that the dashboards are interactive and can lead to multiple insights by clicking on different area of the chart.
- Clone the repo
git clone https://github.com/2mdipro7/Real-Estate-Market-Analysis-UAE.git- Initialize and activate the virtual environment
virtualenv --no-site-packages venv
source venv/bin/activate- Install dependencies
pip install -r requirements.txt-
Download Chrome web driver from - (https://chromedriver.chromium.org/downloads)
-
Run the scraper
python scraper.py-
You will get a file named 'property_data.csv' containing all the raw data.
-
Make sure to import the scraped data as df in the data cleaning notebook.
-
Initiate data cleaning
jupyter nbconvert --execute data_cleanup.ipynbYou will get 7 separate CSV files, that represent separate tables. Combine all the files into a single Excel Worksheet.
- Initiate data segments
jupyter nbconvert --execute data_segmentation.ipynb- You will get a new properties table with its segmented data. Replace all the data in the property sheet with these new segmented data.
- Load this dataset into Tableau.
If you come up with some interesting findings, add the Tableau dashboard link to the repo.
All Rights of this data belong to PropertyFinder.ae