Web_Scrap_e-commerce

It's a simple app which is design to extract the number of products in each category from an e-commerce website using BeautifulSoup(bs4) and Selenium. The website that has been scrapped https://www.harveynorman.com.au/.

Prerequisites

• BeautifulSoup4
• Selenium and Selenium-Webdriver
• Requests
• Pandas

Installation

1. Clone the repo

git clone https:://github.com/udit1999/Web_Scrap_e-commerce.git

2. Install Python packages

pip install -r requirements.txt

3. Download the selenium webdriver(according to your system) and save it on webdriver folder.(this is only for firefox)

Usage

1. To get all the product links

  python scrapper.py

2. To get number of product in each category

  python app.py

The data will be stored in csv file in Data Folder.