It is a python tool which helps to extracts dara like links, subdomains by the help of Sublist3r tool, directories, product price, rating, titles and much more
git clone https://github.com/aboul3la/Web_Scaper.git
Sublist3r currently supports Python 2 and Python 3.
- The recommended version for Python 2 is 2.7.x
- The recommended version for Python 3 is 3.4.x
- Installation on Windows:
Open CMD in the same folder where all this files are present when you download
Type in cmd python.exe -m pip install -r requirements.txt
It will install all modules which requires to run this script
Run the script crawl.py in CMD or python terminal