Skip to content

README.md

Latest commit

 

History

History
20 lines (12 loc) · 732 Bytes

README.md

File metadata and controls

20 lines (12 loc) · 732 Bytes
Raw

Python Webscraper

This is a python project which scrapes every news reported into Aos Fatos website using Scrapy and outputs the data into a CSV file.

Run project

You must have Python 3.7 installed in your environment.

  1. Start Pipenv environment with $ pipenv shell

  2. Install environment depedencies with $ pipenv install

  3. Run aos_fatos scrapy project: $ scrapy crawl fatos -s HTTPCACHE_ENABLED=1, and to generate a file with all the scraped data (as a CSV, for example), use -o {filename}.csv flag.

Libraries