Skip to content

README.md

Latest commit

 

History

History
19 lines (14 loc) · 1.05 KB

README.md

File metadata and controls

19 lines (14 loc) · 1.05 KB
Raw

Crawling_timer

This is a crawler on websites; Jinair, Peach

It's wirtten for Python 2.7.2 and used Redis for a database.

In addtion, Notification will be showed on slack.

Abstract

This is a cyclical crawler worked by Timer.py with slack notification system.

Description File

  • Timer.py : For the variable's value called *fin, crawler can be worked every 1 mins. In this case, crawler worked every minute for 2 mins.
  • connection.py : In this file, you can handle server address, port, urls where you want to run crawling, and database name.
  • JinairCrawler.py : Crawling data on Jinair's main homepage.
  • PeachairCrawler.py : Crawling data on Peachair's main homepage.
  • alarm_in_slack.py : This file is for notification(New promotion and Crawler turned off) on slack. Notification channel on slack can be controled by change of slack_hook