Skip to content
MrYellowOwlPublic

About

Web Crawler

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

9 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

Can Crawl

Description

The Can Crawl tool is a Python script designed to crawl and discover URLs associated with a given domain or IP address. It resolves the IP address of the target and explores the website structure up to a specified number of levels, collecting unique URLs along the way.

Features

  • IP Resolution: Resolves the IP address for the provided domain.
  • Multithreaded Crawling: Uses multithreading to crawl multiple URLs concurrently.
  • Level-Based Crawling: Allows users to specify the number of levels to crawl, providing a structured approach to exploration.

Usage

  1. Clone the repository:

    git clone https://github.com/MrYellowOwl/CanCrawl.git
  2. Navigate to the project directory:

    cd CanCrawl
  3. Run the script:

    python3 CanCrawl.py
  4. Follow the on-screen instructions to enter the target domain, URL, or IP address and the desired number of levels to crawl.

Requirements

  • Python 3.x
  • Requests
  • BeautifulSoup

Acknowledgements

Issues and Contributions

If you encounter any issues or have suggestions for improvements, please open an issue. Contributions are welcome!


MrYellowOwl

GitHub Profile

About

Web Crawler

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages