A Python tool to extract company information, failure reasons, and related data from loot-drop.io by interacting with the website using Selenium.
- Interactive extraction: Uses Selenium to actually click on company cards and extract data from popups
- Smart card detection: Finds cards by locating the
<!-- Cards Grid -->section and identifying cards withcard-yearclass - Robust popup handling: Automatically opens and closes popups for each card
- Extracts all detailed information including:
- Company name
- Value Proposition
- Opportunity Score
- Cause of Death
- The Loot
- Key People & Investors
- Market Today
- New Business Idea & Product Name
- Execution Plan
- Monetization Strategy
- Funding Burned
- Category
- Year Closed
- Exports data to multiple formats (JSON, CSV, Excel)
- Handles 500+ company cards automatically
- Install required dependencies:
pip install -r requirements.txt-
Install ChromeDriver:
- macOS:
brew install chromedriver - Linux: Download from ChromeDriver downloads
- Windows: Download from ChromeDriver downloads and add to PATH
Or use
webdriver-manager: - macOS:
pip install webdriver-managerRun the extractor directly:
python company_extractor.pyOr use the example script:
python example_usage.pyfrom company_extractor import CompanyExtractor
# Create extractor (headless=False to see browser, True to run in background)
extractor = CompanyExtractor(headless=False)
# Extract companies
companies = extractor.extract_all_companies()
# Save to files
extractor.save_to_json("output.json")
extractor.save_to_csv("output.csv")
extractor.save_to_excel("output.xlsx") # Requires openpyxlThe script will:
- Launch a Chrome browser (or run headless)
- Navigate to https://www.loot-drop.io/
- Find the Cards Grid section
- Identify all cards with
card-yearclass - Click each card to open its popup
- Extract detailed information from the popup
- Close the popup and move to the next card
- Save results to:
companies_data.json(JSON format)companies_data.csv(CSV format)companies_data.xlsx(Excel format, if openpyxl is installed)
Each company entry includes:
- company_name: Company name
- value_proposition: The Value Proposition
- opportunity_score: Opportunity Score
- cause_of_death: Cause of Death (why they failed)
- the_loot: The Loot
- key_people_investors: Key People & Investors
- market_today: Market Today
- new_business_idea_product_name: The New Business Idea & Product Name
- execution_plan: Execution Plan
- monetization_strategy: 💰 Monetization Strategy
- funding_burned: Funding amount burned (e.g., "$50M BURNED")
- category: Company category (HARDWARE, SOFTWARE, etc.)
- year_closed: Year the company closed
- Card Detection: The script locates the
<!-- Cards Grid -->HTML comment and finds all cards containing a<span class="card-year" data--h-bstatus="0OBSERVED">element - Card Interaction: For each card, the script:
- Scrolls the card into view
- Clicks the card to trigger the popup
- Waits for the popup to appear
- Data Extraction: Extracts all relevant information from the popup content using BeautifulSoup and regex patterns
- Popup Management: Closes the popup before moving to the next card
You can modify the CompanyExtractor class to:
- Adjust wait times for slower connections
- Change popup detection selectors
- Add custom data extraction logic
- Change output formats
- Add filtering or data processing
- The script uses Selenium WebDriver to interact with the website
- Runs in visible browser mode by default (set
headless=Truefor background operation) - Includes delays between actions to ensure proper page loading
- Handles errors gracefully and continues with next card if one fails
- Respects website structure and includes appropriate wait times
- ChromeDriver not found: Make sure ChromeDriver is installed and in your PATH
- Cards not found: The website structure might have changed. Check the HTML structure
- Popup not opening: Try increasing wait times in the code
- Import errors: Make sure all dependencies are installed:
pip install -r requirements.txt