Booking.com Review Scraper

A powerful and efficient scraper for collecting reviews from Booking.com hotels. This scraper uses Puppeteer and GraphQL to fetch detailed review data including text, scores, dates, and language detection.

Features

• GraphQL Integration: Directly queries Booking.com's GraphQL API for efficient data retrieval
• Language Detection: Automatically detects review language using franc and langs libraries
• Duplicate Prevention: Ensures no duplicate reviews are collected
• Date Range Filtering: Collects reviews within specified date ranges
• Rate Limiting Protection: Implements retry mechanisms and delays to avoid rate limiting
• Comprehensive Data: Collects detailed review information including:
  • Review text (positive and negative aspects)
  • Review scores
  • Dates
  • Accommodation types
  • Reviewer information
  • Hotel responses
  • Language detection

Input Schema

{
  "startDate": "2024-01-01",
  "endDate": "2024-03-20",
  "hotels": [
    {
      "url": "https://www.booking.com/hotel/tr/example.html",
      "hotelName": "Example Hotel"
    }
  ]
}

Output Format

{
  "review_id": "unique_review_id",
  "review_url": null,
  "accommodation_type": "FRIEND|FAMILY|COUPLE|SOLO|BUSINESS",
  "date": "2024-03-20T00:00:00.000Z",
  "hotel_name": "Example Hotel",
  "language": "en|tr|fr|de|...",
  "review_text": "Review content...",
  "response": {
    "replied": true|false,
    "date": null,
    "text": "Hotel response text..."
  },
  "reviewer": "username",
  "score": 9.5,
  "source": "Booking"
}

Dependencies

• apify: ^3.0.0
• puppeteer: 24.9.0
• got: ^11.8.0
• franc: ^6.1.0
• langs: ^2.0.0

Installation

1. Clone the repository:

git clone [repository-url]
cd booking-scraper

2. Install dependencies:

npm install

Usage

1. Create an INPUT.json file with your desired configuration:

{
  "startDate": "2024-01-01",
  "endDate": "2024-03-20",
  "hotels": [
    {
      "url": "https://www.booking.com/hotel/tr/example.html",
      "hotelName": "Example Hotel"
    }
  ]
}

2. Run the scraper:

apify run

Features in Detail

Language Detection

• Uses franc for initial language detection
• Converts 3-letter language codes to 2-letter ISO codes using langs
• Handles short texts and detection errors gracefully

Review Processing

• Normalizes accommodation types (e.g., "GROUP_OF_FRIENDS" → "FRIEND")
• Combines positive and negative review aspects
• Generates unique review IDs using MD5 hashing
• Filters reviews by date range

Error Handling

• Implements retry mechanism for failed requests
• Handles rate limiting with exponential backoff
• Logs errors and warnings for debugging

Performance Considerations

• Implements delays between requests to avoid rate limiting
• Uses batch processing for large datasets
• Saves data periodically to prevent data loss
• Efficient memory usage with Set for duplicate checking

Contributing

Feel free to submit issues and enhancement requests!

License

This project is licensed under the MIT License - see the LICENSE file for details.