Apple Ad Repository Scraper

This is an Apify actor that scrapes advertising data from the Apple Ad Repository. It utilizes Crawlee and Playwright to automate browsing and data extraction.

Use Cases

This actor can be used to:

• Monitor Competitor Ads: Track the advertisements that your competitors are running on the Apple platform.
• Analyze Ad Trends: Collect historical data on ads to identify trends in ad content, placement, and targeting.
• Market Research: Get insights into how apps are advertised across different countries and regions.
• Compliance Monitoring: Ensure that your own ads and those of your partners adhere to the relevant legal and ethical standards.
• Data-Driven Ad Optimization: Use scraped data to optimize your own advertising strategy.

How the Actor Works

The actor performs the following steps:

1. Initial Setup:

   • The actor starts by navigating to the Apple Ad Repository URL specified in the startUrls.
   • It uses the inputSchema file to validate the input parameters.
   • It then reads the environment variables that you provide in the Apify actor user interface.

2. Filtering Ads:

   • It populates the filter form with the user input provided in the environment variables, which are:
     • DEVELOPER_OR_APP: (Optional) Filters ads by a specific developer or app name.
     • COUNTRY_OR_REGION: (Required) Filters ads by a specific country or region. This must be one of the following values:

     Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
     Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Luxembourg,
     Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden
     

     • DATE_RANGE: (Required) Filters ads by a specific date range. This must be one of the following values:

     Last 90 days, Last 180 Days, Last Year
     

   • It applies the filters.

3. Extracting Ad Cards:

   • The actor scrapes all the ad cards from the page, which contain information such as:
     • Picture
     • App Name
     • Developer
     • Legal Name
     • Placement
     • Format
     • Country or Region
     • Parameters
     • First Impression
     • Latest Impression
   • The ad card information is saved into the Apify dataset.

4. Extracting Ad Details:

   • For each ad card, the actor clicks the "View Ad Details" button.
   • It navigates to the ad details page and scrapes:
     • The full image of the ad
     • All the text content on that page.
   • This ad detail information is also saved to the Apify dataset.

5. Data Output:

   • All extracted data (ad cards and ad details) is stored in the Apify dataset for further analysis and export.

Input Parameters

The actor accepts the following input parameters, configured as environment variables on Apify:

• DEVELOPER_OR_APP (Optional):
  • Type: String
  • Description: The name of the developer or app you wish to filter ads by. Leave empty if you want to scrape all ads.
• COUNTRY_OR_REGION (Required):
  • Type: String
  • Description: The country or region you wish to filter ads by. Choose one of the allowed options from the dropdown.
• DATE_RANGE (Required):
  • Type: String
  • Description: The date range you wish to filter by. Choose one of the allowed options from the dropdown.

How to Use

1. Create an Apify Actor: Create a new actor using the "Web Scraper" template on the Apify platform.
2. Upload/Copy Files: Upload/Copy the provided actor.json, inputSchema.json, Dockerfile, package.json, routes.js and src/main.js to the actor's source files.
3. Configure Environment Variables: Under the "Source Code" tab, go to the "Environment variables" section and set the environment variables.
4. Set Start URLs: Ensure the "Start URLs" in the input configuration is set to https://adrepository.apple.com/.
5. Build and Run: Build and run the actor.
6. View Results: Once the actor has completed its run, view the scraped data in the Apify dataset.

Additional Notes

• Error Handling: The actor includes basic error handling but can be improved with additional try...catch blocks.
• Scalability: Apify can be used to scale this actor if you need to scrape large amounts of data or run it frequently.
• Data Storage: The scraped data is stored in the Apify dataset by default. You can configure your actor to save this data to other storage options.
• Rate Limiting: The actor doesn't implement custom rate limiting. Please be mindful of the Apple Ad Repository's terms of service and limits when running.

This README.md file provides a comprehensive overview of the Apple Ad Repository Scraper, covering the use cases, how it works, input parameters, and how to use it. Feel free to customize the Additional Notes as you add to the actor's functionality.