Functional Web Scraper

A functional web scraper built with Apify SDK and TypeScript, using functional programming principles.

Features

• Built with Apify SDK and Crawlee
• Functional programming approach using Ramda
• TypeScript for type safety
• Playwright for browser automation
• Structured data extraction

Installation

npm install

Usage

Basic scraping

npm start

Development mode

npm run dev

Scripts

• npm run build - Compile TypeScript
• npm start - Build and run the scraper
• npm run dev - Watch mode for development
• npm test - Run tests

Requirements

• Node.js >= 16.0.0

Dependencies

• apify - Apify SDK for logging
• crawlee - Web scraping and browser automation library
• playwright - Browser automation
• ramda - Functional programming utilities

--Red ≽^•⩊•^≼