Skip to content

docs(apify-actor-development): add library choice tree — don't reach for Crawlee when you don't need to crawl #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
DaveHanns wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs(apify-actor-development): add library choice tree — don't reach for Crawlee when you don't need to crawl #70

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions skills/apify-actor-development/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,20 @@ Use the appropriate CLI command based on the user's language choice. Additional
- Use `console.log()` or `print()` instead of the Apify logger — these bypass credential censoring
- Disable standby mode without explicit permission

### Choosing the right library for your Actor

Crawlee is designed for crawls — traversing many pages with routing, deduplication, session handling, and concurrency management. For any of the following, do **not** wrap your code in a Crawlee crawler with a single `startUrl`, and do not pull Crawlee in at all:

| Task | Use |
|------|-----|
| Crawl a website across many pages | Crawlee: `CheerioCrawler` for HTML, `PlaywrightCrawler` for JS-rendered |
| Call one HTTP API, transform, write to dataset | Native `fetch()` (Node 18+) or `requests` (Python) + Apify SDK for I/O |
| Receive webhooks or run a long-lived HTTP server | Standby Actor with an HTTP server (`express`, Node `http`, etc.) bound to the platform port |
| Orchestrate other Actors | `Actor.call()` / `Actor.start()` (JS/TS) or `Actor.call()` / `Actor.start()` (Python) |
| AI / LLM work | The model vendor's SDK (e.g. `@anthropic-ai/sdk`, `openai`, `anthropic`) + Apify SDK for I/O |

Crawlee for a one-shot fetch adds startup cost, a request queue, a session pool, and a routing layer you don't need — and obscures the actual work behind a crawler abstraction.

## Logging

See [references/logging.md](references/logging.md) for complete logging documentation including available log levels and best practices for JavaScript/TypeScript and Python.
Expand Down
Loading