Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## v1.26.0-sdk.1

- Regenerated from the public API contract (832 to 837 operations). Adds Google Play ratings, Instagram user dataset search/item/facets, and Reddit trending dataset search.
- Reddit post and comments operations now expose `include_metrics`. The default feed mode costs 1 credit; metrics mode makes one anonymous HTML request and costs 3 credits.

## v1.25.0-sdk.1

- Regenerated from the public API contract (836 to 832 operations). Adds Apple Podcasts chart rankings, new releases, and related shows; App Store editorial collections; and the Apple Podcasts shows dataset family. Removes the TCDB endpoints, which no longer exist upstream.
Expand Down
2 changes: 1 addition & 1 deletion crawlora/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from .operations import GROUPS, OPERATIONS

DEFAULT_BASE_URL = "https://api.crawlora.net/api/v1"
VERSION = "1.25.0-sdk.1"
VERSION = "1.26.0-sdk.1"
DEFAULT_USER_AGENT = f"crawlora-python-sdk/{VERSION}"
DEFAULT_MAX_RETRY_DELAY = 30.0
DEFAULT_RETRY_STATUSES = (408, 409, 425, 429)
Expand Down
372 changes: 372 additions & 0 deletions crawlora/client.pyi

Large diffs are not rendered by default.

153 changes: 150 additions & 3 deletions crawlora/operations.py

Large diffs are not rendered by default.

13 changes: 9 additions & 4 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Generated from `openapi/public.json`. Deprecated, admin, and internal operations are excluded from this SDK contract.

Total operations: `832`
Total operations: `837`

| Group | SDK method | Operation ID | HTTP | Params | Auth | Response | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- |
Expand Down Expand Up @@ -177,6 +177,9 @@ Total operations: `832`
| datasets | `datasets.housing_markets_facets` | `datasets-housing-markets-facets` | `GET /datasets/housing-markets/facets` | `facet` (query Literal['region_type', 'state_code', 'property_type', 'parent_metro', 'parent_metro_code', 'income_vintage', 'is_latest', 'period_begin'] required)<br>`q` (query str)<br>`region_type` (query Literal['national', 'metro', 'county', 'city', 'zip'])<br>`state_code` (query str)<br>`property_type` (query str)<br>`parent_metro_code` (query str)<br>`zip_code` (query str)<br>`period` (query str)<br>`latest` (query bool)<br>`min_median_sale_price` (query float)<br>`max_median_sale_price` (query float)<br>`min_median_list_price` (query float)<br>`max_median_list_price` (query float)<br>`min_price_to_income` (query float)<br>`max_price_to_income` (query float)<br>`min_salary_to_buy` (query int)<br>`max_salary_to_buy` (query int)<br>`min_median_dom` (query float)<br>`max_median_dom` (query float)<br>`min_inventory` (query int)<br>`max_inventory` (query int)<br>`min_homes_sold` (query int) | `ApiKeyAuth` | `DatasetsHousingMarketsFacetsResponse` | |
| datasets | `datasets.housing_markets_item` | `datasets-housing-markets-item` | `GET /datasets/housing-markets/items/{region_type}/{table_id}` | `region_type` (path Literal['national', 'metro', 'county', 'city', 'zip'] required)<br>`table_id` (path int required)<br>`period` (query str)<br>`property_type` (query str)<br>`history` (query bool) | `ApiKeyAuth` | `DatasetsHousingMarketsItemResponse` | |
| datasets | `datasets.housing_markets_search` | `datasets-housing-markets-search` | `GET /datasets/housing-markets/search` | `q` (query str)<br>`region_type` (query Literal['national', 'metro', 'county', 'city', 'zip'])<br>`state_code` (query str)<br>`property_type` (query str)<br>`parent_metro_code` (query str)<br>`zip_code` (query str)<br>`period` (query str)<br>`latest` (query bool)<br>`min_median_sale_price` (query float)<br>`max_median_sale_price` (query float)<br>`min_median_list_price` (query float)<br>`max_median_list_price` (query float)<br>`min_price_to_income` (query float)<br>`max_price_to_income` (query float)<br>`min_salary_to_buy` (query int)<br>`max_salary_to_buy` (query int)<br>`min_median_dom` (query float)<br>`max_median_dom` (query float)<br>`min_inventory` (query int)<br>`max_inventory` (query int)<br>`min_homes_sold` (query int)<br>`sort` (query Literal['relevance', 'price_desc', 'price_asc', 'list_price_desc', 'list_price_asc', 'price_to_income_desc', 'price_to_income_asc', 'salary_to_buy_desc', 'salary_to_buy_asc', 'dom_asc', 'dom_desc', 'inventory_desc', 'homes_sold_desc', 'period_desc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsHousingMarketsSearchResponse` | |
| datasets | `datasets.instagram_users_facets` | `datasets-instagram-users-facets` | `GET /datasets/instagram-users/facets` | `facet` (query Literal['is_verified', 'is_business_account', 'has_bio', 'has_external_url', 'category_name', 'source_tier'] required)<br>`q` (query str)<br>`username` (query str)<br>`category_name` (query str)<br>`source_tier` (query str)<br>`is_verified` (query bool)<br>`is_business_account` (query bool)<br>`has_bio` (query bool)<br>`has_external_url` (query bool)<br>`min_followers` (query int)<br>`max_followers` (query int)<br>`min_ratio` (query float)<br>`max_ratio` (query float)<br>`created_after` (query str)<br>`created_before` (query str)<br>`crawled_after` (query str)<br>`crawled_before` (query str)<br>`sort` (query Literal['relevance', 'followers_desc', 'followers_asc', 'crawled_at_desc', 'crawled_at_asc', 'created_at_desc', 'created_at_asc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsInstagramUsersFacetsResponse` | |
| datasets | `datasets.instagram_users_item` | `datasets-instagram-users-item` | `GET /datasets/instagram-users/items/{username}` | `username` (path str required) | `ApiKeyAuth` | `DatasetsInstagramUsersItemResponse` | |
| datasets | `datasets.instagram_users_search` | `datasets-instagram-users-search` | `GET /datasets/instagram-users/search` | `q` (query str)<br>`username` (query str)<br>`category_name` (query str)<br>`source_tier` (query str)<br>`is_verified` (query bool)<br>`is_business_account` (query bool)<br>`has_bio` (query bool)<br>`has_external_url` (query bool)<br>`min_followers` (query int)<br>`max_followers` (query int)<br>`min_ratio` (query float)<br>`max_ratio` (query float)<br>`created_after` (query str)<br>`created_before` (query str)<br>`crawled_after` (query str)<br>`crawled_before` (query str)<br>`sort` (query Literal['relevance', 'followers_desc', 'followers_asc', 'crawled_at_desc', 'crawled_at_asc', 'created_at_desc', 'created_at_asc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsInstagramUsersSearchResponse` | |
| datasets | `datasets.jobs_companies` | `datasets-jobs-companies` | `GET /datasets/jobs/companies` | `q` (query str)<br>`provider` (query Literal['greenhouse', 'lever', 'ashby', 'workday', 'smartrecruiters', 'workable', 'recruitee', 'rippling', 'personio', 'teamtailor', 'oracle', 'ukg', 'icims', 'eightfold', 'gem', 'pinpoint'])<br>`status` (query Literal['active', 'empty', 'gone', 'blocked', 'pending', 'invalid'])<br>`min_open_roles` (query int)<br>`sort` (query Literal['open_desc', 'company_asc', 'crawled_desc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsJobsCompaniesResponse` | |
| datasets | `datasets.jobs_company_item` | `datasets-jobs-company-item` | `GET /datasets/jobs/companies/{id}` | `id` (path str required) | `ApiKeyAuth` | `DatasetsJobsCompanyItemResponse` | |
| datasets | `datasets.jobs_facets` | `datasets-jobs-facets` | `GET /datasets/jobs/facets` | `size` (query int) | `ApiKeyAuth` | `DatasetsJobsFacetsResponse` | |
Expand Down Expand Up @@ -217,6 +220,7 @@ Total operations: `832`
| datasets | `datasets.producthunt_products_search` | `datasets-producthunt-products-search` | `GET /datasets/producthunt-products/search` | `q` (query str)<br>`topic` (query str)<br>`maker` (query str)<br>`launched_after` (query str)<br>`launched_before` (query str)<br>`min_votes` (query int)<br>`min_rating` (query float)<br>`pricing_type` (query str)<br>`has_website` (query bool)<br>`is_online` (query bool)<br>`sort` (query Literal['relevance', 'votes_desc', 'launched_desc', 'launched_asc', 'rating_desc', 'best_rank_asc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsProducthuntProductsSearchResponse` | |
| datasets | `datasets.producthunt_trends_facets` | `datasets-producthunt-trends-facets` | `GET /datasets/producthunt-trends/facets` | `facet` (query Literal['topic', 'launch_year'] required)<br>`group_by` (query Literal['topic_month', 'topic_year', 'topic'])<br>`topic` (query str)<br>`launched_after` (query str)<br>`launched_before` (query str)<br>`min_votes` (query int)<br>`min_launches` (query int) | `ApiKeyAuth` | `DatasetsProducthuntTrendsFacetsResponse` | |
| datasets | `datasets.producthunt_trends_search` | `datasets-producthunt-trends-search` | `GET /datasets/producthunt-trends/search` | `group_by` (query Literal['topic_month', 'topic_year', 'topic'])<br>`topic` (query str)<br>`launched_after` (query str)<br>`launched_before` (query str)<br>`min_votes` (query int)<br>`min_launches` (query int)<br>`sort` (query Literal['period_desc', 'period_asc', 'launch_count_desc', 'sum_votes_desc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsProducthuntTrendsSearchResponse` | |
| datasets | `datasets.reddit_trending_search` | `datasets-reddit-trending-search` | `GET /datasets/reddit-trending/search` | `q` (query str)<br>`subreddit` (query str)<br>`date` (query str)<br>`sort` (query Literal['rank', 'date_desc'])<br>`page` (query int)<br>`page_size` (query int) | `ApiKeyAuth` | `DatasetsRedditTrendingSearchResponse` | |
| datasets | `datasets.sec_companies_facets` | `datasets-sec-companies-facets` | `GET /datasets/sec-companies/facets` | `facet` (query Literal['sic', 'sic_description', 'exchange', 'state_of_incorporation', 'entity_type', 'reporting_currency', 'revenue_band', 'forms_filed'] required)<br>`q` (query str)<br>`ticker` (query str)<br>`sic` (query str)<br>`exchange` (query str)<br>`state_of_incorporation` (query str)<br>`entity_type` (query str)<br>`reporting_currency` (query str)<br>`has_financials` (query bool)<br>`min_revenue` (query float)<br>`form_filed` (query str) | `ApiKeyAuth` | `DatasetsSecCompaniesFacetsResponse` | |
| datasets | `datasets.sec_companies_financials` | `datasets-sec-companies-financials` | `GET /datasets/sec-companies/financials/{cik}` | `cik` (path str required)<br>`statement` (query Literal['income', 'balance', 'cash_flow'])<br>`period` (query Literal['annual', 'quarterly'])<br>`from` (query int)<br>`to` (query int)<br>`limit` (query int) | `ApiKeyAuth` | `DatasetsSecCompaniesFinancialsResponse` | |
| datasets | `datasets.sec_companies_insider` | `datasets-sec-companies-insider` | `GET /datasets/sec-companies/insider/{cik}` | `cik` (path str required)<br>`from` (query str)<br>`to` (query str)<br>`code` (query str)<br>`limit` (query int) | `ApiKeyAuth` | `DatasetsSecCompaniesInsiderResponse` | |
Expand Down Expand Up @@ -339,8 +343,9 @@ Total operations: `832`
| google_play | `google_play.categories` | `googleplay-categories` | `GET /googleplay/categories` | `country` (query str)<br>`lang` (query str) | `ApiKeyAuth` | `GooglePlayCategoriesResponse` | |
| google_play | `google_play.datasafety` | `googleplay-datasafety` | `GET /googleplay/datasafety` | `app_id` (query str required)<br>`lang` (query str) | `ApiKeyAuth` | `GooglePlayDatasafetyResponse` | |
| google_play | `google_play.developer` | `googleplay-developer` | `GET /googleplay/developer/{dev_id}` | `dev_id` (path str required)<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`full_detail` (query bool) | `ApiKeyAuth` | `GooglePlayDeveloperResponse` | |
| google_play | `google_play.list` | `googleplay-list` | `GET /googleplay/list` | `collection` (query Literal['TOP_FREE', 'TOP_PAID', 'GROSSING', 'NEW_FREE', 'NEW_PAID'])<br>`category` (query str)<br>`age` (query str)<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`full_detail` (query bool) | `ApiKeyAuth` | `GooglePlayListResponse` | |
| google_play | `google_play.list` | `googleplay-list` | `GET /googleplay/list` | `collection` (query Literal['TOP_FREE', 'TOP_PAID', 'GROSSING', 'NEW_FREE', 'NEW_PAID'])<br>`category` (query str)<br>`device` (query Literal['phone', 'tablet', 'tv', 'chromebook', 'watch', 'xr', 'car'])<br>`age` (query str)<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`full_detail` (query bool) | `ApiKeyAuth` | `GooglePlayListResponse` | |
| google_play | `google_play.permissions` | `googleplay-permissions` | `GET /googleplay/permissions` | `app_id` (query str required)<br>`country` (query str)<br>`lang` (query str)<br>`short` (query bool) | `ApiKeyAuth` | `GooglePlayPermissionsResponse` | |
| google_play | `google_play.ratings` | `googleplay-ratings` | `GET /googleplay/ratings` | `app_id` (query str required)<br>`country` (query str)<br>`lang` (query str) | `ApiKeyAuth` | `GooglePlayRatingsResponse` | |
| google_play | `google_play.reviews` | `googleplay-reviews` | `GET /googleplay/reviews` | `app_id` (query str required)<br>`sort` (query Literal['helpfulness', 'newest', 'rating'])<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`paginate` (query bool)<br>`next_pagination_token` (query str) | `ApiKeyAuth` | `GooglePlayReviewsResponse` | |
| google_play | `google_play.search` | `googleplay-search` | `GET /googleplay/search` | `term` (query str required)<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`full_detail` (query bool)<br>`price` (query Literal['all', 'free', 'paid']) | `ApiKeyAuth` | `GooglePlaySearchResponse` | |
| google_play | `google_play.similar` | `googleplay-similar` | `GET /googleplay/similar` | `app_id` (query str required)<br>`num` (query int)<br>`country` (query str)<br>`lang` (query str)<br>`full_detail` (query bool) | `ApiKeyAuth` | `GooglePlaySimilarResponse` | |
Expand Down Expand Up @@ -571,9 +576,9 @@ Total operations: `832`
| product_hunt | `product_hunt.makers` | `producthunt-makers` | `GET /producthunt/product/{id}/makers` | `id` (path str required)<br>`cursor` (query str) | `ApiKeyAuth` | `ProductHuntMakersResponse` | |
| product_hunt | `product_hunt.reviews` | `producthunt-reviews` | `GET /producthunt/product/{id}/reviews` | `id` (path str required) | `ApiKeyAuth` | `ProductHuntReviewsResponse` | |
| product_hunt | `product_hunt.search` | `producthunt-search` | `GET /producthunt/search` | `query` (query str required)<br>`type` (query Literal['product', 'user', 'launch'])<br>`page` (query int)<br>`featured` (query bool)<br>`topics` (query str) | `ApiKeyAuth` | `ProductHuntSearchResponse` | |
| reddit | `reddit.comments` | `reddit-comments` | `GET /reddit/comments/{id}` | `id` (path str required)<br>`sort` (query Literal['confidence', 'top', 'new', 'controversial', 'old', 'qa'])<br>`limit` (query int)<br>`depth` (query int) | `ApiKeyAuth` | `RedditCommentsResponse` | |
| reddit | `reddit.comments` | `reddit-comments` | `GET /reddit/comments/{id}` | `id` (path str required)<br>`sort` (query Literal['confidence', 'top', 'new', 'controversial', 'old', 'qa'])<br>`limit` (query int)<br>`depth` (query int)<br>`include_metrics` (query bool) | `ApiKeyAuth` | `RedditCommentsResponse` | |
| reddit | `reddit.domain_posts` | `reddit-domain-posts` | `GET /reddit/domain/{domain}/posts` | `domain` (path str required)<br>`sort` (query Literal['hot', 'new', 'top', 'rising'])<br>`time` (query Literal['hour', 'day', 'week', 'month', 'year', 'all'])<br>`limit` (query int)<br>`after` (query str) | `ApiKeyAuth` | `RedditDomainPostsResponse` | |
| reddit | `reddit.post` | `reddit-post` | `GET /reddit/post/{id}` | `id` (path str required) | `ApiKeyAuth` | `RedditPostResponse` | |
| reddit | `reddit.post` | `reddit-post` | `GET /reddit/post/{id}` | `id` (path str required)<br>`include_metrics` (query bool) | `ApiKeyAuth` | `RedditPostResponse` | |
| reddit | `reddit.search` | `reddit-search` | `GET /reddit/search` | `q` (query str required)<br>`subreddit` (query str)<br>`sort` (query Literal['relevance', 'hot', 'new', 'top', 'comments'])<br>`time` (query Literal['hour', 'day', 'week', 'month', 'year', 'all'])<br>`limit` (query int)<br>`after` (query str) | `ApiKeyAuth` | `RedditSearchResponse` | |
| reddit | `reddit.subreddit_about` | `reddit-subreddit-about` | `GET /reddit/subreddit/{subreddit}/about` | `subreddit` (path str required)<br>`limit` (query int) | `ApiKeyAuth` | `RedditSubredditAboutResponse` | |
| reddit | `reddit.subreddit_comments` | `reddit-subreddit-comments` | `GET /reddit/subreddit/{subreddit}/comments` | `subreddit` (path str required)<br>`limit` (query int)<br>`after` (query str) | `ApiKeyAuth` | `RedditSubredditCommentsResponse` | |
Expand Down
5 changes: 5 additions & 0 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,14 @@ Newer platforms are grouped like every other endpoint:

```python
posts = crawlora.reddit.search(q="python", subreddit="programming")
post_with_metrics = crawlora.reddit.post(id="1v8hy3q", include_metrics=True)
comments_with_metrics = crawlora.reddit.comments(id="1v8hy3q", include_metrics=True, limit=25)
brand = crawlora.brand.retrieve(domain="stripe.com")
```

Omit `include_metrics` for the 1-credit feed mode. Set it to `True` for the
3-credit anonymous HTML mode with public post and comment engagement metrics.

## Threads Public Lookups

```python
Expand Down
Loading
Loading