Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
connector_org_review_requirements.yaml
**/reference/
190 changes: 189 additions & 1 deletion airbyte-integrations/connectors/source-zoho-creator/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ python = "<3.14,>=3.10"
airbyte-cdk = ">=7.5.0,<8.0.0"
pytest = ">=9.0.1,<10.0.0"
pyyaml = ">=6.0.3,<7.0.0"
zoho-creator-sdk = ">=0.3.1,<1.0.0"

[tool.poetry.scripts]
source-zoho-creator = "source_zoho_creator.run:run"

[tool.poetry.group.dev.dependencies]
pytest = ">=9.0.1,<10.0.0"
pytest-mock = ">=3.0.0,<4.0.0"
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Commands

All commands should be run from `airbyte-integrations/connectors/source-zoho-creator/`.

**Install dependencies:**
```bash
poetry install --with dev
```

**Run the connector locally** (requires `secrets/config.json`):
```bash
poetry run source-zoho-creator spec
poetry run source-zoho-creator check --config secrets/config.json
poetry run source-zoho-creator discover --config secrets/config.json
poetry run source-zoho-creator read --config secrets/config.json --catalog sample_files/configured_catalog.json
```

**Run unit tests:**
```bash
poetry run pytest unit_tests
# Run a single test file:
poetry run pytest unit_tests/test_api.py
# Run a specific test:
poetry run pytest unit_tests/test_api.py::TestZohoCreatorAPITokenRefresh::test_get_access_token_caching
# Run only unit-marked tests:
poetry run pytest unit_tests -m unit
```

**Build Docker image:**
```bash
airbyte-ci connectors --name=source-zoho-creator build --tag=dev
```

**Run full CI test suite:**
```bash
airbyte-ci connectors --name=source-zoho-creator test
```

**Add a dependency:**
```bash
poetry add <package-name>
# Commit both pyproject.toml and poetry.lock
```

## Architecture

This connector follows the Airbyte Python CDK (`airbyte-cdk >= 7.5`) pattern with a custom API client layer rather than declarative YAML.

### Component Overview

**`source_zoho_creator/api.py` — `ZohoCreatorAPI`**
Central API client responsible for:
- OAuth2 token lifecycle: fetches and caches access tokens via refresh token grant, with a 60-second expiry buffer to avoid race conditions.
- Config validation: calls the metadata endpoint (`/creator/v2.1/meta/{owner}/{app}/reports`) to verify credentials and app access.
- Stream/report discovery: `get_application_reports()` returns all report objects for the configured application.
- Schema inference: `get_report_schema()` fetches the first 20 records from a report and infers field types (scalar → `string`, dict → `object`, list → `array`).
- Data fetching: `get_report_data()` returns raw records from a report endpoint.

**`source_zoho_creator/source.py` — `SourceZohoCreator`**
Extends `AbstractSource`. The `streams()` method instantiates one `ReportDataStream` per report returned by `api.get_application_reports()`. Reports are dynamically discovered at runtime — there are no statically defined streams.

**`source_zoho_creator/streams.py` — `ReportDataStream`**
Extends `HttpStream`. One instance is created per Zoho Creator report:
- **URL**: `https://{base_url}/creator/v2.1/data/{owner}/{app}/report/{report_link_name}`
- **Pagination**: Zoho uses a `record_cursor` response header. The stream reads this and passes it as a request header on subsequent pages. Default page size is 1000 records (`MAX_RECORDS`).
- **Schema**: Dynamically generated by calling `api.get_report_schema()` in `get_json_schema()`.
- **Incremental sync**: Cursor fields are `Added_Time` and/or `Modified_Time` if present in the schema. Date format is `"%d-%b-%Y %H:%M:%S"` (e.g., `04-Oct-2023 12:35:24`). The cursor value is passed as a Zoho criteria filter: `Added_Time > "..."`. The `_configured_cursor_field` attribute is set from the configured catalog in the overridden `read()` method.
- **Success check**: Zoho API returns `{"code": 3000, "data": [...]}` for success. Any other code raises `ZohoCreatorAPIError`.

**`source_zoho_creator/exceptions.py`**
Exception hierarchy: `ZohoCreatorAPIError` (base) → `ZohoCreatorAuthError` (auth failures), `ZohoCreatorConfigError` (config issues).

**`source_zoho_creator/spec.yaml`**
Defines the seven required config fields. Also contains an `advanced_auth` block for Airbyte Cloud OAuth2 flow (currently OSS-only per `metadata.yaml`).

### Multi-datacenter Support

All API endpoints are constructed from the configurable `base_accounts_url` (for OAuth token exchange, e.g. `accounts.zoho.com`) and `base_url` (for API calls, e.g. `www.zohoapis.com`). This allows the connector to work across Zoho's regional data centers (US, EU `.eu`, AU `.com.au`, IN `.in`).

### Key Files for Versioning

When releasing a new version, update both `metadata.yaml` (`dockerImageTag`) and `pyproject.toml` (`version`) to keep them in sync.
25,120 changes: 25,120 additions & 0 deletions airbyte-integrations/connectors/source-zoho-creator/reference/read.txt

Large diffs are not rendered by default.

Loading
Loading