add dropcontact integration - GDPR-compliant B2B enrichment#78
Open
jackulau wants to merge 1 commit into
Open
Conversation
Implements the Dropcontact enrich node per issue wespreadjam#16. Submits a single contact to Dropcontact's async /batch endpoint and polls /batch/{request_id} until enrichment results are ready or timeout expires. - dropcontact apiKey credential (X-Access-Token header) - dropcontactEnrichNode with Zod input/output schemas (no z.any) - Fetch-first polling loop (POLL_INTERVAL_MS = 5000), matching apify pattern - Polling terminates on data[0] presence (not just success:true, which Dropcontact returns on both submit-accepted and enrichment-complete) - Explicit field-by-field snake_case -> camelCase mapping with null defaults - Adds dropcontact to NodeCredentials interface in @jam-nodes/core - Registers in builtInNodes and integrations barrel - 42 vitest tests covering credentials, schemas, executor happy path, polling regression, timeout, auth errors, 5xx, 429, partial fields, and executeNode engine integration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #16
Summary
Adds a
dropcontact_enrichnode that integrates the Dropcontact API for GDPR-compliant B2B contact enrichment. Dropcontact is asynchronous:POST /batchreturns arequest_id, and the client must pollGET /batch/{request_id}until enrichment results are ready.The integration follows the existing Apify pattern for in-node polling and the existing Slack/WordPress patterns for credential handling, field mapping, and testing.
Acceptance Criteria (from Issue #16)
What's in this PR
New files
packages/nodes/src/integrations/dropcontact/credentials.tsdropcontactCredential— apiKey type,X-Access-Tokenheader auth, viadefineApiKeyCredentialfrom@jam-nodes/corepackages/nodes/src/integrations/dropcontact/schemas.tsDropcontactEnrichInputSchemaandDropcontactEnrichOutputSchema— fully typed withz.infer<>, noz.any()packages/nodes/src/integrations/dropcontact/enrich.tsdropcontactEnrichNodeviadefineNode— POST + polling loop, explicit field mappingpackages/nodes/src/integrations/dropcontact/index.tspackages/nodes/src/integrations/dropcontact/__tests__/dropcontact.test.tsModified files
packages/core/src/types/node.tsdropcontact?: { apiKey: string }to theNodeCredentialsinterfacepackages/nodes/src/integrations/index.tspackages/nodes/src/index.tsdropcontactEnrichNode, schemas, and credential; adds node tobuiltInNodesDesign decisions
Polling lives inside the node, not the engine
Dropcontact's async submission → poll contract is API-specific, not a generic cross-cutting concern. The CLAUDE.md architectural note says retry/cache/timeout/rate-limiting belong in the execution engine's
ExecutionConfig, but polling for an API-specific completion signal is different: it's the API's normal flow. This matches the existing Apify integration (apify/run-actor.ts::pollUntilFinished).Polling terminates on
data[0]presence, NOTsuccess === truealoneThis is the highest-risk correctness issue in the integration. Dropcontact returns
success: trueon BOTH:POST /batch(meaning "submission accepted"), andGET /batch/{request_id}when results are ready (meaning "enrichment complete")A naive polling loop that exits on
success === truewould terminate on the first poll before any data is available. The implementation checksbody.success === true && Array.isArray(body.data) && body.data.length > 0as the terminal success condition, with a dedicated regression test (poll does NOT terminate on success:true with missing data).Fetch-first polling order
The loop polls immediately on attempt 0 and only sleeps between attempts (skipping the sleep after the last attempt). This matches Apify's pattern and ensures the
timeoutinput reflects real wall-clock semantics — atimeout: 1sends exactly one poll, rather than sleeping 5 seconds first with no budget left.POLL_INTERVAL_MS = 5000is a module constantNot a user input — following the Apify pattern. Tests use
vi.useFakeTimers()+vi.runAllTimersAsync()to fast-forward.Output mapping is explicit field-by-field
mapEnrichedRecorduses a privategetString(obj, key)helper that returnsnullfor missing keys, non-object values, or non-string field values. This:z.nullish()(which would loosen the contract)as anyorz.any()(per CLAUDE.md)Output nullability is split
requestIdandsuccessare non-null (control fields — always present on a valid result)creditsLeftis.nullable()(Dropcontact doesn't always include it).nullable()(the API may resolve some fields but not others — a partial match is a valid, successful result)Single contact per call
Issue #16's input shape describes one contact per
dropcontactEnrichinvocation. Batching multiple contacts (with per-row error correlation) is deferred to a future issue if real workflows need it.Implementation details
HTTP retry and error handling
Both the POST and the polling GETs use
fetchWithRetryfrompackages/nodes/src/utils/http.tswith{ maxRetries: 3, backoffMs: 1000, timeoutMs: 30000 }(same config used by Slack and Apify). That helper handles:{ success: false, error })Retry-After→ retries with the documented delayAbortControllerFailure modes
Every failure path returns
{ success: false, error: "..." }— the executor never throws. Credit exhaustion (success: falseon POST) is surfaced via thereasonfield. Polling timeout surfaces asDropcontact polling timed out after Xs.Credentials
dropcontact?: { apiKey: string }to theNodeCredentialstype in@jam-nodes/coretypeof apiKey !== 'string' || apiKey.length === 0— missing credential object, missingdropcontactkey, and empty-stringapiKeyall short-circuit to a clear error before any network requestSubmitted POST body
buildSubmitBodyonly includes fields the user supplied, so unset fields are never sent to Dropcontact asundefined. Field names are translated to Dropcontact's snake_case convention (firstName→first_name,linkedinUrl→linkedin, etc.).Test plan
All tests are offline (mocked
fetchviavi.stubGlobal) — no real Dropcontact API calls during test runs.Test commands
Results
slack/*andgoogle-sheets/*are not introduced or affected by this PR)Coverage breakdown
dropcontact credentialdropcontact schemasdropcontactEnrichNodedata[0]; polling regression forsuccess:truewithout data;success:falsetermination; credit exhaustion on POST; URL encoding ofrequest_idandapi_key; snake_case → camelCase mapping; missing fields → null; extra fields stripped; emptydata:[]; all-nulldata[0]; polling timeout; 401; 5xx; network error mid-poll; 429 during polling; non-string field values → null;request_idmissing from 2xx success;creditsLeftpreservationexecuteNode integration