Add local-runner for extraction testing and improve test loading by gasperzgonec · Pull Request #74 · devrev/airdrop-template

gasperzgonec · 2026-03-19T13:04:54Z

Fixture-based local testing with MockServer

Replaces the old test runner (which required real DevRev credentials) with a fixture-based system that uses the SDK's MockServer. Snap-in functions can now be tested locally without any external dependencies.

Changes

code/src/test-runner/test-runner.ts -- Full rewrite

Starts a MockServer (from @devrev/ts-adaas) on a dynamic port
Loads state.json from the fixture directory and injects it via the GET /worker_data_url.get route override
Loads event_context.json and builds a complete AirdropEvent with all SDK URLs (callback_url, worker_data_url, devrev_endpoint) pointing at the MockServer
All other DevRev API endpoints (artifact upload, snap-ins.get, domain-mappings.install, etc.) return default 200 OK responses automatically
Supports ${ENV_VAR} templating in fixture JSON files -- variables are resolved from process.env (with .env auto-loaded via dotenv). Missing variables produce a clear error message with the variable name and file path.
Removed the old addCredentials / dotenv-PAT injection flow

code/src/main.ts -- Updated CLI interface

--fixturePath (required): name of the fixture folder inside code/fixtures/
--functionName (optional): which function to run. Can also be set via function_name in event_context.json
--local (optional): enables the SDK's local development mode -- log output is plain text messages instead of full JSON objects with event context tags

code/package.json

Added "fixture" npm script

code/fixtures/ -- Example fixtures

start_extracting_external_sync_units/ -- event_context.json + state.json
start_extracting_data/ -- event_context.json + state.json

Usage

# Run a fixture (JSON log output):
npm run fixture -- --fixturePath start_extracting_external_sync_units

# Run with plain-text logs:
npm run fixture -- --fixturePath start_extracting_external_sync_units --local

# Override function name from CLI:
npm run fixture -- --fixturePath start_extracting_data --functionName extraction --local

Creating new fixtures

Create a folder in code/fixtures/<name>/ with two files:

event_context.json (only event_type is required):

{
  "event_type": "START_EXTRACTING_DATA",
  "function_name": "extraction",
  "connection_data": {
    "key": "${EXTERNAL_SYSTEM_TOKEN}",
    "key_type": "pat"
  }
}

state.json (optional -- defaults to empty state if missing):

{
  "todos": { "completed": false },
  "users": { "completed": false },
  "attachments": { "completed": false }
}

Environment variables can be provided via a .env file or exported in your shell.

How it works

dotenv.config() loads .env into process.env
Fixture JSON files are read and ${VAR} placeholders are resolved
MockServer starts on a random available port
state.json content is injected into the mock GET /worker_data_url.get endpoint
A full AirdropEvent is constructed with test defaults and MockServer URLs
The snap-in function is called with the event
MockServer is stopped after completion

No real credentials or network access to DevRev is needed. External system calls (e.g. the HttpClient in the template) still hit their configured endpoints as normal.

Connected Issues

#ISS-269670

Checklist

Tests added/updated and ran with npm run test OR no tests needed.
Code formatted and checked with npm run lint.
Added "How to test" section to the description OR this section is not needed.

radovanjorgic

I quickly reviewed and gave some comments, but I don't think this is the right direction. Approach is good (mock server, fixtures) but adding so much code in template is not what we want. We should think of abstracting this in SDK.

code/scripts/deploy.sh

code/src/fixtures/custom-sync-unit.json

code/src/functions/external-system/http-client.ts

code/src/local-runner/mock-devrev-server.ts

code/src/functions/extraction/workers/data-extraction.test.ts

code/src/functions/extraction/workers/data-extraction.ts

Copilot

Pull request overview

This PR replaces the credential-dependent local test runner with a fixture-based runner that uses @devrev/ts-adaas MockServer, enabling snap-in functions to be executed locally without needing real DevRev access.

Changes:

Rewrote the test runner to load event_context.json / state.json, resolve ${ENV_VAR} placeholders, and run against a MockServer.
Updated the CLI to accept --fixturePath (required) and optional --functionName, --local, and --printState.
Added example fixture directories and an npm run fixture script.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
code/src/test-runner/test-runner.ts	New fixture-driven runner that builds an `AirdropEvent`, starts/stops `MockServer`, injects fixture state, and supports env templating.
code/src/main.ts	Updates CLI args for fixture execution and injects positional `local` for SDK spawn local-mode logging.
code/package.json	Adds `fixture` script alias to run the fixture CLI via `ts-node`.
code/fixtures/start_extracting_external_sync_units/event_context.json	Example fixture event context for external sync unit extraction start.
code/fixtures/start_extracting_external_sync_units/state.json	Example state fixture for extraction.
code/fixtures/start_extracting_data/event_context.json	Example fixture event context for START_EXTRACTING_DATA.
code/fixtures/start_extracting_data/state.json	Example state fixture for extraction.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

code/src/local-runner/local-runner.ts

code/src/test-runner/test-runner.ts

code/fixtures/start_extracting_data/event_context.json

code/src/main.ts

…to AirdropMessage

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

code/src/test-runner/test-runner.ts

code/src/main.ts

code/src/local-runner/local-runner.ts

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

code/src/main.ts

code/src/test-runner/test-runner.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

code/fixtures/start_extracting_data/event.json

radovanjorgic · 2026-03-25T17:41:41Z

code/fixtures/start_extracting_data/state.json

Please verify if fetch state returns exactly this object or full state object (with fields like lastSuccessfulSyncStarted, attachmentsMetadata and so on). We should have the full state, not sure if we should define only initial or full state here.

The fetch state returns a full object, containing all of these values, if present.

So if we have only {"foo": "bar"} in state.json, it will return full state object, with fields like lastSuccessfulSyncStared and attachmentsMetadata? Those fields will be overriden if explicitly specified in state.json file?

code/fixtures/start_extracting_external_sync_units/state.json

radovanjorgic · 2026-03-25T17:44:09Z

code/fixtures/start_extracting_data/event.json

@@ -0,0 +1,13 @@
+{
+  "event_type": "START_EXTRACTING_DATA",
+  "function_name": "extraction",


Hmm, if we name this event/airdrop message it is a bit confusing to have additional fields like function_name here. I would rather keep it as other snap-ins do - that you need to specify through args the function you want to test offline.

While this is a fair observation, I don't think it's very reproducible. I'd like to have it be stored in the file, as one fixture will be either for extraction or loading, but never both (forcing this to be a CLI arg could add a new problem that could occur when we mismatch and push an extraction event into a loading function, which will never happen in the real life).

I get the point but wanted to have clear separation of concerns here. If I am a developer, I would just copy the content of the real event into event.json and expect to be ready to test offline with it. Without having to add or understand additional fields like function_name. function_name sounds like "how to run" part of the process and is already a familiar dev flow for existing snap-ins (not adaas snapins).

Example from devrev-snaps-typescript-template README.md (https://github.com/devrev/devrev-snaps-typescript-template):

npm install npm run start -- --functionName=on_work_creation --fixturePath=on_work_created_event.json

radovanjorgic · 2026-03-25T17:45:45Z

code/fixtures/start_extracting_data/event.json

+  "event_context": {
+    "external_sync_unit_id": "test_external_sync_unit_id"
+  }
+}


Are all of these fields required? Are all required present here? For example I know request id field (or uuid form event context IIRC) is used when doing some requests, but I don't see it here.

None of these fields are required, this file can be empty.
The defaults here override the missing values in there.

code/src/main.ts

radovanjorgic · 2026-03-25T17:57:52Z

code/src/test-runner/test-runner.ts

+ * Fields from the fixture's airdrop_message.json are merged into the
+ * appropriate sections, with MockServer URLs always taking precedence.
+ */
+function buildEvent(mockServerBaseUrl: string, eventType: EventType, fixture?: FixtureAirdropMessage): AirdropEvent {


We have similar createEvent function in SDK, maybe we can export and use it here as well?

SDK's createEvent is coupled to its internal Jest MockServer singleton — can't inject a custom base URL. Filed as future SDK enhancement.

I would rather delay this and add what is needed to SDK to deliver simplified version of this, then: add to template, announce new feature -> people start pulling the same code, add to sdk -> tell people that this is outdated and can be removed.

code/src/local-runner/local-runner.ts

radovanjorgic · 2026-03-25T18:00:38Z

code/src/local-runner/local-runner.ts

+  console.log(`[fixture] MockServer started on ${mockServer.baseUrl}`);
+
+  // Inject state from state.json (or default to empty state)
+  mockServer.setRoute({


For now we can go with state only, but is there something else that user would like to override to test locally?

Maybe we could add a different configuration options, for example, to support custom responses for some other endpoints (e.g: selective extraction itemTypes).

Yes, we could do that.

code/src/main.ts

…d enum support for event types.

radovanjorgic · 2026-03-30T05:33:15Z

code/src/local-runner/local-runner.ts

+ */
+export interface FixtureEvent {
+  event_type: string;
+  function_name?: FunctionFactoryType;


If we keep function_name (which I would not suggest), this will be required parameter, right?

radovanjorgic · 2026-03-30T05:36:07Z

code/src/local-runner/local-runner.ts

+ * Replaces `${VAR_NAME}` placeholders with values from `process.env`.
+ * Values are JSON-escaped so special characters don't break the JSON structure.
+ */
+function resolveEnvVariables(raw: string, filePath: string): string {


Oh okay so this enables using env variables in fixtures json files like this for example?

{ "foo": "${ENV_VAR_NAME}"? }

Where .env file should be located? In code/?

radovanjorgic · 2026-03-30T05:37:36Z

code/src/local-runner/local-runner.ts

+  throw new Error(`Unknown event_type "${input}". Must be one of: ${Object.values(EventType).join(', ')}`);
+}
+
+function buildEvent(mockServerBaseUrl: string, eventType: EventType, fixture?: FixtureEvent): AirdropEvent {


As said above, if possible let's move this to the sdk and reuse from there (if that is not harmful) and keep template minimal.

radovanjorgic · 2026-03-30T05:40:43Z

code/src/local-runner/local-runner.ts

+  console.log(`[fixture] MockServer started on ${mockServer.baseUrl}`);
+
+  // Inject state from state.json (or default to empty state)
+  mockServer.setRoute({


Yes, we could do that.

radovanjorgic · 2026-03-30T05:45:34Z

code/fixtures/start_extracting_external_sync_units/state.json

There is no state yet in start external sync units phase. It is stateless.

radovanjorgic · 2026-03-30T05:49:40Z

code/src/local-runner/local-runner.ts

+  /** Which function to run — overrides airdrop_message.json's function_name if set */
+  functionName?: FunctionFactoryType;
+  /** Print the adapter state on every worker_data_url.update call */
+  printState?: boolean;


Okay maybe saving to the file is not needed for now (since we don't have any orchestration) but then I think even --printState is not needed as flag. Anyone can just log the adapter.state at the end of process to see what the state looks like. And we anyhow have State is successfully updated to {...state} log from SDK at the end of extraction.

radovanjorgic reviewed Mar 23, 2026

View reviewed changes

gasperzgonec added 2 commits March 24, 2026 14:52

Initial implementation of Fixture testing.

66546b9

Added fixtures and added support for variable interpolation.

6e64ff3

gasperzgonec force-pushed the offline_testing branch from 3870b8f to 6e64ff3 Compare March 24, 2026 14:12

Added printState to enable tracking of state updates.

c419678

gasperzgonec marked this pull request as ready for review March 24, 2026 16:54

Copilot AI review requested due to automatic review settings March 24, 2026 16:54

Copilot started reviewing on behalf of gasperzgonec March 24, 2026 16:55 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

code/src/local-runner/local-runner.ts Show resolved Hide resolved

code/src/test-runner/test-runner.ts Outdated Show resolved Hide resolved

code/fixtures/start_extracting_data/event_context.json Outdated Show resolved Hide resolved

code/src/main.ts Outdated Show resolved Hide resolved

Fixed suggestions reported by Copilot and migrated from EventContext …

524be92

…to AirdropMessage

gasperzgonec requested a review from Copilot March 24, 2026 17:28

Copilot started reviewing on behalf of gasperzgonec March 24, 2026 17:28 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

code/src/test-runner/test-runner.ts Outdated Show resolved Hide resolved

code/src/test-runner/test-runner.ts Outdated Show resolved Hide resolved

code/src/main.ts Outdated Show resolved Hide resolved

code/src/local-runner/local-runner.ts Show resolved Hide resolved

Fixed Copilot notes.

689106d

gasperzgonec requested a review from Copilot March 24, 2026 17:40

Copilot started reviewing on behalf of gasperzgonec March 24, 2026 17:40 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

code/src/main.ts Outdated Show resolved Hide resolved

code/src/test-runner/test-runner.ts Outdated Show resolved Hide resolved

code/src/test-runner/test-runner.ts Outdated Show resolved Hide resolved

Update code/src/main.ts

2a824d8

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

radovanjorgic reviewed Mar 25, 2026

View reviewed changes

gasperzgonec added 3 commits March 26, 2026 14:24

Renamed test-runner to local-runner, renamed fixture files and remove…

a5df57d

…d enum support for event types.

Renamed the actual files

1d56671

Merge branch 'main' into offline_testing

d2d53c8

radovanjorgic reviewed Mar 30, 2026

View reviewed changes

Conversation

gasperzgonec commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixture-based local testing with MockServer

Changes

Usage

Creating new fixtures

How it works

Uh oh!

radovanjorgic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

radovanjorgic Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

gasperzgonec commented Mar 19, 2026 •

edited

Loading

radovanjorgic Mar 30, 2026 •

edited

Loading