How It Works

How It Works

Overview

InMemoryEmulator.BigQuery intercepts all HTTP calls at the HttpMessageHandler level inside the Google BigQuery SDK pipeline. This provides full SDK fidelity with zero production code changes.

Interception Chain

User Code
  → BigQueryClientImpl.ExecuteQuery() / .InsertRows() / .CreateDataset() / etc.
    → BigqueryService.Jobs.Query(request).Execute()
      → ConfigurableHttpClient.SendAsync()
        → ConfigurableMessageHandler.SendAsync()      ← SDK retry/backoff layer
          → FakeBigQueryHandler.SendAsync()            ← OUR INTERCEPTION POINT
            → Route to InMemoryDataStore
            → Build HttpResponseMessage with JSON body
          ← Return response
        ← SDK processes response
      ← SDK deserializes JSON into model types
    ← Return BigQueryResults / BigQueryJob / etc.
  ← User gets real SDK types

Why HTTP-Level Interception

BigQueryClient is abstract with all virtual methods, and BigQueryClientImpl is sealed. While you could subclass BigQueryClient directly, this bypasses the SDK's HTTP serialization/deserialization layer. By intercepting at the HTTP level:

• SDK retry logic is preserved — the ConfigurableMessageHandler sits above our handler
• SDK request serialization → JSON is exercised — catches serialization bugs
• SDK response deserialization ← JSON is exercised — catches format mismatches
• Request/response headers are real — including etags, content types, etc.
• Pagination via pageToken works natively — SDK handles the pageToken round-trip

Wiring

// No credentials needed — we intercept before auth headers matter
var store = new InMemoryDataStore("test-project");
var handler = new FakeBigQueryHandler(store);
var factory = new FakeBigQueryHttpClientFactory(handler);

var initializer = new BaseClientService.Initializer
{
    HttpClientFactory = factory,
    ApplicationName = "InMemoryEmulator.BigQuery",
};
var service = new BigqueryService(initializer);
var client = new BigQueryClientImpl("test-project", service);

Key insight: BaseClientService.Initializer.HttpClientFactory is a public property on Google.Apis.Services.BaseClientService.Initializer. No reflection needed. BigQueryClientImpl has public constructors accepting a BigqueryService. This is cleaner than Cosmos where we needed ConnectionMode.Gateway + CosmosClientOptions.HttpClientFactory.

Auth Bypass

The FakeBigQueryHttpClientFactory implements Google.Apis.Http.IHttpClientFactory and creates a ConfigurableMessageHandler wrapping our fake handler without any credential initializer. The SDK will still set up its message handler pipeline, but since no IConfigurableHttpClientInitializer adds auth interceptors, no Authorization header is added. Our handler ignores auth headers regardless.

Data Model

InMemoryDataStore
  └── string ProjectId
  └── ConcurrentDictionary<string, InMemoryDataset>
        └── string DatasetId
        └── ConcurrentDictionary<string, InMemoryTable>
              └── string TableId
              └── TableSchema Schema
              └── List<InMemoryRow> Rows (thread-safe)

Unlike Cosmos DB (which stores raw JSON strings), BigQuery rows are schema-enforced with typed columns. Each InMemoryRow contains a Dictionary<string, object?> where values are typed CLR objects matching the BigQuery type system.