Skip to content

refactor: use common LDValueConverter and LDContextEncoder in AI SDK#188

Merged
mattrmc1 merged 7 commits into
mainfrom
mmccarthy/AIC-2940/impl-common-context-encoder
Jul 13, 2026
Merged

refactor: use common LDValueConverter and LDContextEncoder in AI SDK#188
mattrmc1 merged 7 commits into
mainfrom
mmccarthy/AIC-2940/impl-common-context-encoder

Conversation

@mattrmc1

@mattrmc1 mattrmc1 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Wires configured judges into completion and agent configs so online evaluation actually runs. Previously completionConfig/agentConfig always attached Evaluator.noop() regardless of a config's judgeConfiguration, so judges were dormant (intentionally descoped in v1.0 — see #180). This completes that wiring.

Applications supply a Runner per judge via a new AIRunnerProvider; the SDK builds a real Evaluator from the judgeConfiguration. Behavior is unchanged when no provider is configured — everything falls back to Evaluator.noop().

New type

@FunctionalInterface
public interface AIRunnerProvider {
  // Returns a Runner for the given judge AI Config, or null to skip this judge.
  Runner create(AIJudgeConfig judgeConfig);
}

The application implements this to wrap any model provider. The client calls it once per judge key when building an Evaluator for a completion or agent config that carries a judgeConfiguration.

New constructor

// Existing one- and two-arg constructors delegate with a null provider (online eval disabled).
public LDAIClientImpl(LDClientInterface client, LDLogger logger, AIRunnerProvider runnerProvider);

LDAIClientImpl changes

The hard-coded Evaluator.noop() in the completion and agent paths (both the resolved-variation and default paths) is replaced with buildEvaluator(judgeConfiguration, context, variables). Judge configs themselves still wire Evaluator.noop() internally — judges do not evaluate themselves.

private Evaluator buildEvaluator(JudgeConfiguration judgeConfig, LDContext context, Map<String, Object> variables) {
  if (runnerProvider == null || judgeConfig == null || judgeConfig.getJudges().isEmpty()) {
    return Evaluator.noop();
  }
  // For each judge key: fetch its AI Config (Mode.JUDGE), get a Runner from the
  // provider, build a Judge. Skip (and log) disabled judges, null runners, or any
  // construction failure so one bad judge never blocks the parent config.
  // ...
  return judges.isEmpty() ? Evaluator.noop() : new Evaluator(judges, judgeConfig, logger);
}

Per-judge sampling rates continue to flow from the JudgeConfiguration through Evaluator.evaluate — the JudgeConfiguration remains the source of truth. Judge configs are fetched through the internal evaluation path, so building an evaluator does not emit judge usage-metric events.

Fallback / isolation behavior

  • Returns Evaluator.noop() when no provider is configured, the judgeConfiguration is absent/empty, or every judge fails to construct.
  • A disabled judge config, a null runner from the provider, or an exception during construction skips only that judge (logged); the parent config is still built with the surviving judges.

Migration

None required. Additive only — a new interface plus a new constructor overload. Existing callers behave identically (noop evaluator). Online evaluation is opt-in by passing an AIRunnerProvider.

Test plan

  • ./gradlew :lib:sdk:server-ai:test passes
  • completionConfigWithJudgeConfigAndProviderBuildsRealEvaluator — real (non-noop) evaluator built
  • agentConfigWithJudgeConfigAndProviderBuildsRealEvaluator — same for agent configs
  • completionConfigWithNoJudgesYieldsNoopEvaluator — no judgeConfiguration → noop
  • completionConfigWithNullRunnerProviderYieldsNoopEvaluator — two-arg constructor → noop even with a judgeConfiguration
  • disabledJudgeConfigIsFilteredAndEvaluatorIsNoop — disabled judge skipped and logged
  • nullRunnerFromProviderSkipsThatJudge — provider returns null → judge skipped, logged
  • throwingRunnerProviderSkipsThatJudgeButKeepsOthers — one bad judge skipped, others survive

Additional context

  • Completes the online-evaluation wiring deferred from v1.0 in feat: add Runner, RunnerResult, Judge, and Evaluator #180.
  • When a provider is configured, each judge in a config's judgeConfiguration is fetched and constructed eagerly at config-retrieval time (one flag evaluation per judge key), even if the evaluator is never invoked.
  • Known follow-up (out of scope): unlike js-core, Java does not yet reserve message_history / response_to_evaluate variables or strip legacy judge template messages before evaluation. Not a regression.

Note

Low Risk
Internal refactor behind existing parsers/interpolation; risk is mainly behavioral parity if shared encoder/converter semantics differ slightly from the removed local code.

Overview
Replaces duplicated server-ai internals with launchdarkly-java-sdk-common (2.5.0).

server-ai now depends on launchdarkly-java-sdk-common and drops the local LDValueConverter implementation (and its unit tests). AIConfigParser still maps model/tool JSON via LDValueConverter.toMap, but that type now comes from com.launchdarkly.sdk.

Interpolator no longer builds the ldctx Mustache variable with private contextToMap / singleContextToMap helpers; it uses LDContextEncoder.encode(context) instead, while keeping the same rule that caller-supplied ldctx is overridden.

No public API changes; this is internal parsing/interpolation alignment with other Java SDKs.

Reviewed by Cursor Bugbot for commit 3ca7b7f. Bugbot is set up for automated code reviews on this repo. Configure here.

@mattrmc1 mattrmc1 marked this pull request as ready for review July 13, 2026 19:26
@mattrmc1 mattrmc1 requested a review from a team as a code owner July 13, 2026 19:26
@mattrmc1 mattrmc1 requested a review from tanderson-ld July 13, 2026 19:27
@mattrmc1 mattrmc1 merged commit c051f2b into main Jul 13, 2026
25 checks passed
@mattrmc1 mattrmc1 deleted the mmccarthy/AIC-2940/impl-common-context-encoder branch July 13, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants