Skip to content

feat(java): add streaming ASR SDK - #1113

Closed
Mason Chen (jiec-msft) wants to merge 18 commits into
microsoft:mainfrom
ghc-jetbrains:java/sdk-preview
Closed

Mason Chen (jiec-msft) wants to merge 18 commits into
microsoft:mainfrom
ghc-jetbrains:java/sdk-preview

Conversation

@jiec-msft

@jiec-msft Mason Chen (jiec-msft) commented Sep 17, 2026

Copy link
Copy Markdown
Member

Adds a Java 17 SDK for Foundry Local streaming ASR through JNA.

  • Streaming API - Supports reusable model and session ownership, PCM and WAV input, cancellation, final results, timing data, and deterministic cleanup.
  • Packaging - Produces standard Maven binary and sources JARs while keeping native runtimes and model weights external.
  • Validation - Adds focused lifecycle, native-contract, audio-validation, and opt-in real-model integration tests.
  • CI - Adds a five-platform Azure matrix, resolves Java 17 for each target architecture, and bootstraps pinned Maven 3.9.9 on self-hosted agents.

The existing preview release remains unchanged.

Earlier PRs: #1094 and #1100.

Add a Java 17 JNA binding for reusable-model PCM transcription, final results, cancellation, and deterministic cleanup. Package a normal Maven SDK and sources JAR while keeping native runtimes and model weights external.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build and test the Java 17 SDK from the team preview branch, derive a fixed version from the source tag, reject shaded JNA or Gson classes, and upload the exact first-party JAR with checksum evidence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move checkout, Java setup, and artifact upload to their current Node 24-based major versions so preview builds do not rely on deprecated action runtimes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vercel

vercel Bot commented Sep 17, 2026

Copy link
Copy Markdown

Mason Chen (@jiec-msft) is attempting to deploy a commit to the MSFT-AIP Team on Vercel.

A member of the Team first needs to authorize it.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@baijumeswani

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jiec-msft Mason Chen (jiec-msft) changed the title feat(java): continue streaming ASR SDK integration feat(java): add streaming ASR SDK Sep 17, 2026
@jiec-msft
Mason Chen (jiec-msft) marked this pull request as ready for review September 17, 2026 07:26
Copilot AI balanced review requested due to automatic review settings September 17, 2026 07:26
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Download cancellation has a race that can misreport success, and model-ID validation diverges from the native contract.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a Java 17/JNA SDK for streaming ASR, including lifecycle management, packaging, validation, and five-platform CI.

Changes:

  • Implements reusable models, audio sessions, streaming PCM/WAV transcription, cancellation, and timing.
  • Adds unit and native integration tests plus Maven packaging.
  • Integrates Java builds into repository development and Azure pipelines.
File summaries
File Description
sdk_v2/java/THIRD_PARTY_NOTICES.md Documents Java dependencies and licensing.
sdk_v2/java/src/test/java/com/microsoft/foundry/local/TranscriptionStateTest.java Tests transcription state and cancellation races.
sdk_v2/java/src/test/java/com/microsoft/foundry/local/NativeContractTest.java Validates native ABI contracts.
sdk_v2/java/src/test/java/com/microsoft/foundry/local/NativeAsrTest.java Adds opt-in native ASR integration coverage.
sdk_v2/java/src/test/java/com/microsoft/foundry/local/AudioValidationTest.java Tests PCM, WAV, and configuration validation.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/WavAudio.java Implements bounded WAV parsing.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/TranscriptionTiming.java Exposes request timing metrics.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/TranscriptionResult.java Defines final transcription results.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/Transcription.java Implements streaming request lifecycle.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/SpeechEvent.java Defines streaming speech events.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/PcmFormat.java Validates supported PCM input.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/NativeApi.java Provides the JNA C-ABI binding.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/ModelNotFoundException.java Adds exact-model lookup errors.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/ModelInfo.java Defines detached model metadata.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/Model.java Implements model operations and downloads.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/FoundryLocalManager.java Manages native ownership and cleanup.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/FoundryLocalException.java Exposes native error codes.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/Configuration.java Defines explicit runtime paths.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/Catalog.java Implements model discovery and lookup.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/CancellationToken.java Adds download cancellation state.
sdk_v2/java/src/main/java/com/microsoft/foundry/local/AudioSession.java Owns sequential transcription sessions.
sdk_v2/java/README.md Documents setup and streaming usage.
sdk_v2/java/pom.xml Configures Maven compilation and packaging.
sdk_v2/java/build.ps1 Adds Maven bootstrap and build support.
sdk_v2/java/.gitignore Ignores Java build artifacts.
sdk_v2/DEVELOPMENT.md Adds Java prerequisites and workflow.
sdk_v2/build_and_test_all.ps1 Includes Java in aggregate validation.
.pipelines/templates/steps-build-java.yml Builds and native-tests Java.
.pipelines/templates/steps-bootstrap-maven.yml Bootstraps pinned Maven.
.pipelines/templates/stages-sdk-v2.yml Registers Java pipeline stages.
.pipelines/templates/stages-java.yml Adds five-platform CI and packaging.
.pipelines/templates/fetch-models-from-artifacts-feed.yml Supports focused ASR model retrieval.
Review details
  • Files reviewed: 32/32 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk_v2/java/src/main/java/com/microsoft/foundry/local/Catalog.java Outdated
Comment thread sdk_v2/java/src/main/java/com/microsoft/foundry/local/Model.java Outdated
Comment thread sdk_v2/java/build.ps1 Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The CI bootstrap can ignore Maven 3.9.9 when an agent already defines M2_HOME.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

.pipelines/templates/steps-bootstrap-maven.yml:40

  • Maven@4 checks M2_HOME before searching PATH when mavenVersionOption is Default. On a self-hosted agent that already defines M2_HOME, both Java Maven tasks will therefore ignore this downloaded 3.9.9 installation, defeating the pinned bootstrap. Publish this directory as M2_HOME (or configure both tasks with mavenVersionOption: Path) so the selected Maven version is deterministic.
  • Files reviewed: 32/32 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread .pipelines/templates/steps-bootstrap-maven.yml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new native ABI, concurrent lifecycle management, and five-platform packaging pipeline warrant final human validation.

Review details

Suppressed comments (3)

sdk_v2/java/README.md:31

  • The immutable-release command also uses a Windows-only POM path and fails when copied into PowerShell on Linux or macOS. Use the cross-platform separator here as well.
mvn -f sdk_v2\java\pom.xml -Drevision=0.1.0 package

sdk_v2/java/README.md:153

  • The opt-in integration-test command has the same cross-platform path issue, so Unix users cannot run the documented validation verbatim. Use forward slashes for the POM path.
mvn -f sdk_v2\java\pom.xml test `

sdk_v2/java/README.md:147

  • This test command passes a backslash-containing filename to Maven, which is not a valid path on the supported Unix platforms. Use a forward-slash path.
mvn -f sdk_v2\java\pom.xml test
  • Files reviewed: 32/32 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread sdk_v2/java/README.md Outdated
Comment thread sdk_v2/java/README.md Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Low-level native ownership, concurrency, and five-platform runtime integration warrant final human validation.

Review details
  • Files reviewed: 32/32 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@baijumeswani

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@baijumeswani

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@baijumeswani

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.

@jiec-msft

Copy link
Copy Markdown
Member Author

Superseded by #1120, which uses the same exact history from a branch in this repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants