You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Kotlin/Native + KMP in rules_kotlin: a synthesis of prior efforts + a working prototype #1682
There have been several parallel runs at this: a tracking issue (#567), a JS re-enable (#1347), an initial native toolchain + kt_library (#1351), a roadmap outline (#1466), a Build Tools API plan (#1481), an external ruleset (kitterion/rules_kotlin_native), and the rules_jvm_external dependency work (bazel-contrib/rules_jvm_external#1357). None has landed, but together they've already established most of the design decisions. I've built a prototype that validates that roadmap and advances past the point where #1351 stalled. Specifically, it solves the "blocking non-hermetic downloads" problem #1466 calls out as painful, and reaches a runnable native binary (cross-compiled Linux ELF, no JVM, run in a bare container).
KMP/Native hasn't been on the core team's roadmap (@Bencodes, 2024: open to community contributions but no team plan). I'm a rules_kotlin maintainer, though to be fair, I've been largely absent for a couple of years. This proposal is part of my return to active maintenance, and it's work I intend to see through. With that on the table, I'm putting KMP/Native on the roadmap by making this bid: this issue (1) consolidates the prior art and the decisions it produced, (2) reports what the prototype demonstrates, and (3) proposes a resolution to the one open API-shape question that #1351 explicitly deferred (the last thing blocking a phased landing), for us to converge on.
This builds directly on @restingbull's roadmap (#1466) rather than competing with it.
1. Prior art and the decisions it already produced
The long-standing tracking issue. @Bencodes (2024): no core-team plan to build KMP, but open to community contributions. This issue is me acting on exactly that: it takes over as the active umbrella, with #567 referencing it and superseded once the roadmap here is accepted.
Decision (@restingbull): start with a Kotlin/Native toolchain and treat JS as an output of the shared IR/klib backend, not a separate JS path. (Extending the same backend to Wasm is my inference, not his.)
The closest prior implementation (@smocherla-brex): a kotlin_native_compiler_repository, a per-(exec,target) kt_native_toolchain, konan.home exposed as a DirectoryInfo artifact, a kt_library rule producing klibs for commonMain-style shared code, provider KtKlibInfo{klibs, transitive_klibs}, integrated into the KotlinBuilder worker (a Platform.NATIVE task executor). Explicit open TODO: "decide if a separate rule is needed for klibs or fit it into existing platform-specific rules."
@restingbull's plans/kmp.md. Phases: toolchain foundation → rules → dependency resolution. Calls out: klib is the intermediate format (he notes it is probably not fully platform-independent, he hedges); blocking the compiler's runtime downloads is painful; BTAPI is required for long-term maintenance; deps via the Gradle resolver in rules_jvm_external. Includes a hard-won list of compiler flags (see Section 3).
@agluszak's plan to route compilation through the Kotlin Build Tools API. Names "easier KMP integration (#1466)" as a motivation. This is the intended long-term compiler-integration layer.
Kotlin 2.0 moved JS/Native artifacts to klib (no more JARs), which is why JS was dropped and why a klib-producing core is the unlock for JS+Native+Wasm.
KMP Maven artifacts (Gradle Module Metadata) are an rules_jvm_external concern; @smocherla-brex's bazel-contrib/rules_jvm_external#1357 adds Gradle-resolver support (ArtifactView, defaults to JVM, can select platform variants). Cross-repo coordination needed.
Net of the decisions: native-first; klib as the shared IR artifact (JS/Wasm later); konan.home as a directory artifact; BTAPI as the eventual compiler API; deps through the rules_jvm_external Gradle resolver; and an unresolved choice between a dedicated klib
rule vs. folding klibs into the existing platform rules.
2. What the prototype demonstrates (and where it stands vs. the prior art)
kt_native_library:.kt → .klib for a konan target, host and cross-target,
with transitive deps. Verified hermetic: builds under darwin-sandbox with --sandbox_default_allow_network=false; the compiler's dependency downloader is never
invoked for klib compilation (a fresh KONAN_DATA_DIR stays empty).
kt_native_binary: links a real native executable (-produce program). The
~650–720 MB LLVM/sysroot/cross-toolchain bundle is pre-fetched by a repo rule and
staged into a sandboxed KONAN_DATA_DIR (regenerating the compiler's .extracted
marker), so the link runs sandboxed, no network. linux_x64 verified with the
network disabled; macos_arm64 links in-sandbox using the host Xcode SDK. This is the
item [kmp] Outline of a roadmap #1466 flags as "this will be painful," demonstrated working.
e2e: a Bazel sh_test builds+runs a host binary; a script cross-compiles a linux_x64 binary on macOS and runs it in a bare ubuntu container with no JRE: a true
native ELF, zero JVM references.
This advances past #1351 (which reached klib-only and stalled before hermetic dep
provisioning and binary linking). It is a prototype, though: several things are
deliberate shortcuts that should be replaced before this is the final form. Calling them
out explicitly so the gap to a mergeable design is clear:
konan.home exposure → use DirectoryInfo. I used filegroup(glob(["**"])); Add initial kotlin native toolchain and kt_library rule #1351 + [kmp] Outline of a roadmap #1466 expose it as a bazel-skylib DirectoryInfo (passed as a system
property). The directory artifact is the right answer: it avoids enumerating tens of
thousands of inputs and is far friendlier to RBE. The prototype's filegroup is a
shortcut to drop.
Compiler invocation → BTAPI seam.Add initial kotlin native toolchain and kt_library rule #1351 integrated native into the KotlinBuilder worker (Platform.NATIVE); I shell konanc via a small launcher. The
launcher is a deliberate, thin seam that's easy to swap for the BTAPI (Build Tools API plan #1481),
which [kmp] Outline of a roadmap #1466 says is the long-term target and "not worth undertaking [other
compiler-integration] effort while pending." I've run this past Eugene
Zhuravlev (@eugenezh) at JetBrains (their point person for the Bazel Kotlin rules, working
on the Build Tools API integration), flagging both the wrapped-compiler question and the
deliberate choice here to keep K/N compiler invocation isolated behind a thin seam while
the Build Tools API is still stabilizing; he didn't raise concerns. That isolation is the point: it keeps the eventual BTAPI migration a
localized swap rather than a rewrite, so the two efforts don't have to move in lockstep.
Whether the interim seam should be the launcher or the worker is still an open call
(tracked in the Compiler invocation sub-issue, Section 6).
Reproducibility flags → not yet applied. I haven't yet wired [kmp] Outline of a roadmap #1466's -Xklib-relative-path-base / -Xdebug-prefix-map (needed for cross-machine remote
cache hits) or -Xoverride-konan-properties=airplaneMode=true (belt-and-suspenders
download disable on top of pre-staging). All should go in.
Unpinned dependency hashes → must pin. The konan dist + LLVM/sysroot bundle are
fetched without sha256 pins in the prototype (fast iteration). The final form needs
pinned, integrity-checked downloads. Non-negotiable for a hermetic, reproducible repo
rule.
Single-host coverage → needs the full matrix. Only the macos-aarch64 host dep
table is filled in; the rule shape is host-generic but the other host entries
(linux-x86_64, etc.) and their hashes are stubs. Cross-target (--platforms) is
verified; cross-host provisioning is not yet populated.
In other words: the prototype is empirical confirmation that #1466's plan works, plus a
solution to its hardest Phase-1 item. But it is a prototype, and the items above (plus
the API-shape decision in Section 4) are what stand between it and a form worth merging. It
should be re-based onto #1351/#1466's conventions, not kept as a parallel third thing.
3. The flags/lessons, reconciled
#1466 lists the hard-won compiler flags; the prototype exercised several. Consolidated,
the working recipe is:
Distribution: expose konan.home as a DirectoryInfo; pass via system property.
Block downloads: pre-stage LLVM/sysroot/cross-toolchain as a repo rule with the dependencies/.extracted marker and set -Xoverride-konan-properties=airplaneMode=true.
Cache isolation:-Xauto-cache-dir / -Xauto-cache-from to keep the compiler's
internal cache out of the read-only external repo (the prototype hit exactly the klib/cache/<target>STATIC write-back [kmp] Outline of a roadmap #1466 predicts; redirecting fixes it).
Reproducible klibs (RBE):-Xklib-relative-path-base, -Xdebug-prefix-map;
explicit stdlib as a declared input (-no-default-libs + explicit -library).
Cross-compile: ship a set of platform()s; konan target comes from --platforms
4. The open API-shape question (tracked as a sub-issue)
This is #1351's deferred TODO and the crux. The constraint that decides it concerns how
"common" code can be packaged. Two things observed/known:
Platform klibs are target-tagged, directly verified in the prototype: the same
sources compiled for different targets produce klibs whose manifests read native_targets=macos_arm64 vs. native_targets=linux_x64. So a platform klib is not
a single artifact consumable by every target. ([kmp] Outline of a roadmap #1466 flags this too, though @restingbull hedges whether it's still strictly true; the prototype manifests are the
concrete evidence.)
Common/commonMain code is a separate question I have not yet built: KMP models
it as a metadata klib plus per-target compilations, and expect/actual requires the actual to be compiled in the same module as its expect, so "common" is not a
standalone linked artifact once expect/actual is used; it's source co-compiled per target
(or a metadata klib). This is unverified in the prototype; it's the KMP-common work
(phase P6 in Section 5 below).
Given that, the realistic options:
(A) Dedicated klib rule.kt_native_library/kt_library produces klibs;
platform-specific rules consume them. This is what Add initial kotlin native toolchain and kt_library rule #1351 built and what @restingbull's
native-first steer (Re-enable Javascript support #1347) points to. Clean; covers pure-common well; needs an
expect/actual story (a metadata klib or a friend-association, and note rules_kotlin
already has associates).
(B) Platform-as-configuration. One rule, backend chosen by --platforms. Most
Bazel-native; expect/actual "just works" because each platform build co-compiles
common+platform; but bazel build //:x produces different artifacts by config and it
fights the rich existing kt_jvm_library surface.
My current thinking: (A) as the core (it matches the prior decisions and the
klib-not-portable reality), with target selection by --platforms (the configuration
half of (B)), and a later kt_multiplatform_library (or metadata-klib support) for
expect/actual + hierarchical source sets. Naming needs a call too: #1351 used kt_library for the neutral klib rule; the prototype uses kt_native_library. We should
converge on one. Decision tracked in the sub-issue "API shape + naming" (Section 6).
4a. JVM consumption of a common kt_library (a goal for (A))
A goal for (A): a klib-producing kt_library should be usable as an upstream dependency of kt_jvm_*, with the JVM rule getting real JVM artifacts out of it. A klib is never a JVM
classpath input, so this isn't automatic: it needs a JVM facet (the common sources compiled
to bytecode, surfaced as JavaInfo). Pure-common code can do this cleanly (a likely modest
change to kt_jvm_library, which already understands JavaInfo), and it needs no
co-compilation machinery, so it sequences early, right after the klib rule (P2a in Section 5).
It's a cross-rule commitment, so it's tracked as a spike, with mechanics, prototype scope, and
success criteria there: sub-issue "Spike: JVM facet for common kt_library" (Section 6).
A second, related-but-distinct case is defining JVM actuals for common expects (a JVM variant
of an expect-bearing library). Kotlin requires expect and actual in the same module, so
this can't be facet consumption at all: it needs the common expect sources and the JVM actual
sources co-compiled into one artifact. (Friend/associates visibility doesn't satisfy
expect/actual matching, so it's no shortcut.) That shares the co-compilation machinery of the
broader KMP-common work, so it lives in P6, but as P6's first slice, pulled to the front as
far as the machinery preconditions allow (JVM being the cheapest second target). This facet spike
scopes only the pure-common case.
4b. C interop and external native linking (the OpenSSL case)
Consuming a C library (the canonical example is linking OpenSSL into a Linux binary) is on
the roadmap but not built in the prototype, and it's the difference between toy native rules
and ones that build real software. Kotlin/Native does it in two steps: cinterop reads a .def
and emits a target-specific bindings klib (it runs libclang from the heavy konan deps,
so it inherits P3's provisioning, which is why it sequences after P3), then the final link must
reach the actual native library. The open question is where the headers and the compiled lib
come from: a .def+sysroot/prebuilt interim, vs. the CcInfo-bridged final form (a
Bazel cc_library feeding both cinterop and the link, hermetic and cross-target-correct). I lean toward making the CcInfo-bridged form work, largely for coherence with the rest of
the build: a C dependency should surface the ordinary Bazel way, as a cc_library (wrapping a
prebuilt binary dep, vendored sources, or whatever), carry its headers and linkable artifacts as CcInfo, and be consumed by a kt_native target like any other dep. One firm constraint on
that: it has to work by consumingcc_common/CcInfo as they stand. If wiring it into cinterop and the konanc link turns out to require changes to rules_cc itself, that's a
strong signal it's the wrong approach, and we stay on the .def+sysroot/prebuilt interim until
there's a cleaner path. That tension (konanc's self-driven link vs. Bazel cc_common;
cross-target lib availability) is exactly the genuine research item, so it's tracked as a
spike, with full mechanics and success criteria there: sub-issue "Spike: C interop + external
linking via CcInfo" (Section 6).
4c. Dependency resolution and native Maven coordinates
Pulling KMP/native dependencies in is its own design problem, and it deserves directed
attention rather than being treated as a downstream detail. Maven isn't inherently JVM, but it's
substantially JVM-oriented in practice, and native code surfaces in Maven coordinates in several
ways (KMP's Gradle Module Metadata variants, platform/classifier suffixes like -iosarm64, and
so on). The task is to research the community-standard representations first, then design K/N
dependency consumption to integrate with them, and with @smocherla-brex's Gradle-resolver
work in rules_jvm_external (#1357: ArtifactView, platform-variant selection), rather than
inventing a parallel scheme. Same guardrail as the rest of the interop work: keep it consistent
and "bazely", and push changes upstream (into rules_jvm_external or the resolver) only if we
hit a real design flaw or an unanticipated second-order effect with no clean alternative.
Tracked as a spike, with the #1357 coordination folded in: sub-issue "Spike: native
dependency resolution & Maven coordinates" (Section 6).
P2a: JVM facet for pure-common code. The common rule also emits a JVM facet (a JAR
surfaced as JavaInfo), so a kt_jvm_* target can depend on a common kt_library and get
real JVM artifacts out of it (Section 4a). No expect/actual and no co-compilation
involved, so this needs none of the P6 machinery: it can land as soon as the common rule
exists and the API shape is settled, which is why it sits here rather than with the
KMP-common work. Prototype: not built; tracked as the JVM-facet spike (Section 6).
P3: Binary linking.kt_native_binary (executables; later static/dynamic/
framework). Prototype: linux_x64 + macos_arm64 working; Apple frameworks/xcframework
later (macOS-host-locked).
P4: C interop & external native linking. A kt_native_cinterop rule (.def →
bindings klib via the bundled cinterop/libclang) plus wiring external native libraries
into the kt_native_binary link: the OpenSSL case (Section 4b). Sequenced here, right after
binary linking and ahead of the JS/Wasm and KMP expansions, because real native binaries
need it sooner. Prototype: not built; design in Section 4b. Interim .def+sysroot/prebuilt;
the CcInfo-bridged, cross-target form is the open research item.
P6: KMP common / expect-actual. First slice, pulled to the front of the phase as far
as the machinery preconditions allow: the JVM variant of an expect-bearing common
library (common expect + JVM actual co-compiled in one module), since JVM is the
cheapest second target (no native provisioning, no metadata-klib work). Then the general
case: metadata klibs, hierarchical source sets, native/JS actuals (see Section 4a). The
hardest phase overall, gated on the K2 metadata-compilation flags.
P7: Dependency resolution via the rules_jvm_external Gradle resolver
(Add support for gradle resolver rules_jvm_external#1357); add kotlin_native_library support there.
Native-in-Maven representation and the community standards for it need their own research
pass (Section 4c). Coordinate w/ @smocherla-brex.
P8: BTAPI migration (Build Tools API plan #1481): swap the konanc launcher for the Build Tools API
once its native facade ships with published coordinates. A follow-up after the swap: drop
the interim seam and integrate more deeply wherever the performance gains justify it.
Definition of done (the acceptance bar): a set of real examples build green in CI: an
iOS Compose-Multiplatform + Metro app, the same as a macOS desktop app, a Linux binary
linking a C library (e.g. OpenSSL) via cinterop, a Windows native artifact, and a shared
KMP library consumed across JVM + native.
6. Sub-issues (tracked work)
This issue is the umbrella (with #567 referencing it, and superseded by it if this roadmap
is accepted). The open decisions and spikes are broken out as child issues so each can be
discussed and closed independently; this issue tracks them. (Numbers filled in once the
children are filed.)
Decisions / discussion
Approve the roadmap & first increment(approval): sign-off on phased Native/KMP per Section 5, reconciled with @restingbull's [kmp] Outline of a roadmap #1466 (I'll drive it as that roadmap, rebased onto its conventions, not a fork), and on a hermetic source-only kt_native_library (P1+P2, no external deps, no expect/actual) as the first PR.
API shape + naming(decision; Section 4): dedicated klib rule (A) vs. fold-in (C), and kt_library vs. kt_native_library. I've landed on (A); this unblocks everything downstream.
Compiler invocation(discussion; Section 2.2): launcher-behind-a-seam now (BTAPI-swappable) vs. the worker-integration Add initial kotlin native toolchain and kt_library rule #1351 started vs. waiting on BTAPI (Build Tools API plan #1481, @agluszak). I've surfaced this to Eugene Zhuravlev (JetBrains, Build Tools API) with no pushback; isolating invocation behind a thin seam while the BTAPI stabilizes keeps the migration localized and fits the Build Tools API plan #1481 direction. I lean toward keeping the interim free of deep worker coupling.
Spike: JVM facet for common kt_library(Section 4a): prove kt_jvm_* consuming a kt_library's JavaInfo JVM facet for pure-common code. The expect/actual JVM variant (co-compiled actuals) is a distinct case, tracked as P6's first slice, not this spike.
Spike: C interop + external linking via CcInfo(Section 4b): the OpenSSL case, bridge a Bazel cc_library/CcInfo into cinterop + the konanc link, cross-target. The one genuinely unsolved piece of native linking. Lean: make the CcInfo-bridged form work by consumingcc_common as it stands; if it needs changes to rules_cc, fallback to the .def+sysroot interim.
Spike: native dependency resolution & Maven coordinates(Section 4c; coordination @smocherla-brex): how native code is represented in Maven coordinates (KMP Gradle Module Metadata variants, platform suffixes) and the community-standard way to consume it; design K/N dep resolution to integrate with those and with the how do the custom toolchain and kotlinc distribution work together? #1357 Gradle resolver, pushing
upstream only if a real design flaw forces it.
I'll drive this: split it into reviewable PRs and keep it aligned with #1466 and #1481. The
prototype branch is up at cgruber/rules_kotlin@kotlin_native if anyone wants to walk
through it. Let's land the approval and the API-shape call first; those unblock the rest.
TL;DR
There have been several parallel runs at this: a tracking issue (#567), a JS re-enable (#1347), an initial native toolchain +
kt_library(#1351), a roadmap outline (#1466), a Build Tools API plan (#1481), an external ruleset (kitterion/rules_kotlin_native), and therules_jvm_externaldependency work (bazel-contrib/rules_jvm_external#1357). None has landed, but together they've already established most of the design decisions. I've built a prototype that validates that roadmap and advances past the point where #1351 stalled. Specifically, it solves the "blocking non-hermetic downloads" problem #1466 calls out as painful, and reaches a runnable native binary (cross-compiled Linux ELF, no JVM, run in a bare container).KMP/Native hasn't been on the core team's roadmap (@Bencodes, 2024: open to community contributions but no team plan). I'm a rules_kotlin maintainer, though to be fair, I've been largely absent for a couple of years. This proposal is part of my return to active maintenance, and it's work I intend to see through. With that on the table, I'm putting KMP/Native on the roadmap by making this bid: this issue (1) consolidates the prior art and the decisions it produced, (2) reports what the prototype demonstrates, and (3) proposes a resolution to the one open API-shape question that #1351 explicitly deferred (the last thing blocking a phased landing), for us to converge on.
This builds directly on @restingbull's roadmap (#1466) rather than competing with it.
1. Prior art and the decisions it already produced
kt_librarykotlin_native_compiler_repository, a per-(exec,target)kt_native_toolchain,konan.homeexposed as aDirectoryInfoartifact, akt_libraryrule producing klibs forcommonMain-style shared code, providerKtKlibInfo{klibs, transitive_klibs}, integrated into theKotlinBuilderworker (aPlatform.NATIVEtask executor). Explicit open TODO: "decide if a separate rule is needed for klibs or fit it into existing platform-specific rules."plans/kmp.md. Phases: toolchain foundation → rules → dependency resolution. Calls out: klib is the intermediate format (he notes it is probably not fully platform-independent, he hedges); blocking the compiler's runtime downloads is painful; BTAPI is required for long-term maintenance; deps via the Gradle resolver inrules_jvm_external. Includes a hard-won list of compiler flags (see Section 3).define_kt_toolchainalready grewexec_compatible_with/target_compatible_with: the per-target toolchain selection KMP needs is already supported.rules_jvm_externalconcern; @smocherla-brex's bazel-contrib/rules_jvm_external#1357 adds Gradle-resolver support (ArtifactView, defaults to JVM, can select platform variants). Cross-repo coordination needed.Net of the decisions: native-first; klib as the shared IR artifact (JS/Wasm later);
konan.homeas a directory artifact; BTAPI as the eventual compiler API; deps through therules_jvm_externalGradle resolver; and an unresolved choice between a dedicated klibrule vs. folding klibs into the existing platform rules.
2. What the prototype demonstrates (and where it stands vs. the prior art)
Built on
masterat84fe8d3d, Bazel 9.1.0, Kotlin/Native 2.3.21. Branch:cgruber/rules_kotlin@kotlin_native.
kt_native_library:.kt → .klibfor a konan target, host and cross-target,with transitive deps. Verified hermetic: builds under
darwin-sandboxwith--sandbox_default_allow_network=false; the compiler's dependency downloader is neverinvoked for klib compilation (a fresh
KONAN_DATA_DIRstays empty).kt_native_binary: links a real native executable (-produce program). The~650–720 MB LLVM/sysroot/cross-toolchain bundle is pre-fetched by a repo rule and
staged into a sandboxed
KONAN_DATA_DIR(regenerating the compiler's.extractedmarker), so the link runs sandboxed, no network.
linux_x64verified with thenetwork disabled;
macos_arm64links in-sandbox using the host Xcode SDK. This is theitem [kmp] Outline of a roadmap #1466 flags as "this will be painful," demonstrated working.
sh_testbuilds+runs a host binary; a script cross-compiles alinux_x64binary on macOS and runs it in a bareubuntucontainer with no JRE: a truenative ELF, zero JVM references.
This advances past #1351 (which reached klib-only and stalled before hermetic dep
provisioning and binary linking). It is a prototype, though: several things are
deliberate shortcuts that should be replaced before this is the final form. Calling them
out explicitly so the gap to a mergeable design is clear:
konan.homeexposure → useDirectoryInfo. I usedfilegroup(glob(["**"]));Add initial kotlin native toolchain and
kt_libraryrule #1351 + [kmp] Outline of a roadmap #1466 expose it as a bazel-skylibDirectoryInfo(passed as a systemproperty). The directory artifact is the right answer: it avoids enumerating tens of
thousands of inputs and is far friendlier to RBE. The prototype's
filegroupis ashortcut to drop.
kt_libraryrule #1351 integrated native into theKotlinBuilderworker (Platform.NATIVE); I shellkonancvia a small launcher. Thelauncher is a deliberate, thin seam that's easy to swap for the BTAPI (Build Tools API plan #1481),
which [kmp] Outline of a roadmap #1466 says is the long-term target and "not worth undertaking [other
compiler-integration] effort while pending." I've run this past Eugene
Zhuravlev (@eugenezh) at JetBrains (their point person for the Bazel Kotlin rules, working
on the Build Tools API integration), flagging both the wrapped-compiler question and the
deliberate choice here to keep K/N compiler invocation isolated behind a thin seam while
the Build Tools API is still stabilizing; he didn't raise concerns. That isolation is the point: it keeps the eventual BTAPI migration a
localized swap rather than a rewrite, so the two efforts don't have to move in lockstep.
Whether the interim seam should be the launcher or the worker is still an open call
(tracked in the Compiler invocation sub-issue, Section 6).
-Xklib-relative-path-base/-Xdebug-prefix-map(needed for cross-machine remotecache hits) or
-Xoverride-konan-properties=airplaneMode=true(belt-and-suspendersdownload disable on top of pre-staging). All should go in.
fetched without
sha256pins in the prototype (fast iteration). The final form needspinned, integrity-checked downloads. Non-negotiable for a hermetic, reproducible repo
rule.
macos-aarch64host deptable is filled in; the rule shape is host-generic but the other host entries
(
linux-x86_64, etc.) and their hashes are stubs. Cross-target (--platforms) isverified; cross-host provisioning is not yet populated.
In other words: the prototype is empirical confirmation that #1466's plan works, plus a
solution to its hardest Phase-1 item. But it is a prototype, and the items above (plus
the API-shape decision in Section 4) are what stand between it and a form worth merging. It
should be re-based onto #1351/#1466's conventions, not kept as a parallel third thing.
3. The flags/lessons, reconciled
#1466 lists the hard-won compiler flags; the prototype exercised several. Consolidated,
the working recipe is:
konan.homeas aDirectoryInfo; pass via system property.dependencies/.extractedmarker and set-Xoverride-konan-properties=airplaneMode=true.-Xauto-cache-dir/-Xauto-cache-fromto keep the compiler'sinternal cache out of the read-only external repo (the prototype hit exactly the
klib/cache/<target>STATICwrite-back [kmp] Outline of a roadmap #1466 predicts; redirecting fixes it).-Xklib-relative-path-base,-Xdebug-prefix-map;explicit stdlib as a declared input (
-no-default-libs+ explicit-library).platform()s; konan target comes from--platformskt_libraryrule #1351 and the prototype both work).4. The open API-shape question (tracked as a sub-issue)
This is #1351's deferred TODO and the crux. The constraint that decides it concerns how
"common" code can be packaged. Two things observed/known:
sources compiled for different targets produce klibs whose manifests read
native_targets=macos_arm64vs.native_targets=linux_x64. So a platform klib is nota single artifact consumable by every target. ([kmp] Outline of a roadmap #1466 flags this too, though
@restingbull hedges whether it's still strictly true; the prototype manifests are the
concrete evidence.)
commonMaincode is a separate question I have not yet built: KMP modelsit as a metadata klib plus per-target compilations, and
expect/actualrequires theactualto be compiled in the same module as itsexpect, so "common" is not astandalone linked artifact once expect/actual is used; it's source co-compiled per target
(or a metadata klib). This is unverified in the prototype; it's the KMP-common work
(phase P6 in Section 5 below).
Given that, the realistic options:
kt_native_library/kt_libraryproduces klibs;platform-specific rules consume them. This is what Add initial kotlin native toolchain and
kt_libraryrule #1351 built and what @restingbull'snative-first steer (Re-enable Javascript support #1347) points to. Clean; covers pure-common well; needs an
expect/actual story (a metadata klib or a friend-association, and note rules_kotlin
already has
associates).--platforms. MostBazel-native; expect/actual "just works" because each platform build co-compiles
common+platform; but
bazel build //:xproduces different artifacts by config and itfights the rich existing
kt_jvm_librarysurface.kt_libraryrule #1351's TODO; least new surface,but muddies
kt_jvm_library.My current thinking: (A) as the core (it matches the prior decisions and the
klib-not-portable reality), with target selection by
--platforms(the configurationhalf of (B)), and a later
kt_multiplatform_library(or metadata-klib support) forexpect/actual + hierarchical source sets. Naming needs a call too: #1351 used
kt_libraryfor the neutral klib rule; the prototype useskt_native_library. We shouldconverge on one. Decision tracked in the sub-issue "API shape + naming" (Section 6).
4a. JVM consumption of a common
kt_library(a goal for (A))A goal for (A): a klib-producing
kt_libraryshould be usable as an upstream dependency ofkt_jvm_*, with the JVM rule getting real JVM artifacts out of it. A klib is never a JVMclasspath input, so this isn't automatic: it needs a JVM facet (the common sources compiled
to bytecode, surfaced as
JavaInfo). Pure-common code can do this cleanly (a likely modestchange to
kt_jvm_library, which already understandsJavaInfo), and it needs noco-compilation machinery, so it sequences early, right after the klib rule (P2a in Section 5).
It's a cross-rule commitment, so it's tracked as a spike, with mechanics, prototype scope, and
success criteria there: sub-issue "Spike: JVM facet for common
kt_library" (Section 6).A second, related-but-distinct case is defining JVM
actuals for commonexpects (a JVM variantof an expect-bearing library). Kotlin requires
expectandactualin the same module, sothis can't be facet consumption at all: it needs the common
expectsources and the JVMactualsources co-compiled into one artifact. (Friend/
associatesvisibility doesn't satisfyexpect/actual matching, so it's no shortcut.) That shares the co-compilation machinery of the
broader KMP-common work, so it lives in P6, but as P6's first slice, pulled to the front as
far as the machinery preconditions allow (JVM being the cheapest second target). This facet spike
scopes only the pure-common case.
4b. C interop and external native linking (the OpenSSL case)
Consuming a C library (the canonical example is linking OpenSSL into a Linux binary) is on
the roadmap but not built in the prototype, and it's the difference between toy native rules
and ones that build real software. Kotlin/Native does it in two steps:
cinteropreads a.defand emits a target-specific bindings klib (it runs libclang from the heavy konan deps,
so it inherits P3's provisioning, which is why it sequences after P3), then the final link must
reach the actual native library. The open question is where the headers and the compiled lib
come from: a
.def+sysroot/prebuilt interim, vs. theCcInfo-bridged final form (aBazel
cc_libraryfeeding bothcinteropand the link, hermetic and cross-target-correct).I lean toward making the
CcInfo-bridged form work, largely for coherence with the rest ofthe build: a C dependency should surface the ordinary Bazel way, as a
cc_library(wrapping aprebuilt binary dep, vendored sources, or whatever), carry its headers and linkable artifacts as
CcInfo, and be consumed by akt_nativetarget like any other dep. One firm constraint onthat: it has to work by consuming
cc_common/CcInfoas they stand. If wiring it intocinteropand thekonanclink turns out to require changes torules_ccitself, that's astrong signal it's the wrong approach, and we stay on the
.def+sysroot/prebuilt interim untilthere's a cleaner path. That tension (konanc's self-driven link vs. Bazel
cc_common;cross-target lib availability) is exactly the genuine research item, so it's tracked as a
spike, with full mechanics and success criteria there: sub-issue "Spike: C interop + external
linking via
CcInfo" (Section 6).4c. Dependency resolution and native Maven coordinates
Pulling KMP/native dependencies in is its own design problem, and it deserves directed
attention rather than being treated as a downstream detail. Maven isn't inherently JVM, but it's
substantially JVM-oriented in practice, and native code surfaces in Maven coordinates in several
ways (KMP's Gradle Module Metadata variants, platform/classifier suffixes like
-iosarm64, andso on). The task is to research the community-standard representations first, then design K/N
dependency consumption to integrate with them, and with @smocherla-brex's Gradle-resolver
work in
rules_jvm_external(#1357: ArtifactView, platform-variant selection), rather thaninventing a parallel scheme. Same guardrail as the rest of the interop work: keep it consistent
and "bazely", and push changes upstream (into
rules_jvm_externalor the resolver) only if wehit a real design flaw or an unanticipated second-order effect with no clean alternative.
Tracked as a spike, with the #1357 coordination folded in: sub-issue "Spike: native
dependency resolution & Maven coordinates" (Section 6).
5. Proposed phased plan (reconciled with #1466)
This mirrors #1466's phases; the prototype already has working code for P1–P3.
DirectoryInfo), per-(exec,target)toolchains, blocking non-hermetic downloads (pre-staged deps + airplaneMode).
Prototype: working + network-off verified; remaining-for-final:
DirectoryInfo(vs.the prototype
filegroup), pinned depsha256s, full host matrix (see Section 2).kt_native_library(source-only klib, transitive deps),reproducibility flags, cache isolation. Prototype: done; add
-Xklib-relative-path-base/-Xdebug-prefix-map.surfaced as
JavaInfo), so akt_jvm_*target can depend on a commonkt_libraryand getreal JVM artifacts out of it (Section 4a). No
expect/actualand no co-compilationinvolved, so this needs none of the P6 machinery: it can land as soon as the common rule
exists and the API shape is settled, which is why it sits here rather than with the
KMP-common work. Prototype: not built; tracked as the JVM-facet spike (Section 6).
kt_native_binary(executables; later static/dynamic/framework). Prototype: linux_x64 + macos_arm64 working; Apple frameworks/xcframework
later (macOS-host-locked).
kt_native_cinteroprule (.def→bindings klib via the bundled
cinterop/libclang) plus wiring external native librariesinto the
kt_native_binarylink: the OpenSSL case (Section 4b). Sequenced here, right afterbinary linking and ahead of the JS/Wasm and KMP expansions, because real native binaries
need it sooner. Prototype: not built; design in Section 4b. Interim
.def+sysroot/prebuilt;the
CcInfo-bridged, cross-target form is the open research item.as the machinery preconditions allow: the JVM variant of an expect-bearing common
library (common
expect+ JVMactualco-compiled in one module), since JVM is thecheapest second target (no native provisioning, no metadata-klib work). Then the general
case: metadata klibs, hierarchical source sets, native/JS
actuals (see Section 4a). Thehardest phase overall, gated on the K2 metadata-compilation flags.
rules_jvm_externalGradle resolver(Add support for gradle resolver rules_jvm_external#1357); add
kotlin_native_librarysupport there.Native-in-Maven representation and the community standards for it need their own research
pass (Section 4c). Coordinate w/ @smocherla-brex.
once its native facade ships with published coordinates. A follow-up after the swap: drop
the interim seam and integrate more deeply wherever the performance gains justify it.
Definition of done (the acceptance bar): a set of real examples build green in CI: an
iOS Compose-Multiplatform + Metro app, the same as a macOS desktop app, a Linux binary
linking a C library (e.g. OpenSSL) via cinterop, a Windows native artifact, and a shared
KMP library consumed across JVM + native.
6. Sub-issues (tracked work)
This issue is the umbrella (with #567 referencing it, and superseded by it if this roadmap
is accepted). The open decisions and spikes are broken out as child issues so each can be
discussed and closed independently; this issue tracks them. (Numbers filled in once the
children are filed.)
Decisions / discussion
Approve the roadmap & first increment (approval): sign-off on phased Native/KMP per Section 5, reconciled with @restingbull's [kmp] Outline of a roadmap #1466 (I'll drive it as that roadmap, rebased onto its conventions, not a fork), and on a hermetic source-only
kt_native_library(P1+P2, no external deps, no expect/actual) as the first PR.Approve phased Kotlin/Native + KMP support along this roadmap #1683
API shape + naming (decision; Section 4): dedicated klib rule (A) vs. fold-in (C), and
kt_libraryvs.kt_native_library. I've landed on (A); this unblocks everything downstream.API shape + rule naming for the native/common rules #1684
Compiler invocation (discussion; Section 2.2): launcher-behind-a-seam now (BTAPI-swappable) vs. the worker-integration Add initial kotlin native toolchain and
kt_libraryrule #1351 started vs. waiting on BTAPI (Build Tools API plan #1481, @agluszak). I've surfaced this to Eugene Zhuravlev (JetBrains, Build Tools API) with no pushback; isolating invocation behind a thin seam while the BTAPI stabilizes keeps the migration localized and fits the Build Tools API plan #1481 direction. I lean toward keeping the interim free of deep worker coupling.Compiler invocation: launcher seam vs. worker integration vs. await BTAPI #1685
Spikes / research
Spike: JVM facet for common
kt_library(Section 4a): provekt_jvm_*consuming akt_library'sJavaInfoJVM facet for pure-common code. The expect/actual JVM variant (co-compiledactuals) is a distinct case, tracked as P6's first slice, not this spike.Spike: JVM consumption of a common
kt_library(theJavaInfofacet) #1686Spike: C interop + external linking via
CcInfo(Section 4b): the OpenSSL case, bridge a Bazelcc_library/CcInfointocinterop+ thekonanclink, cross-target. The one genuinely unsolved piece of native linking. Lean: make theCcInfo-bridged form work by consumingcc_commonas it stands; if it needs changes torules_cc, fallback to the.def+sysroot interim.Spike: C interop + external native linking via
CcInfo(the OpenSSL case) #1687Spike: native dependency resolution & Maven coordinates (Section 4c; coordination @smocherla-brex): how native code is represented in Maven coordinates (KMP Gradle Module Metadata variants, platform suffixes) and the community-standard way to consume it; design K/N dep resolution to integrate with those and with the how do the custom toolchain and kotlinc distribution work together? #1357 Gradle resolver, pushing
upstream only if a real design flaw forces it.
Spike: native dependency resolution & Maven coordinates (↔ rules_jvm_external#1357) #1688
I'll drive this: split it into reviewable PRs and keep it aligned with #1466 and #1481. The
prototype branch is up at cgruber/rules_kotlin@kotlin_native if anyone wants to walk
through it. Let's land the approval and the API-shape call first; those unblock the rest.