Skip to content

[jitlayers] add missing JD parameter to remaining external API call#60988

Merged
IanButterworth merged 1 commit into
masterfrom
jn/orcapi-jd
May 12, 2026
Merged

[jitlayers] add missing JD parameter to remaining external API call#60988
IanButterworth merged 1 commit into
masterfrom
jn/orcapi-jd

Conversation

@vtjnash
Copy link
Copy Markdown
Member

@vtjnash vtjnash commented Feb 10, 2026

This previously relied on the user making all symbol names globally unique, despite that being essentially impossible to query or satisfy, and despite the API also requiring the user to provide a private namespace to track it in anyways. Also have claude add a few tests.

@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Feb 10, 2026

Fixes the external jitdylib design issues here: #60031 (comment)

@gbaraldi
Copy link
Copy Markdown
Member

So users of this API have to create their own JDs?

@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Feb 11, 2026

Obviously that is how you avoid name conflict mistakes, yes. It is also just a simple function renaming, since you always had to pass in the JD (JLJITCreateJITDylib), you just couldn't make or query it previously

@topolarity topolarity requested a review from xal-0 February 12, 2026 15:19
@topolarity
Copy link
Copy Markdown
Member

Test failures seem related https://buildkite.com/julialang/julia-master/builds/54534#019c49e7-80a1-49a9-98c9-a8581d2f1202/L1105-L1108:

llvmcall2                                                   (6) |         failed at 2026-02-10T15:40:47.761
Test Failed at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-P2C257CXGN.0/build/default-honeycrisp-P2C257CXGN-0/julialang/julia-master/julia-5fc9df738a/share/julia/test/llvmcall2.jl:122
  Expression: unsafe_string(LLVMGetErrorMessage(err)) == "Symbols not found: [ jl_get_ptls_states ]"
   Evaluated: "" == "Symbols not found: [ jl_get_ptls_states ]"

Otherwise in favor.

@gbaraldi gbaraldi requested a review from vchuravy February 12, 2026 16:40
@vtjnash vtjnash added the merge me PR is reviewed. Merge when all tests are passing label May 4, 2026
@gbaraldi
Copy link
Copy Markdown
Member

gbaraldi commented May 6, 2026

@vchuravy or @wsmoses this will matter for enzyme so please double check

@vchuravy
Copy link
Copy Markdown
Member

vchuravy commented May 7, 2026

this will matter for enzyme so please double check

Can't really estimate the impact yet, since we are not yet ready for 1.13, yet alone nightly.

I put up a PR to LLVM.jl to adjust to the change here JuliaLLVM/LLVM.jl#557

This previously relied on the user making all symbol names globally
unique, despite that being essentially impossible to query or satisfy,
and despite the API also requiring the user to provide a private
namespace to track it in anyways. Now users are expected to properly use
OrcJIT namespaces. This affects ccall users too, since we only want
explicitly named libraries to find symbols consistently across all
platforms. But this wasn't consistent about the handling on Windows.
Also have claude add a few tests for this API.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
@IanButterworth IanButterworth merged commit d5f1466 into master May 12, 2026
9 checks passed
@IanButterworth IanButterworth deleted the jn/orcapi-jd branch May 12, 2026 00:50
@IanButterworth IanButterworth removed the merge me PR is reviewed. Merge when all tests are passing label May 12, 2026
maleadt pushed a commit to JuliaLLVM/LLVM.jl that referenced this pull request May 12, 2026
JLJITGetExternalJITDylib is replaced by JLJITCreateJITDylib (creates a
fresh JITDylib linked to GlobalJD/SessionJD), and JLJITLookup is replaced
by JLJITJDLookup which takes an explicit JITDylib.

The public API is made consistent across Julia versions: lookup always
requires a JITDylib argument. On pre-1.13 Julia the JD is accepted but
internally ignored (the old symbol-table search is used). On 1.13+ the JD
is passed through to the new API. The old no-JD lookup(jljit, name) is
deprecated on pre-1.13 Julia and removed on 1.13+.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
maleadt added a commit to JuliaLLVM/LLVM.jl that referenced this pull request May 12, 2026
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Tim Besard <tim.besard@gmail.com>
gbaraldi pushed a commit to JuliaLang/AllocCheck.jl that referenced this pull request May 15, 2026
LLVM.jl v9.8.1 deprecated the 2-arg `lookup(jljit::JuliaOJIT, name)` form
in favor of `lookup(jljit, jd, name)` (the old form was removed in
JuliaLang/julia#60988 for Julia 1.14.0-DEV.2171). The depwarn fired
during the JIT lookup inside `@check_allocs`-generated code, which
caused downstream packages testing `@check_allocs` with `@test_nowarn`
to fail.

Pass the already-constructed `jd::JITDylib` explicitly. Bumps LLVM
compat to 9.8.1 since the 3-arg JuliaOJIT method was added in that
release.

Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com>
wsmoses pushed a commit to EnzymeAD/Enzyme.jl that referenced this pull request May 16, 2026
`lookup(jljit, name)` is deprecated in LLVM.jl 9.8.0+ (adapting to
JuliaLang/julia#60988). Update both call sites in orcv2.jl to pass an
explicit JITDylib and bump the compat bound to 9.8.1.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
hardikxk pushed a commit to hardikxk/julia that referenced this pull request May 18, 2026
…uliaLang#60988)

This previously relied on the user making all symbol names globally
unique, despite that being essentially impossible to query or satisfy,
and despite the API also requiring the user to provide a private
namespace to track it in anyways. Also have claude add a few tests.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
maleadt added a commit to JuliaGPU/GPUCompiler.jl that referenced this pull request May 19, 2026
JuliaLang/julia#60988 changed `JITDylib(jljit)` to return a fresh
private dylib that's only searched outward (to GlobalJD/SessionJD), so
defining "deferred_codegen" there is invisible to JIT-compiled user
code in JD.

Use `LLVM.lookup_dylib(es, "JuliaGlobals")` to grab the existing
GlobalJD — which JD links to with `MatchExportedSymbolsOnly` — and
define the absolute symbol there. JD's lookup chain then resolves
"deferred_codegen" through GlobalJD.

The pre-1.14 branch is unchanged.
maleadt added a commit to JuliaGPU/GPUCompiler.jl that referenced this pull request May 19, 2026
JuliaLang/julia#60988 changed `JITDylib(jljit)` to return a fresh
private dylib that's only searched outward (to GlobalJD/SessionJD), so
defining "deferred_codegen" there is invisible to JIT-compiled user
code in JD.

Use `LLVM.lookup_dylib(es, "JuliaGlobals")` to grab the existing
GlobalJD, which JD links to with `MatchExportedSymbolsOnly`, and
define the absolute symbol there. JD's lookup chain then resolves
"deferred_codegen" through GlobalJD.
mkitti pushed a commit to mkitti/julia that referenced this pull request May 22, 2026
…uliaLang#60988)

This previously relied on the user making all symbol names globally
unique, despite that being essentially impossible to query or satisfy,
and despite the API also requiring the user to provide a private
namespace to track it in anyways. Also have claude add a few tests.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants