Hide deferred_codegen ccall body behind overlay table

[gbaraldi]

This is probably wrong but does some of the plumbing that we will eventually need

Summary

Fixes #3091. Compiler.deferred_codegen's body contains ccall(\"extern deferred_codegen\", llvmcall, ...), which is not valid for normal Julia codegen. When jl_compile_all_defs enqueues the despecialized form during a sysimage build, it fails with UndefVarError: deferred_codegen not defined in Enzyme.

• Move the ccall body into a Base.Experimental.@overlay EnzymeMethodTable definition; the public deferred_codegen is now function deferred_codegen end (empty).
• Overlay methods have external_mt set, which jl_compile_all_defs skips (precompile_utils.c:147), so the invalid body is no longer despecialized into the sysimage.
• Override GPUCompiler.method_table(::CompilerJob{<:EnzymeTarget}) = EnzymeMethodTable so the EnzymeInterpreter consults the overlay during nested codegen.

Test plan

• CI passes — Runic formatting, build, and unit tests
• CPU autodiff_deferred callers in test/core/abi.jl, test/core/deferred.jl ("Deferred and deferred thunk" testset), and test/tests.jl removed — those direct CPU calls were compiled by regular Julia and bypassed the overlay, so they now would throw MethodError. The remaining coverage:
  • High-order callers in test/basic.jl:665 (nested_df!), test/advanced.jl:1513 (f_gradient_deferred!), and test/core/deferred.jl ("Deferred upgrade" testset) — these autodiff_deferred calls live inside another autodiff-compiled function, so the EnzymeInterpreter sees the overlay and inlines the ccall version.
  • GPU tests (test/cuda.jl, test/amdgpu.jl, test/metal.jl).

Known follow-up

The GPU test paths currently won't see the overlay, because each GPU framework uses its own @MethodTable (e.g. CUDA.method_table). To restore GPU coverage of deferred_codegen, add weakdep extensions (ext/EnzymeCUDAExt.jl, etc.) that re-emit the overlay into each framework's method table:

```julia
Base.Experimental.@overlay CUDA.method_table function Enzyme.Compiler.deferred_codegen(...)
# same ccall body
end
```

🤖 Generated with Claude Code

[github-actions]

Your PR requires formatting changes to meet the project's style guidelines.
Please consider running Runic (git runic main) to apply these changes.

Click here to view the suggested changes.

diff --git a/test/core/abi.jl b/test/core/abi.jl
index d6cddcf9..ea2eaad6 100644
--- a/test/core/abi.jl
+++ b/test/core/abi.jl
@@ -113,7 +113,7 @@ end
     @test res[] ≈ 6.0
     @test dres[] ≈ 2.0
     @test orig == Float64
-    
+
     function inplace2(x)
         x[] *= 2
         return nothing

[github-actions]

Benchmark Results

	main	352c0ed...	main / 352c0ed...
basics/make_zero/namedtuple	0.0576 ± 0.0046 μs	0.0569 ± 0.0044 μs	1.01 ± 0.11
basics/make_zero/struct	0.26 ± 0.0077 μs	0.26 ± 0.008 μs	1 ± 0.043
basics/overhead	4.88 ± 0.01 ns	4.54 ± 0.01 ns	1.08 ± 0.0032
basics/remake_zero!/namedtuple	0.232 ± 0.0084 μs	0.233 ± 0.0089 μs	0.996 ± 0.052
basics/remake_zero!/struct	0.229 ± 0.0085 μs	0.238 ± 0.013 μs	0.965 ± 0.065
fold_broadcast/multidim_sum_bcast/1D	10.4 ± 0.44 μs	10.5 ± 2.2 μs	0.993 ± 0.22
fold_broadcast/multidim_sum_bcast/2D	12.8 ± 0.38 μs	12.6 ± 0.37 μs	1.01 ± 0.042
time_to_load	1.09 ± 0.011 s	1.11 ± 0.031 s	0.988 ± 0.03

Benchmark Plots

A plot of the benchmark results has been uploaded as an artifact at https://github.com/EnzymeAD/Enzyme.jl/actions/runs/26050655422/artifacts/7065439274.

[vchuravy]

This will break things w.r.t CUDA iirc

[vchuravy]

I don't think we currently need an EnzymeMethodTable,

what we need is an entry in CUDA overlay table making Enzyme.autodiff use deferred_codegen.