When Jitting with Numba, since we send a new function for each execution in the payload, the JIT cache gets invalidated. One way of solving this is by making it possible to cache a function and then reuse the same function (without unpickling it) if it's already been executed in a worker. (This doesn't involve any complex caching, just store the functions that have been pickled in a function_name -> function dict.
When Jitting with Numba, since we send a new function for each execution in the payload, the JIT cache gets invalidated. One way of solving this is by making it possible to cache a function and then reuse the same function (without unpickling it) if it's already been executed in a worker. (This doesn't involve any complex caching, just store the functions that have been pickled in a function_name -> function dict.