Possible numba version conflict

[HenryRWinterbottom]

The following exception is being raised when executing the PI application.

Traceback (most recent call last):
  File "/scratch/henrywinterbottom/trunk/stoa/stoa_diags/sorc/stoa_diags/tc/be2002_pi.py", line 161, in compute
    results = pi(
              ^^^
  File "/scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/numba/core/dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function min>) found for signature:
 
 >>> min(LiteralList((Array(float64, 1, 'C', False, aligned=True), float64)))
 
There are 8 candidate implementations:
     - Of which 4 did not match due to:
     Overload in function 'MinMaxBase.generic': File: numba/core/typing/builtins.py: Line 906.
       With argument(s): '(Poison<LiteralList((Array(float64, 1, 'C', False, aligned=True), float64))>)':
      Rejected as the implementation raised a specific error:
        TypingError: Poison type used in arguments; got Poison<LiteralList((Array(float64, 1, 'C', False, aligned=True), float64))>
  raised from /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/numba/core/types/functions.py:236
     - Of which 2 did not match due to:
     Overload of function 'min': File: <numerous>: Line N/A.
       With argument(s): '(LiteralList((Array(float64, 1, 'C', False, aligned=True), float64)))':
      No match.
     - Of which 1 did not match due to:
     Overload in function 'indval_min': File: numba/cpython/builtins.py: Line 550.
       With argument(s): '(LiteralList((Array(float64, 1, 'C', False, aligned=True), float64)))':
      Rejected as the implementation raised a specific error:
        TypingError: missing a required argument: 'indval2'
  raised from /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/numba/core/typing/templates.py:784
     - Of which 1 did not match due to:
     Overload in function 'boolval_min': File: numba/cpython/builtins.py: Line 579.
       With argument(s): '(LiteralList((Array(float64, 1, 'C', False, aligned=True), float64)))':
      Rejected as the implementation raised a specific error:
        TypingError: missing a required argument: 'val2'
  raised from /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/numba/core/typing/templates.py:784

During: resolving callee type: Function(<built-in function min>)
During: typing of call at /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/tcpyPI/pi.py (162)


File "../venv/lib/python3.11/site-packages/tcpyPI/pi.py", line 162:
def cape(TP,RP,PP,T,R,P,ascent_flag=0,ptop=50,miss_handle=1):
    <source elided>
    RH=EVP/ESP                              # Parcel's relative humidity
    RH=min([RH,1.0])                        # ensure that the relatively humidity does not exceed 1.0
    ^

During: resolving callee type: type(CPUDispatcher(<function cape at 0x743bcce47560>))
During: typing of call at /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/tcpyPI/pi.py (610)

During: resolving callee type: type(CPUDispatcher(<function cape at 0x743bcce47560>))
During: typing of call at /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/tcpyPI/pi.py (610)

During: resolving callee type: type(CPUDispatcher(<function cape at 0x743bcce47560>))
During: typing of call at /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages/tcpyPI/pi.py (610)


File "../venv/lib/python3.11/site-packages/tcpyPI/pi.py", line 610:
def pi(SSTC,MSL,P,TC,R,CKCD=0.9,ascent_flag=0,diss_flag=1,V_reduc=0.8,ptop=50,miss_handle=1):
    <source elided>
        RP=constants.EPS*R[NK]*MSL/(PP*(constants.EPS+R[NK])-R[NK]*MSL)
        result = cape(TP,RP,PP,T,R,P,ascent_flag,ptop,miss_handle)

The versions of numpy and numba related to this exception are:

(venv) (base) henrywinterbottom@lilith: pip show numpy
Name: numpy
Version: 1.24.4
Summary: Fundamental package for array computing in Python
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: 
License: BSD-3-Clause
Location: /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages
Requires: 
Required-by: astropy, Cartopy, cftime, cgen, contourpy, copernicusmarine, gsw, h5py, matplotlib, MetPy, netCDF4, numba, numcodecs, pandas, parcels, pyerfa, pyspharm, scipy, shapely, tcpyPI, windspharm, xarray, zarr

and

Name: numba
Version: 0.57.0
Summary: compiling Python code using LLVM
Home-page: https://numba.pydata.org
Author: 
Author-email: 
License: BSD
Location: /scratch/henrywinterbottom/trunk/stoa/stoa_diags/venv/lib/python3.11/site-packages
Requires: llvmlite, numpy
Required-by: tcpyPI

[maresb]

Thanks @HenryRWinterbottom for the report! I believe it's not a versioning issue. Instead, it seems that RH is somehow entering as a single-element array instead of a scalar, and that's causing numba to choke.

Because the types are not strictly enforced, it's hard to distinguish between bad input data and a bug in the code. Would it be possible for you to share a minimal working example, including data, of how to reproduce this so that we can figure out how to fix it?

[HenryRWinterbottom]

@maresb Thank you for quickly following up.

You are correct, it is not a versioning issue but a data format issue. I am guessing, and I think as you are suggesting, that some of the legacy numpy data handling is no longer supported. Once I converted everything from nump.array to DataArray, things worked as expected. And, your run_sample.py script was very helpful for my debug.

I am happy to close this if you do not need any additional information from me.

Thank you again for you help.

[maresb]

Wonderful, I am glad you got it working!

I doubt you will be the only one to encounter this, and I would like to make our code more robust so that we at least can provide an informative error. Depending on your availability of course, anything you could do to provide instructions to reproduce your original problem would be appreciated!

[HenryRWinterbottom]

@maresb No problem. Let me get caught up on a few other things and I will dump what I did in a comment.

[Chwithin]

I also encountered the same problem, but I really couldn’t figure out how to set and convert the data types since I had never used Python before. In the end, I chose to disable numba, and then the code was able to run. Although it runs quite slowly, it feels much better than struggling endlessly with data types.
The specific setting is to add two lines of code at the beginning of run_sample.py
import os
os.environ['TCPYPI_DISABLE_NUMBA'] = '1'

[HenryRWinterbottom]

I also encountered the same problem, but I really couldn’t figure out how to set and convert the data types since I had never used Python before. In the end, I chose to disable numba, and then the code was able to run. Although it runs quite slowly, it feels much better than struggling endlessly with data types. The specific setting is to add two lines of code at the beginning of run_sample.py import os os.environ['TCPYPI_DISABLE_NUMBA'] = '1'

@Chwithin This was my workaround. I didn't have time to investigate further, but I think this workaround could simply be a note for the user. Also, the calculations are slow, especially with larger grid sizes. I was able to improve the processing time by using xarray and asyncio in my pipeline, which uses this package. If you haven't considered that option, it might be helpful. Just my thoughts.

[maresb]

If anyone could give me steps to reproduce this it would be greatly appreciated. I need a specific dataset, plus exact instructions of all the steps leading up to the error.

[HenryRWinterbottom]

I don't have specific input data or instructions, but the most recent commit that does not produce this issue is ba97934cabfdb4dd5333e9450f0c1407340ec5dc.

[Chwithin]

@maresb I just directly ran run_sample.py from https://github.com/dgilford/tcpyPI, using the sample data sample_data.nc provided in the repository. Then the error occurred, and the Numba version is shown in the screenshot below.

However, if I add these two lines in the code (run_sample.py):

import os
os.environ['TCPYPI_DISABLE_NUMBA'] = '1'

it runs normally. The speed is a bit slower, but still acceptable — it takes about 25 minutes.

[maresb]

@Chwithin Thanks for the detailed report! I've been trying to reproduce this but haven't been able to so far with numba 0.60.0 + Python 3.9 + various numpy/xarray combinations.

Could you please share the output of pip freeze (or pip list) from your py39 environment? I need the exact versions of all installed packages to reproduce your setup. You can run:

pip freeze > requirements.txt

and paste the contents here (or attach the file). Thanks!

[Chwithin]

@maresb Thank you for your attention to this issue. Here is my current configuration: requirements.txt