[BUG] Errors when building PyCall.jl

[MilesCranmer]

tldr, delete the folders:

• ~/.julia/packages/PyCall
• ~/.julia/prefs/PyCall
• ~/.julia/environments/pysr-{your-pysr-version}

and re-run python -m pysr install.

---

I am documenting this error because it has shown up a lot for people, and I want to document the solution.

The error is basically that after upgrading Python or Julia or both, and running PySR or re-installing PySR, you get the following error:

[ Info: Trying to import PyCall...
┌ Info: PyCall is already installed but not compatible with this Python
└ executable. Re-building PyCall...
[ Info: Run `Pkg.build(“PyCall”; verbose=true)`
ERROR: LoadError: The following package names could not be resolved:
 * PyCall (not found in project or manifest)

The solution to this is to delete your ~/.julia/packages/PyCall folder. Then, simply re-run python -m pysr install.

If this gives you further errors in building, you should also delete ~/.julia/prefs/PyCall, and re-run the install step. For further issues, you could delete ~/.julia/environments folder (or more specifically, the folders ~/.julia/environments/pysr-{version} and ~/.julia/environments/{julia_version}, if you want to keep your other environments).

---

This should fix the problem. This issue is basically that the old installed version of PyCall.jl is attached to a specific combination of Julia and Python. And, for whatever reason, re-running julia.install() doesn't update the built version of PyCall.jl.

This is a temporary manual solution. In the future PySR should handle this automatically.

[MilesCranmer]

It might be good to automatically reset ~/.julia/prefs/PyCall whenever julia.install() is called? What do you think @mkitti?

[MilesCranmer]

I think PySR should catch julia.tools.PyCallInstallError and print out some PySR-relevant information. Especially the above debugging tips.

I think another couple of changes would simplify the install process for the user:

1. Have the PySR environment include the hash of the python path, so that using a different python would result in a new PySR install.
2. Have the PyCall.jl variables overwritten if PyCall is already installed in, e.g., @v1.8. Right now it seems like install PyCall.jl globally will just freeze the python path.

[mkitti]

Couldn't we just check for a mismatch between PyCall.python and whatever the current Python executable is? How do you find the latter?

[MilesCranmer]

I'm not sure, because the Python executable could have the same name. So ideally we would want to also store the version of Python and any other meta information that might affect things.

[MilesCranmer]

Also, a note to myself: should add unit tests for issues like #287 and #222. Would probably need to perform the test within a docker container with pyenv installed, and switch pyenv partway.

[mkitti]

We need more diagnostics here.

Can you catch this error and do one of the following?

println(stderr, Base.active_project())
println(stderr, read(Base.active_project()), String))
manifest = joinpath(dirname(Base.active_project()), "Manifest.toml"))
if isfile(manifest)
    println(stderr, read(manifest, String))
end

Somehow either Project.toml or Manifest.toml had been corrupted or the active environment is not what we think it is.

If there is something to delete, it might be the deps folder within thr package.