-
Notifications
You must be signed in to change notification settings - Fork 0
Installing and Running Python
What machines will python run on? We have run python on Sun Unix, on various version of Redhat linux, and on MAC OSX (Tiger --> Mountain Lion). It is compiled using mpicc, with an option to compile with gcc. It uses the Gnu Scientific Libraries (gsl)
This more or less works in the structure branch, which has a makefile and install.txt, but requires a release of Python76b to work. once this is done I will finalise this.
Python and the various routines associated are set up in a self-contained directory structure. The basic directory structure and the data files that one needs to run Python need to be retrieved (and most likely recompiled).
If you have git installed: To obtain the directory structure, simply retrieve it using git as follows to clone the directory structure:
$ git clone https://github.com/agnwinds/python.git -b structure
You then need to cd to the new directory and set your environment variables
$ export PYTHON = /path/to/python/
$ cd $PYTHON
$ make install
$ make clean
note that export syntax is for bash- for csh use
$ setenv PYTHON /path/to/python/
Without git: Use the ZIP function under the structures branch, and then download .tar.gz versions of the python source under releases.
Once you have a directory /path/to/python/ which contains the structure, place the unpacked tar.gz python source folder under /path/to/python/progs/
$ export PYTHON = /path/to/python/
$ cd $PYTHON
$ make GIT=False install
$ make clean
$ cd progs/python_xx #replace xx version you download
$ make clean
$ make CC=gcc python # if you want to use mpicc, ignore the CC=gcc
$ make clean
Again, for csh use
$ setenv PYTHON /path/to/python/
As you can tell, the git install is simpler!
The only environment variables that is needed is to define PYTHON. For bash or sh users simply place the following:
export PYTHON=/path/to/python/
where $HOME is you would modify $HOME to read something else if you are not installing putting this in your top level directory. You also need to add $PYTHON/bin to your path, e.g::
PATH = $PATH:$PYTHON/bin
for c shell, use the following commands
setenv PYTHON /path/to/python/
setenv PATH "${PATH}:${PYTHON}"
Once you have added PYTHON to your path, copy the script Setup_Py_Dir to $PYTHON/bin/:
$ cp $PYTHON/py_progs/setup_scripts/Setup_Py_Dir $PYTHON/bin
Then go to a directory where you want to do a run with parameter file root.pf
$ Setup_Py_Dir
$ py root.pf
The code will now run!
The python directory structure is fairly simple:
- progs: location of source code for various fully debugged version of the code
- bin: location of excutables
- data: location for all datafiles. Files that are mainly for reference should be gzipped to save space. Such files are not recreated in
- bin: The location of the executables. (It is a good idea to put this directory in your path)
- software: This directory contains libraries which are used in in python that must be recompiled when creating an installation on a new machine, primarily Bill Pence's cfitsio package and the GNU scientific library gsl
- Example: A directory with a few examples of python runs. (Note that the input files will have changed and so one may not be able to run these examples without some changes in the input files.)
The Makefile installation has been tested on Linux and Mac OSX systems.
It may or may not be necessary to recompile python and the associated programs on your machine depending on how closely it matches the linux cluster at STScI . This section describes how to do a clean install on your machine.
- Compile Bill Pence's cfitsio library Cfitsio is a a c library for reading and writing fits files. It is used in python by the routine py_wind. Newer versions of the library can be found here
To install the version which is part of the Python directory tree, cd software and then to the highest version number of the cfitsio library. There is a README file with instruction about how to do this, but basically the commands are as follows::
./configure
make
mv libcfitsio.a ../../lib
make clean - Assure that you have left a clean directory
The last set is required, because although the README indicates you can set the install directory as part of the configuration step, ksl has never succeeded in accomplishing this.
- Install the GNU scientific libraries if this is necessary. The GNU scientific libararies are used in python, and must be either installed or available on your machine. If they are already available you may be able to skip this step. The latest version of gal can be retrieved from "here": http://www.gnu.org/software/gsl/. If you need to install gsl yourself, here is the basic procedure
cd software, and then to the highest version of gsl which you see there.
Once in the gsl directory, the basic procedure for installation is:
./configure --disable-shared --prefix=/Users/long/Python/gsl
or, if $PYTHON is defined
./configure --disable-shared --prefix=$PYTHON/gsl
where --disable-shared eliminates shared libraries, /Users/long/Python/ must be replaced with the appropriate path to your location. ksl thinks we might be able to create binaries that are more resistant to configuration changes if we do not use shared libraries, but has had problems succeeding with shared libraries, in part because there are other shared libraries. --prefix indicates where gsl is going to be installed. If missing there is a default location which requires root priveleges. The remainder of the installation is::
make
make check 2>&1 - this is csh version of the command to check that everything is OK, and can probably be skipped
make install
make clean - Make sure you have left a pristine directory. If the install fails, the person before you may not have done this, and so do a clean and try again.
Note that the gsl libraries take a while to compile If you are suspicious that the process of compilation proceeded too quickly., you are probably write and some miscreant did not clean the previous installation
At this point you should check to see that the $PYTHON/gsl directory has been updated.
Note that this has been been the biggest problem we have had installing python, and so if you have problems or solutions please comment on them on this page.
At this point you should be able to recompile python, py_wind or any other program you would like to be using, so:
-
Finally, compile python and associated programs
cd $PYTHON/progs git clone https://github.com/agnwinds/python.git
You can then make the various routines with:
make [CC=...] [D] python
make py_wind
... etc
Note that for python and py_wind, the makefile copies two versions of the executable to ../../bin, one with the version number attached and one without. This is so that one can run multiple versions of the code.
** This completes the installation**
** At this point you should be ready to try Running Python.**
64 bits
If you have tried to install and run python on a 64 bit machine and get errors quicly, including a segmentation fault, this is most likely because you are not using the latest versions of kpar. The solution is to go to Python Versions and download a more recent version. The older versions of kpar had an error in that they did not quite conform to ansi standards in one line of the code. The 64 bit gcc compiler does not recognize this as a problem during compilation but it generates a segmentation fault when the program is run.
GSL
Most of the installation problems that have been uncovered have involved gsl. This section gives additional information on how such problems might be debugged. If we can understand the various issues surrounding install then, we will of course update the instructions. It's possible that the Makefile needs to be made more robust.
We need a fool proof method to use gsl, since it is not always installed on one's machine.
There is a lot of information on libraries "here": http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html
It;s possible that if there are problems one can use the LD_LIBRARY_PATH variable to organize the order in which libraries are loaded::
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/workdir gcc -l libName.so -L /home/user/workdir -o testprog testprog.c
or adjusting what I have been trying to do, e.g.::
gcc -L /path/to/the/sofile -lName -o testprog testprog.c
There are tools on linux and Mac OSX for determining what shared libraries are loaded with a file. For linux and for sun the command is ldd and one obtains something like the following::
[long@sierra bin] ? $ ldd -d py libg2c.so.0 => /usr/lib/libg2c.so.0 (0x40018000) libm.so.6 => /lib/tls/libm.so.6 (0x009b9000) libgsl.so.0 => /usr/lib/libgsl.so.0 (0x00a0f000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00cbb000) libc.so.6 => /lib/tls/libc.so.6 (0x0088b000) /lib/ld-linux.so.2 (0x00871000)
when running on py.
For OS - X the command is apparent otool, and hre one obtains the following::
~/Python/bin : otool -L py py: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.9) /usr/local/lib/libg2c.0.dylib (compatibility version 1.0.0, current version 1.0.0)
For OS-X, I believe I had succeeded in loading the gls libraries not as a shared library, which presumably explains why it is missing.
This can be done by submitting a bug under the Issues page