Skip to content

CI: Fix the relocated installation test and instrument the Windows failure - #33

Open
petrasovaa wants to merge 3 commits into
preload-dynamic-librariesfrom
preload-dynamic-libraries-ci
Open

CI: Fix the relocated installation test and instrument the Windows failure#33
petrasovaa wants to merge 3 commits into
preload-dynamic-librariesfrom
preload-dynamic-libraries-ci

Conversation

@petrasovaa

Copy link
Copy Markdown
Owner

Experimental branch for iterating on the CI failures of OSGeo#7733. Not meant for OSGeo/grass as it is.

Linux and macOS: the new step could not import grass at all

The step Run pytest for loading C libraries into the process failed with exit code 4:

ImportError while loading conftest '.../python/grass/script/tests/conftest.py'.
python/grass/script/__init__.py:4: in <module>
E   ModuleNotFoundError: No module named 'grass'

PYTHONPATH was set to ${HOME}/install-relocated/etc/python, but Autotools installs into the prefix plus a version directory, so the real path is ${HOME}/install-relocated/grass86/etc/python. With a non-existent entry there, grass is not importable and pytest falls back to importing the repository copy of grass/script/__init__.py by file path, which then does not find grass.exceptions. The path is now taken from grass --config python_path before the move (the command does not work from the moved installation) and rebased onto the new prefix.

Windows: instrumented, not fixed yet

test_grass_lib_usable_with_load_libs fails in the main pytest step (which is why the new step never runs there). The subprocess exits with 1 and empty stdout and stderr - no Python traceback, so this looks like a fatal error in the C library, whose message is lost. Timing suggests it got as far as the grass.lib.raster calls.

The sibling test with a custom environment passes even though its session variables exist only in a dict that the C library cannot read, which means G_gisinit there succeeds against some other session. That is consistent with the limitation recorded in lib/gis/tests/lib_gis_env_test.py (the C runtime keeps its own copy of the environment on Windows), but the logs do not prove it.

So this round only makes the failure legible:

  • the failure message includes both output streams,
  • the first test writes the session the C library resolved (GISDBASE, LOCATION_NAME, MAPSET) to stderr, flushed, before the raster is opened,
  • the custom environment test asserts that this session is the one init() created instead of accepting any working session.

Depending on what the Windows run reports, the next step is either skipping that test on Windows with the cached-environment reason, or pushing GISRC, GISBASE and GIS_LOCK into the C runtime with libgis.G_putenv at the end of init() when load_libs is set.

The analysis of the logs and these changes were done with Claude Code.

The installation directory is the configured prefix plus a directory
with the version (grass86), so the etc/python subdirectory is not
directly in the prefix and the step failed to import grass at all.
Ask the grass command for the path before the installation is moved,
since the command does not work afterwards.
…ests

On Windows, the first test fails with the subprocess exiting with 1 and
no output at all, which is what a fatal error in the C library looks
like from Python. To tell a wrong session from a failed call, the
session the library resolved is written to stderr before the raster is
opened, and the test with a custom environment now checks that the
library works with the session which init created (it can pass without
checking it because another session in the environment works too).
The failure message now also includes both output streams.
The libraries read GISBASE and GISRC with getenv from the environment of
the process, which is not the session environment (the env parameter of
init) and, on Windows, not even the global environment, because the C
runtime keeps a copy of the environment made when the process started.
The variables are therefore set with G_putenv, which runs inside the
libraries and so writes the environment they read.

Without GISBASE, a library which reports an error fails again in the
reporting itself, and the re-entrancy guard of G_fatal_error ends the
process without a message, which is what the Windows and macOS failures
looked like. A test now checks that a message from a library is
reported, so that a silent exit is not mistaken for a passing test.

The tests keep the completed process so that both output streams can be
shown when the code in the subprocess fails.

Written with the help of Claude Code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant