Skip to content

Multi-threaded build sometimes fails to build Python client #49

@etremel

Description

@etremel

Sometimes when I attempt to do an incremental rebuild of Cascade after making some changes, the Python client (which I did not change) will fail to build with an error message like this:

[build] * Building wheel from sdist
[build] * Creating virtualenv isolated environment...
[build] * Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
[build]   Determining projects to restore...
[build]   Restored /home/edward/research/cascade/src/service/cs/CascadeClient.csproj (in 124 ms).
[build] error: [Errno 2] No such file or directory: 'derecho.cascade-1.0rc0/derecho/cascade/udl.py'
[build] 
[build] ERROR Backend subproccess exited when trying to invoke build_sdist
[build] gmake[2]: *** [src/service/python/CMakeFiles/member_client_py.dir/build.make:107: src/service/python/derecho/cascade/member_client.cpython-310-x86_64-linux-gnu.so] Error 1
[build] gmake[2]: *** Deleting file 'src/service/python/derecho/cascade/member_client.cpython-310-x86_64-linux-gnu.so'
[build] gmake[1]: *** [CMakeFiles/Makefile2:559: src/service/python/CMakeFiles/member_client_py.dir/all] Error 2

Apparently the Python package builder failed to find the file udl.py when it needed it. This file is located in the Python client's source directory (in src/service/python/derecho/cascade/udl.py), and the CMakeLists.txt in the python folder specifies that it should be copied to the binary (build) directory before attempting to build the Python client:

add_custom_command(TARGET member_client_py PRE_LINK
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}/derecho ${CMAKE_CURRENT_BINARY_DIR}/derecho)

But for some reason this copy_directory command is not executed before attempting to build the member_client_py target, so the directory (and udl.py) is not found.

I believe this error happens due to a race condition in multi-threaded make/CMake, because I can reliably make it go away (and build successfully) by trying again with a single-threaded build, i.e. make -j1.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions