criu: introduce LUO image backend - #2947
Closed
Thesola10 wants to merge 89 commits into
Closed
Conversation
Fixes: checkpoint-restore#2121 Signed-off-by: Pengda Yang <daz-3ux@proton.me>
When enabling parallel restore, the target process and the main CRIU process need an IPC interface to communicate and transfer restore commands. This patch adds a Unix domain TCP socket and stores this socket in `fdstore`. Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
Currently the restore of buffer object comsumes a significant amount of time. However, this part has no logical dependencies with other restore operations. This patch introduce some structures and some helper functions for the target process to offload this task to the main CRIU process. Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
This patch implements the entire logic to enable the offloading of buffer object content restoration. The goal of this patch is to offload the buffer object content restoration to the main CRIU process so that this restoration can occur in parallel with other restoration logic (mainly the restoration of memory state in the restore blob, which is time-consuming) to speed up the restore phase. The restoration of buffer object content usually takes a significant amount of time for GPU applications, so parallelizing it with other operations can reduce the overall restore time. It has three parts: the first replaces the restoration of buffer objects in the target process by sending a parallel restore command to the main CRIU process; the second implements the POST_FORKING hook in the amdgpu plugin to enable buffer object content restoration in the main CRIU process; the third stops the parallel thread in the RESUME_DEVICES_LATE hook. This optimization only focuses on the single-process situation (common case). In other scenarios, it will turn to the original method. This is achieved with the new `parallel_disabled` flag. Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
amdgpu represents allocated device memory as a memory mapping of the device file. This is a non-standard VMA that must be handled by the plugin, not the normal VMA code. Ignore all VMAs on device files. Signed-off-by: David Francis <David.Francis@amd.com>
amdgpu dmabuf CRIU requires the ability of the amdgpu plugin to retry. Change files_ext.c to read a response of 1 from a plugin restore function to mean retry. Signed-off-by: David Francis <David.Francis@amd.com>
For amdgpu plugin to call the new amdgpu drm CRIU ioctls, it needs the amdgpu drm header file, copied from the kernel's includes. Signed-off-by: David Francis <David.Francis@amd.com>
The amdgpu plugin usually calls drm ioctls through the libdrm wrappers. However, amdgpu restore requires dealing with dmabufs and gem handles directly, which means drm ioctls must be called directly. Add the drm.h header (from the kernel's uapi). Signed-off-by: David Francis <David.Francis@amd.com>
Buffer objects held by the amdgpu drm driver are checkpointed with the new BO_INFO and MAPPING_INFO ioctls/ioctl options. Handling is in amdgpu_plugin_drm.h Handling of imported buffer objects may require dmabuf fds to be transferred between processes. These occur over fdstore, with the handle-fstore id relationships kept in shread memory. There is a new plugin callback: RESTORE_INIT to create the shared memory. During checkpoint, track shared buffer objects, so that buffer objects that are shared across processes can be identified. During restore, track which buffer objects have been restored. Retry restore of a drm file if a buffer object is imported and the original has not been exported yet. Skip buffer objects that have already been completed or cannot be completed in the current restore. So drm code can use sdma_copy_bo, that function no longer requires kfd bo structs Update the protobuf messages with new amdgpu drm information. Signed-off-by: David Francis <David.Francis@amd.com>
The amdgpu plugin was counting how many files were checkpointed to determine when it should close the device files. The number of device files is not consistent; a process may have multiple copies of the drm device files open. Instead of doing this counting, add a new callback after all files are checkpointed, so plugins can clean up their resources at an appropriate time. Signed-off-by: David Francis <David.Francis@amd.com>
amdgpu libraries that use dmabuf fd to share GPU memory between processes close the dmabuf fds immediately after using them. However, it is possible that checkpoint of a process catches one of the dmabuf fds open. In that case, the amdgpu plugin needs to handle it. The checkpoint of the dmabuf fd does require the device file it was exported from to have already been dumped To identify which device this dmabuf fd was exprted from, attempt to import it on each device, then record the dmabuf handle it imports as. This handle can be used to restore it. Signed-off-by: David Francis <David.Francis@amd.com>
These pr_info lines begin with "CC3" and "TWI" were not meant to be included in the patch. Co-authored-by: Andrei Vagin <avagin@google.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Use `return 0` on success in `post_dump_dmabuf_check()` for consistency with other functions. Co-authored-by: Andrei Vagin <avagin@google.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Co-authored-by: Andrei Vagin <avagin@google.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Add a comment that explains the purpose of `retry_needed`. Co-authored-by: Andrei Vagin <avagin@google.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
amdgpu_plugin_drm.c:167:6: error: variable 'num_bos' set but not used [-Werror,-Wunused-but-set-variable]
167 | int num_bos = 0;
|
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
This patch updates drm.h and amdgpu_drm.h kernel headers, and adds drm_mode.h (included by drm.h) from the rocm-7.1.0 release tag. Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
These header files are copied directly from the Linux kernel and contain typos. We skip these files in codespell to simplify maintenance. Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Use local copies of amdgpu and DRM headers for consistency. Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
On AMD Instinct MI300 systems, restoring a large GPU application can fail because the checkpoint size is too large and the maximum value of an offset (with integer type) is insufficient. This problem occurs when the total size of all buffer objects exceeds int max, not because any single buffer is too large, but it can also happen with a large number of small buffers. Fixes: checkpoint-restore#2812 Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
The syntax of the inherit-fd functionality for unix socket and pipe includes a colon. Fixes: 0df3f79 ("criu(8): fix --inherit-fd description") Fixes: c37324b ("crtools: describe the inherit-fd option") Signed-off-by: Mark Polyakov <mark@thundercompute.com> Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Static code analysis reported: criu/cr-dump.c:2328:2: identical_branches: The same code is executed when the condition "ret" is true or false, because the code in the if-then branch and after the if statement is identical. Should the if statement be removed? This is a fix for the warning. Signed-off-by: Adrian Reber <areber@redhat.com>
Static code analysis reported:
1. criu/cr-restore.c:2438:2: var_decl: Declaring variable "end_vma"
without initializer.
4. criu/cr-restore.c:2451:5: assign: Assigning: "s_vma" = "&end_vma",
which points to uninitialized data.
7. criu/cr-restore.c:2449:4: uninit_use: Using uninitialized value
"s_vma->list.next".
This tries to fix it by initializing the variable.
Signed-off-by: Adrian Reber <areber@redhat.com>
At least on tests running on Fedora rawhide following error could be
seen:
```
criu/tty.c: In function 'pts_fd_get_index':
criu/tty.c:262:21: error: initialization discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
262 | char *pos = strrchr(link->name, '/');
|
```
This fixes it.
Signed-off-by: Adrian Reber <areber@redhat.com>
Refactor user_fpregs_struct_t to wrap user_fpsimd_state in a dedicated struct, preparing for future extending by just adding new members Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com> [ alex: fixes ] Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Acked-by: Mike Rapoport <rppt@kernel.org>
Introduce ARM64 Guarded Control Stack (GCS) constants and macros in a new uapi header for use in both CRIU and compel. Includes: - NT_ARM_GCS type - prctl(2) constants for GCS enable/write/push modes - Capability token helpers (GCS_CAP, GCS_SIGNAL_CAP) - HWCAP_GCS definition These are based on upstream Linux definitions Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com> Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Acked-by: Mike Rapoport <rppt@kernel.org>
Add basic prerequisites for Guarded Control Stack (GCS) state on AArch64. This adds a gcs_context to the signal frame and extends user_fpregs_struct_t to carry GCS metadata, preparing the groundwork for GCS in the parasite. For now, the GCS fields are zeroed during compel_get_task_regs(), technically ignoring GCS since it does not reach the control logic yet; that will be introduced in the next commit. The code path is gated and does not affect normal tests. Can be explicitly enabled and tested via: make -C infect GCS_ENABLE=1 && make -C infect run Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com> [ alex: clean up fixes ] Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Acked-by: Mike Rapoport <rppt@kernel.org>
When GCS is enabled, the kernel expects a capability token at GCSPR_EL0-8 and sa_restorer at GCSPR_EL0-16 on rt_sigreturn. The sigframe must be consistent with the kernel’s expectations, with GCSPR_EL0 advanced by -8 having it point to the token on signal entry. On rt_sigreturn, the kernel verifies the cap at GCSPR_EL0, invalidates it and increments GCSPR_EL0 by 8 at the end of gcs_restore_signal() . Implement parasite_setup_gcs() to: - read NT_ARM_GCS via ptrace(PTRACE_GETREGSET) - write (via ptrace) the computed capability token and restorer address - update GCSPR_EL0 to point to the token's location Call parasite_setup_gcs() into parasite_start_daemon() so the sigreturn frame satisfies kernel's expectation Tests with GCS remain opt‑in: make -C compel/test/infect GCS_ENABLE=1 && make -C compel/test/infect run Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com> [ alex: cleanup fixes ] Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Acked-by: Mike Rapoport <rppt@kernel.org>
Introduce an opt-in mode for building and running compel tests
with Guarded Control Stack (GCS) enabled on AArch64.
Changes:
- Extend compel/test/infect to support `GCS_ENABLE=1` builds,
adding `-mbranch-protection=standard` and
`-z experimental-gcs=check` to CFLAGS/LDFLAGS.
- Export required GLIBC_TUNABLES at runtime via `TEST_ENV`.
Usage:
make -C compel/test/infect GCS_ENABLE=1
make -C compel/test/infect GCS_ENABLE=1 run
By default (`GCS_ENABLE` unset or 0), builds and runs are unchanged.
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
Signed-off-by: Lorenzo Fontana <fontanalorenz@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
In unix_collect_one(), icons (in-flight connections) are added to the global unix_listen_icons hash table. Previously, if xzalloc() failed mid-loop while allocating an icon, the code could leave partially inserted icons in the global hash, resulting in dangling pointers and potential use-after-free crashes. This patch changes the logic to pre-allocate all unix_sk_listen_icon nodes before inserting any of them into unix_listen_icons. If allocation fails, the temporary list is freed and the function aborts before mutating the global hash. This ensures that the hash is only updated when all allocations succeed, and avoids the need to traverse and remove icons from the hash on error. Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
prepare_cgroup_namespace() can call userns_call() to move the task into a cgroup. userns_call() uses sendmsg() with SCM_CREDENTIALS to communicate with usernsd. If the user namespace maps are configured such that the parent's UID (typically 0) is not mapped to the same UID in the child namespace (i.e., lower_first != 0), the getuid() syscall in the child will return the overflow UID (65534) until the child's credentials are explicitly set within the namespace. When sendmsg() is called with SCM_CREDENTIALS containing the overflow UID, the kernel's __scm_send function returns -EINVAL, causing the restore to fail. [avagin: tweaked the commit message] Signed-off-by: Efim Verzakov <efimverzakov@gmail.com> Signed-off-by: Andrei Vagin <avagin@gmail.com>
We saw compilation errors like: error: call to ‘__open_missing_mode’ declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments Let's fix them by adding mode 0777 everywhere. Before this change the mode was taken randomly from stack (according to man 2 open) and that is likely not what we want. Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
After recvmsg() with MSG_PEEK, dump_sk_queue() checks for MSG_TRUNC (data truncation) but not MSG_CTRUNC (control data truncation). When the control message buffer is too small, the kernel sets MSG_CTRUNC and silently discards the overflowing ancillary data. For SCM_RIGHTS, this means passed file descriptors are lost without any error. Add a MSG_CTRUNC check right after the existing MSG_TRUNC check so that dump fails explicitly instead of silently producing an incomplete image. Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
Fix several memory leaks in sk-queue error handling: - collect_one_packet(): Free pkt->data when n_scm > 1 check fails. - collect_one_packet(): Move list_add_tail() after read_img_buf() so that a packet with freed data is never left on the list. - dump_scm_rights(): Free scme when dump_my_file() fails. The allocation from xmalloc() was leaked on this error path. - send_one_pkt(): Free pkt->scm (allocated in prepare_scms()) after sendmsg(), alongside the existing xfree(pkt->data). Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
kfd_criu_device_bucket.drm_fd is an unsigned int, so the comparison device_bucket->drm_fd < 0 was always false. Fix that, and also make the error message a bit more descriptive. Signed-off-by: David Francis <David.Francis@amd.com>
pr_perror already appends a newline; the explicit \n in the
format string violates the lint rule and breaks CI.
Fixes: 555b257c5e77 ("plugin/amdgpu: Catch error for failure to open drm device")
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
Rework GitHub workflows to consolidate all test-related jobs into a single ci.yml file. This ensures that the alpine-test job runs first, and all other tests are executed only if it passes. This reduces redundant triggers and provides a clear dependency path for CI runs. Signed-off-by: Andrei Vagin <avagin@gmail.com>
gcc doesn't report any warnings if there are two identical enums with the same name. Actually, we wanted to detect if a specific enum is already exists. By using a different member name in the test, we ensure that a conflict occurs if the enum is already defined in the system headers. Signed-off-by: Andrei Vagin <avagin@gmail.com>
Install libcriu into a local staging directory (.install/) using
'make install-lib' and reference only that directory during build
and test:
CFLAGS := -I$(INSTALL_DIR)/include/criu
LDFLAGS := -L$(INSTALL_DIR)/lib -lcriu
LD_LIBRARY_PATH=${MAIN_DIR}/.install/lib
Nothing from the non-test part of the repo is used at compile,
link, or run time.
The compile rule is also corrected to use $< instead of $^
for gcc -c.
Fixes: checkpoint-restore#2927
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
restore_userns_binfmt_misc() was opening the userns image a second time, but criu-image-streamer rejects duplicate file requests: criu-image-streamer Error: CRIU is requesting the image file userns-15.img multiple times. This is not allowed to keep the memory usage low Fix by pre-reading the image once in a new read_user_ns_img() helper, called from prepare_namespace_before_tasks() which runs before the first clone(). Following the same pattern as NetnsEntry on net namespace ids, the UsernsEntry is stored in a new 'user.e' field on the ns_id structure for the user namespace. Both the CRIU coordinator (prepare_userns) and the forked root task (restore_userns_binfmt_misc) look up the ns_id and access the entry from there -- no second open_image() call is needed. Tested with: sudo ./test/zdtm.py run --stream -t zdtm/static/env00 --ignore-taint sudo ./test/zdtm.py run --stream -t zdtm/static/env00 --ignore-taint -f uns Both pass with the fix applied. Fixes: e6510a3 ("criu: Support binfmt_misc sandboxing") Fixes: checkpoint-restore#2924 Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
VMA_AREA_GUARD areas are added to the end of the vma_area_list and are not necessarily sorted by address. pmc_fill_cache() expects VMAs to be sorted and triggers BUG_ON() if it meets a VMA with a lower address than the current cache window. Since guard VMAs don't have pagemap entries and are skipped during dump, just stop the lookahead loop when we encounter one. Signed-off-by: Andrei Vagin <avagin@gmail.com>
This is a simple approach to introduce LUO support into CRIU, by storing our checkpoint images into memfds, and persisting them into a single LUO session. If the session descriptor is created by CRIU, we will halt instead of exiting since session descriptors must remain open when kexec is called in order to be preserved. Signed-off-by: Karim Vergnes <me@thesola.io>
|
A friendly reminder that this PR had no activity for 30 days. |
Member
|
Is it better than using a dax file system? (https://www.kernel.org/doc/Documentation/filesystems/dax.txt) |
Contributor
Author
|
The idea of using LUO as an explicit backend is that we can then add LUO-specific optimisations, such as adding open FDs to the session instead of serialising them, or writing memory pages zero-copy into the memfd images. |
Member
I mean why we may want to do that? Where are you going to store memfd luo ids? |
|
A friendly reminder that this PR had no activity for 30 days. |
|
A friendly reminder that this PR had no activity for 30 days. |
Currently, the LUO feature only supports backup and recovery for shmem memory. Will support for backup and recovery of anonymous memory be added in the future? |
Member
|
The LUO integration will be implemented based on #3119. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch adds Live Update Orchestrator as a storage backend for CRIU images.
What this PR does
It adds two flags,
--luo-sessionand--luo-session-fd, which specify a LUO session to save CRIU images into.If CRIU owns the session file descriptor, that is the flag
--luo-sessionis used, CRIU will remain open in order for the session file descriptor to be preserved whenkexecis called.How it works
A new module,
luo.c, has been introduced to manage LUO session file descriptors as a storage medium, by preservingmemfds as is recommended in the kernel documentation.Dump side:
--luo-sessionis specified, a new LUO session is allocated with the name given as argument.memfdis opened and returned as the target file descriptor--luo-sessionis specified, CRIU hangs on a no-op in order to keep the session FD open untilkexecRestore side:
--luo-sessionis specified, CRIU will query/dev/liveupdatefor a session with that name.--luo-sessionis specified, the LUO session is finalized and its resources are freed.Implementation notes
criu/include/linux/liveupdate.h--luo-session-fdis intended for an externally managed file descriptor (ideallysystemd?) and CRIU will not attempt to close or preserve it.TODO:
memfd