The check-commit target verifies that each commit builds successfully
on its own. However, without cleaning previously generated build artifacts
(e.g. auto-generated `*.o` files) this can cause the check to fail:
make[2]: *** No rule to make target 'compel/include/uapi/compel/asm/breakpoints.h', needed by 'compel/arch/x86/src/lib/infect.o'. Stop.
make[1]: *** [Makefile.compel:35: compel/libcompel.a] Error 2
To fix this, we add 'git clean -dfx' to remove all untracked and ignored
files before attempting the build.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Update the state machine to track entry and exit stages for all
syscalls, not just the target one. This allows skipping
ptrace_get_regs() calls on the exit stage of syscalls.
Signed-off-by: Andrei Vagin <avagin@google.com>
The restorer used a futex to synchronize threads right before sigreturn.
This could lead to a deadlock when compel_stop_tasks_on_syscall executes
all restored tasks/threads sequentially, allowing each to execute one
syscall per iteration. One task might wait for another one that hasn't
had a chance to run yet.
To avoid this, this patch replaces the futex with an atomic variable.
The last thread to finish the restoration process handles the final
cleanup tasks, such as closing the log and unmapping restorer memory.
Signed-off-by: Andrei Vagin <avagin@google.com>
compel_stop_on_syscall is only ever called for a single task. Simplify
its implementation by removing the 'tasks' parameter and the associated
logic for handling multiple tasks. It now takes a 'pid' directly and
waits for that specific task to exit the required syscall.
Signed-off-by: Andrei Vagin <avagin@google.com>
Hardware breakpoints were originally intended to speed up the resume process
by stopping the process at a specific point in the pie code. However, it
turned out that they don't provide a significant speedup and, in some cases,
can even slow it down. This is especially critical for hosts with a large
number of CPUs.
Hardware Breakpoint Restore Performance Benchmark
==================================================
System: Linux 6.17.0-1007-gcp x86_64
CPU: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.10GHz
Virt: google
CRIU: Version: 4.2
Iterations per data point: 5
Threads With BP (us) Without BP (us) Diff (%)
-------- ------------ --------------- --------
>>> Benchmarking with 1 thread(s)...
1 391 326 19.9%
>>> Benchmarking with 10 thread(s)...
10 1098 695 58.0%
>>> Benchmarking with 50 thread(s)...
50 3772 2344 60.9%
>>> Benchmarking with 100 thread(s)...
100 6740 4504 49.6%
>>> Benchmarking with 500 thread(s)...
500 31382 19982 57.1%
>>> Benchmarking with 1000 thread(s)...
1000 58617 40568 44.5%
Notes:
'With BP' = hardware breakpoints enabled (current default)
'Without BP' = CRIU_FAULT=130 (FI_NO_BREAKPOINTS, uses PTRACE_SYSCALL)
Positive diff% means breakpoints are slower
Signed-off-by: Andrei Vagin <avagin@google.com>
Splitting the ptrace interrupt and wait loops allows us to send the
interrupt to all tasks and threads first, so they can start stopping in
parallel while we're still sending interrupts to the remaining ones.
This should improve restore performance for large process trees.
The redundant parse_threads calls in catch_tasks are removed as threads
are already parsed in attach_to_tasks which is called earlier.
Signed-off-by: Andrei Vagin <avagin@google.com>
When CRIU traces a large number of tasks, calling wait4(-1, ...) can
be slow because the kernel has to iterate over many tasks to find one
that has changed state. This overhead becomes significant when
thousands of tasks are involved. The time complexity of the old approach
was O(n^2) because we called wait4(-1, ...) N times, and each call
took O(N) time in the kernel.
This patch introduces compel_stop_tasks_on_syscall which takes an
array of PIDs and waits for each one specifically. This avoids the
O(N) search in the kernel's do_wait implementation and significantly
improves performance during the restore phase when many tasks are
being resumed and stopped at the rt_sigreturn syscall.
Some parallelism still exists in this approach. The loop waits for each
process to stop on a syscall, verifies it, and then lets it continue
running. While we move to wait for the next process in the array, the
previously processed ones continue their execution in parallel.
The performance profile of the bottleneck showed:
- 70.45% entry_SYSCALL_64
- do_syscall_64
- 68.29% __do_sys_wait4
- kernel_wait4
- 68.28% do_wait
- 68.26% __do_wait
- 65.69% wait_consider_task
8.71% _raw_spin_lock_irq
2.39% _raw_spin_unlock_irq
Signed-off-by: Andrei Vagin <avagin@google.com>
Commit 6b29103 adds the function read_user_ns_img to read user namespace
image. It looks up the user namespace by id. However, user namespace
can not be found because only mnt, net and pid namespaces are added to
the nsid list. The user namespace is not read from pstree ids.
This commit adds user namespace to the nsid list from the pstree ids
image.
Fixes: 6b29103 ("criu: fix double-open of userns image in --stream mode")
Fixes: #2938
Signed-off-by: Efim Verzakov <efimverzakov@gmail.com>
restore_userns_binfmt_misc() was opening the userns image a second
time, but criu-image-streamer rejects duplicate file requests:
criu-image-streamer Error: CRIU is requesting the image file
userns-15.img multiple times. This is not allowed to keep the
memory usage low
Fix by pre-reading the image once in a new read_user_ns_img()
helper, called from prepare_namespace_before_tasks() which runs
before the first clone(). Following the same pattern as NetnsEntry
on net namespace ids, the UsernsEntry is stored in a new 'user.e'
field on the ns_id structure for the user namespace. Both the CRIU
coordinator (prepare_userns) and the forked root task
(restore_userns_binfmt_misc) look up the ns_id and access the
entry from there -- no second open_image() call is needed.
Tested with:
sudo ./test/zdtm.py run --stream -t zdtm/static/env00 --ignore-taint
sudo ./test/zdtm.py run --stream -t zdtm/static/env00 --ignore-taint -f uns
Both pass with the fix applied.
Fixes: e6510a338a ("criu: Support binfmt_misc sandboxing")
Fixes: #2924
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
Rework GitHub workflows to consolidate all test-related jobs into a
single ci.yml file. This ensures that the alpine-test job runs first,
and all other tests are executed only if it passes.
This reduces redundant triggers and provides a clear dependency path
for CI runs.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
pr_perror already appends a newline; the explicit \n in the
format string violates the lint rule and breaks CI.
Fixes: 555b257c5e77 ("plugin/amdgpu: Catch error for failure to open drm device")
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
kfd_criu_device_bucket.drm_fd is an unsigned int, so the comparison
device_bucket->drm_fd < 0
was always false.
Fix that, and also make the error message a bit more descriptive.
Signed-off-by: David Francis <David.Francis@amd.com>
Fix several memory leaks in sk-queue error handling:
- collect_one_packet(): Free pkt->data when n_scm > 1 check fails.
- collect_one_packet(): Move list_add_tail() after read_img_buf() so
that a packet with freed data is never left on the list.
- dump_scm_rights(): Free scme when dump_my_file() fails. The
allocation from xmalloc() was leaked on this error path.
- send_one_pkt(): Free pkt->scm (allocated in prepare_scms()) after
sendmsg(), alongside the existing xfree(pkt->data).
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
After recvmsg() with MSG_PEEK, dump_sk_queue() checks for MSG_TRUNC
(data truncation) but not MSG_CTRUNC (control data truncation). When
the control message buffer is too small, the kernel sets MSG_CTRUNC
and silently discards the overflowing ancillary data. For SCM_RIGHTS,
this means passed file descriptors are lost without any error.
Add a MSG_CTRUNC check right after the existing MSG_TRUNC check so
that dump fails explicitly instead of silently producing an incomplete
image.
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
We saw compilation errors like:
error: call to ‘__open_missing_mode’ declared with attribute error:
open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
Let's fix them by adding mode 0777 everywhere.
Before this change the mode was taken randomly from stack (according to
man 2 open) and that is likely not what we want.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
prepare_cgroup_namespace() can call userns_call() to move
the task into a cgroup. userns_call() uses
sendmsg() with SCM_CREDENTIALS to communicate with usernsd.
If the user namespace maps are configured such that the parent's UID
(typically 0) is not mapped to the same UID in the child namespace
(i.e., lower_first != 0), the getuid() syscall in the child will return
the overflow UID (65534) until the child's credentials are explicitly
set within the namespace.
When sendmsg() is called with SCM_CREDENTIALS containing the overflow UID,
the kernel's __scm_send function returns -EINVAL, causing the restore
to fail.
[avagin: tweaked the commit message]
Signed-off-by: Efim Verzakov <efimverzakov@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
In unix_collect_one(), icons (in-flight connections) are added to the
global unix_listen_icons hash table. Previously, if xzalloc() failed
mid-loop while allocating an icon, the code could leave partially
inserted icons in the global hash, resulting in dangling pointers and
potential use-after-free crashes.
This patch changes the logic to pre-allocate all unix_sk_listen_icon
nodes before inserting any of them into unix_listen_icons. If allocation
fails, the temporary list is freed and the function aborts before
mutating the global hash. This ensures that the hash is only updated
when all allocations succeed, and avoids the need to traverse and remove
icons from the hash on error.
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
Pavel Emelyanov, the founder and long-time leader of the CRIU project,
is moving to a retired role. As the project's inventor, Pavel's vision
and leadership over many years were fundamental in bringing CRIU to its
current status as a production-ready solution.
Andrey Vagin is appointed as the chief maintainer.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
We need to support restoring processes on different nodes or subnets
where the original socket IPs may no longer be valid. This change
introduces a new plugin hook, CR_PLUGIN_HOOK__UPDATE_INETSK, and wires
it into both the plugin API and the legacy descriptor autogen so that
existing plugins remain compatible.
The hook is invoked from collect_one_inetsk() before address validation,
port reservation, or bind()/connect(). A plugin may rewrite the IPv4 or
IPv6 src/dst address fields in InetSkEntry in‑place and return 0.
Returning -ENOTSUP skips to the next plugin; any other negative value
aborts the restore for that socket. Only the IP addresses may change —
family, ports, ifname and length constraints remain enforced.
This mechanism is mainly intended for cross‑node or Kubernetes‑style
scenarios where the final IP is only known after the target network
namespace is created and configured. Pre‑editing image files would be
awkward or infeasible otherwise.
Note: for established TCP connections, external coordination or preserving
the original 4‑tuple is still required. This hook only adjusts addresses
and does not modify TCP state.
Signed-off-by: Rowan-Ye <rowenye1@gmail.com>
mnt_id was declared as uint32_t but assigned -1 when absent, which yields
UINT32_MAX. When passed to mntns_get_root_by_mnt_id(int), the conversion
from UINT32_MAX to int is implementation-defined and can become a large
positive value, potentially causing lookup_nsid_by_mnt_id() to return NULL
and trigger BUG_ON().
Change mnt_id to int with an explicit cast from f_handle->mnt_id when
present, so the sentinel -1 is preserved reliably across platforms.
Signed-off-by: 3idey <elaidya225@gmail.com>
Add checks for mntns_get_root_by_mnt_id() return value before using it
in openat() calls. This prevents using an invalid file descriptor if
the mount namespace root lookup fails.
Add error logging when mntns_get_root_by_mnt_id() fails to provide
diagnostic context about which mount namespace and path were being
accessed.
Also rename local variable 'path' to 'fpath' to avoid shadowing the
outer scope variable of the same name.
Signed-off-by: 3idey <elaidya225@gmail.com>
Before that patch we always had exactly the same magic and extentsion
patterns generated.
While on it let's fix the data restrictions:
- The length of extension and magic should be non-zero.
- Let's explicitly wrap extension characters with 256.
Fixes: #2886
Co-developed-by: Andrei Vagin <avagin@google.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
In ns flavor for instance the binfmt_misc is not c/r-ed, but without
this cleanup the test will not detect that as it would inherit correct
unchanged binfmt_misc step which was there on dump.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
In zdtm/static/binfmt_misc we print the string which contains random
bytes, some of which may not map to utf-8. So let's ignore those bytes
we can't show in utf-8.
Else we can get:
File ".../criu/test/zdtm.py", line 660, in print_output
print(output.read())
~~~~~~~~~~~^^
File "<frozen codecs>", line 325, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe3 in position 180: invalid continuation byte
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
The Linux kernel already supports per user namespace sandboxed mounts:
21ca59b365c0 ("binfmt_misc: enable sandboxed mounts")
The overall dump/restore logic is based on the fact that a binfmt_misc
superblock is static and is allocated once per a binfmt_misc mount
within a given user namespace. So the code makes a temporary mount to
read/write the entries and then removes it. The binfmt_misc data is
stored to the USERNS image.
Signed-off-by: Dmitry Sepp <dmitry.sepp@virtuozzo.com>
Detect if the kernel supports binfmt_misc sandboxing and store the
result in kerndat.
The result will be used by the code that dumps and restores binfmt_misc
entries.
Signed-off-by: Dmitry Sepp <dmitry.sepp@virtuozzo.com>
Remove the OpenVZ specific code that was not used outside of the OpenVZ
infrastructure in preparation for implementing support for upstream
Linux kernel binfmt_misc sandboxing.
Signed-off-by: Dmitry Sepp <dmitry.sepp@virtuozzo.com>
Add a test that creates a SIGEV_THREAD_ID posix timer targeting the
thread leader in a multithreaded process. This exercises the code path
where encode_notify_thread_id() must look up the thread leader's vtid
in threads[0].ns[0].virt.
Without the fix in the previous commit, this test fails during dump
with:
Error (criu/timer.c:329): Unable to convert the notify thread id <pid>
The test creates a worker thread (so the process is multithreaded, which
triggers the collect_threads() path) and a SIGEV_THREAD_ID timer on the
main thread, then verifies the timer still fires after C/R.
Signed-off-by: 3idey <elaidya225@gmail.com>
In collect_threads(), threads[0] (the thread leader) gets its .real pid
set but .ns[0].virt is left uninitialized. Later in dump_one_task(),
item->pid->ns[0].virt is set from the parasite misc data, but
threads[0].ns[0].virt is not populated until dump_task_threads() runs.
The problem is that parasite_dump_posix_timers_seized() is called
*before* dump_task_threads(). When a posix timer has SIGEV_THREAD_ID
notification targeting the thread leader, encode_notify_thread_id()
looks up threads[i].ns[0].virt and finds an uninitialized (garbage)
value for threads[0]. This causes:
- On dump: the garbage vtid is written into the image as
notify_thread_id.
- On restore: timer_create() fails with -EINVAL because the
notify_thread_id is invalid.
Fix this by initializing threads[0].ns[0].virt right after
item->pid->ns[0].virt is set, before the posix timer dump.
Fixes: #2887
Signed-off-by: 3idey <elaidya225@gmail.com>
page_pipe_from_pagemap() opens a page_read via open_page_read() but
never calls pr.close() on error or success paths. This leaks the
pagemap and pages image file descriptors, and any parent page_read
chain.
Add proper cleanup using goto err pattern to ensure pr.close() is
always called before returning. Also destroy the page_pipe on
fill_page_pipe() failure to avoid leaking its memory and pipe FDs.
Signed-off-by: 3idey <elaidya225@gmail.com>
In container setups, particularly k8s, cgroup may be used to
restrict what GPUs a process has access to. CRIU may then be used
to checkpoint that process while CRIU itself has full access to
all GPUs.
In this case, the kfd CRIU ioctl will return only a subset of the
ioctls visible through sysfs.
Don't write these devices or any links connected to them to the
tology dump files.
Signed-off-by: David Francis <David.Francis@amd.com>
When irmap_queue_cache fails, the remaining fanotify mark entries
and the mark array are not freed, causing a memory leak.
Add cleanup label to ensure proper deallocation of remaining entries
and the mark array on error paths.
Signed-off-by: 3idey <elaidya225@gmail.com>
When irmap_queue_cache fails, the remaining inotify watch descriptors
and the wd array are not freed, causing a memory leak.
Add cleanup label to ensure proper deallocation of remaining entries
and the wd array on error paths.
Signed-off-by: 3idey <elaidya225@gmail.com>
Error messages now include additional context to aid debugging:
- Device and inode numbers for file handles
- Watch descriptor (wd) for inotify operations
- Mask values for fanotify marks
- Mount ID for mount-related operations
This makes it much easier to diagnose checkpoint/restore failures
by providing complete information about which watch or mark failed
and why, without needing to add debug logging.
Signed-off-by: 3idey <elaidya225@gmail.com>
When restoring inotify watches, the restore_one_inotify() function opens
a file descriptor via get_mark_path() and stores it in the 'target'
variable. However, in two error paths, the function returns directly
without properly closing this file descriptor:
1. When INOTIFY_IOC_SETNEXTWD ioctl fails (line 556)
2. When has_inotify_setnextwd is true but watch descriptor mismatch
occurs (line 577)
Both cases bypass the cleanup code at the 'err' label which calls
close_safe(&target), resulting in a file descriptor leak.
Fix this by using 'goto err' instead of 'return -1' in both error paths,
ensuring proper cleanup of the target file descriptor.
Signed-off-by: 3idey <elaidya225@gmail.com>
Add repository-specific guidance for GitHub Copilot in
.github/copilot-instructions.md. This file includes information about:
- Coding style (Linux Kernel Coding Style)
- Architectural overview of the project
- PIE code requirements (must be self-contained and depend on compel)
- Descriptions of CRIU commands
- ZDTM test suite details
- Commit message formatting guidelines
This is just initial skeleton designed to optimize GitHub Copilot
reviews.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
When `opts.pid` is explicitly set to `os.getpid()`, `pycriu` fails to
daemonize the `criu` process. This causes `criu` to run as a child of
the dumped process, leading to the error "The criu itself is within
dumped tree".
This can be fixed by modifying `_send_req_and_recv_resp` to check if the
target PID matches the current process PID. If so, it enables daemon
mode, ensuring `criu` is detached and the dump succeeds.
Signed-off-by: unichronic <ishuvam.pal@gmail.com>
In a simple case where the parent process and the child one are in one
pid namespace we can safely use vpid(item) to prace the child. But, for
the cases where the child is a pid namespace init, or the child is put
into external pid namespace, the parent and the child have different pid
namespaces and using pid vpid(item) (which e.g. for init will always be
1 here) to ptrace the child process is inorrect.
Let's use the pid reported to us from clone as it's always the right pid
of the child from the parent's point of view.
Fixes: 7dd583002 ("restore: add infrastructure to enable shadow stack")
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
The return value of sys_rseq was previously ignored during
unregistration, under the assumption that it would not fail if the rseq
structure was properly registered.
However, if sys_rseq fails, the kernel retains the registration. If the
memory containing the rseq structure is subsequently unmapped or reused,
kernel updates to the rseq area can cause the process to crash (e.g.,
via SIGSEGV).
Check the return value of sys_rseq. If it fails, log the error code and
abort the restoration process. This makes rseq unregistration failures
fatal and explicit, aiding in debugging and preventing later obscure
crashes.
Signed-off-by: liqiang2020 <liqiang64@huawei.com>
In some cases, CRIU can observe tasks that exit during checkpointing,
and sets the state of these tasks to COMPEL_TASK_DEAD.
This patch adds a string representation of this value that can be used
by CRIT when decoding the images.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
CRIU defines the following constants for task state in compel/include/uapi/task-state.h
COMPEL_TASK_ALIVE = 0x01
COMPEL_TASK_STOPPED = 0x03
COMPEL_TASK_ZOMBIE = 0x06
Thus, we need to swap the values for "zombie" and "stopped" used in CRIT.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
When running the command 'make docker-test', almost all zdtm tests fail,
logging 'ip: not found'. 'ip' command of the iproute2 package was missing.
So added the package to the list of dependencies in 'apt-packages.sh'. Now
tests run
Signed-off-by: ImranullahKhann <imranullahkhann2004@gmail.com>
As of Alpine Linux 3.19, musl libc no longer contains separate
fopen64(), fseeko64(), or ftello64() functions. This causes building
CRIU with amdgpu plugin to fail with the following error:
amdgpu_plugin.c: In function 'parallel_restore_bo_contents':
amdgpu_plugin.c:2286:17: error: implicit declaration of function 'fseeko64'; did you mean 'fseeko'? [-Wimplicit-function-declaration]
2286 | fseeko64(bo_contents_fp, entry->read_offset + offset, SEEK_SET);
| ^~~~~~~~
| fseeko
make[2]: *** [Makefile:31: amdgpu_plugin.so] Error 1
make[1]: *** [Makefile:363: amdgpu_plugin] Error 2
To fix this, add the missing $(DEFINES) to plugin builds, and since we
always compile with _FILE_OFFSET_BITS=64, we don't need the 64 suffix.
Fixes: #2826
Suggested-by: Andrei Vagin <avagin@google.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
The "man 2 close":"Dealing with error returns from close()" says:
"Retrying the close() after a failure return is the wrong thing to do"
We should not leave the fd there, attempting to close it again on next
close()/close_safe() may lead to accidentally closing something else.
It confirms with the kernel code where sys_close() removes fd from
fdtable in this stack:
+-> sys_close
+-> file_close_fd
+-> file_close_fd_locked
+-> rcu_assign_pointer(fdt->fd[fd], NULL)
If there was an fd this stack is always reached and fd is always
removed.
Let's replace the fd with -1 after close no matter what.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
