- Define user_aarch64_gcs_entry in core-aarch64.proto to store
Guarded Control Stack state (gcspr_el0, features_enabled).
- Extend thread_info_aarch64 with an optional gcs field
Also extend thread_info_aarch64 with an optional gcs field
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
Introduce an opt-in mode for building and running compel tests
with Guarded Control Stack (GCS) enabled on AArch64.
Changes:
- Extend compel/test/infect to support `GCS_ENABLE=1` builds,
adding `-mbranch-protection=standard` and
`-z experimental-gcs=check` to CFLAGS/LDFLAGS.
- Export required GLIBC_TUNABLES at runtime via `TEST_ENV`.
Usage:
make -C compel/test/infect GCS_ENABLE=1
make -C compel/test/infect GCS_ENABLE=1 run
By default (`GCS_ENABLE` unset or 0), builds and runs are unchanged.
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
When GCS is enabled, the kernel expects a capability token at GCSPR_EL0-8
and sa_restorer at GCSPR_EL0-16 on rt_sigreturn. The sigframe must be
consistent with the kernel’s expectations, with GCSPR_EL0 advanced by -8
having it point to the token on signal entry. On rt_sigreturn, the kernel
verifies the cap at GCSPR_EL0, invalidates it and increments GCSPR_EL0 by 8
at the end of gcs_restore_signal() .
Implement parasite_setup_gcs() to:
- read NT_ARM_GCS via ptrace(PTRACE_GETREGSET)
- write (via ptrace) the computed capability token and restorer address
- update GCSPR_EL0 to point to the token's location
Call parasite_setup_gcs() into parasite_start_daemon() so the sigreturn
frame satisfies kernel's expectation
Tests with GCS remain opt‑in:
make -C compel/test/infect GCS_ENABLE=1 && make -C compel/test/infect run
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
[ alex: cleanup fixes ]
Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
Acked-by: Mike Rapoport <rppt@kernel.org>
Add basic prerequisites for Guarded Control Stack (GCS) state on AArch64.
This adds a gcs_context to the signal frame and extends user_fpregs_struct_t to
carry GCS metadata, preparing the groundwork for GCS in the parasite.
For now, the GCS fields are zeroed during compel_get_task_regs(), technically
ignoring GCS since it does not reach the control logic yet; that will be
introduced in the next commit.
The code path is gated and does not affect normal tests. Can be explicitly
enabled and tested via:
make -C infect GCS_ENABLE=1 && make -C infect run
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
[ alex: clean up fixes ]
Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
Acked-by: Mike Rapoport <rppt@kernel.org>
Introduce ARM64 Guarded Control Stack (GCS) constants and macros
in a new uapi header for use in both CRIU and compel.
Includes:
- NT_ARM_GCS type
- prctl(2) constants for GCS enable/write/push modes
- Capability token helpers (GCS_CAP, GCS_SIGNAL_CAP)
- HWCAP_GCS definition
These are based on upstream Linux definitions
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
Acked-by: Mike Rapoport <rppt@kernel.org>
Refactor user_fpregs_struct_t to wrap user_fpsimd_state in a
dedicated struct, preparing for future extending by just
adding new members
Signed-off-by: Igor Svilenkov Bozic <svilenkov@gmail.com>
[ alex: fixes ]
Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
Acked-by: Mike Rapoport <rppt@kernel.org>
Static code analysis reported:
1. criu/cr-restore.c:2438:2: var_decl: Declaring variable "end_vma"
without initializer.
4. criu/cr-restore.c:2451:5: assign: Assigning: "s_vma" = "&end_vma",
which points to uninitialized data.
7. criu/cr-restore.c:2449:4: uninit_use: Using uninitialized value
"s_vma->list.next".
This tries to fix it by initializing the variable.
Signed-off-by: Adrian Reber <areber@redhat.com>
Static code analysis reported:
criu/cr-dump.c:2328:2: identical_branches: The same code is executed
when the condition "ret" is true or false, because the code in the
if-then branch and after the if statement is identical. Should the if
statement be removed?
This is a fix for the warning.
Signed-off-by: Adrian Reber <areber@redhat.com>
The syntax of the inherit-fd functionality for unix socket and pipe
includes a colon.
Fixes: 0df3f79fc0 ("criu(8): fix --inherit-fd description")
Fixes: c37324b6d0 ("crtools: describe the inherit-fd option")
Signed-off-by: Mark Polyakov <mark@thundercompute.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
On AMD Instinct MI300 systems, restoring a large GPU application can
fail because the checkpoint size is too large and the maximum value of
an offset (with integer type) is insufficient. This problem occurs when
the total size of all buffer objects exceeds int max, not because any
single buffer is too large, but it can also happen with a large number
of small buffers.
Fixes: #2812
Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
These header files are copied directly from the Linux kernel and contain
typos. We skip these files in codespell to simplify maintenance.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
This patch updates drm.h and amdgpu_drm.h kernel headers,
and adds drm_mode.h (included by drm.h) from the rocm-7.1.0
release tag.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
amdgpu_plugin_drm.c:167:6: error: variable 'num_bos' set but not used [-Werror,-Wunused-but-set-variable]
167 | int num_bos = 0;
|
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Add a comment that explains the purpose of `retry_needed`.
Co-authored-by: Andrei Vagin <avagin@google.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Co-authored-by: Andrei Vagin <avagin@google.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Use `return 0` on success in `post_dump_dmabuf_check()` for consistency
with other functions.
Co-authored-by: Andrei Vagin <avagin@google.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
These pr_info lines begin with "CC3" and "TWI" were not meant to be
included in the patch.
Co-authored-by: Andrei Vagin <avagin@google.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
amdgpu libraries that use dmabuf fd to share GPU memory between
processes close the dmabuf fds immediately after using them.
However, it is possible that checkpoint of a process catches one
of the dmabuf fds open. In that case, the amdgpu plugin needs
to handle it.
The checkpoint of the dmabuf fd does require the device file
it was exported from to have already been dumped
To identify which device this dmabuf fd was exprted from, attempt
to import it on each device, then record the dmabuf handle
it imports as. This handle can be used to restore it.
Signed-off-by: David Francis <David.Francis@amd.com>
The amdgpu plugin was counting how many files were checkpointed
to determine when it should close the device files.
The number of device files is not consistent; a process may
have multiple copies of the drm device files open.
Instead of doing this counting, add a new callback after all
files are checkpointed, so plugins can clean up their
resources at an appropriate time.
Signed-off-by: David Francis <David.Francis@amd.com>
Buffer objects held by the amdgpu drm driver are checkpointed with
the new BO_INFO and MAPPING_INFO ioctls/ioctl options. Handling
is in amdgpu_plugin_drm.h
Handling of imported buffer objects may require dmabuf fds to be
transferred between processes. These occur over fdstore, with the
handle-fstore id relationships kept in shread memory. There is a
new plugin callback: RESTORE_INIT to create the shared memory.
During checkpoint, track shared buffer objects, so that buffer objects
that are shared across processes can be identified.
During restore, track which buffer objects have been restored. Retry
restore of a drm file if a buffer object is imported and the
original has not been exported yet. Skip buffer objects that have
already been completed or cannot be completed in the current restore.
So drm code can use sdma_copy_bo, that function no longer requires
kfd bo structs
Update the protobuf messages with new amdgpu drm information.
Signed-off-by: David Francis <David.Francis@amd.com>
The amdgpu plugin usually calls drm ioctls through the libdrm
wrappers. However, amdgpu restore requires dealing with dmabufs
and gem handles directly, which means drm ioctls must be
called directly.
Add the drm.h header (from the kernel's uapi).
Signed-off-by: David Francis <David.Francis@amd.com>
For amdgpu plugin to call the new amdgpu drm CRIU ioctls, it needs
the amdgpu drm header file, copied from the kernel's includes.
Signed-off-by: David Francis <David.Francis@amd.com>
amdgpu dmabuf CRIU requires the ability of the amdgpu plugin
to retry.
Change files_ext.c to read a response of 1 from a plugin restore
function to mean retry.
Signed-off-by: David Francis <David.Francis@amd.com>
amdgpu represents allocated device memory as a memory mapping
of the device file. This is a non-standard VMA that must
be handled by the plugin, not the normal VMA code.
Ignore all VMAs on device files.
Signed-off-by: David Francis <David.Francis@amd.com>
This patch implements the entire logic to enable the offloading of
buffer object content restoration.
The goal of this patch is to offload the buffer object content
restoration to the main CRIU process so that this restoration can occur
in parallel with other restoration logic (mainly the restoration of
memory state in the restore blob, which is time-consuming) to speed up
the restore phase. The restoration of buffer object content usually
takes a significant amount of time for GPU applications, so
parallelizing it with other operations can reduce the overall restore
time.
It has three parts: the first replaces the restoration of buffer objects
in the target process by sending a parallel restore command to the main
CRIU process; the second implements the POST_FORKING hook in the amdgpu
plugin to enable buffer object content restoration in the main CRIU
process; the third stops the parallel thread in the RESUME_DEVICES_LATE
hook.
This optimization only focuses on the single-process situation (common
case). In other scenarios, it will turn to the original method. This is
achieved with the new `parallel_disabled` flag.
Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
Currently the restore of buffer object comsumes a significant amount of
time. However, this part has no logical dependencies with other restore
operations. This patch introduce some structures and some helper
functions for the target process to offload this task to the main CRIU
process.
Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
When enabling parallel restore, the target process and the main CRIU
process need an IPC interface to communicate and transfer restore
commands. This patch adds a Unix domain TCP socket and stores this
socket in `fdstore`.
Signed-off-by: Yanning Yang <yangyanning@sjtu.edu.cn>
Add a transition test to verify that CRIU correctly avoids active rseq
critical sections when selecting an address for parasite code injection
and execution during checkpointing.
If CRIU were to inject parasite blobs or execute parasite syscalls
inside an active restartable sequence critical section, kernel
preemption hooks (`rseq_handle_notify_resume`) would trigger sequence
aborts or fixups, clobbering userspace memory or breaking parasite
execution.
- Emulates a spinning worker loop (`while (futex_get(f) == 0)`) inside
an active rseq critical section so the thread is consistently inside
`[start_ip, start_ip + post_commit_offset)` when CRIU initiates
checkpointing.
- Dynamically sets and clears `__rseq_abi.rseq_cs` inside the assembly
block (`start_ip`), adhering to the rseq protocol where critical
section descriptors must only be active during critical section
execution.
- Configures `abort_ip` to return (`ret`) out of the assembly function
back to `test_waitsig_rseq()`, where a wrapper loop re-invokes the
function if C/R is still in progress or an interruption occurs.
Signed-off-by: Andrei Vagin <avagin@google.com>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
compel has to abort all rseq critical sections.
Signed-off-by: Andrei Vagin <avagin@google.com>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
A parasite code is injected into a target process and executed it via
ptrace. If the instruction pointer is inside an rseq critical section
when parasite code starts executing, the critical section can be aborted
and IP will be redirected to rseq_cs->abort_ip, which isn't expected.
Here is one more issue is that the kernel zeroes out rseq->rseq_cs in
userspace memory, destroying the rseq state before CRIU can record it.
To avoid critical sections during parasite execution, compel aborts a
critical section when it is required.
Reported-by: Alberto Ventafridda <alberto.ventafridda@gmail.com>
Signed-off-by: Andrei Vagin <avagin@google.com>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Commit 2d2168fc9 hardcodes ELFDATA2LSB in place of a previously used
endian-aware macro BORD, causing issues on big endian architectures,
e.g., on s390x:
$ ./criu/criu check -v4
(00.000000) CRIU run id = 6b0cc8b9-0cbe-44e7-aeb4-7387862d9670
(00.000016) Version: 4.2 (gitid v4.2-211-g4d76d1acd)
[...]
(00.061309) vdso: Parsing at 3ffc75ae000 3ffc75b0000
(00.061319) Error (criu/pie-util-vdso.c:126): vdso: Unsupported ELF data encoding: 2
(00.061320) Error (criu/vdso.c:633): vdso: Failed to fill self vdso symtable
(00.061321) Error (criu/kerndat.c:2043): kerndat_vdso_fill_symtable failed when initializing kerndat.
(00.061367) Adjust mmap_min_addr 0x1000 -> 0x10000
(00.061368) Found mmap_min_addr 0x10000
(00.061374) files stat: fs/nr_open 1073741816
(00.061375) Error (criu/crtools.c:280): Could not initialize kernel features detection.
Fixes: 2d2168fc9 ("vdso: relax EI_OSABI check to support linux in ELF header")
Signed-off-by: Čestmír Kalina <ckalina@redhat.com>
Commit 2e30db5c3d added --network-lock handling and a CLI restore
warning that checks argv[optind] after option parsing.
However, CRIU can also call parse_options() from the RPC config parsing
path with argc == 0 and argv == NULL. This path was added by 99d52ec7dd
for req->config_file handling.
If an RPC configuration file contains "network-lock iptables",
has_network_lock_opt is set and the restore warning dereferences
argv[optind], crashing criu swrk.
Guard the restore warning with argv && optind < argc before accessing
argv[optind].
Fixes: 2e30db5c3d ("criu: add --network-lock option to allow nftables alternative")
Signed-off-by: Zhang Xuedong <2190206983@qq.com>
Linking on arm64 using lld fails with:
LINK criu/pie/parasite.built-in.o
ld: error: discarding .shstrtab section is not allowed
keep the missing sections to enhance portability
Fixes: #2978
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
In restorer_get_vma_hint, if a VMA from self_vma_list or tgt_vma_list
overlaps with the current candidate region (prev_vma_end + vma_len), we
advance prev_vma_end and move to the next VMA in the list. However, we
must then restart the check from the beginning of the while loop to
ensure that the updated candidate region doesn't overlap with any other
VMAs.
Fixes#3018
Fixes: 2c9ee0abd2 ("restore: update prev_vma_end after processing the last vma in a list")
Signed-off-by: Andrei Vagin <avagin@google.com>
Here is actually two issues:
First, prev_vma_end isn't updated properly after processing the last vma
in a list.
Second, a dummy end_vma isn't fully initialized and vma_area_is can
access uninitilized data.
Signed-off-by: Andrei Vagin <avagin@google.com>
VMA_AREA_GUARD areas are added to the end of the vma_area_list and
are not necessarily sorted by address. pmc_fill_cache() expects
VMAs to be sorted and triggers BUG_ON() if it meets a VMA with
a lower address than the current cache window.
Since guard VMAs don't have pagemap entries and are skipped during
dump, just stop the lookahead loop when we encounter one.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
Install libcriu into a local staging directory (.install/) using
'make install-lib' and reference only that directory during build
and test:
CFLAGS := -I$(INSTALL_DIR)/include/criu
LDFLAGS := -L$(INSTALL_DIR)/lib -lcriu
LD_LIBRARY_PATH=${MAIN_DIR}/.install/lib
Nothing from the non-test part of the repo is used at compile,
link, or run time.
The compile rule is also corrected to use $< instead of $^
for gcc -c.
Fixes: #2927
Signed-off-by: Ahmed Elaidy <elaidya225@gmail.com>
Default to the pure-python protobuf implementation
on Python 3.14+ when the user has not explicitly
selected a backend. This avoids known protobuf
C-extension import failures seen in some distro
packaging setups.
Fixes#2848
Signed-off-by: Farzan Aman Khan <farzanaman99@gmail.com>
Change compel_task_size() to probe the actual kernel address space
limit at runtime via mmap() with MAP_FIXED_NOREPLACE flag.
On systems where the CPU supports 5-level paging but the kernel
doesn't have it enabled, we need to detect the actual task size the
kernel is using.
On x86-64, task_size is either (1UL << 47) for 4-level paging or
(1UL << 56) for 5-level paging. The function probes by attempting
to mmap at the 4-level boundary and if it succeeds or the address
is already occupied, the kernel supports 5-level paging.
Fixes: #2877
Signed-off-by: Tushar Mohapatra <tusharmohapatra.gig@gmail.com>
Fedora rawhide ships a pre-release of GCC 16 which produces following
error:
uprobes.c:34:22: error: variable ‘dummy’ set but not used [-Werror=unused-but-set-variable=]
34 | volatile int dummy = 0;
| ^~~~~
Marking this variable as "__maybe_unused" to fix the error.
Signed-off-by: Adrian Reber <areber@redhat.com>
After commit [1] we accidentally stopped reporting the errors from
kerndat_has_timer_cr_ids(), let's fix that.
Fixes: 1eaa870cc ("kerndat: check that hardware breakpoints work") [1]
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
ctrl_dir_and_opt() builds comma-separated controller directory and
mount option strings by repeatedly appending via snprintf() into
stack-allocated buffers. The offsets (doff, ooff) are advanced by
the snprintf() return value, but snprintf() returns the number of
bytes that *would* have been written on truncation, not the number
actually written. When a controller name is long enough to cause
truncation, the offset advances past the buffer end. On the next
iteration, the remaining-length calculation (e.g. ds - doff) goes
negative and is implicitly cast to a large size_t, disabling the
truncation guard entirely. Subsequent snprintf() calls and the
trailing-comma removal then write out of bounds on the stack
(CWE-121: Stack-based Buffer Overflow).
A crafted cgroup.img with oversized cnames entries triggers this
during restore in any code path that calls ctrl_dir_and_opt():
prepare_cgns(), move_in_cgroup(), prepare_cgroup_properties(),
prepare_cgroup_sfd(), and cgroupd().
Fix this by checking every snprintf() call for truncation before
advancing the offset. Each append now verifies that remaining
space is positive and that snprintf() did not return a value
greater than or equal to the remaining space. On any overflow
the function returns -1 and all five callers now propagate this
error. Also change the type of `off` in prepare_cgroup_properties()
from unsigned int to int so the -1 return is not silently converted
to a large positive value.
This vulnerability was identified and analyzed by an AI security
analysis tool.
Assisted-by: claude-opus-4-6
Signed-off-by: Adrian Reber <areber@redhat.com>
The service socket now defaults to 0600 (owner-only). Since the
test runs the setuid criu binary as a non-root user (uid 1000),
the socket ends up owned by root. Chown the socket to the test
user after service startup so it can connect, mirroring what an
administrator would do in production.
Assisted-by: Claude Code (claude-opus-4-6):claude-opus-4-6@default
Generated with Claude Code (https://claude.ai/code)
Signed-off-by: Adrian Reber <areber@redhat.com>
Change the service socket permissions from 0666 (world-accessible)
to 0600 (owner-only) to prevent unauthorized local users from
connecting to the CRIU service socket.
Print a message informing the administrator that the socket is
restricted and that chmod should be used to widen access if needed.
Assisted-by: Claude Code (claude-opus-4-6):claude-opus-4-6@default
Generated with Claude Code (https://claude.ai/code)
Signed-off-by: Adrian Reber <areber@redhat.com>
Starting with Linux 7.1, the kernel enforces strict read-only field
protection for the rseq V2 ABI. Userspace is no longer allowed to
modify kernel-managed fields (cpu_id_start, cpu_id, node_id, mm_cid)
while rseq is registered. Violations are detected on the next context
switch and the offending process is killed with SIGSEGV.
The rseq01 test was zeroing the entire glibc-registered rseq area
with memset() before calling test_init(), which internally calls
fork(). This corrupted the read-only fields while the glibc rseq
registration was still active, causing the kernel to send SIGSEGV
during the fork.
Fix this by calling unregister_old_rseq() before the memset in both
main() and thread_routine(), so the kernel is no longer monitoring
those fields when they are zeroed. The subsequent register_thread()
call re-registers rseq with the clean area.
Link: 7f00232152
Assisted-by: Claude Code (https://claude.ai/code):claude-opus-4-6
Signed-off-by: Adrian Reber <areber@redhat.com>
Kernel 7.1 removed IPPROTO_UDPLITE support. Add a checkskip
script that probes for a UDPLITE socket and skips the test
with EPROTONOSUPPORT instead of failing.
Assisted-by: Claude Code (claude-opus-4-6):claude-opus-4-6@default
Signed-off-by: Adrian Reber <areber@redhat.com>
