From 39f7aa6801901fcf56173bbc4f47de9cf994c99c Mon Sep 17 00:00:00 2001 From: Andrei Vagin Date: Sat, 7 Mar 2026 22:44:48 +0000 Subject: [PATCH] docs: update Restartable Sequences documentation - Explain the sensitivity of rseq state to process execution - Document the use of PTRACE_GET_RSEQ_CONF and external peeking - Detail the critical requirement to unregister the restorer's own rseq - Explain how re-registration and rseq_cs restoration ensure automatic kernel fixups - Update kernel requirements (v5.13 for automated detection) Signed-off-by: Andrei Vagin --- .../under-the-hood/restartable-sequences.md | 90 ++++++------------- 1 file changed, 25 insertions(+), 65 deletions(-) diff --git a/Documentation/under-the-hood/restartable-sequences.md b/Documentation/under-the-hood/restartable-sequences.md index 39ce59d7e..50fe3b0b2 100644 --- a/Documentation/under-the-hood/restartable-sequences.md +++ b/Documentation/under-the-hood/restartable-sequences.md @@ -1,78 +1,38 @@ -# Restartable Sequences +# Restartable Sequences (rseq) -Restartable sequences (RSEQ) are short, carefully defined sections of userspace code that enable efficient access to per-CPU data structures without the overhead of heavyweight synchronization primitives like mutexes or atomic operations. +Restartable Sequences (rseq) is a Linux kernel feature (introduced in v4.18) that enables high-performance userspace operations on per-CPU data without requiring atomic instructions or traditional locking. Each thread registers a `struct rseq`, and the kernel ensures that if a thread is preempted or interrupted while inside a critical section, it is "restarted" by jumping to a predefined abort handler. -Introduced in Linux kernel version 4.18, RSEQ allows userspace programs to register critical code paths that the kernel can safely restart if a CPU migration or preemption occurs. This mechanism enables high-performance, scalable data access while ensuring correctness. For more background, see [The 5-year journey to bring restartable sequences to Linux](https://www.efficios.com/blog/2019/02/08/linux-restartable-sequences/). +## The Challenge of C/R with rseq -## Linux Kernel Interface +Checkpointing and restoring rseq is exceptionally delicate because the kernel's rseq state is tightly coupled with process execution. -The kernel interface for RSEQ is minimal, consisting of a single system call: -`sys_rseq(struct rseq *rseq, uint32_t rseq_len, int flags, uint32_t sig)` +1. **Dumping Sensitivity**: If an infected thread is allowed to run even briefly (e.g., to execute parasite code), the kernel's `rseq_handle_notify_resume` hook may be triggered. This would cause the kernel to "fix up" the rseq state, clearing critical section pointers in userspace memory and losing the very state CRIU needs to capture. +2. **Restoration Morphing**: During restoration, CRIU "morphs" into the target process. If the CRIU binary itself was compiled with rseq support (common in modern distributions), it may have an active rseq registration that must be carefully managed before the memory layout is swapped. -The data structures and flags are defined in [`include/uapi/linux/rseq.h`](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/rseq.h): +## How CRIU Handles rseq -```c -enum rseq_cs_flags { - RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT = (1U << RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT_BIT), - RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL = (1U << RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL_BIT), - RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE = (1U << RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE_BIT), -}; +CRIU provides robust support for rseq, ensuring that threads interrupted within a critical section correctly restart after restoration. -struct rseq_cs { - __u32 version; /* always 0 currently */ - enum rseq_cs_flags flags; - void *start_ip; - intptr_t post_commit_offset; - void *abort_ip; -}; +### 1. Checkpointing (Dumping) +CRIU captures the rseq configuration without disturbing the thread's execution state: -struct rseq { - __u32 cpu_id_start; - __u32 cpu_id; - struct rseq_cs *rseq_cs; - enum rseq_cs_flags flags; -}; -``` +* **PTRACE_GET_RSEQ_CONF**: CRIU uses this ptrace command (Kernel v5.13+) to retrieve the address, size, and signature of the `struct rseq` registered for each thread. +* **External Peeking**: To avoid triggering kernel fixups, CRIU **does not** use its standard parasite code to read rseq-related memory. Instead, it uses `PTRACE_PEEKDATA` to read the `struct rseq` and `struct rseq_cs` (critical section descriptor) directly from the outside while the task is frozen. +* **Critical Section Detection**: By reading the `rseq_cs` pointer within the `struct rseq`, CRIU identifies if a thread was in the middle of a sequence at the time of the snapshot. -To use RSEQ, an application must maintain a `struct rseq` and register it with the kernel. Before entering an RSEQ critical section, the application populates a `struct rseq_cs` with the start address and an abort handler (to be called if the section is interrupted) and updates the `rseq_cs` pointer in the registered `struct rseq`. +### 2. Restoration +The restoration process involves two critical rseq-related steps: -## Handling RSEQ Flags +* **Unregistering Restorer rseq**: Before CRIU performs the final "morphing" (unmapping its own memory and mapping the application's memory), it must explicitly **unregister** any rseq area used by the CRIU process itself. Failing to do so would cause the kernel to attempt to update a `cpu_id` field in memory that has been unmapped, resulting in an immediate segmentation fault. +* **Re-establishing Application rseq**: Once the application's memory layout and thread registers are restored, CRIU calls the `rseq()` system call for each thread. It re-registers the original `struct rseq` at its original address. +* **Automatic Restart**: Because the `rseq_cs` pointer is restored as part of the thread's memory, the kernel will detect the active critical section upon the first resumption and automatically trigger the application's restart/abort logic, ensuring data integrity. -Flags can be specified in both `struct rseq` and `struct rseq_cs`. The kernel combines these flags to determine restart behavior. Typically, `flags` is zero, meaning the critical section is interrupted and the instruction pointer (IP) is redirected to the abort handler if preemption, migration, or a signal occurs. Applications can use specific flags to allow completion even during certain events. +## Kernel Requirements -Note that `RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL` must be used alongside both `RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT` and `RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE` to ensure consistent restart semantics. +* **rseq support**: Linux Kernel v4.18+ +* **PTRACE_GET_RSEQ_CONF**: Linux Kernel v5.13+ (Required for reliable automated detection). -## Checkpoint/Restore of RSEQ - -CRIU handles RSEQ based on the process's execution state at checkpoint time: - -### Case 1: Outside a Critical Section -This is the simplest scenario. The process has a registered `struct rseq`, but the IP is not within a critical section. - -- **Checkpoint**: CRIU uses `PTRACE_GET_RSEQ_CONFIGURATION` to locate the `struct rseq` and record its address, length, and signature. -- **Restore**: CRIU retrieves the `struct rseq` information from the image and re-registers it from the restorer context. - -### Case 2: Inside an Abortable Critical Section -If the IP is within an RSEQ critical section with standard flags (e.g., `0`), RSEQ semantics require that an interruption redirects the IP to the abort handler. Simply restoring the process with its original IP would violate these semantics. - -- **Checkpoint**: CRIU records the `struct rseq` configuration and explicitly adjusts the saved IP to point to the RSEQ abort handler. -- **Restore**: The process resumes execution at the abort handler, ensuring a semantically correct state. - -The `fixup_thread_rseq` function implements this logic by detecting if the IP falls within the registered `rseq_cs` range and rewriting it as needed. - -### Case 3: Inside a Non-Abortable Critical Section -When `RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL` is set, the section is non-abortable. However, special handling is still required because the kernel clears the `rseq_cs` pointer when CRIU transfers execution to the parasite code. - -- **Checkpoint**: CRIU saves the `struct rseq` configuration and the IP without modification, but also explicitly records the `rseq_cs` field. -- **Restore**: CRIU re-registers the `struct rseq` and then uses `PTRACE_POKEAREA` to restore the `rseq_cs` pointer, re-establishing the kernel's RSEQ execution context. - -## TODO - -- Test support for all architectures (currently x86_64 only). -- Improve support for built-in RSEQ in non-Glibc libraries. -- Add comprehensive tests for pre-dump, leave-running, and multi-threaded scenarios. - -## Useful links -- https://github.com/torvalds/linux/blob/master/kernel/rseq.c -- https://www.efficios.com/blog/2019/02/08/linux-restartable-sequences/ -- https://lwn.net/Articles/883104/ +## See also +* [Checkpoint/Restore Architecture](checkpointrestore.md) +* [Parasite Code](parasite-code.md) +* [Restorer Context](restorer-context.md)