Mirrors/criu
1
0
Fork 0
mirror of https://github.com/checkpoint-restore/criu.git synced 2026-07-17 16:47:50 +00:00
Code Issues Projects Releases Wiki Activity

docs: Clarify memory consistency and performance in pre-dump documentation

Browse source 
Correct the documentation regarding `vmsplice` and `SPLICE_F_GIFT`
behavior.  `SPLICE_F_GIFT` does not enforce Copy-on-Write (COW) when the
user-space process modifies the gifted memory. Instead, it is a
zero-copy mechanism where post-resume modifications can lead to
inconsistent intermediate dumps.

Explain how CRIU handles this inconsistency via its iterative design,
relying on the soft-dirty tracker to catch post-resume modifications and
re-dump them in subsequent iterations, ensuring final consistency.

Also emphasize that the `splice` scheme relies heavily on `vmsplice`
performance to minimize freeze time, making the migration almost
invisible to the process.

Signed-off-by: Andrei Vagin <avagin@google.com>
This commit is contained in:
Andrei Vagin 2026-06-23 16:57:03 +00:00 • committed by Andrei Vagin
parent d87bc8c0fc
commit f3b293fd85
1 changed files with 13 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

17
Documentation/under-the-hood/optimizing-pre-dump-algorithm.md
View file

@ -5,9 +5,9 @@ Pre-dumping is the process of capturing dirty memory pages while an application
## Traditional vs. Optimized Pre-dump
### The `read` Mode (Traditional)
In this mode, CRIU uses the `process_vm_readv()` system call to read memory from the target process.
In this mode, CRIU uses the `process_vm_readv` system call to read memory from the target process.
* **Workflow**: Tasks are briefly frozen to identify dirty pages and reset the soft-dirty bit, then resumed. CRIU then reads the pages from the running process's address space.
* **Challenge**: Reading memory while a process is running can lead to minor inconsistencies if the process modifies a page *while* it is being read. Furthermore, `process_vm_readv()` requires the target process to be alive and its memory mappings to remain stable during the read.
* **Challenge**: Reading memory while a process is running can lead to minor inconsistencies if the process modifies a page *while* it is being read (see [Memory Consistency](#memory-consistency) below). Furthermore, `process_vm_readv` requires the target process to be alive and its memory mappings to remain stable during the read.
### The `splice` Mode (Optimized & Default)
The `splice` mode (enabled via `--pre-dump-mode=splice`) uses a zero-copy "gift" mechanism to further reduce freeze time and improve reliability.
@ -19,8 +19,17 @@ The `splice` mode (enabled via `--pre-dump-mode=splice`) uses a zero-copy "gift"
4. **Parallel Draining**: While the tasks are running, the main CRIU process "drains" the data from the pipes and writes it to the image files or sends it to the page server.
#### Why `splice` is Better:
* **Consistency via COW**: The `SPLICE_F_GIFT` flag ensures that if the process modifies a "gifted" page after resuming, the kernel performs a **Copy-on-Write (COW)**. The pipe buffer continues to hold the *original* version of the page as it existed at the moment of the `vmsplice()` call, ensuring a perfectly consistent snapshot of that page.
* **Minimized Downtime**: The "freeze" duration is reduced to just the time needed for the parasite to execute the `vmsplice()` system calls, rather than the time needed to transfer gigabytes of memory data over the network or to disk.
* **Minimized Downtime**: The "freeze" duration is reduced to just the time needed for the parasite to execute the `vmsplice()` system calls, rather than the time needed to transfer memory data over the network or to disk. This scheme relies entirely on `vmsplice()` being extremely fast. Because the target process is frozen during these calls, minimizing this duration is critical to the primary goal of pre-dumping: reducing process downtime during live migration and making the migration process almost invisible to the application.
* **Zero-Copy Transfer**: By gifting pages directly from the target process to the pipe, `splice` mode avoids copying memory to CRIU user-space buffers (unlike `read` mode which uses `process_vm_readv` to copy data). While this zero-copy mechanism does not use COW (meaning intermediate dumps can be inconsistent if pages are modified after resume), CRIU's iterative design handles this inconsistency (see below) while maximizing transfer performance.
## Memory Consistency
Because the target process is resumed while CRIU is still writing the memory data (in both `read` and `splice` modes), intermediate pre-dump images may contain inconsistent memory states.
This inconsistency is expected and handled by CRIU's iterative design:
1. **Tracking Changes**: CRIU uses the kernel's soft-dirty tracker to monitor memory writes after the process is resumed.
2. **Subsequent Dumps**: Any page modified after it has been pre-dumped is marked dirty again and will be captured in the next pre-dump iteration or during the final dump.
3. **Restoring Consistency**: During restore, CRIU applies the dump images in sequence (from oldest pre-dump to the final dump). The final dump is taken while the process is fully frozen, ensuring that the final state of all memory pages is consistent.
## Usage