Since #3022 restore reads the pages image with O_DIRECT unconditionally. That
helps a cold or cross-host restore, but on a same-host restore it bypasses the
still-warm cache and reads from disk, which made maps04 much slower (#3053).
Add an --image-io-mode option to select the pages-image I/O method: writeback
(default) keeps the buffered behaviour, direct uses O_DIRECT. It applies to both
dump and restore and can be set on the command line or over RPC.
Gate the restore side at the probe's call sites: open_page_read_at() and the
restore vma-io builder in mem.c switch the pages fd to O_DIRECT only when direct
mode was requested, so writeback keeps reading through the host page cache. The
probe itself no longer inspects the mode; as it now runs only on an explicit
request, its buffered-fallback log becomes a warning. The dump side follows in
the next commit.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Dan Feigin <dfeigin@nvidia.com>
Correct the documentation regarding `vmsplice` and `SPLICE_F_GIFT`
behavior. `SPLICE_F_GIFT` does not enforce Copy-on-Write (COW) when the
user-space process modifies the gifted memory. Instead, it is a
zero-copy mechanism where post-resume modifications can lead to
inconsistent intermediate dumps.
Explain how CRIU handles this inconsistency via its iterative design,
relying on the soft-dirty tracker to catch post-resume modifications and
re-dump them in subsequent iterations, ensuring final consistency.
Also emphasize that the `splice` scheme relies heavily on `vmsplice`
performance to minimize freeze time, making the migration almost
invisible to the process.
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the profile identification and namespace dumping process
- Document the use of the 'parasite profile' for non-disruptive dumping
- Detail policy loading via apparmor_parser and namespace reconstruction
- Support for modern features like Profile Stacking
- List kernel and filesystem requirements
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the hardware-assisted shadow stack mechanism
- Document the state capture via NT_ARM_GCS ptrace regset
- Detail restoration using map_shadow_stack and sigframe integration
- List kernel requirements for AArch64 hosts
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain how zombies are identified and their exit codes captured
- Describe the 'helper technique' for restoring zombies via immediate exit
- Detail parent-child coordination to prevent premature reaping
- Add references to related technical documentation
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the decouplling of socket paths and inodes
- Document the SIOCUNIXFILE ioctl for stable handle retrieval
- Describe the restoration workflow (tmpfs yard, peer coordination)
- Explain the capture and redelivery of in-flight file descriptors
- Clarify handling of external Unix sockets
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the address and ABI mismatch challenges
- Detail the Proxy (Patching) method for older kernels
- Document the modern arch_prctl method for native vDSO mapping
- Explain the role and restoration of the VVAR data region
- Mention automatic feature detection via Kerndat
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain Build-ID extraction (ELF magic, 1MB mapping)
- Document 'buildid' (default) vs 'filesize' methods
- Explain the automatic fallback mechanism
- Describe the importance for security and memory pointer integrity
- Detail usage via the --file-validation flag
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the mechanics of Lazy Migration and on-demand page loading
- Detail the Lazy Pages Daemon and the UFFD descriptor handover (SCM_RIGHTS)
- Document the use of non-cooperative UFFD features (Fork, Remap, Unmap)
- Describe the page fault handling loop and page server integration
- Clarify benefits and trade-offs of the lazy approach
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the capture of device attributes (TUN vs TAP, Flags)
- Explain index preservation using TUNSETIFINDEX
- Document multi-queue support and re-attachment via TUNSETQUEUE
- Clarify current limitations (BPF filters, in-flight packets)
- Explain persistency management during restoration
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the PTY index restoration 'brute-force' strategy
- Detail the capture of termios, winsize, and ownership
- Describe the restoration workflow for master and slave peers
- Clarify the status of buffered data and legacy BSD PTYs
- Document the re-binding of controlling terminals (TIOCSCTTY)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the mechanics of TCP Repair Mode and state manipulation
- Explain the role of libsoccr in capturing sequence numbers and options
- Document the network locking workflow using nftables/iptables
- Describe the 'Silent Close' technique to preserve peer connections
- Highlight the importance of sequence number and window restoration
Signed-off-by: Andrei Vagin <avagin@google.com>
- Formalize the CR_STATE_* state machine and synchronization mechanism
- Detail the multi-stage restoration workflow (Root Task, NS Prep, Forking, etc.)
- Explain the security rationale for Stage 6 (Credentials and Seccomp)
- Document the final transition via sigreturn and thread restoration
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the use of sock_diag for kernel state extraction
- Describe the SCM_RIGHTS mechanism for queue inspection
- Detail TCP Repair Mode for connection restoration
- List supported families including Netlink and Packet sockets
- Improve overall structure and technical depth
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the use of internal inode numbers (shmid) for anonymous sharing
- Detail the restoration of shared anonymous regions via memfd_create()
- Describe the 'master' vs 'slave' roles and futex synchronization
- Document System V IPC and file-backed shared mapping restoration
- Add references to kcmp and memory dumping optimizations
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the top-down allocation strategy using RLIMIT_NOFILE
- Explain per-process isolation (service_fd_id) for shared FD tables
- Document the relocation mechanism (F_DUPFD_CLOEXEC, dup3)
- Describe the 'sfds_protected' flag and safety invariants
- List common Service FD types (LOG, IMG, RPC, TRANSPORT, etc.)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the necessity of a dedicated context for memory swapping
- Describe the shared restorer mapping and mremap-based re-positioning
- Detail the safe hole detection strategy to avoid VMA conflicts
- Document the final transition via sigreturn
- Highlight the characteristics of the freestanding PIE blob
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the sensitivity of rseq state to process execution
- Document the use of PTRACE_GET_RSEQ_CONF and external peeking
- Detail the critical requirement to unregister the restorer's own rseq
- Explain how re-registration and rseq_cs restoration ensure automatic kernel fixups
- Update kernel requirements (v5.13 for automated detection)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain stable identification vs numeric PIDs
- Detail restoration of alive vs dead processes
- Document the 'helper process' trick for dead pidfds
- Explain the transition from anonymous inodes to pidfs (kernel v6.9+)
- Clarify current limitations (PIDFD_THREAD)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the PID reuse problem during iterative migration
- Document the use of pidfd_open() for race-free identification
- Detail the 'socket trick' for persistent FD storage via SCM_RIGHTS
- Explain the identity verification process in subsequent iterations
- List required kernel features (pidfd_open, pidfd_getfd)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the legacy ns_last_pid interface and its limitations
- Detail the modern clone3() with set_tid mechanism (kernel v5.5+)
- Describe the benefits of atomic PID assignment and nested namespace support
- Mention automatic feature detection via Kerndat
- Document implementation using architecture-specific assembly wrappers
Signed-off-by: Andrei Vagin <avagin@google.com>
- Document the use of PTRACE_PEEKSIGINFO (kernel v3.10+) for dumping
- Explain the distinction between shared and private pending signals
- Detail the batch processing of signal queues
- Explain the usage of rt_sigqueueinfo and rt_tgsigqueueinfo for restoration
- Clarify the importance of PTRACE_GETSIGMASK
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the multi-stage infection process using ptrace
- Explain memory exchange optimization via memfd
- Describe the PIE relocation and GOT patching
- Explain the daemon mode and request-response control loop
- Document the 'cure' operation for returning tasks to their original state
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the 2MB sliding window (PMC_SIZE) mechanism
- Describe greedy prefetching for adjacent small VMAs
- Document the usage of ioctl(PAGEMAP_SCAN) on modern kernels
- Clarify cache invalidation strategies during dump phases
- Mention the CRIU_PMC_OFF environment variable for debugging
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain 'splice' (default) vs 'read' (traditional) pre-dump modes
- Detail the zero-copy gifting mechanism using vmsplice() and SPLICE_F_GIFT
- Explain how COW ensures consistency while capturing memory from running tasks
- Describe parallel draining of pages to minimize freeze time
Signed-off-by: Andrei Vagin <avagin@google.com>
- Formalize the stages of the Mount V2 engine
- Explain the role of the Root Yard (root_yard_mp)
- Describe pre-fork plain mounting and source resolution
- Detail propagation restoration via MOVE_MOUNT_SET_GROUP
- Explain the final pivot_root and post-fork fixup stages
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the shift from path-based to FD-based mounting
- Detail the use of detached mounts via open_tree() and fsopen()
- Describe propagation grouping via MOVE_MOUNT_SET_GROUP (v5.15+)
- Explain the tree construction and atomic final attachment process
- Update kernel requirements and fallback behavior
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail key information captured from /proc/pid/mountinfo
- Explain the restoration challenges (dependencies, propagation)
- Document the Mount V2 engine and its benefits
- Explain external mount mapping and auto-detection
- List common issues like overmounts and unsupported filesystems
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the in_parent flag in pagemap entries
- Detail detection of unchanged pages via soft-dirty bit
- Document the --auto-dedup mode for dump and restore
- Describe online disk space reclamation using FALLOC_FL_PUNCH_HOLE
- Clarify image chaining and sparse file support
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the multi-stage dumping approach involving parasite injection
- Explain zero-copy dumping using vmsplice() and SPLICE_F_GIFT
- Describe the use of splice() for efficient image writing and page server transport
- Document VMA re-mapping and content filling during restoration
- Add references to COW preservation and lazy migration (userfaultfd)
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the soft-dirty bit mechanism for tracking modified pages
- Document the usage of ioctl(PAGEMAP_SCAN) for efficient scanning (kernel v6.7+)
- Describe the iterative pre-dump workflow and image chaining
- Detail the consolidation of pages during restoration
- Mention the role of the page server in minimizing disk I/O
Signed-off-by: Andrei Vagin <avagin@google.com>
- Clarify feature detection for system calls, filesystems, and namespaces
- Update persistent caching locations (/run/criu.kdat vs XDG_RUNTIME_DIR)
- Distinguish between kerndat (host capabilities) and inventory (checkpoint metadata)
- Mention 'criu check --extra' for runtime inspection
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the kernel pointer comparison mechanism of kcmp()
- Describe the two-level red-black tree optimization (genid + kcmp sub-tree)
- List all supported KCMP_* types (FILE, VM, FILES, FS, EPOLL_TFD, etc.)
- Clarify how genid minimizes expensive system calls
Signed-off-by: Andrei Vagin <avagin@google.com>
- Describe the (inode, device) to path resolution problem
- List default heuristic scan hints (/etc, /var/log, etc.)
- Explain user-defined scan paths via --irmap-scan-path
- Detail the pre-dump optimization and irmap-cache.img
- Clarify the status of Irmap vs open_by_handle_at on modern kernels
Signed-off-by: Andrei Vagin <avagin@google.com>
- Formalize Master and Slave descriptor roles
- Explain the SCM_RIGHTS distribution mechanism
- Document transport socket naming and 'criu_run_id' usage
- Detail deterministic master selection to avoid deadlocks
- Explain dynamic service FD relocation during collisions
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the challenges of finding the 'watchee' path
- Explain the use of open_by_handle_at() and Irmap
- Explicitly document that pending events are dropped with a warning
- Explain how spurious events are generated during restore (ghost files)
- Add details for Fanotify inode and mount marks
Signed-off-by: Andrei Vagin <avagin@google.com>
- Explain the PTRACE_SEIZE and PTRACE_INTERRUPT sequence
- Detail the transparency of ptrace-stop (TRAP_STOP)
- Document cgroup v1 and v2 freezer mechanisms
- Mention kernel kludges for v1 freezer unreliability
- Clarify the relationship between freezer and ptrace
Signed-off-by: Andrei Vagin <avagin@google.com>
- Formalize TASK_ALIVE, TASK_STOPPED, and TASK_DEAD states
- Explain the rationale for default behaviors in dump/restore
- Mention pre-dump enforcement of the Running state
- Document the use of --leave-stopped for debugging
- Add instructions for resuming trees via SIGCONT and pstree_cont.py
Signed-off-by: Andrei Vagin <avagin@google.com>
- Formalize the Master and Slave descriptor concepts
- Describe the 'open()' state machine and early FD distribution via SCM_RIGHTS
- Document the inter-process synchronization (set_fds_event, futexes)
- List key dependencies (TTYs, Unix Sockets, Epoll)
- Add notes on Service FDs and restoration ordering
Signed-off-by: Andrei Vagin <avagin@google.com>
- Detail the Linux file object hierarchy (Inode, Dentry, File)
- Explain the SCM_RIGHTS mechanism for retrieving local FD copies
- Describe the gen_id and kcmp optimization for shared file detection
- Clarify the two-tier image storage structure (fdinfo vs specialized images)
Signed-off-by: Andrei Vagin <avagin@google.com>
