compel/infect: Use waitpid with specific PIDs in compel_stop_tasks_on_syscall

When CRIU traces a large number of tasks, calling wait4(-1, ...) can
be slow because the kernel has to iterate over many tasks to find one
that has changed state. This overhead becomes significant when
thousands of tasks are involved. The time complexity of the old approach
was O(n^2) because we called wait4(-1, ...) N times, and each call
took O(N) time in the kernel.

This patch introduces compel_stop_tasks_on_syscall which takes an
array of PIDs and waits for each one specifically. This avoids the
O(N) search in the kernel's do_wait implementation and significantly
improves performance during the restore phase when many tasks are
being resumed and stopped at the rt_sigreturn syscall.

Some parallelism still exists in this approach. The loop waits for each
process to stop on a syscall, verifies it, and then lets it continue
running. While we move to wait for the next process in the array, the
previously processed ones continue their execution in parallel.

The performance profile of the bottleneck showed:
   - 70.45% entry_SYSCALL_64
      - do_syscall_64
         - 68.29% __do_sys_wait4
            - kernel_wait4
               - 68.28% do_wait
                  - 68.26% __do_wait
                     - 65.69% wait_consider_task
                          8.71% _raw_spin_lock_irq
                          2.39% _raw_spin_unlock_irq

Signed-off-by: Andrei Vagin <avagin@google.com>
This commit is contained in:
Andrei Vagin 2026-03-09 00:19:00 +00:00 committed by Alexander Mikhalitsyn
parent 3d242917fa
commit 03d41eff5d
No known key found for this signature in database
GPG key ID: B1F47F5CB05B4FA3
3 changed files with 108 additions and 3 deletions

View file

@ -93,6 +93,7 @@ enum trace_flags {
};
extern int __must_check compel_stop_on_syscall(int tasks, int sys_nr, int sys_nr_compat);
extern int compel_stop_tasks_on_syscall(int tasks, pid_t *pids, const int sys_nr, const int sys_nr_compat);
extern int __must_check compel_stop_pie(pid_t pid, void *addr, bool no_bp);