Commit graph

11326 commits

Author SHA1 Message Date
Pavel Tikhomirov
5d6c8bc584 clang-format: disable column limit constraint
The "ColumnLimit: 120" is not only allowing lines to be longer than 80
characters but it also forces line wrapping at 120 characters. If total
expression length is more than 120 characters, clang-format will try to
wrap it as close to 120 as it can, it would not even allow to wrap at 80
characters if we really want it. But as we all know 80 characters is
Linux kernel coding style default and as far as our coding style is
based on it it is really strange to prohibit wrapping lines at 80
characters...

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
81a30c3206 zdtm/memfd04: check execveat on memfd that has memory mappings
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
e86e8dac0d memfd: don't reopen file descriptors for memory mappings
One memfd can be shared by a few restored files. Only of these files is
restored with a file created with memfd_open. Others are restored by reopening
memfd files via /proc/self/fd/.

It seems unnecessary for restoring memfd memory mappings. We can always use the
origin file.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
28e854d662 amdgpu: fix clang warnings
amdgpu_plugin.c:930:6: error: variable 'buffer' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
        if (ret) {
            ^~~
amdgpu_plugin.c:988:8: note: uninitialized use occurs here
        xfree(buffer);

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
ba168ab78c ci: enable build with amdgpu plugin
This patch adds the `libdrm-dev` package to the list of CRIU
dependencies installed in CI to build CRIU with amdgpu plugin.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Andrei Vagin
aa38a59899 amdgpu: print an error if the dup syscall fails
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
940a05c0ba amdgpu: don't leak fd on an error path in open_img_file
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
a68975c06d plugins: the UPDATE_VMA_MAP callback returns fd with the full control
It means CRIU has to close it when it is not needed.

It looks more logically correct and matches the behaviour of
the RESTORE_EXT_FILE callback.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Michal Clapinski
66f39adf12 criu: change the comment about magic numbers
Signed-off-by: Michal Clapinski <mclapinski@google.com>
2023-10-22 13:29:25 -07:00
Vladislav Khmelevsky
9b4e8292cd vma: Add !VVAR condition to vma_entry_can_be_lazy
Currently most of the times we don't have problems with VVAR segment and
lazy restore because when VDSO is parked there is an munmap call that
calls UFFDIO_UNREGISTER on the destination address.
But we don't want to enable userfaultfd for VDSO and VVAR at the first
place.

Signed-off-by: Vladislav Khmelevsky <och95@yandex.ru>
2023-10-22 13:29:25 -07:00
Vladislav Khmelevsky
28adebefb7 Return page size as unsigned long
Currently page_size() returns unsigned int value that is after "bitwise
not" is promoted to unsigned long value e.g. in uffd.c
handle_page_fault. Since the value is unsigned promotion is done with 0
MSB that results in lost of MSB pagefault address bits. So make
page_size to return  unsigned long to avoid such situation.

Signed-off-by: Vladislav Khmelevsky <och95@yandex.ru>
2023-10-22 13:29:25 -07:00
Andrei Vagin
00f8a56b6e zdtm: check userns once
All test logs are flooded with the "userns is supported" messages...

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
ba48ceb575 zdtm: socket_udp_shutdown: Make the test fail instead of timing out.
When -- after restore -- sockets can't communicate, the test times out
while waiting on recvfrom(). Since the communication is local, send()
works instantaneously - so mark sockets with SOCK_NONBLOCK and report
failure if the message is not received immediately.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
f6e820bedb zdtm: Treat ESRCH from kill() as success.
This fixes a failure to clean up after a failed test, where CRIU didn't start properly.

```
===================== Run zdtm/transition/socket-tcp in h ======================
Start test
./socket-tcp --pidfile=socket-tcp.pid --outfile=socket-tcp.out
Traceback (most recent call last):
  File ".../zdtm_py.py", line 1906, in do_run_test
    cr(cr_api, t, opts)
  File ".../zdtm_py.py", line 1584, in cr
    cr_api.dump("dump")
  File ".../zdtm_py.py", line 1386, in dump
    self.__dump_process = self.__criu_act(action,
  File ".../zdtm_py.py", line 1224, in __criu_act
    raise test_fail_exc("CRIU %s" % action)
test_fail_exc: CRIU dump

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<embedded module '_launcher'>", line 182, in run_filename_from_loader_as_main
  File "<embedded module '_launcher'>", line 34, in _run_code_in_main
  File ".../zdtm_py.py", line 2790, in <module>
    fork_zdtm()
  File ".../zdtm_py.py", line 2782, in fork_zdtm
    do_run_test(tinfo[0], tinfo[1], tinfo[2], tinfo[3])
  File ".../zdtm_py.py", line 1922, in do_run_test
    t.kill()
  File ".../zdtm_py.py", line 509, in kill
    os.kill(int(self.__pid), sig)
ProcessLookupError: [Errno 3] No such process
```

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
c29c5a1df0 zdtm: cgroup04: Improve skip check's robustness.
cgroup04 test needs full control over mem and devices cgroup hierarchies.
Make the test's .checkskip script better at detecting if the cgroups are
available for use.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
131e464a05 zdtm: cgroup04: Improve error messages.
Make the errno values reported by cgroup04 always correct and showing
relevant parameters.
Constify constant strings, while at it.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
785d97a619 zdtm: If ignoring kernel taint, also ignore taint changes.
At least in Google's VM environment, the kernel taints are unrelated to CRIU
runs.  Don't fail tests if taints change, if kernel taints are ignored.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
5e544dc449 ci: stop testing ubuntu overlayfs
They break it with each kernel rebase. More details are here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257

Last time, it was fixed a few month ago and it has been broken again in
5.15.0-1046-azure.

Let's bind-mount the CRIU directory into a test container to make it
independent of a container file system.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
7a2910f897 py/cli: add --version option
This patch implements the '--version' for the crit tool.

$ crit --version
3.17

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
61d9cf6f90 crit/setup.py: use __version__ from pycriu
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
8adefc90d2 lib/pycriu: generate version.py
The version of CRIU is specified in the Makefile.versions file.
This patch generates '__varion__' value for the pycriu module.
This value can be used by crit to implement `--version`.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Younes Manton
59fcfa80d8 compel: Add support for ppc64le scv syscalls
Power ISA 3.0 added a new syscall instruction. Kernel 5.9 added
corresponding support.

Add CRIU support to recognize the new instruction and kernel ABI changes
to properly dump and restore threads executing in syscalls. Without this
change threads executing in syscalls using the scv instruction will not
be restored to re-execute the syscall, they will be restored to execute
the following instruction and will return unexpected error codes
(ERESTARTSYS, etc) to user code.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-10-22 13:29:25 -07:00
David Francis
d06c9b5cda criu/plugin: Add environment variable to cap size of buffers.
The amdgpu plugin would create a memory buffer at the size
of the largest VRAM bo (buffer object). On some systems, VRAM
size exceeds RAM size, so the largest bo might be larger than
the available memory.

Add an environment variable KFD_MAX_BUFFER_SIZE, which caps the
size of this buffer. By default, it is set to 0, and has no
effect. When active, any bo larger than its value will be
saved to/restored from file in multiple passes.

Signed-off-by: David Francis <David.Francis@amd.com>
2023-10-22 13:29:25 -07:00
Michal Clapinski
8caf460b9c zdtm: test MEMBARRIER_CMD_GLOBAL_EXPEDITED migration
Check membarrier registration both ways:
1. By issuing membarrier commands and checking if they succeed.
2. By issuing MEMBARRIER_CMD_GET_REGISTRATIONS.

The first way is needed for older kernels. The second way is needed to test
MEMBARRIER_CMD_GLOBAL_EXPEDITED.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
2023-10-22 13:29:25 -07:00
Michal Clapinski
2ba8727822 dump: use MEMBARRIER_CMD_GET_REGISTRATIONS when available
MEMBARRIER_CMD_GET_REGISTRATIONS can tell us whether or not the process used
MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED unlike the old probing method.

Falls back to the old method when MEMBARRIER_CMD_GET_REGISTRATIONS is
unavailable.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
32241b00de vagrant: run tests with fedora 38
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
862cb5c1cb vagrant: update to version 2.3.7
This patch also updated the download URL format

from
    https://releases.hashicorp.com/vagrant/2.3.7/vagrant_2.3.7_x86_64.deb

to
    https://releases.hashicorp.com/vagrant/2.3.7/vagrant_2.3.7-1_amd64.deb

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Pavel Tikhomirov
d2a0d1fa64 lint: don't fail workflow on indent fail
There are multiple cases where good human readable code block is
converted to an unreadable mess by clang-format, so we don't want to
rely on clang-format completely. Also there is no way, as far as I can
see, to make clang-format only fix what we want it to fix without
breaking something.

So let's just display hints inline where clang-format is unhappy. When
reviewer sees such a warning it's a good sign that something is broken
in coding-style around this warning.

We add special script which parses diff generated by indent and
generates warning for each hunk.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Pavel Tikhomirov
08f286ed96 CONTRIBUTING.md: improve coding-style related sections
This is highlight that code readability is the real goal of all the
coding-style rules. We should not do coding-style just for coding-style,
e.g. when clang-format suggests crazy formating we should not follow it
if we feel it is bad.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Pavel Tikhomirov
38bf7f42e5 CONTRIBUTING.md: don't mention ctags
Ctags is mentioned in the beginning of the "Edit the source code" which
is really confusing: Do you need ctags to edit CRIU code? - No. It is
just one helpful tool to browse the code, and we do not want to enforce
it. So, what is it doing in contribution guide? People who really need
it should be able to find it in Makefile or just write oneliner of their
own to collect tags...

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
b7a8bb0889 zdtm: test execveat(memfd)
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
c9b2633ca3 memfd: return original memfd fd for execveat()
If there is only a single RW opened fd for a memfd, it can be used
to pass it to execveat() with AT_EMPTY_PATH to have its contents
executed.  This currently works only for the original fd from
memfd_create().  For now we ignore processes that reopen the memfd's
rw and expect a particular executability trait of it.  (Note: for
security purposes recent kernels have SEAL_EXEC to make memfds
non-executable.)

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
4805654370 kerndat: check_pagemap: close(fd) on error path
Plug a fd leak when returning error from check_pagemap().
(Cosmetic, as the process will exit soon anyway.)

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
d5284313f5 kerndat: Make errors from clone3() check more precise.
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
badf8060c6 proc_parse: Log smaps entry while dumping VMA.
Help debugging problems with restoring custom VMAs.

From: Michał Cłapiński <mclapinski@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
1800018bc1 test/other: add test for action-script
This commit is introducing a test for the action-script functionality
of CRIU to verify that pre-dump, post-dump, pre-restore, pre-resume,
post-restore, post-resume hooks are executed during dump/restore.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Michał Mirosław
ea05b06ac2 proc_parse: remove trivial goto from vma_get_mapfile_user()
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
f7d7dc9c08 compel/infect: include the relevant pid in "no-breakpoints restore" debug message
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
b56a9cef32 kerndat: Make pagemap check more robust against swapped out pages.
Fix test of whether the kernel exposes page frame numbers to cope with the
possibility that the top of the stack is swapped out, which was happening
in about one 1 out of 3 million runs.  This lead to a later failure when
trying to read the PFN of the zero page, after which criu would exit with
no error message.

Original-From: Ambrose Feinstein <ambrose@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
86ad52bc2d ci/loongarch64: compile tests before running zdtm.py
Otherwise tests fail by timeout.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
0085f992cb memfd: don't set fd attributes not needed for vma mapping
There is only one user of memfd_open() outside of memfd.c: open_filemap().
It is restoring a file-backed mapping and doesn't need nor expect to
update F_SETOWN nor the fd's position.  Check the inherited_fd() handling
in the callers to simplify the code.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
8c17535f3f loongarch64: fix syscall_64.tbl
The 288d6a61e2 change broke all the syscall numbers.

Reported-by: Michał Mirosław <emmir@google.com>
Fixes: (288d6a61e2 "loongarch64: reformat syscall_64.tbl for 8-wide tabs")
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
6ea60d6ef7 github: auto-remove changes requested and awaiting reply labels
Labels are removed when new comments are posted.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
620e8c0a61 Put a cap on the size of single preadv in restore operation.
While each preadv() is followed by a fallocate() that removes the data
range from image files on tmpfs, temporarily (between preadv() and
fallocate()) the same data is in two places; this increases the memory
overhead of restore operation by the size of a single preadv.
Uncapped preadv() would read up to 2 GiB of data, thus we limit that to
a smaller block size (128 MiB).

Based-on-work-by: Paweł Stradomski <pstradomski@google.com>
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
2547ac8ac1 zdtm: membarrier: test migration of membarrier() registration
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Michał Mirosław
e07155e194 dump+restore: Implement membarrier() registration c/r.
Note: Silently drops MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED as it's
not currently detectable. This is still better than silently dropping
all membarrier() registrations.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
5b790aa181 loongarch64: reformat syscall_64.tbl for 8-wide tabs
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
d1096e3b31 lib/py: add VMA_AREA_MEMFD constant
The VMA_AREA_MEMFD constant was introduced with commit

29a1a88bce
memfd: add memory mapping support

This patch extends the status map used in CRIT and coredump with the
value of this constant to recognize it.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Andrei Vagin
d3b955e578 ci/docker: install all required packages
This change fixes the issue:
```
The following packages have unmet dependencies:
 docker-ce : Depends: containerd.io (>= 1.6.4)
E: Unable to correct problems, you have held broken packages.
```

Signed-off-by: Andrei Vagin <avagin@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
9477354def scripts/apt: don't hide apt output
It is required to investigate issues.

Signed-off-by: Andrei Vagin <avagin@google.com>
2023-10-22 13:29:25 -07:00