Commit graph

11120 commits

Author SHA1 Message Date
Andrei Vagin
60ca30145b ci: run rootless tests in the main job
Signed-off-by: Andrei Vagin <avagin@google.com>
2023-02-08 23:30:40 -08:00
Pavel Tikhomirov
06516ca0ae clang-format: use IndentGotoLabels to get rid of goto label indentation
This is done to follow 'Linux kernel coding style', same change was
added to .clang-format in linux kernel source recently:
d7f6604341

We don't change it in current code base but let's follow it in all
future uses.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-02-07 16:58:28 -08:00
Pavel Tikhomirov
4837343200 namespaces: cleanup switch_mnt_ns and restore_mnt_ns
Simplify code a bit: make exit codes of those functions more
transparent, rename ret to exit_code.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-02-07 16:57:46 -08:00
Pavel Tikhomirov
6cbeaeb899 mount: simplify code around mount_cr_time_mount
Checking errno in outer function is really strange, also saving errno of
mount syscall after calling pr_perror is completely wrong. So let's try
to simplify things.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-02-07 16:57:46 -08:00
Michał Mirosław
7285f0f3c5 remap: refactor goto jumping to a while loop
Make the code a bit more readable by uncovering a while loop from
a if() goto sequence.

Signed-off-by: Michał Mirosław <emmir@google.com>
2023-02-07 16:51:08 -08:00
Michał Mirosław
0192e90fbc remap: Fix typo
Fixes: 237bd26982 ("remap: Rename global lock", 2017-05-18)
Signed-off-by: Michał Mirosław <emmir@google.com>
2023-02-07 16:51:08 -08:00
Pavel Tikhomirov
c3a74ffc3b ci/lint: make push action have at least too commits depth
We see that when lint is called for push action git has only one last
commit which makes make indent with git-clang-format fail to operate.

Fix it by increasing fetch depth to one more commit.

Fixes: #2066
Fixes: d6db3333a ("clang-format: rework make indent to check specific commits")
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-02-07 16:50:24 -08:00
Younes Manton
317170ce09 criu(8): Add info about unprivileged mode limitations
Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-01-30 18:52:15 -08:00
Younes Manton
d088a711b0 proc_parse: Don't bail out on is_memfd() VMAs
Co-authored-by: Ivanq <imachug@yandex.ru>
Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-01-30 18:52:08 -08:00
Younes Manton
d46c74a061 shmem: Fall back to /proc/$pid/mem if no map_files
If trying to open /proc/$pid/map_files/x-x for a given VMA fails with
EPERM (can happen in unprivileged mode when running in a non-init user
ns), fall back to reading the content from /proc/$pid/mem.

Co-authored-by: Ivanq <imachug@yandex.ru>
Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-01-30 18:47:40 -08:00
Younes Manton
b46d8429cc proc_parse: Set VMA_AREA_REGULAR where needed
This patch sets VMA_AREA_REGULAR on hugetlb and anon shmem VMAs since
they can be handled the same way as other kinds of regular memory.

Co-authored-by: Ivanq <imachug@yandex.ru>
Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-01-30 18:47:40 -08:00
Pavel Tikhomirov
2640489b7a string: define wrapers __setproctitle and __setproctitle_init to hide bsd headers
We see that libbsd redefines __has_include to be always true, which
breaks such checks for rseq. The idea behind this patch is to put all
uses of libbsd functions to separate c files and only export wrapper
functions for them.

Using __setproctitle and __setproctitle_init everywhere in existing
code:

git grep --files-with-matches "setproctitle" | xargs sed -i 's/setproctitle/__setproctitle/g'
git grep --files-with-matches "setproctitle_init" | xargs sed -i 's/setproctitle_init/__setproctitle_init/g'

Fixes: #2036
Suggested-by: Andrei Vagin <avagin@google.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-30 18:30:00 -08:00
Pavel Tikhomirov
0927e87be4 string: use our own __strlcpy and __strlcat to remove bsd headers
We see that libbsd redefines __has_include to be always true, which
breaks such checks for rseq. The idea behind this patch is remove the
use of libbsd functions and always export our replacement functions.

Using __strlcat and __strlcpy everywhere in existing code:
git grep --files-with-matches "strlcat" | xargs sed -i 's/strlcat/__strlcat/g'
git grep --files-with-matches "strlcpy" | xargs sed -i 's/strlcpy/__strlcpy/g'

Fixes: #2036
Suggested-by: Andrei Vagin <avagin@google.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-30 18:30:00 -08:00
Pavel Tikhomirov
ac78cbe770 log: remove all uses of %m specifier in pr_* functions
As our pr_* functions are complex and can call different system calls
inside before actual printing (e.g. gettimeofday for timestamps) actual
errno at the time of printing may be changed.

Let's just use %s + strerror(errno) instead of %m with pr_* functions to
be explicit that errno to string transformation happens before calling
anything else.

Note: tcp_repair_off is called from pie with no pr_perror defined due to
CR_NOGLIBC set and if I use errno variable there I get "Unexpected
undefined symbol: `__errno_location'. External symbol in PIE?", so it
seems there is no way to print errno there, so let's just skip it.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-25 09:37:36 -08:00
Pavel Tikhomirov
f684c74fca lint: prohibit to use %m specifier in pr_* functions
As our pr_* functions are complex and can call different system calls
inside before actual printing (e.g. gettimeofday for timestamps) actual
errno at the time of printing may be changed.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-25 09:37:36 -08:00
Adrian Reber
b7e5910691 ci: switch from lgtm to codeql
Signed-off-by: Adrian Reber <areber@redhat.com>
2023-01-24 11:30:11 +00:00
Michal Clapinski
e3d62f6877 restorer: add logging on prctl PR_SET_MM_MAP failure
This kernel feature contained some bugs initially. Those logs are useful in identifing what the
underlaying issue is and which kernel patch to backport.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
2023-01-23 12:03:32 -08:00
Pavel Tikhomirov
54888c487f zdtm: add mntns_compare check to mount_complex_sharing
This way we can check that mount tree topology (including sharing
groups) is the same before and after c/r.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-20 09:22:59 -08:00
Pavel Tikhomirov
ad349597b5 zdtm: add mountinfo topology compare to test lib
Now we can compare mount tree and sharing group tree topology before and
after c/r with mntns_compare() helper.

Algorithm here is:

1) build mount tree based on mnt_id and parent_mnt_id from mountinfo
2) sort mount tree children based on path comparison
3) at the same time set topology_id for mounts by DFS order and order
   mounts in list accordingly
4) build shared groups tree based on sharing_id and master_id
5) at the same time set topology_id for sharings as smallest topology_id
   of its mounts, also sharings are put in their list in order of
   their topology_id
6) walk sorted mounts lists for both namespaces simultaneously each
   pair of moutns should have matching ids and parent ids
7) walk sorted sharings lists for both namespaces simultaneously each
   pair of sharings should have matching ids and parent ids

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-20 09:22:59 -08:00
Pavel Tikhomirov
e753695fd2 zdtm: add mountinfo parsing to test lib
For mount testing it is nice to be able to parse mountinfo from zdtm
test itself, for instance to be able to compare mountinfo topology
before and after c/r, or for anything else. So let's add a helper
mntns_parse_mountinfo() which parses current mount namespace mountinfo.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-20 09:22:59 -08:00
Pavel Tikhomirov
796cf2d6ac zdtm/lib: copy xmalloc.h
Need to use xzalloc in zdtm lib.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-20 09:22:59 -08:00
Pavel Tikhomirov
4dc8fbab58 zdtm/lib: copy list.h
Need it to use linux lists in zdtm.

Also copy container_of from comiler.h to zdtmtst.h like we already do
for e.g. __stack_aligned__ macro.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-20 09:22:59 -08:00
Pavel Tikhomirov
d6db3333aa clang-format: rework make indent to check specific commits
Previousely "make indent" checked all files in criu source directory for
codding style flaws. We have several problems with it:

- clang-format default format sometimes changes in new versions of the
package and we need to reformat all our code base each time it happens
- on different systems we may have different versions of clang-format
and on latest criu-dev "make indent" may be still unhappy on your system
- when we want to update clang-format rules ourselves we need to update
all our code base each time
- sometimes clang-format rules are not fitting all our cases, (e.g.: an
option IndentGotoLabels works nice for simple C code, but is a no go for
assembler and C macros) and putting "clang-format off" everywhere is a
mess
- sometimes we intentionally want to break clang-format rules (e.g.:
we want to put function arguments on a new line separating them
"logically" not "mechanically" following 120-char rule like clang-format
does).

This adds a BASE option for "make indent" where all commits in range
BASE..HEAD would be checked with git-clang-format for codding style
flaws. For instance when developing on top of criu-dev, one can use
"make BASE=origin/criu-dev indent" to check all their commits for
compliance with the clang-format rules. Default base is HEAD~1 to make
last commit checked when "make indent" is called. The closest thing to
the old behaviour would then be "make indent BASE=init", note that only
commited files would be checked.

Extra options to git-clang-format may be passed through OPTS variable.

Also reuse "make indent" in github lint workflow.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-17 14:11:05 -08:00
Adrian Reber
a8b06d03a0 scripts/ci: use Fedora 37 for vagrant based tests
Signed-off-by: Adrian Reber <areber@redhat.com>
2023-01-17 13:56:47 -08:00
Pavel Tikhomirov
e209d6a27c tty: fix codding-style around for_each_bit call
Wraping "{" to next line after for-each macros is wrong.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-17 13:56:24 -08:00
Pavel Tikhomirov
a30b59d70b clang-format: add for_each_bit macros to ForEachMacros
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-17 13:56:24 -08:00
Pavel Tikhomirov
4109cfb206 autofs: fix a frankenstein auto-created by clang-format
Fixes: 93dd984ca ("Run 'make indent' on all C files")

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-12 11:28:01 -08:00
Cyrill Gorcunov
f7bcd1c8c3 log: Write more details in write_pidfile
Signed-off-by: Cyrill Gorcunov <gorcunov@virtuozzo.com>
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-11 09:40:31 -08:00
Michal Clapinski
a55e7289e2 asm: fix for_each_bit macro
find_next_bit operates on a bit instead of byte positions/sizes.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
2023-01-10 08:36:55 -08:00
Pavel Tikhomirov
f41bb33542 crtools: fix help message alignment for --network-lock
Fixes: 2e30db5c3 ("criu: add --network-lock option to allow nftables alternative")

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-01-06 09:41:37 -08:00
Radostin Stoyanov
b7e3e3f94a zdtm: fix 'zdtm.py list' command
The command ./zdtm.py list currently fails with

    if opts['rootless']:
       ~~~~^^^^^^^^^^^^
    KeyError: 'rootless'

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-01-06 15:47:08 +01:00
Andrei Vagin
a987223208 cgroups: ignore EOPNOTSUPP on setting memory.kmem.limit_in_byte
memory.kmem.limit_in_bytes has been deprecated. Look at e7c4184164f7
("memcg, kmem: further deprecate kmem.limit_in_bytes") for more details.

Signed-off-by: Andrei Vagin <avagin@google.com>
2023-01-02 16:26:06 -08:00
Andrei Vagin
008c2b9c7f test/javaTests: update org.testng:testng (Maven)
TestNG is vulnerable to Path Traversal

Fixes https://github.com/checkpoint-restore/criu/security/dependabot/1.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2022-12-13 14:25:07 -08:00
Andrei Vagin
6e11e7f081 sockets: tiny style fix
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2022-12-12 09:32:58 -08:00
Younes Manton
318ff08619 non-root: Don't dump socket option SO_MARK if 0
Restoring SO_MARK requires root or CAP_NET_ADMIN. If the value
is 0 we will avoid dumping it so that we don't need to do a
privileged call on restore.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2022-12-12 09:17:10 -08:00
Younes Manton
b3c7286175 non-root: Rework socket bufs for unprivileged mode
SO_SNDBUFFORCE/SO_RCVBUFFORCE require root or CAP_NET_ADMIN.
We can use SO_SNDBUF/SO_RCVBUF in some cases and avoid
needing elevated privileges.

This patch renames sk_setbufs() to sk_setbufs_ns() and
makes sk_setbufs() a general helper that sets socket
send and receive buffer sizes. The helper tries to use
SO_SNDBUFFORCE/SO_RCVBUFFORCE first and falls back to
SO_SNDBUF/SO_RCVBUF if we're in unprivileged mode.

The existing sk_setbufs_ns() which takes a pid parameter
and is intended to be called via userns_call() is rewritten
to call sk_setbufs().

Existing code that sets buffer sizes via setsockopt() is
modified to call sk_setbufs() instead.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2022-12-12 09:17:10 -08:00
Shubham Verma
50db2be1a7 Fix typo in comment
Signed-off-by: Shubham Verma <shubhamv.sv@gmail.com>
2022-12-03 10:26:35 -08:00
Liang-Chun Chen
f4a91fc401 zdtm: add two tests for highly sparse ghost file
ghost_multi_hole00 and ghost_multi_hole01 are tests which create a ghost file
with a lot of holes, there are 4K data and 4K hole inside every 8K length.

The only difference between them is ghost-fiemap option, 01 is a
test for the fiemap dumping algorithm, and we want to test the
behavior of EXTENT_MAX_COUNT part, so the file size should be 8M, thus there
will be 1024 chunks in the ghost file.

In some file system, such as xfs, we somehow can not easily create highly sparse
file as in ext4 or btrfs, therefore we need `fallocate` to forcibly create holes.

Signed-off-by: Liang-Chun Chen <featherclc@gmail.com>
2022-11-30 09:28:02 -08:00
Liang-Chun Chen
7819a11e94 files-reg.c: fiemap algorithm for ghost file
In order to reduce the frequency of using system call, based on
https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/misc/create_inode.c#n519,
I created a new algorithm of dumping chunk via fiemap.(copy_file_to_chunks_fiemap)

Also, I added another BOOL_OPT for users to determine which algorithm they
want to use. Moreover, for those filesystem not supporting fiemap, criu
will fall back to the original algorithm(SEEK_HOLE/SEEK_DATA).

v2: don't call copy_chunk_from_file on outstanding extent; rearange
headers to workaround "redeclaration of ‘enum fsconfig_command’" problem

Signed-off-by: Liang-Chun Chen <featherclc@gmail.com>
2022-11-30 09:28:02 -08:00
Radostin Stoyanov
4f659d591f ci: fix make indent
This patch fixes applies the changes required by clang-format v15.0.5
for `make indent`.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2022-11-25 18:42:50 +01:00
Radostin Stoyanov
5b9f7a93fc ci/alpine: remove symlink for /usr/bin/python
The python3 package in Alpine has recently been updated to install
symbolic link for /usr/bin/python.

https://git.alpinelinux.org/aports/commit/main/python3?id=d91da210b1614eb75517d59b7f348fee01699f35

This causes the following error in CI:

  Step 10/11 : RUN ln -s /usr/bin/python3 /usr/bin/python
   ---> Running in a5a94be9dc93
  ln: failed to create symbolic link '/usr/bin/python': File exists
  The command '/bin/sh -c ln -s /usr/bin/python3 /usr/bin/python' returned a non-zero code: 1

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2022-11-24 08:47:31 +01:00
Radostin Stoyanov
b50d3d7b4e ci/lint: install ShellCheck with dnf
The way ShellCheck is installed was changed in commit c056f99
(ci/gha/lint: install a recent shellcheck) to use the latest version
v0.8.0 and remove some of the "shellcheck disable=..." annotations.
Since then, Fedora 37 has been released and the ShellCheck package
has been updated to v0.8.0.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2022-11-24 08:29:40 +01:00
Drew Wock
6f3b81d0da Fix warnings from -Wstrict-prototypes in clang 16.0.0
While building on a machine that has a HOL clang compiler,
I ran into warnings regarding the changed line.  It appears
this warning is on by default because of anticipated changes
to the C standard.

Signed-off-by: Drew Wock <ajwock@gmail.com>
2022-11-21 14:30:34 -08:00
Radostin Stoyanov
7fee7d263c amdgpu: define __nmk_dir if missing
This patch adds a missing definition for `__nmk_dir` in the Makefile
for the amdgpu plugin. This definition is required, for example, when
building the `test_topology_remap` target:

	make -C plugins/amdgpu/ test_topology_remap

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2022-11-21 00:15:50 -08:00
Mathias Gibbens
1c6517a88c Remove execute bit from source file
Signed-off-by: Mathias Gibbens <mathias@calenhad.com>
2022-11-18 10:18:13 +00:00
Bui Quang Minh
52435d72b8 cgroup: Remove redundant code that handles zombie tasks
Zombie tasks are dumped in dump_zombies() so it is redundant to handle them
in dump_one_task().

Deprecate cg_set in task_core_entry as this field must be per thread now.

Signed-off-by: Bui Quang Minh <minhquangbui99@gmail.com>
2022-11-11 17:35:34 -08:00
Bui Quang Minh
614fb7de02 kerndat: Mark memfd_create(MFD_HUGETLB) unavailable when ENOSYS is returned
Some users on Raspberry Pi report that the kerndat checking for
memfd_create(MFD_HUGETLB) support returns ENOSYS even when memfd_create
syscall is available. We currently treat this error as unexpected and
return error. This commit marks the memfd_create(MFD_HUGETLB) as
unavailable when ENOSYS is returned.

Signed-off-by: Bui Quang Minh <minhquangbui99@gmail.com>
2022-11-11 11:20:22 -08:00
Adrian Reber
979c84209b ci: move cgroup unmounting to run-ci-tests.sh
A previous commit added a cgroup cpuset unmounting to
scripts/ci/Makefile. We are sometimes running in a container without the
necessary privileges to unmount certain cgroups.

This commit moves the cgroup unmounting to a place in run-ci-tests.sh
which already requires privileged access and does not break unprivileged
build-only CI runs.

Signed-off-by: Adrian Reber <areber@redhat.com>
2022-11-08 11:24:01 +00:00
Adrian Reber
f47f5c0841 ci: Do not fail if latest epel repository definition is already installed
Signed-off-by: Adrian Reber <areber@redhat.com>
2022-11-08 11:23:19 +00:00
Bui Quang Minh
d3ed3e9087 ci: Make cpuset move to cgroup-v2 hierarchy
As cgroupv2_00, cgroupv2_01 need cpuset in cgroup-v2 hierarchy to check CRIU
handle cgroup-v2 properly, umount cpuset in cgroup-v1 to make it move to
cgroup-v2.

Signed-off-by: Bui Quang Minh <minhquangbui99@gmail.com>
2022-11-01 21:35:04 -07:00