Commit graph

549 commits

Author SHA1 Message Date
Andrei Vagin
e076c11e22 ci: fix codespell errors
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-11-27 16:47:16 -08:00
Pavel Tikhomirov
5d6c8bc584 clang-format: disable column limit constraint
The "ColumnLimit: 120" is not only allowing lines to be longer than 80
characters but it also forces line wrapping at 120 characters. If total
expression length is more than 120 characters, clang-format will try to
wrap it as close to 120 as it can, it would not even allow to wrap at 80
characters if we really want it. But as we all know 80 characters is
Linux kernel coding style default and as far as our coding style is
based on it it is really strange to prohibit wrapping lines at 80
characters...

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
ba168ab78c ci: enable build with amdgpu plugin
This patch adds the `libdrm-dev` package to the list of CRIU
dependencies installed in CI to build CRIU with amdgpu plugin.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Andrei Vagin
5e544dc449 ci: stop testing ubuntu overlayfs
They break it with each kernel rebase. More details are here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257

Last time, it was fixed a few month ago and it has been broken again in
5.15.0-1046-azure.

Let's bind-mount the CRIU directory into a test container to make it
independent of a container file system.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
32241b00de vagrant: run tests with fedora 38
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
862cb5c1cb vagrant: update to version 2.3.7
This patch also updated the download URL format

from
    https://releases.hashicorp.com/vagrant/2.3.7/vagrant_2.3.7_x86_64.deb

to
    https://releases.hashicorp.com/vagrant/2.3.7/vagrant_2.3.7-1_amd64.deb

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Pavel Tikhomirov
d2a0d1fa64 lint: don't fail workflow on indent fail
There are multiple cases where good human readable code block is
converted to an unreadable mess by clang-format, so we don't want to
rely on clang-format completely. Also there is no way, as far as I can
see, to make clang-format only fix what we want it to fix without
breaking something.

So let's just display hints inline where clang-format is unhappy. When
reviewer sees such a warning it's a good sign that something is broken
in coding-style around this warning.

We add special script which parses diff generated by indent and
generates warning for each hunk.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
1800018bc1 test/other: add test for action-script
This commit is introducing a test for the action-script functionality
of CRIU to verify that pre-dump, post-dump, pre-restore, pre-resume,
post-restore, post-resume hooks are executed during dump/restore.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Andrei Vagin
86ad52bc2d ci/loongarch64: compile tests before running zdtm.py
Otherwise tests fail by timeout.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
d3b955e578 ci/docker: install all required packages
This change fixes the issue:
```
The following packages have unmet dependencies:
 docker-ce : Depends: containerd.io (>= 1.6.4)
E: Unable to correct problems, you have held broken packages.
```

Signed-off-by: Andrei Vagin <avagin@google.com>
2023-10-22 13:29:25 -07:00
Andrei Vagin
9477354def scripts/apt: don't hide apt output
It is required to investigate issues.

Signed-off-by: Andrei Vagin <avagin@google.com>
2023-10-22 13:29:25 -07:00
znley
e25a243b28 ci: add workflow for loongarch64
Signed-off-by: znley <shanjiantao@loongson.cn>
2023-10-22 13:29:25 -07:00
znley
c9df09eeab compel: add loongarch64 support
Signed-off-by: znley <shanjiantao@loongson.cn>
2023-10-22 13:29:25 -07:00
Prajwal S N
8a24d4872e ci: add workflow to ensure self-contained commits
Signed-off-by: Prajwal S N <prajwalnadig21@gmail.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
ede018176c make: remove checks for python 2 binary
This commit removes the checks for the Python 2 binary in the makefile
and makes sure that ZDTM tests always use python3. Since support for
Python 2 has been dropped, these checks are no longer needed.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
642fd99bfd remove python-future dependency
This commit removes the dependency on the __future__ module, which was
used to enable Python 3 features in Python 2 code. With support for
Python 2 being dropped, it is no longer necessary to maintain backward
compatibility.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
362d8fa5c2 ci: disable CentOS 7 test in Cirrus CI
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
82a0db036e docker/podman: test c/r with action-script
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-10-22 13:29:25 -07:00
Dhanuka Warusadura
fc08fa9077 criu-ns: Update shebang line to python
CentOS 7 CI environment uses Python 2. To execute criu-ns
script in CentOS 7 changing the current shebang line to
python is required.

This reverse the changes made in a15a63fce0

Signed-off-by: Dhanuka Warusadura <csx@tuta.io>
2023-10-22 13:29:25 -07:00
Dhanuka Warusadura
9c9e8ea3f2 criu-ns: Add tests for criu-ns script
These changes add test implementations for criu-ns script.

Fixes: #1909

Signed-off-by: Dhanuka Warusadura <csx@tuta.io>
2023-10-22 13:29:25 -07:00
Dhanuka Warusadura
e4b6fb2d1f criu-ns: Add support for older Python version in CI
These changes remove and update the changes introduced in
7177938e60 in favor of the
Python version in CI.

os.waitstatus_to_exitcode() function appeared in Python 3.9

Related to: #1909

Signed-off-by: Dhanuka Warusadura <csx@tuta.io>
2023-10-22 13:29:25 -07:00
Dhanuka Warusadura
733f165512 criu-ns: Add --criu-binary argument to run_criu()
--criu-binary argument provides a way to supply the CRIU binary
location to run_criu().

Related to: #1909

Signed-off-by: Dhanuka Warusadura <csx@tuta.io>
2023-10-22 13:29:25 -07:00
Pavel Tikhomirov
cc607f8103 criu-ns: make --pidfile option show pid in caller pidns
Using the fact that we know criu_pid and criu is a parent of restored
process we can create pidfile with pid on caller pidns level.

We need to move mount namespace creation to child so that criu-ns can
see caller pidns proc.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-10-22 13:29:25 -07:00
Adrian Reber
50e17a1cf3 scripts: make newer versions of shellcheck happy
Signed-off-by: Adrian Reber <areber@redhat.com>
2023-10-22 13:29:25 -07:00
Radostin Stoyanov
7f0f07599a crit: fix compatibility with Python 3.12
Python 3.12 includes a few breaking changes, such as the removal of the
distutils module [1] and the deprecation of `setup.py install` in
favour of pip install [2]. This patch updates the installation script
for crit to reflect these changes by replacing the use of
`setup.py install` with `pip install` and `distutils` with
`setuptools`. In addition, a minimal pyproject.toml file has
been added as it is required by the new version of pip [3].

It is worth noting that with this change we are switching from the egg
packaging format to wheel [4] and add pip as a build dependency.

[1] https://www.python.org/downloads/release/python-3120a2/
[2] https://github.com/pypa/setuptools/pull/2824
[3] https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/
[4] https://packaging.python.org/en/latest/discussions/wheel-vs-egg/

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
65407616e0 ci/archlinux: initialize machine ID
When installing packages within Archlinux container, pacman fails with
the following errors:

(3/7) Creating temporary files...
/usr/lib/tmpfiles.d/journal-nocow.conf:26: Failed to replace specifiers in '/var/log/journal/%m': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:23: Failed to replace specifiers in '/run/log/journal/%m': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:25: Failed to replace specifiers in '/run/log/journal/%m': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:26: Failed to replace specifiers in '/run/log/journal/%m/*.journal*': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:29: Failed to replace specifiers in '/var/log/journal/%m': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:30: Failed to replace specifiers in '/var/log/journal/%m/system.journal': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:32: Failed to replace specifiers in '/var/log/journal/%m': No such file or directory
/usr/lib/tmpfiles.d/systemd.conf:33: Failed to replace specifiers in '/var/log/journal/%m/system.journal': No such file or directory

To solve this problem we need to initialize the machine ID.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
85b5c1e451 ci/podman-test: drop crun installation script
In a previous commit, we set the default runtime to runc and
"manage-cgroups" to ignore. We remove the installation script
for crun as it is not used with this test.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
91ff24b47e ci: disable c/r of cgroups with podman
This patch disables the checkpoint/restore of cgroups for
the tests using Podman as a temporary workaround for
https://github.com/checkpoint-restore/criu/issues/2091

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Younes Manton
d7da4a69af ci: Add maps00 test in unprivileged mode in user namespace
CAP_CHECKPOINT_RESTORE does not give access to /proc/$pid/map_files in
user namespaces. In order to test that CRIU in unprivileged mode can
dump and restore anonymous shared memory pages we will run the maps00
tests in a user namespace.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-04-15 21:17:21 -07:00
Pavel Tikhomirov
7280e96a79 clang-format: use IndentGotoLabels to get rid of goto label indentation
This is done to follow 'Linux kernel coding style', same change was
added to .clang-format in linux kernel source recently:
d7f6604341

We don't change it in current code base but let's follow it in all
future uses.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-04-15 21:17:21 -07:00
Adrian Reber
a918093ceb scripts/ci: use Fedora 37 for vagrant based tests
Signed-off-by: Adrian Reber <areber@redhat.com>
2023-04-15 21:17:21 -07:00
Pavel Tikhomirov
abfe0b5d24 clang-format: add for_each_bit macros to ForEachMacros
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-04-15 21:17:21 -07:00
Andrei Vagin
9686693aa6 test/javaTests: update org.testng:testng (Maven)
TestNG is vulnerable to Path Traversal

Fixes https://github.com/checkpoint-restore/criu/security/dependabot/1.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
d0c64b7b34 ci/alpine: remove symlink for /usr/bin/python
The python3 package in Alpine has recently been updated to install
symbolic link for /usr/bin/python.

https://git.alpinelinux.org/aports/commit/main/python3?id=d91da210b1614eb75517d59b7f348fee01699f35

This causes the following error in CI:

  Step 10/11 : RUN ln -s /usr/bin/python3 /usr/bin/python
   ---> Running in a5a94be9dc93
  ln: failed to create symbolic link '/usr/bin/python': File exists
  The command '/bin/sh -c ln -s /usr/bin/python3 /usr/bin/python' returned a non-zero code: 1

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Adrian Reber
153614cb1d ci: move cgroup unmounting to run-ci-tests.sh
A previous commit added a cgroup cpuset unmounting to
scripts/ci/Makefile. We are sometimes running in a container without the
necessary privileges to unmount certain cgroups.

This commit moves the cgroup unmounting to a place in run-ci-tests.sh
which already requires privileged access and does not break unprivileged
build-only CI runs.

Signed-off-by: Adrian Reber <areber@redhat.com>
2023-04-15 21:17:21 -07:00
Bui Quang Minh
2ebce92333 ci: Make cpuset move to cgroup-v2 hierarchy
As cgroupv2_00, cgroupv2_01 need cpuset in cgroup-v2 hierarchy to check CRIU
handle cgroup-v2 properly, umount cpuset in cgroup-v1 to make it move to
cgroup-v2.

Signed-off-by: Bui Quang Minh <minhquangbui99@gmail.com>
2023-04-15 21:17:21 -07:00
Adrian Reber
1cba559da4 non-root: add non-root test case to cirrus runs
Run env00 and pthread00 test as non-root as initial proof of concept.

Signed-off-by: Adrian Reber <areber@redhat.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
0a872ccf16 scripts/protobuf-gen.sh: fix (not ignore) shellcheck warnings
This basically replaces

	for x in $(sed ...); do

with

	sed ... | while IFS= read -r x; do

The only caveat is, sed program was amended to remove empty lines
(there was one right above the PB_AUTOGEN_STOP).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
75b859f23f scripts/ci: rm shellcheck disable annotations
Those are no longer needed with shellcheck 0.8.0.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
aeb6961f3d scripts/ci/run-ci-tests: use bash arrays
This is a preferred way of fixing SC2086 shellcheck warning.

Note that since ZDTM_OPTS is passed as a string (via make or docker),
we are converting it to an array using read -a.

Remove all "shellcheck disable=SC2086" annotations.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
b1fb9f2f0b Fix, not ignore, shellcheck SC1091 warnings
This is easy to fix (but we have to specify -x).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
9d2948b239 scripts/ci/asan.sh: fix, not ignore, shellcheck warning
We can use globstar bash feature instead of find in this case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Kir Kolyshkin
968eec0d59 scripts/ci/apt-install: fix (not ignore) shellcheck warning
It is ok to quote $@, as it expands to "$1" "$2" ...

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-04-15 21:17:21 -07:00
Pavel Tikhomirov
9e91e62a7c criu-ns: capture controlling tty
When we are restoring in new pidns we specifically do setsid() from
criu-ns init so that sids of restored tasks are non-zero in this pidns
and on next dump CRIU would not have problems with zero sids, see [1].

But after this CRIU tries to inherit and setup a tty for the restored
process, and it fails to set it's process group via TIOCSPGRP to be a
foreground group for it's tty, because tty already is a controlling tty
for other session (which we had before setsid).

So to make it restore we need to reset tty to be a controlling tty of
criu-ns init via TIOCSCTTY before calling criu.

Else when restoring first time via criu-ns (from criu-ns dump) we get:

Error (criu/tty.c:689): tty: Failed to set group 40816 on 0: Inappropriate ioctl for device

https://github.com/checkpoint-restore/criu/issues/232 [1]

v2: add why and what comment in code, set controlling tty only for
--shell-job and fail if stdin is not a tty.

Fixes: #1893
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2023-04-15 21:17:21 -07:00
Younes Manton
7bc24688d6 ci: Clean up and improve Java testing
This patch changes top-level OpenJ9 filename and data references to Java
to make them generic and launches tests against both HotSpot and OpenJ9
JVMs.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-04-15 21:17:21 -07:00
Younes Manton
0178f2f990 ci: Add Dockerfile for openj9 on Ubuntu
Semeru builds (which use OpenJ9 instead of HotSpot) are the successors
of AdoptOpenJDK's OpenJ9 builds.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-04-15 21:17:21 -07:00
Younes Manton
39b3de60b6 ci: Rename openj9 Dockerfiles to hotspot
We used to pull AdoptOpenJDK's OpenJ9 builds but switched to
Eclipse Temurin, which uses the HotSpot VM instead of OpenJ9.
Rename the corresponding Dockerfiles to hotspot.

Signed-off-by: Younes Manton <ymanton@ca.ibm.com>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
2642b657da docker-test: handle race condition error
There is a race condition in docker/containerd that causes docker to
occasionally fail when starting a container from a checkpoint immediately
after the checkpoint has been created.

This problem is unrelated to criu and has been reported in
https://github.com/moby/moby/issues/42900

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
f9bc0a750a docker-test: use containerd installed from package
In commits [1, 2] the version of containerd installed by default in the
GitHub CI virtual environment was replaced with the latest release from
GitHub as a workaround to a bug in containerd.  This bug has been fixed
sometime ago and the current default version of containerd (1.6.6) does
not require this workaround. However, with the latest release, the
containerd binaries uploaded on GitHub have been built for Ubuntu 22.04
[3]. Our tests are still running on Ubuntu 20.04 and this results in the
following error:

/usr/bin/containerd: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/bin/containerd)
/usr/bin/containerd: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /usr/bin/containerd)

[1] 046cad8
[2] 81a68ad
[3] 6b2dc9a37

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00
Radostin Stoyanov
750acec25f Revert "ci: Switch to non overlaysfs tests"
This reverts commit 8bb05e3bf3.

The following bug has been fixed:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1967924

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2023-04-15 21:17:21 -07:00