Currently, adding a package which is required either for development or testing
requires it to be added in multiple places due to many duplicated Dockerfiles
and installation scripts. This makes it difficult to ensure that all scripts
are updated appropriately and can lead to some places being missed.
This patch consolidates the list of dependencies and adds installation
scripts for each package-manager used in our CI (apk, apt, dnf, pacman).
This change also replaces the `debian/dev-packages.lst` as this subfolder
conflicts with the Ubuntu/Debian packing scripts used for CRIU:
https://github.com/rst0git/criu-deb-packages
This patch also removes the CentOS 8 build scripts as it is EOL
and the container registry is no longer available.
Signed-off-by: Shashank Balaji <shashank.mahadasyam@sony.com>
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
The CI tests with CentOS 7 have been disabled and removed [1,2].
This patch removes the obsolete Makefile targets for these tests.
[1] 24bc083653
[2] f8466ca798
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
We added cross-compile tests with testing debian release to be able to
replicate the error reported in #1653, however, installing build
dependencies in this release currently fails with the following error:
libc6-dev:armhf : Breaks: libc6-dev-armhf-cross (< 2.33~) but 2.32-1cross4 is to be installed
This is not something we can fix, therefore using the debian unstable
release (instead of testing) could be more reliable option for our CI.
This would still replicate the problem reported in #1653.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Debian testing has newer compiler version and running
cross compilation tests would allow us to catch any compilation
errors early.
Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
The current debian stable release is Bullseye, not Buster. However, we
can use the 'stable' release instead. This would allow the CI to
automatically pick up updates in the future.
Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
Fedora Rawhide updated to a glibc using clone3(). clone3() is, however,
not yet part of the seccomp filter. Unfortunately 'docker build' does
not allow dropping seccomp but luckily 'podman build' does.
This switches the Fedora Rawhide test to use Podman. Podman is part of
GitHub Actions and no additional packages need to be installed.
Signed-off-by: Adrian Reber <areber@redhat.com>
The cross compile container definitions for each architecture were
almost the same files except for the architecture.
This moves the architecture to variables so that all cross compile
setups can use the same container definition.
Signed-off-by: Adrian Reber <areber@redhat.com>
ccache was set up in Travis to speed up compilation by re-using the
.ccache directory from previous CI runs. As we are no longer using
Travis we can remove all CI related ccache setup.
Signed-off-by: Adrian Reber <areber@redhat.com>
Our CentOS based CI run is based on CentOS 7. CentOS 8 exists already
for some time and CentOS 7 will probably go end of life at some point.
This adds a CentOS 8 based CI run to be prepared for the time CentOS 7
goes away.
Signed-off-by: Adrian Reber <areber@redhat.com>
Now that Travis also supports ppc64le and s390x we can remove all qemu
based docker emulation from our test setup. This now runs ppc64le and
s390x tests on real hardware (LXD containers).
Signed-off-by: Adrian Reber <areber@redhat.com>
This switches all arm related tests (32bit and 64bit) to the aarch64
systems Travis provides. For arm32 we are running in a armv7hf container
on aarch64 with 'setarch linux32'.
The main changes are that docker on Travis aarch64 cannot use
'--privileged' as Travis is using unprivileged LXD containers to setup
the testing environment.
Signed-off-by: Adrian Reber <areber@redhat.com>
With the newly introduced aarch64 at Travis it is possible for the CRIU
test-cases to switch to aarch64.
Travis uses unprivileged LXD containers on aarch64 which blocks many of
the kernel interfaces CRIU needs. So for now this only tests building
CRIU natively on aarch64 instead of using the Docker+QEMU combination.
All tests based on Docker are not working on aarch64 is there currently
seems to be a problem with Docker on aarch64. Maybe because of the
nesting of Docker in LXD.
Signed-off-by: Adrian Reber <areber@redhat.com>
To be able to run the test containers in scripts/build with podman this
puts the name of the container runtime into $CONTAINER_RUNTIME.
Now it can be overridden with
make fedora-rawhide CONTAINER_RUNTIME=podman
Signed-off-by: Adrian Reber <areber@redhat.com>
It is one of our target platforms.
Cc: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Acked-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Also, use more official / better maintained Docker image repo.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
This comes with newer libc headers so we can catch errors earlier.
Also, clang 4.0.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Let's use Ubuntu 17.04 "Zesty" for ARM-64 bit CI.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
We have to use "debian.jessie" instead of "ubuntu.xenial" because
qemu-s390x-static can only run z900 instructions and Ubuntu is
built with a more recent architecture level set.
We first enable only gcc because there are still some issues with clang.
Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
v2: Use the devicemapper storage driver for docker containers to
run more tests. AUFS doesn't support file handles, so a few tests
(e.g. inotify00 can't be run).
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
As we compile-test non-x86_64 architectures under qemu emulation,
it works pretty slow.
Dmitry Safonov suggested, and Andrey Vagin initially implemented
supporting ccache for such builds. This patch is based heavily
on Andrey's work -- all the bugs added are purely mine though.
Performance results: in an ideal environment (two builds of the same
code, one with cold (empty) ccache, another with the hot one)
I saw compile time improvements of 4x to 5x, and total test run time
improvement up to 2x to 2.5x. In layman terms, the complete test run
that was taking more than 50 minutes now takes about 25!
Notes on handling .ccache directory:
1. Before running docker build, .ccache directory (saved in between
runs by Travis) is moved to criu source code root, from where it
is copied by docker together with criu sources.
2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
is set accordingly.
3. After running docker build, .ccache is copied out from docker
container back to the host (from where it is saved by Travis).
Ccache envorinment notes:
1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
on compression/decompression (big performance hit under qemu).
2. CCACHE_CPP2 is required with clang, see detailed explanation at
http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
unfortunately I was not able to come up with a simpler approach.
Misc:
1. Travis runs "ccache -s" after the build is completed. A call to
"ccache -s" is called to Dockerfile before make, so one can
compare before/after numbers.
2. make invocations are surrounded by "date" calls so one can get the
compilation times.
Changes in v2:
- consolidate Dockerfile statements (ENV, RUN)
- single object make test is no longer commented out
- simplify "make mrproper" test
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Reviewed-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Unify alpine and non-alpine builds. The only difference is foreign
arch builds need some preparation -- separate that to a dependency.
Unfortunately we can't use wildcard targets ("%: ") as non-wildcard
ones are prevaling. Therefore, a somewhat ugly hack to generate
$arch: Dockerfile.$arch dependency is added.
While at it:
- rename DB_ARGS to DB_CC
- mark clean as phony
- make "all" really run all the builds we can
Nice side effect: autocompletion ("make -C scripts/build <TAB>")
now works!
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Apparently, there's no clang-3.8 for Ubuntu Trusty arm64. The latest
available version is 3.6, let's try to use it.
https://travis-ci.org/kolyshkin/criu/builds/173168198
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
With this, one can do a CRUI build via QEMU+Docker using clang
as a compiler, for example:
make -C scripts/build alpine-clang
make -C scripts/build ppc64le-clang
For alpine, default clang version is used. For others, a version has
to be specified explicitly, as there is no package called "clang" in
Ubuntu Trusty (which is used as a base for most arches). Since
clang-3.8 appears to be available from Trusty, and this is the current
stable version of clang, let's use it.
travis-ci: success for QEMU+Dockerfile improvements + clang build
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
With GNU make, using .PHONY leads to the same result as using
the "FORCE" hack, but unlike FORCE is not a hack. Since we do not
plan to use any other make than GNU make, let's just mark phony
targets as such.
For more details about why PHONY is better than FORCE, see
http://www.gnu.org/software/make/manual/make.html#index-FORCE
[v2: rebased and splitted]
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
Make the Makefile leaner and cleaner.
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Acked-by: Andrew Vagin <avagin@virtuozzo.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
