criu/.github/workflows/ci.yml
Adrian Reber 4d76d1acdc ci: shard alpine-test into parallel jobs to reduce CI time
The alpine-test CI job runs all ~483 zdtm tests sequentially three
times (normal, mntns-compat-mode, criu-config), followed by many
non-shardable tests. This dominates overall CI wait time. With only
2 jobs running in parallel (GCC and CLANG) the alpine tests take
around 30 minutes.

Use the existing --test-shard-index and --test-shard-count flags
already built into test/zdtm.py to split the zdtm test suite across
four parallel runners (shards 0-3). A fifth shard runs all
non-shardable tests (lazy pages, fault injection, test/others/*,
rootless, compel, plugins, etc.) independently and in parallel with
the zdtm shards. This increases parallelism from 2 to 10 jobs and
reduces the alpine test wall-clock time from ~30 to ~10 minutes.

Changes:
- run-ci-tests.sh: Build SHARD_OPTS from ZDTM_SHARD_INDEX/COUNT
  env vars and pass them to zdtm.py. Extract all non-shardable
  tests into a run_non_shardable_tests() function. Dispatch based
  on shard index: 0-3 run zdtm slices, 4 runs non-shardable
  tests, unset runs everything sequentially (preserving existing
  behavior). Validate that ZDTM_SHARD_INDEX is set when
  ZDTM_SHARD_COUNT is set.
- Makefile: Pass ZDTM_SHARD_INDEX and ZDTM_SHARD_COUNT into the
  container when set. Split long container run command across
  multiple lines for readability.
- ci.yml: Add shard: [0, 1, 2, 3, 4] to the alpine-test matrix,
  producing 10 jobs (2 compilers x 5 shards). Job labels now show
  descriptive shard names (e.g. "zdtm 1/4", "non-zdtm") instead
  of raw indices.

When sharding is not configured the script behaves identically to
before, so other CI jobs (aarch64, compat, gcov, etc.) are
unaffected.

Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Adrian Reber <areber@redhat.com>
2026-05-24 07:19:51 +01:00

275 lines
8.1 KiB
YAML

name: CI
on: [push, pull_request]
# Cancel any preceding run on the pull request.
concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/criu-dev' }}
jobs:
alpine-test:
name: Alpine Test (${{ matrix.target }}, ${{ matrix.shard_name }})
strategy:
matrix:
os: [ubuntu-22.04]
target: [GCC=1, CLANG=1]
shard: [0, 1, 2, 3, 4]
include:
- shard: 0
shard_name: zdtm 1/4
- shard: 1
shard_name: zdtm 2/4
- shard: 2
shard_name: zdtm 3/4
- shard: 3
shard_name: zdtm 4/4
- shard: 4
shard_name: non-zdtm
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Run Alpine ${{ matrix.target }} ${{ matrix.shard_name }} Test
run: >
sudo -E make -C scripts/ci alpine ${{ matrix.target }}
ZDTM_SHARD_INDEX=${{ matrix.shard }}
ZDTM_SHARD_COUNT=4
alpine-test-arm64:
name: Alpine Test ARM64
needs: [alpine-test]
strategy:
matrix:
os: [ubuntu-22.04-arm]
target: [GCC=1, CLANG=1]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Run Alpine ${{ matrix.target }} Test
run: sudo -E make -C scripts/ci alpine ${{ matrix.target }}
aarch64-test:
needs: [alpine-test]
strategy:
matrix:
os: [ubuntu-24.04-arm, ubuntu-22.04-arm]
target: [GCC=1, CLANG=1]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Run Tests ${{ matrix.target }} on ${{ matrix.os }}
run: |
# The 'sched_policy00' needs the following:
sudo sysctl -w kernel.sched_rt_runtime_us=-1
# etc/hosts entry is needed for netns_lock_iptables
echo "127.0.0.1 localhost" | sudo tee -a /etc/hosts
sudo -E make -C scripts/ci local ${{ matrix.target }} RUN_TESTS=1 \
ZDTM_OPTS="-x zdtm/static/change_mnt_context -x zdtm/static/maps05"
archlinux-test:
needs: [alpine-test]
# archlinux:latest + pacman -Syu is a rolling-release build; failures
# caused by upstream package churn are outside CRIU's control.
continue-on-error: true
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run Arch Linux Test
run: sudo -E make -C scripts/ci archlinux
compat-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
strategy:
matrix:
target: [GCC, CLANG]
steps:
- uses: actions/checkout@v4
- name: Run Compat Tests (${{ matrix.target }})
run: sudo -E make -C scripts/ci local COMPAT_TEST=y ${{ matrix.target }}=1
cross-compile:
needs: [alpine-test]
runs-on: ubuntu-latest
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
experimental: [false]
target: [
armv7-stable-cross,
aarch64-stable-cross,
ppc64-stable-cross,
riscv64-stable-cross,
]
include:
- experimental: true
target: armv7-unstable-cross
- experimental: true
target: aarch64-unstable-cross
- experimental: true
target: ppc64-unstable-cross
steps:
- uses: actions/checkout@v4
- name: Run Cross Compilation Targets
run: >
sudo make -C scripts/ci ${{ matrix.target }}
docker-test:
needs: [alpine-test]
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-22.04]
steps:
- uses: actions/checkout@v4
- name: Run Docker Test (${{ matrix.os }})
run: sudo make -C scripts/ci docker-test
fedora-asan-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run Fedora ASAN Test
run: sudo -E make -C scripts/ci fedora-asan
fedora-rawhide-test:
name: ${{ matrix.name }}
needs: [alpine-test]
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
- os: ubuntu-22.04
name: x86_64 Fedora Rawhide
- os: ubuntu-24.04-arm
name: aarch64 Fedora Rawhide
steps:
- uses: actions/checkout@v4
- name: Run Fedora Rawhide Test
# We need to pass environment variables from the CI environment to
# distinguish between CI environments. However, we need to make sure that
# XDG_RUNTIME_DIR environment variable is not set due to a bug in Podman.
# FIXME: https://github.com/containers/podman/issues/14920
run: sudo -E XDG_RUNTIME_DIR= make -C scripts/ci fedora-rawhide CONTAINER_RUNTIME=podman BUILD_OPTIONS="--security-opt seccomp=unconfined"
vagrant-fedora-rawhide-test:
name: Vagrant Fedora ${{ matrix.name }} based test
needs: [alpine-test]
runs-on: ubuntu-24.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- variant: fedora-stable
name: Stable
- variant: fedora-next
name: Next
steps:
- uses: actions/checkout@v4
- name: Install Lima
uses: lima-vm/lima-actions/setup@v1
- name: Cache Lima images
uses: actions/cache@v4
with:
path: ~/.cache/lima
key: lima-fedora-${{ github.sha }}
restore-keys: lima-fedora-
- name: Start Fedora VM
run: limactl start --plain --name=default --cpus=4 --memory=12 template://fedora
- name: Copy source into VM
run: |
lima sudo mkdir -p /home/criu
lima sudo chown "$(lima whoami)" /home/criu
limactl copy -r . default:/home/criu
- name: Setup VM
run: lima sudo /home/criu/scripts/ci/lima.sh ${{ matrix.variant }}-setup
- name: Reboot VM to activate new kernel
run: |
limactl stop default
limactl start default
- name: Show VM info
run: |
lima uname -a
lima cat /proc/cmdline
- name: Run tests
run: ssh -tt lima-default sudo -i /home/criu/scripts/ci/lima.sh ${{ matrix.variant }}-test
gcov-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run Coverage Tests
run: sudo -E make -C scripts/ci local GCOV=1
- name: Run gcov
run: sudo -E find . -name '*gcda' -type f -print0 | sudo -E xargs --null --max-args 128 gcov
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
java-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run Java Test
run: sudo make -C scripts/ci java-test
loongarch64-qemu-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- run: sudo make -C scripts/ci loongarch64-qemu-test
nftables-test:
needs: [alpine-test]
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Remove iptables
run: sudo apt remove -y iptables
- name: Install libnftables-dev
run: sudo contrib/apt-install libnftables-dev
- name: chmod 755 /home/runner
# CRIU's tests are sometimes running as some random user and need
# to be able to access the test files.
run: sudo chmod 755 /home/runner
- name: Build with nftables network locking backend
run: sudo make -C scripts/ci local COMPILE_FLAGS="NETWORK_LOCK_DEFAULT=NETWORK_LOCK_NFTABLES"
podman-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run Podman Test
run: sudo make -C scripts/ci podman-test
stream-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run CRIU Image Streamer Test
run: sudo -E make -C scripts/ci local STREAM_TEST=1
x86-64-clang-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run X86_64 CLANG Test
run: sudo make -C scripts/ci x86_64 CLANG=1
x86-64-gcc-test:
needs: [alpine-test]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Run X86_64 GCC Test
run: sudo make -C scripts/ci x86_64