Commit graph

9895 commits

Author SHA1 Message Date
Sebastiaan van Stijn
2a76ecc9fd README: fix broken links to github.com/xemul/criu
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-07 15:59:56 +03:00
Sebastiaan van Stijn
1356a1def3 Replace references to github.com/xemul/criu
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-07 15:59:56 +03:00
Andrei Vagin
4e84d11c1f kerndat: remove unused code
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Andrei Vagin
25460af822 kerndat: mark functions as static which are used in kerndat.c only
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
f6ab462074 vdso: Correctly track vdso position without vvar
If vvar is absent vdso_before_vvar is initialized by "false".
Which means that the check that supposed to track vdso/vvar pair went
into wrong brackets. In result it broke CRIU on kernels that don't have
vvar mapping.

Simpilfy the code by moving the check for VVAR_BAD_SIZE outside of
conditional for vdso_before_vvar.

Reported-by: Cyrill Gorcunov <gorcunov@gmail.com>
Fixes: 0918c76676 ("vdso/restorer: Always track vdso/vvar positions in
vdso_maps_rt")
Signed-off-by: Dmitry Safonov <dima@arista.com>
Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>
Tested-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Andrei Vagin
5f91f920a8 test: bring the lo interface up in each network namespace
This is needed to workaround the problem with "ip route save":
(00.113153) 	Running ip route save
Error: ipv4: FIB table does not exist.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Pavel Tikhomirov
e5bdcbbd1d zdtm/inotify: add a test that no unexpected events appear after c/r
Just create two inotify watches on a testfile, and do nothing except
c/r, it is expected that there is no events in queue after these.

before "inotify: cleanup auxiliary events from queue":

[root@snorch criu]# ./test/zdtm.py run -t zdtm/static/inotify04
=== Run 1/1 ================ zdtm/static/inotify04
======================== Run zdtm/static/inotify04 in h ========================
 DEP       inotify04.d
 CC        inotify04.o
 LINK      inotify04
Start test
./inotify04 --pidfile=inotify04.pid --outfile=inotify04.out --dirname=inotify04.test
Run criu dump
Run criu restore
Send the 15 signal to  60
Wait for zdtm/static/inotify04(60) to die for 0.100000
=============== Test zdtm/static/inotify04 FAIL at result check ================
Test output: ================================
18:37:14.279:    60: Event       0x10
18:37:14.280:    60: Event       0x20
18:37:14.280:    60: Event       0x10
18:37:14.280:    60: Read 3 events
18:37:14.280:    60: FAIL: inotify04.c:105: Found 3 unexpected inotify events (errno = 11 (Resource temporarily unavailable))

<<< ================================

v2: make two inotifies on the same file

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>

zdtm: inotify04 add another inotify on the same file
2019-09-07 15:59:56 +03:00
Pavel Tikhomirov
96992883ca inotify: cleanup auxiliary events from queue
I've mentioned the problem that after c/r each inotify receives one or
more unexpected events.

This happens because our algorithm mixes setting up an inotify watch on
the file with opening and closing it.

We mix inotify creation and watched file open/close because we need to
create the inotify watch on the file from another mntns (generally). And
we do a trick opening the file so that it can be referenced in current
mntns by /proc/<pid>/fd/<id> path.

Moreover if we have several inotifies on the same file, than queue gets
even more events than just one which happens in a simple case.

note: For now we don't have a way to c/r events in queue but we need to
at least leave the queue clean from events generated by our own.

These, still, looks harder to rewrite wd creation without this proc-fd
trick than to remove unexpected events from queues.

So just cleanup these events for each fdt-restorer process, for each of
its inotify fds _after_ restore stage (at CR_STATE_RESTORE_SIGCHLD).
These is a closest place where for an _alive_ process we know that all
prepare_fds() are done by all processes. These means we need to do the
cleanup in PIE code, so need to add sys_ppoll definitions for PIE and
divide process in two phases: first collect and transfer fds, second do
real cleanup.

note: We still do prepare_fds() for zombies. But zombies have no fds in
/proc/pid/fd so we will collect no in collect_fds() and therefore we
have no in prepare_fds(), thus there is no need to cleanup inotifies for
zombies.

v2: adopt to multiple unexpected events
v3: do not cleanup from fdt-receivers, done from fdt-restorer
v4: do without additional fds restore stage
v5: replace sys_poll with sys_ppoll and fix minor nits

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>

use ppoll always and remove poll
2019-09-07 15:59:56 +03:00
Dmitry Safonov
10a831689e restorer: Use gettimeofday() from rt-vdso for log timings
Omit calling raw syscalls and use vdso for the purpose of logging.
That will eliminate as much as one-syscall-per-PIE-message.
Getting time without switching to kernel will speed up C/R,
keeping logs as informative as they were.

Fixes: #346

I haven't enabled vdso timings for ia32 applications as it needs more
changes and complexity.. Maybe later.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
9e5c0634ff vdso: Add compatible property to vdso_maps
We need to differ compatible (ia32) vdso maps from x86_64.
That dictates ABI on vdso code.
According to that, the decision to (not) use gettimeofday() from vdso in
64-bit restorer.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
23960fe60e seccomp/restorer: Disable gtod from vdso in strict mode
Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
90ecb82202 restorer/parasite-vdso: Don't move vvar if failed to move vdso
Also slight refactor.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
53c2fdc955 vdso/restorer: Always track vdso/vvar positions in vdso_maps_rt
For simplicity, make them always valid in restorer.
rt->vdso_start will be used to calculate gettimeofday() address.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:56 +03:00
Dmitry Safonov
2d521f3c93 vdso/restorer: Try best to preserve vdso during restore
vdso will be used in restorer for timings in logs - try to keep it
during restore process.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Dmitry Safonov
28949d5fb8 compel/std/uapi: Provide setter for gettimeofday()
Provide a way to set gettimeofday() function for an infected task.
CRIU's parasite & restorer are very voluble as more logs are better
than lesser in terms of bug investigations.
In all modern kernels there is a way to get time without entering
kernel: vdso. So, add a way to reduce the cost of logging without making
it less valuable.

[I'm not particularly fond of std_log_set_gettimeofday() name, so
 if someone can come with a better naming - I'm up for a change]

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Dmitry Safonov
d2d6e3f537 compel/log: Use enum as parameter for std_log_set_loglevel()
Doesn't change uapi, but makes it a bit more friendly and documented
which loglevel means what for foreign user.

Signed-off-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Radostin Stoyanov
b25d1facae pb2dict: Disable undefined name 'basestring'
The following error is falsely reported by flake8:

lib/py/images/pb2dict.py:266:24: F821 undefined name 'basestring'

This error occurs because `basestring` is not available in Python 3,
however the if condition on the line above ensures that this error
will not occur at run time.

Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2019-09-07 15:59:55 +03:00
Radostin Stoyanov
5721e61000 scripts: Install flake8 with dnf in Fedora
In the Fedora tests we install python3-pip only to install flake8.

This is not necessary as there is a Fedora package for flake8.

Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2019-09-07 15:59:55 +03:00
Radostin Stoyanov
2a683849b9 scripts: Set PYTHON=python3 in Fedora Dockerfiles
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2019-09-07 15:59:55 +03:00
Radostin Stoyanov
cd87a628e1 scripts: Remove yaml/ipaddress Py2 fedora modules
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2019-09-07 15:59:55 +03:00
Pavel Tikhomirov
77efcde96d mount: fix inconsistent return and goto err alternation
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2019-09-07 15:59:55 +03:00
Adrian Reber
229a8ab06b scripts: remove python2 from Fedora Dockerfiles
More and more python2 packages are being removed from future Fedora
releases. This removes python2 packages explicitly listed in CRIU's
Dockerfiles, which all are not required for the current level of
testing.

Signed-off-by: Adrian Reber <areber@redhat.com>
2019-09-07 15:59:55 +03:00
Adrian Reber
9fe8960d89 scripts: add possibility to override docker with podman
To be able to run the test containers in scripts/build with podman this
puts the name of the container runtime into $CONTAINER_RUNTIME.

Now it can be overridden with

make fedora-rawhide CONTAINER_RUNTIME=podman

Signed-off-by: Adrian Reber <areber@redhat.com>
2019-09-07 15:59:55 +03:00
Andrei Vagin
1e2647f123 images: convert type of child_subreaper from int32 to bool
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Andrei Vagin
104aaf383e restorer: print errors if prctl syscalls failed
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Radostin Stoyanov
ba454407bf make: Insert version macros in criu.h
Including the version information of CRIU in criu.h is required by
projects that use libcriu to preserve backward compatibility.

Closes #738

Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
c01da212da mem/shmem: Fix typos for_each_shmem macro
Since we use _i as a counter in macro declaration
we should use it as a reference. This macro simply
happen to work now because of being called with
variable i declarated in the caller code.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
92717977cd mem/shmem: Use xmalloc in collect_sysv_shmem
To get error message in log if no memory available.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
67d5dbb995 mem/shmem: More elegant entries declaration
Signed-off-by: Cyrill Gorcunov <gorcunov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
d999a9e079 mem/shmem: Use xrealloc_safe in expand_shmem
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
0319d1bf2e mem/page-pipe: Use xrealloc_safe in page_pipe_add_hole
To shrink code a bit.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
cb7b013dd0 mem/pmc: Print pid for debug sake
When logs are massive it is convenient for grepping.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
7eec03bd95 mem/pmc: Use pr_warn_once if cache is disabled
No need to spam on every pmc_init call.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:55 +03:00
Cyrill Gorcunov
43ac528cb9 mem/page-xfer: Add log prefix
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
8f9ae895d0 mem/vma: Sanitize struct vm_area_list
- make names more descriptive
 - add comments
 - use union for nr_priv_pages and rst_priv_size since
   former priv_size has been used with different meaning:
   number of pages during checkpoint time and size in bytes
   on restore moment

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
0ee3d0764d mem/vma: Drop never used VM_AREA_LIST macro
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
32e9f15525 mem/vma: Use vm_area_list_init where appropriate
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
6e86b9eb01 mem/vma: Use memset for vm_area_list_init
To eliminate side effects, in particular setting nr_aios
is already missing here.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
f3b8371c30 mem/page-pipe: Use ssize_t for splice/tee results
Integer value is too short.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
692fdada56 mem/page-pipe: Align members for readability sake
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
088b086bfe mem/page-pipe: create_page_pipe -- Drop redundant zero assignment
We allocate with xzalloc, no need for additional zero assignemtns.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Cyrill Gorcunov
cb94616436 mem/page-pipe: Eliminate redundant pipe_off setup
In case if we may use previous pipe the pipe_off
get set directly so no need for redundat unconditional
assignment.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Michał Cłapiński
6606f246c2 Add ZDTM tests for child subreaper property
1. Basic check if property is migrated
2. Check that property is restored for existing children
3. Check that child subreaper does not affect reparenting

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Michał Cłapiński <mclapinski@google.com>
Reviewed-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2019-09-07 15:59:54 +03:00
Michał Cłapiński
db2777e73c Add support for migrating CHILD_SUBREAPER prctl
1. Checkpoint it via parasite.
2. Restore it after forking.

Signed-off-by: Michał Cłapiński <mclapinski@google.com>
Reviewed-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2019-09-07 15:59:54 +03:00
Sebastiaan van Stijn
b758e4b476 Replace libprotobuf-c0-dev with libprotobuf-c-dev
The `libprotobuf-c0-dev` virtual package is no longer available
in Debian Buster, but is provided by `libprotobuf-c-dev`, which
is available.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-07 15:59:54 +03:00
Andrei Vagin
a82275f3d4 zdtm: use a proper page size for the host
In zdtm.py, the page size is hardcoded as 4096, but on ppc64le, is is equal
to 64K and all test fail with errors like this:

ERROR: bad page counts, stats = 13 real = 208(0)

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Andrei Vagin
cb6768b62c test/packet_sock_mmap: parse inode as unsigned long long
7f95a16df000-7f95a16e1000 rw-p 00000000 00:09 2183152397                 socket:[2183152397]

Reported-by: Mr Jenkins
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2019-09-07 15:59:54 +03:00
Pavel Tikhomirov
3ca4c73e47 zdtm: make grep_errors also grep warnings
It is inspired by the discussion about inotify fix:
https://github.com/checkpoint-restore/criu/pull/728#issuecomment-506929427

From one point of view, warnings might be important to understand why we
detect some visible change in the environment after c/r-ing the process,
and if this change is expected or not. So we should add "Warn" messages
to the output.

From over point, these warnings if they are expected, can spoil our
final logs with a lot of unnecessary details, so add changes in previous
patches to silence the most noisy of these warnings.

Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
2019-09-07 15:59:54 +03:00
Pavel Tikhomirov
facfebee22 inventory: skip warning in case of no parent directory
We want to grep warnings from zdtm tests to travis final logs.
And I see a lot of these:

Warn  (criu/image.c:137): Failed to open parent directory

If there is no parent images directory then there is no previous dump
and no pid-reuse problem with pagemaps possible, so it is fine to have
no parent inventory image at the same time which is used here to fix the
problem. These always hapens on the first iteration of iterative dump.
So don't warn here.

While on it also fix error message in detect_pid_reuse.

v2: add detect_pid_reuse part
v3: improve comments
2019-09-07 15:59:54 +03:00
Pavel Tikhomirov
72da499a27 dedup: convert noisy warning to debug and improve messages
We want to grep warnings from zdtm tests to travis final logs.
And I see a lot of these:

(00.250989) Warn  (criu/pagemap.c:90): Missing 7f84103e3000 in parent pagemap
(00.250999) 	p 0x7f84103f5000 [1]

We do a lookup of an intersecting pagemap entry with a memory region we
want to dedup, it is expected that sometimes we don't have some subrange
in pagemap entries. So these should not be a warning, make it debug
message.

While on it change the message to save us from been confused with other
"Missing..." error messages, and change abstract "parent image" message
to the IDs of pages image in all messages in dedup_one_iovec().

v2: print image ids
2019-09-07 15:59:54 +03:00