Commit graph

8910 commits

Author SHA1 Message Date
Andrei Vagin
e85646875f zdtm/maps006: modify test so that file and anon vma-s are mixed
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-12-12 09:12:16 +03:00
Andrei Vagin
cef1efe5bd page-pipe: allow to share pipes between page pipe buffers
Now criu create a new pipe buffer, if a previous one has another set of
flags. In this case, a pipe is not full and we can use it for the
next page buffer.

We need 88 pipes to pre-dump the zdtm/static/fork test without this
patch, and we need only 17 pipes with this patch.

Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-12-12 09:12:16 +03:00
Andrei Vagin
6feebf3aad page-pipe: move code to resize a pipe in a separate function
v2: and move it upper, because it is going to be used in ppb_alloc()

Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-12-12 09:12:16 +03:00
Andrei Vagin
e3c6784b8f parasite: remove restriction to a number of iovec-s
vmsplice can't splice more than UIO_MAXIOV, but we can
call it a few times from a parasite.

v2: s/nr/nr_segs/

Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-12-12 09:12:16 +03:00
Adrian Reber
c949e75968 crtools: also print the current kernel version
In addition to writing the CRIU version to the log file this adds the
current kernel version to the log file:

(00.000008) Version: 3.5 (gitid v3.5-511-ga8cc6cf)
(00.000303) Running on node01 Linux 3.10.0-513.el7.x86_64 #1 SMP Tue Feb 29 06:78:90 EST 2017 x86_64

v2:
 - small changes as suggested by Dmitry (thanks)

Signed-off-by: Adrian Reber <areber@redhat.com>
Reviewed-by: Dmitry Safonov <0x7f454c46@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-30 01:16:33 +03:00
Andrei Vagin
dc384c0e30 netfilter: use ipv4 iptables rules to block IPv4-mapped IPv6 addresses
If ipv6 socket has an IPv4-mapped address, it is used to handle ipv4
connection, so we have to use ipv4 iptables rules to block this
connection.

Reported-by: Mr Jenkins
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-30 01:14:05 +03:00
Andrei Vagin
c31d423eeb compel: use a correct name format for vma files in /proc/pid/map_files/
Currently we use the "map_files/%p-%p" format, but actually it should
be "map_files/%lx-%lx".

The kernel could handle both formats, but recently Alexey Dobriyan fixed
the kernel and it accept only the second format.

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-30 01:13:50 +03:00
Andrei Vagin
3121d90de0 criu: print a criu version with the info level
We always ask users what version of criu they use to investigate a problem,
so it better to have it in a log.

Signed-off-by: Andrei Vagin <avagin@openvz.org>
Reviewed-by: Dmitry Safonov <0x7f454c46@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:26:02 +03:00
Andrei Vagin
ffee07723e criu: remap soccr log levels to criu levels
criu and soccr has different values for log levels, so
someone has to remap them.

Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Reported-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:25:49 +03:00
root
638c14f2ed zdtm: grep errors from page-server.log and lazy-pages.log
This can help to investigate logs from Mr Jenkins.

Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Cyrill Gorcunov
e6537f3d8d fsnotify: open_handle -- Handle multiple mounts with same s_dev
When inotify is laying on uovermounted fs we should walk over
all mountpoints with same s_dev to find openable path.

Note on restore the path is usually already allocated during
dump stage so get_mark_path won't call for open_handle(), in
turn on dump stage the positive return from open_handle()
will cause fsnotify engine to find openable path, thus there
is kind of double work to be optimized in future.

For example we got a container where systemd-udevd inside
opens inotify for /dev/X entry then overmount ./dev path
with slave option and in result irmap engine on predump
can't figure out where the inotify is sitting causing
migrtion to abort.

Signed-off-by: Cyrill Gorcunov <gorcunov@virtuozzo.com>
Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Dmitry Safonov
d541bc797c build: Move generated config.h into include/common/
config.h is a generated file with "build-features" defines.
We use it for several purposes:
o to check that compiler can do it's job
o to complement user-visible API between distributions
o to add compile-time options from .config global file

It's used in criu and soccr, but compel also needs such thing.

Previously, soccr has a link to config.h in criu includes,
but it would be much cleaner to move it to other headers,
that are shared between sub-projects into include/common.

Reported-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Tested-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Dmitry Safonov
585dda236c ia32: Get rid of R_X86_64_32S relocation
Distributions starts to supply GCC that is configured to compile
-pie and -fPIC code by default due to security reasons.

CONFIG_COMPAT was unfriendy to -pie by the reason of R_X86_64_32S
relocation in call32.S helper:
  LINK     criu/criu
/usr/bin/ld: criu/arch/x86/crtools.built-in.o: relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
make[1]: *** [criu/Makefile:92: criu/criu] Error 1
make: *** [Makefile:225: criu] Error 2

Use %rip-relative addressing to avoid ld errors for shared binary linking.
Puff, all needs to be done with bare hands!

Now CONFIG_COMPAT can be used with -pie binaries and all should
also work for debian toolchain (#315).

Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Andrei Vagin
008db0cb7a zdtm: run page-server via rpc
v2: typo fix
v3: run criu pre-dump via rpc
v4: don't use status-fd for rpc

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Andrei Vagin
397df9c035 lib/py: allow to execute page-server as a child process
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Andrei Vagin
ebd64bddfe service: allow to execute page-server as a child process
In this case we can wait it and get an exit code.

For example, it will be useful for p.haul where one connection
is used several times, so we need a way how to understand  that
page-server exited unexpectedly.

v2: don't write ps_info if a start descriptor isn't set

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:23 +03:00
Cyrill Gorcunov
0b6f9c7975 build: Reused .FORCE from nmk
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
8db0f03758 build: nmk -- Move phony targets to include.mk
So they can be reused.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
d62e01929e build: nmk -- Add .FORCE target
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Avindra Goolcharan
8e45ce4905 images.py: remove shebang
This file is not executable directly, so it should not have the shebang.

Signed-off-by: Avindra Goolcharan <aavindraa@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Pavel Begunkov
3e0200e571 build: get rid of warnings with sysmacro
warning: In the GNU C Library, "major" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "major", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "major", you should undefine it after including <sys/types.h>.
  if (major(st.st_rdev) != major(st_rtc.st_rdev) ||

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
30b2630e80 test: static,aio01 -- Use proper type for context
aio_context_t is 8 byte long so on 32 bit mode it might be
strippped off when unsigned long used instead. Fix this typo.

Signed-off-by: Cyrill Gorcunov <gorcunov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
cae6262ce9 zdtm: add an option to show criu statistics
v2: defining crit_bin and using it for Popen() // Mike

Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
7d6d795d28 stats: add counters for pipes and page_pipe_bufs
The number of pipes are limited in a system, so it is better to know how
many we use.

Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
00413cb5a7 test: check ipv6 sockets which handle ipv4 connections
A server socket is created with AF_INET6, but a client
socket is create with AF_INET.

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
6e770e50b8 soccr: c/r ipv6 sockets which handles ipv4 connections
IPv6 listening sockets can accept both ipv4 and ipv6 connections,
in both cases a family of an accepted socket will be AF_INET6.

But we have to send tcp packets accoding with a connection type.

------------------------ grep Error ------------------------
(00.002320)     53: Debug: 		Will set rcv_wscale to 7
(00.002325)     53: Debug: 		Will turn timestamps on
(00.002331)     53: Debug: Will set mss clamp to 65495
(00.002338)     53: Debug: 	Restoring TCP 1 queue data 2 bytes
(00.002403)     53: Error (soccr/soccr.c:673): Unable to send a fin packet: libnet_write_raw_ipv6(): -1 bytes written (Network is unreachable)

(00.002434)     53: Error (criu/files.c:1191): Unable to open fd=3 id=0x6
(00.002506) Error (criu/cr-restore.c:2171): Restoring FAILED.
------------------------ ERROR OVER ------------------------

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
09b93e0ac5 sk-inet: restore a value of SO_REUSEPORT
The SO_REUSEPORT option allows multiple sockets on the same
host to bind to the same port. This option has to ve restored when all
sockets are bound to a port. The same logic is already used to restore
SO_REUSEADDR.

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Andrei Vagin
3503332086 zdtm: check a case when one port is shared between two sockets
SO_REUSEPORT allows multiple sockets on the same host to bind to the
same port.

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
2e16cc1e4f compel: Do not loose sign of result in compat syscall
Regs are present in unsigned format so convert them
into signed first to provide results.

In particular if memfd_create syscall failed we won't
notice -ENOMEM error but rather treat it as unsigned
hex value

 | (05.303002) Putting parasite blob into 0x7f1c6ffe0000->0xfffffff4
 | (05.303234) Putting tsock into pid 42773

Signed-off-by: Cyrill Gorcunov <gorcunov@virtuozzo.com>
Reviewed-by: Dmitry Safonov <dima@arista.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
fc21d6fb53 crit: Add socket states decoding
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
4d0fc1a496 crit: Add socket types decoding
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
d4c29ab7cb crit: Add protocols decoding
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
56cd56706d crit: Add more families into socket decoding
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
ec273275fe crit: Add INET6 familiy
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Pavel Tikhomirov
926e42ac63 zdtm: test overmounting with shared parent works
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Pavel Tikhomirov
2d2381e01d zdtm: test shared mount propagation is preserved
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Pavel Tikhomirov
bf1f9c61c1 mount: allow overmount on parent with shared group
In CT, we do:

mkdir -p /a/b/c1
mkdir -p /c2
mount --bind /c2 /a/b/c1
mount --rbind /a/b /a

And after that container is not dumpable with error:

mnt: Unable to handle mounts under 146:./a

Just because overmounts with shared parent group are prohibited,
but I can't see any problem with enabling them.

https://jira.sw.ru/browse/PSBM-69501
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Pavel Tikhomirov
2214568de9 mount: fix parent shared group dependency in can_mount_now
What we do before patch:

1) If we are NOT in the same shared group - if we have some parent's
shared group member unmounted, we just wait for it.
2) If we are in the same group - we wait only for members with root
path len shorter than ours.

That is done to make child mount propagate in all shared group,
but I think it is wrong, e.g.:

mkdir -p /dir/a/b/c /d /e /f
mount --bind /dir/a /d
mount --bind /dir/a/b /e
mount --bind /f /e/c

Before c/r we have:

507 114 182:1017985 / / rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
144 507 182:1017985 /dir/a /d rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
146 507 182:1017985 /dir/a/b /e rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
148 146 182:1017985 /f /e/c rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
150 507 182:1017985 /f /dir/a/b/c rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
149 144 182:1017985 /f /d/b/c rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12

After c/r we have:

600 132 182:1017985 / / rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
602 600 182:1017985 /f /dir/a/b/c rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
603 600 182:1017985 /dir/a /d rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12
604 600 182:1017985 /dir/a/b /e rw,relatime shared:63 master:60 - ext4 /dev/ploop63624p1 rw,data=ordered,balloon_ino=12

There is no propagation as all mounts are in same shared group and
602(150) has shorter root than 603(144) and 604(146).

What we should do:

Wait member of our parent's shared group only if it has our 'sibling'
mount in it. Sibling mount is the one which had propagated to shared
mount of our parent for us when we were mounted. We need to enforce
propagation only for these case.

https://jira.sw.ru/browse/PSBM-69501
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:14 +03:00
Cyrill Gorcunov
b2d2574955 image-desc: Encode pagemap in unsigned long format
The anonymous shared memory are using shmid for image
name encoding which is unsigned long and we already
met scenario where high bits get strippped off thus
the restore failed.

Lets use unsigned long here, and because pagemap code
is shared between plain memory and anon shared memory
use unsigned long every where.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
bf341b463e image-desc: Use unsigned format for tmpfs image
The index comes from mnt_id which is signed integer
both in kernel and in userspace, but negative value
is never valid, thus don't use it.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
d9952dcd4f image-desc: Use unsigned format for pid driven entries
For images which are using pid as id for image names
use unsigned format since here is no negative pid
in real system.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
582f104d04 image-desc: Use unsigned format for userns
It uses ns->id

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
af5a6b52cb image-desc: Use unsigned format for netns
Since it uses ns->id

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
a8d7857303 image-desc: Use unsigned format for netdev
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
ba10ae1ac0 image-desc: Use unsigned for mountpoints
Just as we declare it in ns_id structure.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
6a08b0cdf1 image-desc: Use unsigned int for old binfmt-misc
Same as for autofs.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
9020d18d44 image-desc: Use unsigned int for tmpfs dev image
Same as for autofs.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
2ca78e9a00 image-desc: Use unsigned format for autofs
Both the kernel and criu uses unsigned int
for it, make the format appropriate.

	| struct mount_info {
	|	...
	|	unsigned int		s_dev;
	|	...
	| }

We didn't see negative number here in real life so
I don't think if such %d to %u convention cause
backward compatibility problem ever.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
1fbc51f8e2 image-desc: Fix fdinfo format
We pass unsigned 4 byte integer here, so
use appropriate format.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00
Cyrill Gorcunov
57e4ad5ba8 namespaces: __get_ns_id -- Use safe snprintf
Namespace descriptors are not promised to have
constant short names, so just to be on a safe
side.

Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2017-11-23 20:23:13 +03:00