Running crit tool 4 times per test (decode, encode, decode --pretty
and encode back again) is way too slow. The majority of time, as
it turned out, goes on python load and arguments parsing. The en-
and de-coding works pretty fast.
So doing re-code logic in one python script for ALL images is way
way faster -- ~1 hour vs ~1 minute on my box.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
There are loads and dumps method in pycriu.images that work with
strings, instead of open()-ed files. For simplicity strings are
turned into streams with io.BytesIO and the files are then pushed
into regular load/dump methods.
The problem is that array.array object doesn't work with io object
in .fromfile/.tofile methods, so we have to read/write the data
from them explicitly and use .fromstring/.tostring for arrays.
With this the crit test finally passes :D
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
- need to add encoding method
- also need to do the same for ipc queues and shmem
Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
The native pb engine doesn't accept types other than int or long:
...
File "/root/src/criu/pycriu/images/pb2dict.py", line 264, in dict2pb
pb_val.append(_dict2pb_cast(field, v))
File "/usr/lib/python2.7/site-packages/google/protobuf/internal/containers.py", line 111, in append
self._type_checker.CheckValue(value)
File "/usr/lib/python2.7/site-packages/google/protobuf/internal/type_checkers.py", line 104, in CheckValue
raise TypeError(message)
TypeError: 1.1258999068426252e+16 has type <type 'float'>, but expected one of: (<type 'int'>, <type 'long'>)
In particular, this is seen when encoding back so_filter field from
inetsk image.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Ruslan Kuprieiev <kupruser@gmail.com>
To better understand the content of mm-<ID>.img and pagemap-<ID>.img
additional constant names have been added to better resolve the hex
value to symbolical names.
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Currently decoded with --pretty image cannot be encoded back if there's
an IP address inside. "Just decoded" can.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Currently device numbers are shown as plain integers, but in
pretty output it's nice to see the major:minor pairs.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Ruslan Kuprieiev <rkuprieiev@cloudlinux.com>
This commit adds basic support for dumping and restoring seccomp filters
via the new ptrace interface. There are two current known limitations with
this approach:
1. This approach doesn't support restoring tasks who first do a seccomp()
and then a setuid(); the test elaborates on this and I don't think it is
tough to do, but it is not done yet.
2. Filters are compared via memcmp(), so two tasks which have the same
parent task and install identical (via memory) filters will have those
filters considered to be the "same". Since we force all tasks to have
the same creds (including seccomp filters) right now, this isn't a
problem.
The approach used here is very similar to the cgroup approach: the actual
filters are stored in a seccomp.img, and each task has an id that points to
the part of the filter tree it needs to restore. This keeps us from dumping
the same filter multiple times, since filters are inherited on fork.
v2:
* remove unused seccomp_filters field from struct rst_info
* rework memory layout for passing filters to restorer blob
* add a sanity check when finding inherited filters
Signed-off-by: Tycho Andersen <tycho.andersen@canonical.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
This class is essentially libcriu in python(yet for now it
has only some basic functions such as check/dump/restore).
It is needed to make life for python users even more easier,
i.e. hiding some nasty connection stuff. It is also using
criu swrk(COMM_BIN) communication method, instead of an old
system service, as we no longer recommend it.
Signed-off-by: Ruslan Kuprieiev <rkuprieiev@cloudlinux.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Chris Lamb <lamby@debian.org> reported in Debian that criu is not
building reproducible while working on the "reproducible builds" effort
[0].
[0] https://wiki.debian.org/ReproducibleBuilds
[1] https://bugs.debian.org/801211
Signed-off-by: Salvatore Bonaccorso <carnil@debian.org>
Acked-by: Andrew Vagin <avagin@odin.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Some flags have known text description, e.g. flags on a file
or flags on vma. Add ability to show these flags in decoded
form (and encode them back into binary).
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
This should print some information about image file itself, e.g.
statistical information. Right now print magic and the number of
object written in the file.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Using collections.OrderedDict allows us to keep fields in the
same order as they appear in corresponding proto files, which
helps to impove readability. In non-pretty mode we still use
regular dict.
Signed-off-by: Ruslan Kuprieiev <rkuprieiev@cloudlinux.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
These images have common magic in front of per-image one. With
this we have 3 "types" of images -- inventory (head), other
images, service files. The latter would be stats (not an image,
just happen to be in PB format) and irmap cache (not an image
again, just auxiliary thing which is in PB for convenience).
Since inventory file is the first one we read on restore it's
OK to set the global "new images" flag there. Dump (write) is
always in new format.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Ruslan Kuprieiev <rkuprieiev@cloudlinux.com>
Acked-by: Andrew Vagin <avagin@odin.com>
Acked-by: Cyrill Gorcunov <gorcunov@openvz.org>
There are two places where we store IP addresses (both IPv4 and IPv6).
Mark them with custom option and print them in compressed form for
--pretty output.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Ruslan Kuprieiev <kupruser@gmail.com>
I plan to mark some fields as IP address and print them respectively.
The --format hex is not nice switch for this and introducing one more
(--format hex ipadd) is too bad.
So let's fix the cirt API to be simple and stupid. By default crit
generates canonical one-line JSON. With --pretty option it splits the
output into lines, adds indentation and prints hex as hex and IP as
IP.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Acked-by: Ruslan Kuprieiev <kupruser@gmail.com>
Pavel reported that decimal values for some fields are hard to read,
because people used to see hex values in there. Unfortunately, json
doesn't support hex representation of integers, so we can only store
them as hex strings. Not all field need to be represented as hex
strings, so this set introduces a custom field option called "criu"
to use in our proto files. One should use [(criu).hex = true] to mark
which field should be represented as a hex string. pb2dict module
from pycriu package will look into field options and if he finds that
criu.hex is set to True, it will convert such field to/from hex string.
Though, such behaviour is optional and user can request it by specifying
--format hex when calling crit decode("crit encode" in its turn, detects
such fields automatically and doesn't require any special cmdline options
to be set).
We need our proto files to compile with both protoc and
protoc-c compilers, which requires creating google/protobuf
directory with a symlink to /usr/include/google/protobuf/
descriptor.proto to make protoc-c and generated c files happy.
Reported-by: Pavel Emelyanov <xemul@parallels.com>
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
Currently we are handling conversion in the simplest way, by
just parsing protobuf text format representation of pb message,
splitting string in half and storing values wrapped in ""-s
in python dict. Which leads to a bunch of errors, like,
for example, when converting bytes fields. It also doesn't
handle types properly.
This patch introduces a proper way to handle pb<->json conversion
by iterating through pb fields and properly storing them in dict
or properly extracting values from dict.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
pagemap.img is the special one, as it starts with an entry
of pagemap_head type and is followed by entries of
pagemap_entry type. Currently we treat pagemap as if
pagemap_head is a payload and pagemap_entry's are
an extra, which is wrong.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
pycriu is a python package, that, for now, consists
of rpc module and images package. rpc module contains
data structures for interacting with CRIU RPC.
images package contains methods for loading\dumping
criu images.
See a big comment below in pycriu/images/images.py file.
Signed-off-by: Ruslan Kuprieiev <kupruser@gmail.com>
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
