[CRIU] BUG: CRIU corrupt floating point state after checkpoint

Andrei Vagin avagin at gmail.com
Wed Sep 25 00:09:17 MSK 2019


On Mon, Sep 23, 2019 at 09:35:14PM -0700, Diyu Zhou wrote:
> Hey CRIU,
>
> It seems to me that CRIU corrupts the floating point of the process after
> checkpointing. I was wondering if I did something wrong with CRIU or it is a
> bug.


(00.000193) cpu: x86_family 6 x86_vendor_id GenuineIntel x86_model_id
Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
(00.000198) cpu: fpu: x87 FPU will use FXSAVE
(00.000201) cpu: fpu:1 fxsr:1 xsave:0 xsaveopt:0 xsavec:0 xgetbv1:0 xsaves:0

Cyrill or Duma, could you take a look at this?

>
> To illustrate it, I wrote a simple floating point program that keeps
> calculating the average value of three floating point array elements. I use
> CRIU to checkpoint it as follows:
>
> $criu dump -t pid --leave-running --shell-job
>
> After several iterations, I can observe that some of the floating point
> registers are modified to 0 and causes the program to output incorrect result.
>
> One thing I have observed is that the floating point corruption does not occur
> on the main thread but only on other threads.
>
> I have attached my /proc/cpuinfo output, the dump log of CRIU and the
> simple benchmark I have used to illustrate the program (please run it with
> ./ftress1 8 1000 1), together with a bash script I wrote to invoke CRIU to
> checkpoint the program constantly.
>
> I'm on an x86 machine installed with Fedora 29, Kernel version: 4.20.13.  The
> version of CRIU is shown below (just downloaded from github)
>
> $criu -v
>
> Version: 3.12
> GitID: v3.12-259-g4d7f74aa
> Thank you for your help and I'm happy to provide more information.
>
> Best,
> Diyu




> (00.000033) Version: 3.12 (gitid v3.12-259-g4d7f74aa)
> (00.000060) Running on dash Linux 4.20.13-200.fc29.x86_64 #1 SMP Wed Feb 27 19:42:55 UTC 2019 x86_64
> (00.000078) Loaded kdat cache from /run/criu.kdat
> (00.000083) ========================================
> (00.000091) Dumping processes (pid: 2160)
> (00.000095) ========================================
> (00.000103) rlimit: RLIMIT_NOFILE unlimited for self
> (00.000112) Running pre-dump scripts
> (00.000153) irmap: Searching irmap cache in work dir
> (00.000171) No irmap-cache image
> (00.000178) irmap: Searching irmap cache in parent
> (00.000183) irmap: No irmap cache
> (00.000193) cpu: x86_family 6 x86_vendor_id GenuineIntel x86_model_id Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
> (00.000198) cpu: fpu: x87 FPU will use FXSAVE
> (00.000201) cpu: fpu:1 fxsr:1 xsave:0 xsaveopt:0 xsavec:0 xgetbv1:0 xsaves:0
> (00.000354) cg-prop: Parsing controller "cpu"
> (00.000364) cg-prop: 	Strategy "replace"
> (00.000370) cg-prop: 	Property "cpu.shares"
> (00.000374) cg-prop: 	Property "cpu.cfs_period_us"
> (00.000377) cg-prop: 	Property "cpu.cfs_quota_us"
> (00.000381) cg-prop: 	Property "cpu.rt_period_us"
> (00.000385) cg-prop: 	Property "cpu.rt_runtime_us"
> (00.000389) cg-prop: Parsing controller "memory"
> (00.000393) cg-prop: 	Strategy "replace"
> (00.000396) cg-prop: 	Property "memory.limit_in_bytes"
> (00.000400) cg-prop: 	Property "memory.memsw.limit_in_bytes"
> (00.000404) cg-prop: 	Property "memory.swappiness"
> (00.000408) cg-prop: 	Property "memory.soft_limit_in_bytes"
> (00.000412) cg-prop: 	Property "memory.move_charge_at_immigrate"
> (00.000415) cg-prop: 	Property "memory.oom_control"
> (00.000419) cg-prop: 	Property "memory.use_hierarchy"
> (00.000423) cg-prop: 	Property "memory.kmem.limit_in_bytes"
> (00.000426) cg-prop: 	Property "memory.kmem.tcp.limit_in_bytes"
> (00.000430) cg-prop: Parsing controller "cpuset"
> (00.000434) cg-prop: 	Strategy "replace"
> (00.000438) cg-prop: 	Property "cpuset.cpus"
> (00.000441) cg-prop: 	Property "cpuset.mems"
> (00.000445) cg-prop: 	Property "cpuset.memory_migrate"
> (00.000449) cg-prop: 	Property "cpuset.cpu_exclusive"
> (00.000452) cg-prop: 	Property "cpuset.mem_exclusive"
> (00.000456) cg-prop: 	Property "cpuset.mem_hardwall"
> (00.000460) cg-prop: 	Property "cpuset.memory_spread_page"
> (00.000464) cg-prop: 	Property "cpuset.memory_spread_slab"
> (00.000468) cg-prop: 	Property "cpuset.sched_load_balance"
> (00.000472) cg-prop: 	Property "cpuset.sched_relax_domain_level"
> (00.000476) cg-prop: Parsing controller "blkio"
> (00.000480) cg-prop: 	Strategy "replace"
> (00.000483) cg-prop: 	Property "blkio.weight"
> (00.000487) cg-prop: Parsing controller "freezer"
> (00.000491) cg-prop: 	Strategy "replace"
> (00.000495) cg-prop: Parsing controller "perf_event"
> (00.000499) cg-prop: 	Strategy "replace"
> (00.000503) cg-prop: Parsing controller "net_cls"
> (00.000507) cg-prop: 	Strategy "replace"
> (00.000511) cg-prop: 	Property "net_cls.classid"
> (00.000514) cg-prop: Parsing controller "net_prio"
> (00.000518) cg-prop: 	Strategy "replace"
> (00.000522) cg-prop: 	Property "net_prio.ifpriomap"
> (00.000526) cg-prop: Parsing controller "pids"
> (00.000530) cg-prop: 	Strategy "replace"
> (00.000533) cg-prop: 	Property "pids.max"
> (00.000537) cg-prop: Parsing controller "devices"
> (00.000541) cg-prop: 	Strategy "replace"
> (00.000544) cg-prop: 	Property "devices.list"
> (00.000595) Preparing image inventory (version 1)
> (00.000629) Add pid ns 1 pid 2203
> (00.000643) Add net ns 2 pid 2203
> (00.000655) Add ipc ns 3 pid 2203
> (00.000666) Add uts ns 4 pid 2203
> (00.000677) Add mnt ns 5 pid 2203
> (00.000688) Add user ns 6 pid 2203
> (00.000700) Add cgroup ns 7 pid 2203
> (00.000705) cg: Dumping cgroups for 2203
> (00.000734) cg:  `- New css ID 1
> (00.000739) cg:     `- [blkio] -> [/] [0]
> (00.000743) cg:     `- [cpu,cpuacct] -> [/] [0]
> (00.000746) cg:     `- [cpuset] -> [/] [0]
> (00.000750) cg:     `- [devices] -> [/system.slice/sshd.service] [0]
> (00.000753) cg:     `- [freezer] -> [/] [0]
> (00.000756) cg:     `- [hugetlb] -> [/] [0]
> (00.000760) cg:     `- [memory] -> [/user.slice/user-0.slice/session-4.scope] [0]
> (00.000763) cg:     `- [name=systemd] -> [/user.slice/user-0.slice/session-4.scope] [0]
> (00.000774) cg:     `- [net_cls,net_prio] -> [/] [0]
> (00.000778) cg:     `- [perf_event] -> [/] [0]
> (00.000781) cg:     `- [pids] -> [/user.slice/user-0.slice/session-4.scope] [0]
> (00.000785) cg: Set 1 is criu one
> (00.001061) Seized task 2160, state 1
> (00.001083) seccomp: Collected tid_real 2160 mode 0
> (00.001200) 	Seizing 2160's 2161 thread
> (00.001336) seccomp: Collected tid_real 2161 mode 0
> (00.001362) Collected (3 attempts, 0 in_progress)
> (00.001389) Collected (4 attempts, 0 in_progress)
> (00.001402) Collected 2160 in 1 state
> (00.001461) Lock network
> (00.001555) 	type sysfs source sysfs mnt_id 19 s_dev 0x13 / @ ./sys flags 0x30000e options
> (00.001572) 	type proc source proc mnt_id 20 s_dev 0x4 / @ ./proc flags 0x30000e options
> (00.001584) 	type devtmpfs source devtmpfs mnt_id 21 s_dev 0x6 / @ ./dev flags 0x1100002 options size=4065312k,nr_inodes=1016328,mode=755
> (00.001595) 	type securityfs source securityfs mnt_id 22 s_dev 0x7 / @ ./sys/kernel/security flags 0x30000e options
> (00.001606) 	type tmpfs source tmpfs mnt_id 23 s_dev 0x14 / @ ./dev/shm flags 0x1100006 options
> (00.001620) 	type devpts source devpts mnt_id 24 s_dev 0x15 / @ ./dev/pts flags 0x30000a options gid=5,mode=620,ptmxmode=000
> (00.001641) 	type tmpfs source tmpfs mnt_id 25 s_dev 0x16 / @ ./run flags 0x1100006 options mode=755
> (00.001655) 	type tmpfs source tmpfs mnt_id 26 s_dev 0x17 / @ ./sys/fs/cgroup flags 0x110000f options mode=755
> (00.001668) 	type cgroup2 source cgroup2 mnt_id 27 s_dev 0x18 / @ ./sys/fs/cgroup/unified flags 0x30000e options nsdelegate
> (00.001680) 	type cgroup source cgroup mnt_id 28 s_dev 0x19 / @ ./sys/fs/cgroup/systemd flags 0x30000e options xattr,name=systemd
> (00.001690) 	type pstore source pstore mnt_id 29 s_dev 0x1a / @ ./sys/fs/pstore flags 0x30000e options
> (00.001739) 	type bpf source bpf mnt_id 30 s_dev 0x1b / @ ./sys/fs/bpf flags 0x30000e options mode=700
> (00.001751) 	type cgroup source cgroup mnt_id 31 s_dev 0x1c / @ ./sys/fs/cgroup/cpuset flags 0x30000e options cpuset
> (00.001762) 	type cgroup source cgroup mnt_id 32 s_dev 0x1d / @ ./sys/fs/cgroup/net_cls,net_prio flags 0x30000e options net_cls,net_prio
> (00.001776) 	type cgroup source cgroup mnt_id 33 s_dev 0x1e / @ ./sys/fs/cgroup/perf_event flags 0x30000e options perf_event
> (00.001788) 	type cgroup source cgroup mnt_id 34 s_dev 0x1f / @ ./sys/fs/cgroup/cpu,cpuacct flags 0x30000e options cpu,cpuacct
> (00.001799) 	type cgroup source cgroup mnt_id 35 s_dev 0x20 / @ ./sys/fs/cgroup/devices flags 0x30000e options devices
> (00.001809) 	type cgroup source cgroup mnt_id 36 s_dev 0x21 / @ ./sys/fs/cgroup/memory flags 0x30000e options memory
> (00.001820) 	type cgroup source cgroup mnt_id 37 s_dev 0x22 / @ ./sys/fs/cgroup/blkio flags 0x30000e options blkio
> (00.001831) 	type cgroup source cgroup mnt_id 38 s_dev 0x23 / @ ./sys/fs/cgroup/freezer flags 0x30000e options freezer
> (00.001845) 	type cgroup source cgroup mnt_id 39 s_dev 0x24 / @ ./sys/fs/cgroup/pids flags 0x30000e options pids
> (00.001892) 	type cgroup source cgroup mnt_id 40 s_dev 0x25 / @ ./sys/fs/cgroup/hugetlb flags 0x30000e options hugetlb
> (00.001904) 	type configfs source configfs mnt_id 87 s_dev 0x26 / @ ./sys/kernel/config flags 0x300000 options
> (00.001917) 	type ext4 source /dev/sda12 mnt_id 91 s_dev 0x80000c / @ ./ flags 0x300000 options
> (00.001930) 	type autofs source systemd-1 mnt_id 18 s_dev 0x12 / @ ./proc/sys/fs/binfmt_misc flags 0x300000 options fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16937
> (00.001943) 	type hugetlbfs source hugetlbfs mnt_id 41 s_dev 0x28 / @ ./dev/hugepages flags 0x300000 options pagesize=2M
> (00.001953) 	type debugfs source debugfs mnt_id 42 s_dev 0x8 / @ ./sys/kernel/debug flags 0x300000 options
> (00.001963) 	type mqueue source mqueue mnt_id 43 s_dev 0x11 / @ ./dev/mqueue flags 0x300000 options
> (00.001973) 	type tmpfs source tmpfs mnt_id 44 s_dev 0x29 / @ ./tmp flags 0x1100006 options
> (00.001982) 	type ext4 source /dev/sda13 mnt_id 107 s_dev 0x80000d / @ ./m/local1 flags 0x300000 options
> (00.001993) 	type rpc_pipefs source sunrpc mnt_id 172 s_dev 0x2a / @ ./var/lib/nfs/rpc_pipefs flags 0x300000 options
> (00.002016) 	type tmpfs source tmpfs mnt_id 242 s_dev 0x2b / @ ./run/user/0 flags 0x300006 options size=815288k,mode=700
> (00.002027) mnt: Building mountpoints tree
> (00.002032) mnt: 	Building plain mount tree
> (00.002035) mnt: 		Working on 242->25
> (00.002039) mnt: 		Working on 172->91
> (00.002042) mnt: 		Working on 107->91
> (00.002046) mnt: 		Working on 44->91
> (00.002049) mnt: 		Working on 43->21
> (00.002052) mnt: 		Working on 42->19
> (00.002056) mnt: 		Working on 41->21
> (00.002059) mnt: 		Working on 18->20
> (00.002063) mnt: 		Working on 91->0
> (00.002066) mnt: 		Working on 87->19
> (00.002069) mnt: 		Working on 40->26
> (00.002073) mnt: 		Working on 39->26
> (00.002076) mnt: 		Working on 38->26
> (00.002080) mnt: 		Working on 37->26
> (00.002083) mnt: 		Working on 36->26
> (00.002086) mnt: 		Working on 35->26
> (00.002090) mnt: 		Working on 34->26
> (00.002093) mnt: 		Working on 33->26
> (00.002096) mnt: 		Working on 32->26
> (00.002100) mnt: 		Working on 31->26
> (00.002103) mnt: 		Working on 30->19
> (00.002107) mnt: 		Working on 29->19
> (00.002110) mnt: 		Working on 28->26
> (00.002113) mnt: 		Working on 27->26
> (00.002117) mnt: 		Working on 26->19
> (00.002127) mnt: 		Working on 25->91
> (00.002131) mnt: 		Working on 24->21
> (00.002134) mnt: 		Working on 23->21
> (00.002138) mnt: 		Working on 22->19
> (00.002141) mnt: 		Working on 21->91
> (00.002144) mnt: 		Working on 20->91
> (00.002148) mnt: 		Working on 19->91
> (00.002151) mnt: 	Resorting siblings on 91
> (00.002155) mnt: 	Resorting siblings on 172
> (00.002158) mnt: 	Resorting siblings on 107
> (00.002161) mnt: 	Resorting siblings on 44
> (00.002165) mnt: 	Resorting siblings on 25
> (00.002168) mnt: 	Resorting siblings on 242
> (00.002171) mnt: 	Resorting siblings on 21
> (00.002175) mnt: 	Resorting siblings on 43
> (00.002178) mnt: 	Resorting siblings on 41
> (00.002182) mnt: 	Resorting siblings on 24
> (00.002185) mnt: 	Resorting siblings on 23
> (00.002189) mnt: 	Resorting siblings on 20
> (00.002192) mnt: 	Resorting siblings on 18
> (00.002196) mnt: 	Resorting siblings on 19
> (00.002199) mnt: 	Resorting siblings on 42
> (00.002202) mnt: 	Resorting siblings on 87
> (00.002206) mnt: 	Resorting siblings on 30
> (00.002209) mnt: 	Resorting siblings on 29
> (00.002212) mnt: 	Resorting siblings on 26
> (00.002216) mnt: 	Resorting siblings on 40
> (00.002219) mnt: 	Resorting siblings on 39
> (00.002222) mnt: 	Resorting siblings on 38
> (00.002226) mnt: 	Resorting siblings on 37
> (00.002229) mnt: 	Resorting siblings on 36
> (00.002233) mnt: 	Resorting siblings on 35
> (00.002236) mnt: 	Resorting siblings on 34
> (00.002240) mnt: 	Resorting siblings on 33
> (00.002245) mnt: 	Resorting siblings on 32
> (00.002248) mnt: 	Resorting siblings on 31
> (00.002252) mnt: 	Resorting siblings on 28
> (00.002256) mnt: 	Resorting siblings on 27
> (00.002259) mnt: 	Resorting siblings on 22
> (00.002263) mnt: Done:
> (00.002266) mnt: [./](91->0)
> (00.002270) mnt:  [./var/lib/nfs/rpc_pipefs](172->91)
> (00.002274) mnt:  <--
> (00.002277) mnt:  [./m/local1](107->91)
> (00.002280) mnt:  <--
> (00.002284) mnt:  [./tmp](44->91)
> (00.002287) mnt:  <--
> (00.002290) mnt:  [./run](25->91)
> (00.002294) mnt:   [./run/user/0](242->25)
> (00.002297) mnt:   <--
> (00.002301) mnt:  <--
> (00.002304) mnt:  [./dev](21->91)
> (00.002307) mnt:   [./dev/mqueue](43->21)
> (00.002311) mnt:   <--
> (00.002314) mnt:   [./dev/hugepages](41->21)
> (00.002318) mnt:   <--
> (00.002321) mnt:   [./dev/pts](24->21)
> (00.002324) mnt:   <--
> (00.002327) mnt:   [./dev/shm](23->21)
> (00.002331) mnt:   <--
> (00.002334) mnt:  <--
> (00.002337) mnt:  [./proc](20->91)
> (00.002341) mnt:   [./proc/sys/fs/binfmt_misc](18->20)
> (00.002344) mnt:   <--
> (00.002347) mnt:  <--
> (00.002350) mnt:  [./sys](19->91)
> (00.002354) mnt:   [./sys/kernel/debug](42->19)
> (00.002358) mnt:   <--
> (00.002361) mnt:   [./sys/kernel/config](87->19)
> (00.002364) mnt:   <--
> (00.002368) mnt:   [./sys/fs/bpf](30->19)
> (00.002371) mnt:   <--
> (00.002374) mnt:   [./sys/fs/pstore](29->19)
> (00.002378) mnt:   <--
> (00.002381) mnt:   [./sys/fs/cgroup](26->19)
> (00.002385) mnt:    [./sys/fs/cgroup/hugetlb](40->26)
> (00.002388) mnt:    <--
> (00.002397) mnt:    [./sys/fs/cgroup/pids](39->26)
> (00.002401) mnt:    <--
> (00.002405) mnt:    [./sys/fs/cgroup/freezer](38->26)
> (00.002408) mnt:    <--
> (00.002412) mnt:    [./sys/fs/cgroup/blkio](37->26)
> (00.002415) mnt:    <--
> (00.002419) mnt:    [./sys/fs/cgroup/memory](36->26)
> (00.002422) mnt:    <--
> (00.002425) mnt:    [./sys/fs/cgroup/devices](35->26)
> (00.002429) mnt:    <--
> (00.002432) mnt:    [./sys/fs/cgroup/cpu,cpuacct](34->26)
> (00.002436) mnt:    <--
> (00.002439) mnt:    [./sys/fs/cgroup/perf_event](33->26)
> (00.002443) mnt:    <--
> (00.002446) mnt:    [./sys/fs/cgroup/net_cls,net_prio](32->26)
> (00.002449) mnt:    <--
> (00.002452) mnt:    [./sys/fs/cgroup/cpuset](31->26)
> (00.002456) mnt:    <--
> (00.002459) mnt:    [./sys/fs/cgroup/systemd](28->26)
> (00.002463) mnt:    <--
> (00.002466) mnt:    [./sys/fs/cgroup/unified](27->26)
> (00.002469) mnt:    <--
> (00.002473) mnt:   <--
> (00.002476) mnt:   [./sys/kernel/security](22->19)
> (00.002479) mnt:   <--
> (00.002483) mnt:  <--
> (00.002486) mnt: <--
> (00.002505) Collecting netns 2/2203
> (00.002648) unix: 	Collected: ino 17427 peer_ino 0 family    1 type    2 state  7 name /run/systemd/notify
> (00.002657) unix: 	Collected: ino 16931 peer_ino 0 family    1 type    5 state 10 name /run/systemd/coredump
> (00.002662) unix: 	Collected: ino 17442 peer_ino 0 family    1 type    1 state 10 name /run/systemd/journal/stdout
> (00.002667) unix: 	Collected: ino 2597 peer_ino 0 family    1 type    1 state 10 name /run/systemd/private
> (00.002672) unix: 	Collected: ino 17446 peer_ino 0 family    1 type    2 state  7 name /run/systemd/journal/socket
> (00.002678) unix: 	Collected: ino 17449 peer_ino 0 family    1 type    2 state  7 name /run/systemd/journal/dev-log
> (00.002683) unix: 	Collected: ino 16963 peer_ino 0 family    1 type    5 state 10 name /run/udev/control
> (00.002688) unix: 	Collected: ino 24133 peer_ino 0 family    1 type    2 state  7 name /run/user/0/systemd/notify
> (00.002693) unix: 	Collected: ino 24137 peer_ino 0 family    1 type    1 state 10 name /run/user/0/systemd/private
> (00.002698) unix: 	Collected: ino 439 peer_ino 0 family    1 type    1 state 10 name
> (00.002703) unix: 	Collected: ino 24143 peer_ino 0 family    1 type    1 state 10 name /run/user/0/bus
> (00.002708) unix: 	Collected: ino 21600 peer_ino 0 family    1 type    1 state 10 name /var/lib/gssproxy/default.sock
> (00.002713) unix: 	Collected: ino 21601 peer_ino 0 family    1 type    1 state 10 name /run/gssproxy.sock
> (00.002718) unix: 	Collected: ino 432 peer_ino 0 family    1 type    1 state 10 name /var/run/.heim_org.h5l.kcm-socket
> (00.002723) unix: 	Collected: ino 434 peer_ino 0 family    1 type    1 state 10 name /run/dbus/system_bus_socket
> (00.002729) unix: 	Collected: ino 438 peer_ino 0 family    1 type    1 state 10 name
> (00.002735) unix: 	Collected: ino 21275 peer_ino 21274 family    1 type    1 state  1 name (null)
> (00.002740) unix: 	Collected: ino 16976 peer_ino 17446 family    1 type    2 state  7 name (null)
> (00.002745) unix: 	Collected: ino 17429 peer_ino 17430 family    1 type    2 state  7 name (null)
> (00.002750) unix: 	Collected: ino 17719 peer_ino 17427 family    1 type    2 state  7 name (null)
> (00.002755) unix: 	Collected: ino 22378 peer_ino 22377 family    1 type    1 state  1 name (null)
> (00.002760) unix: 	Collected: ino 21197 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002765) unix: 	Collected: ino 2020 peer_ino 21575 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.002770) unix: 	Collected: ino 22377 peer_ino 22378 family    1 type    1 state  1 name (null)
> (00.002774) unix: 	Collected: ino 17430 peer_ino 17429 family    1 type    2 state  7 name (null)
> (00.002780) unix: 	Collected: ino 21318 peer_ino 19362 family    1 type    1 state  1 name /run/dbus/system_bus_socket
> (00.002785) unix: 	Collected: ino 19849 peer_ino 19848 family    1 type    2 state  7 name (null)
> (00.002789) unix: 	Collected: ino 21274 peer_ino 21275 family    1 type    1 state  1 name (null)
> (00.002794) unix: 	Collected: ino 22256 peer_ino 3730 family    1 type    1 state  1 name (null)
> (00.002805) unix: 	Collected: ino 779 peer_ino 778 family    1 type    1 state  1 name (null)
> (00.002810) unix: 	Collected: ino 19362 peer_ino 21318 family    1 type    1 state  1 name (null)
> (00.002815) unix: 	Collected: ino 772 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002820) unix: 	Collected: ino 21598 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002824) unix: 	Collected: ino 778 peer_ino 779 family    1 type    1 state  1 name (null)
> (00.002829) unix: 	Collected: ino 21597 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002834) unix: 	Collected: ino 21316 peer_ino 450 family    1 type    1 state  1 name /run/dbus/system_bus_socket
> (00.002839) unix: 	Collected: ino 3730 peer_ino 22256 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.002844) unix: 	Collected: ino 19848 peer_ino 19849 family    1 type    2 state  7 name (null)
> (00.002849) unix: 	Collected: ino 22371 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002853) unix: 	Collected: ino 21575 peer_ino 2020 family    1 type    1 state  1 name (null)
> (00.002858) unix: 	Collected: ino 450 peer_ino 21316 family    1 type    1 state  1 name (null)
> (00.002868) unix: 	Collected: ino 3480 peer_ino 19830 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.002874) unix: 	Collected: ino 22703 peer_ino 0 family    1 type    1 state  1 name (null)
> (00.002879) unix: 	Collected: ino 2879 peer_ino 2880 family    1 type    1 state  1 name (null)
> (00.002883) unix: 	Collected: ino 25116 peer_ino 3772 family    1 type    1 state  1 name (null)
> (00.002888) unix: 	Collected: ino 21071 peer_ino 21072 family    1 type    1 state  1 name (null)
> (00.002893) unix: 	Collected: ino 21072 peer_ino 21071 family    1 type    1 state  1 name (null)
> (00.002898) unix: 	Collected: ino 24139 peer_ino 25880 family    1 type    1 state  1 name (null)
> (00.002902) unix: 	Collected: ino 21314 peer_ino 21315 family    1 type    1 state  1 name (null)
> (00.002912) unix: 	Collected: ino 19830 peer_ino 3480 family    1 type    1 state  1 name (null)
> (00.002918) unix: 	Collected: ino 3772 peer_ino 25116 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.002923) unix: 	Collected: ino 25902 peer_ino 0 family    1 type    1 state  1 name (null)
> (00.002928) unix: 	Collected: ino 24722 peer_ino 24721 family    1 type    1 state  1 name (null)
> (00.002933) unix: 	Collected: ino 647 peer_ino 2847 family    1 type    1 state  1 name /run/dbus/system_bus_socket
> (00.002938) unix: 	Collected: ino 25130 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002942) unix: 	Collected: ino 24908 peer_ino 3745 family    1 type    1 state  1 name (null)
> (00.002947) unix: 	Collected: ino 2849 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002952) unix: 	Collected: ino 19832 peer_ino 17446 family    1 type    2 state  7 name (null)
> (00.002957) unix: 	Collected: ino 2847 peer_ino 647 family    1 type    1 state  1 name (null)
> (00.002961) unix: 	Collected: ino 35960 peer_ino 0 family    1 type    5 state  7 name (null)
> (00.002966) unix: 	Collected: ino 21320 peer_ino 21687 family    1 type    1 state  1 name (null)
> (00.002971) unix: 	Collected: ino 20309 peer_ino 2022 family    1 type    1 state  1 name (null)
> (00.002975) unix: 	Collected: ino 24909 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002980) unix: 	Collected: ino 21070 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.002987) unix: 	Collected: ino 24136 peer_ino 24135 family    1 type    2 state  7 name (null)
> (00.002993) unix: 	Collected: ino 2880 peer_ino 2879 family    1 type    1 state  1 name (null)
> (00.002998) unix: 	Collected: ino 2873 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.003002) unix: 	Collected: ino 25880 peer_ino 24139 family    1 type    1 state  1 name /run/dbus/system_bus_socket
> (00.003008) unix: 	Collected: ino 2022 peer_ino 20309 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.003013) unix: 	Collected: ino 22998 peer_ino 0 family    1 type    1 state  1 name (null)
> (00.003022) unix: 	Collected: ino 21687 peer_ino 21320 family    1 type    1 state  1 name /run/dbus/system_bus_socket
> (00.003028) unix: 	Collected: ino 2008 peer_ino 20247 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.003033) unix: 	Collected: ino 22328 peer_ino 17446 family    1 type    2 state  7 name (null)
> (00.003037) unix: 	Collected: ino 24721 peer_ino 24722 family    1 type    1 state  1 name (null)
> (00.003042) unix: 	Collected: ino 20247 peer_ino 2008 family    1 type    1 state  1 name (null)
> (00.003047) unix: 	Collected: ino 20316 peer_ino 17446 family    1 type    2 state  7 name (null)
> (00.003051) unix: 	Collected: ino 24135 peer_ino 24136 family    1 type    2 state  7 name (null)
> (00.003056) unix: 	Collected: ino 644 peer_ino 17449 family    1 type    2 state  7 name (null)
> (00.003061) unix: 	Collected: ino 3745 peer_ino 24908 family    1 type    1 state  1 name /run/systemd/journal/stdout
> (00.003066) unix: 	Collected: ino 21315 peer_ino 21314 family    1 type    1 state  1 name (null)
> (00.003409) inet: 	Collected: ino   0x5e17 family AF_INET    type SOCK_STREAM    port       22 state TCP_LISTEN       src_addr 0.0.0.0
> (00.003420) inet: 	Collected: ino   0x5771 family AF_INET    type SOCK_STREAM    port     6010 state TCP_LISTEN       src_addr 127.0.0.1
> (00.003426) inet: 	Collected: ino    0xb4f family AF_INET    type SOCK_STREAM    port     6011 state TCP_LISTEN       src_addr 127.0.0.1
> (00.003432) inet: 	Collected: ino    0xf4f family AF_INET    type SOCK_STREAM    port     6012 state TCP_LISTEN       src_addr 127.0.0.1
> (00.003437) inet: 	Collected: ino   0x5e89 family AF_INET    type SOCK_STREAM    port       22 state TCP_ESTABLISHED  src_addr 131.179.116.119
> (00.003443) inet: 	Collected: ino   0x5e61 family AF_INET    type SOCK_STREAM    port       22 state TCP_ESTABLISHED  src_addr 131.179.116.119
> (00.003449) inet: 	Collected: ino   0x6987 family AF_INET    type SOCK_STREAM    port       22 state TCP_ESTABLISHED  src_addr 131.179.116.119
> (00.003485) inet: 	Collected: ino    0x364 family AF_INET    type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr 192.168.116.5
> (00.003492) inet: 	Collected: ino   0x623d family AF_INET    type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr 131.179.116.119
> (00.003498) inet: 	Collected: ino   0x5316 family AF_INET    type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr 127.0.0.1
> (00.003507) inet: 	Collected: ino   0x5312 family AF_INET    type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr 0.0.0.0
> (00.003823) inet: 	Collected: ino   0x5e19 family AF_INET6   type SOCK_STREAM    port       22 state TCP_LISTEN       src_addr ::
> (00.003833) inet: 	Collected: ino   0x5770 family AF_INET6   type SOCK_STREAM    port     6010 state TCP_LISTEN       src_addr ::1
> (00.003839) inet: 	Collected: ino    0xb4e family AF_INET6   type SOCK_STREAM    port     6011 state TCP_LISTEN       src_addr ::1
> (00.003844) inet: 	Collected: ino    0xf4e family AF_INET6   type SOCK_STREAM    port     6012 state TCP_LISTEN       src_addr ::1
> (00.003874) inet: 	Collected: ino    0x369 family AF_INET6   type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr fe80::3aea:a7ff:fe93:b0cd
> (00.003881) inet: 	Collected: ino   0x5d71 family AF_INET6   type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr fe80::2e0:81ff:feb8:8547
> (00.003887) inet: 	Collected: ino   0x588a family AF_INET6   type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr fe80::2e0:81ff:feb8:8547
> (00.003893) inet: 	Collected: ino   0x5318 family AF_INET6   type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr ::1
> (00.003898) inet: 	Collected: ino   0x530d family AF_INET6   type SOCK_DGRAM     port      123 state TCP_CLOSE        src_addr ::
> (00.003964) netlink: Collect netlink sock 0x11
> (00.003971) netlink: Collect netlink sock 0x5319
> (00.003975) netlink: Collect netlink sock 0x5319
> (00.003979) netlink: Collect netlink sock 0x8c77
> (00.003982) netlink: Collect netlink sock 0x4002
> (00.003986) netlink: Collect netlink sock 0x4003
> (00.003996) netlink: Collect netlink sock 0x3ebd
> (00.003999) netlink: Collect netlink sock 0x524c
> (00.004003) netlink: Collect netlink sock 0x12
> (00.004007) netlink: Collect netlink sock 0xa2a
> (00.004011) netlink: Collect netlink sock 0x4436
> (00.004014) netlink: Collect netlink sock 0x4436
> (00.004018) netlink: Collect netlink sock 0x3698
> (00.004021) netlink: Collect netlink sock 0xc01
> (00.004025) netlink: Collect netlink sock 0x4bcc
> (00.004028) netlink: Collect netlink sock 0x801
> (00.004032) netlink: Collect netlink sock 0xa24
> (00.004036) netlink: Collect netlink sock 0x4246
> (00.004040) netlink: Collect netlink sock 0x4ba0
> (00.004044) netlink: Collect netlink sock 0x5e3e
> (00.004048) netlink: Collect netlink sock 0x4b9e
> (00.004051) netlink: Collect netlink sock 0x4b9f
> (00.004055) netlink: Collect netlink sock 0x4ba1
> (00.004058) netlink: Collect netlink sock 0x5e3e
> (00.004062) netlink: Collect netlink sock 0x4ba1
> (00.004066) netlink: Collect netlink sock 0x4ba0
> (00.004070) netlink: Collect netlink sock 0x4b9f
> (00.004074) netlink: Collect netlink sock 0x4b9e
> (00.004077) netlink: Collect netlink sock 0x4246
> (00.004081) netlink: Collect netlink sock 0xa24
> (00.004084) netlink: Collect netlink sock 0xc0a
> (00.004088) netlink: Collect netlink sock 0xc04
> (00.004169) ========================================
> (00.004175) Dumping task (pid: 2160)
> (00.004179) ========================================
> (00.004182) Obtaining task stat ...
> (00.004216)
> (00.004221) Collecting mappings (pid: 2160)
> (00.004224) ----------------------------------------
> (00.004350) Dumping path for -3 fd via self 12 [/root/container/rootfs/fstress1]
> (00.004420) vma 401000 borrows vfi from previous 400000
> (00.004436) vma 402000 borrows vfi from previous 401000
> (00.004448) vma 403000 borrows vfi from previous 402000
> (00.004460) vma 404000 borrows vfi from previous 403000
> (00.004662) Dumping path for -3 fd via self 12 [/usr/lib64/libc-2.28.so]
> (00.004684) vma 7facd13d8000 borrows vfi from previous 7facd13b6000
> (00.004697) vma 7facd1525000 borrows vfi from previous 7facd13d8000
> (00.004709) vma 7facd1571000 borrows vfi from previous 7facd1525000
> (00.004749) vma 7facd1572000 borrows vfi from previous 7facd1571000
> (00.004766) vma 7facd1576000 borrows vfi from previous 7facd1572000
> (00.004817) Dumping path for -3 fd via self 12 [/usr/lib64/libpthread-2.28.so]
> (00.004836) vma 7facd1582000 borrows vfi from previous 7facd157c000
> (00.004849) vma 7facd1592000 borrows vfi from previous 7facd1582000
> (00.004892) vma 7facd1598000 borrows vfi from previous 7facd1592000
> (00.004905) vma 7facd1599000 borrows vfi from previous 7facd1598000
> (00.004955) Dumping path for -3 fd via self 12 [/usr/lib64/libm-2.28.so]
> (00.004974) vma 7facd15ab000 borrows vfi from previous 7facd159e000
> (00.004986) vma 7facd164b000 borrows vfi from previous 7facd15ab000
> (00.005028) vma 7facd1720000 borrows vfi from previous 7facd164b000
> (00.005041) vma 7facd1721000 borrows vfi from previous 7facd1720000
> (00.005090) Dumping path for -3 fd via self 12 [/usr/lib64/ld-2.28.so]
> (00.005109) vma 7facd1736000 borrows vfi from previous 7facd1735000
> (00.005136) vma 7facd1756000 borrows vfi from previous 7facd1736000
> (00.005174) vma 7facd175f000 borrows vfi from previous 7facd1756000
> (00.005187) vma 7facd1760000 borrows vfi from previous 7facd175f000
> (00.005252) Collected, longest area occupies 4005 pages
> (00.005259) 0x400000-0x401000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x1
> (00.005264) 0x401000-0x402000 (4K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x1000 reg fp  shmid: 0x1
> (00.005269) 0x402000-0x403000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x2000 reg fp  shmid: 0x1
> (00.005273) 0x403000-0x404000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x2000 reg fp  shmid: 0x1
> (00.005278) 0x404000-0x405000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x3000 reg fp  shmid: 0x1
> (00.005282) 0xe45000-0xe66000 (132K) prot 0x3 flags 0x22 fdflags 0 st 0x221 off 0 reg heap ap  shmid: 0
> (00.005287) 0x7facd0410000-0x7facd0411000 (4K) prot 0 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005297) 0x7facd0411000-0x7facd13b6000 (16020K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005302) 0x7facd13b6000-0x7facd13d8000 (136K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x2
> (00.005307) 0x7facd13d8000-0x7facd1525000 (1332K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x22000 reg fp  shmid: 0x2
> (00.005311) 0x7facd1525000-0x7facd1571000 (304K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x16f000 reg fp  shmid: 0x2
> (00.005316) 0x7facd1571000-0x7facd1572000 (4K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x1bb000 reg fp  shmid: 0x2
> (00.005320) 0x7facd1572000-0x7facd1576000 (16K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x1bb000 reg fp  shmid: 0x2
> (00.005325) 0x7facd1576000-0x7facd1578000 (8K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1bf000 reg fp  shmid: 0x2
> (00.005329) 0x7facd1578000-0x7facd157c000 (16K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005333) 0x7facd157c000-0x7facd1582000 (24K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x3
> (00.005338) 0x7facd1582000-0x7facd1592000 (64K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x6000 reg fp  shmid: 0x3
> (00.005342) 0x7facd1592000-0x7facd1598000 (24K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x16000 reg fp  shmid: 0x3
> (00.005346) 0x7facd1598000-0x7facd1599000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x1b000 reg fp  shmid: 0x3
> (00.005351) 0x7facd1599000-0x7facd159a000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1c000 reg fp  shmid: 0x3
> (00.005355) 0x7facd159a000-0x7facd159e000 (16K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005359) 0x7facd159e000-0x7facd15ab000 (52K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x4
> (00.005364) 0x7facd15ab000-0x7facd164b000 (640K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0xd000 reg fp  shmid: 0x4
> (00.005368) 0x7facd164b000-0x7facd1720000 (852K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0xad000 reg fp  shmid: 0x4
> (00.005372) 0x7facd1720000-0x7facd1721000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x181000 reg fp  shmid: 0x4
> (00.005377) 0x7facd1721000-0x7facd1722000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x182000 reg fp  shmid: 0x4
> (00.005381) 0x7facd1722000-0x7facd1724000 (8K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005386) 0x7facd1735000-0x7facd1736000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x5
> (00.005390) 0x7facd1736000-0x7facd1756000 (128K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x1000 reg fp  shmid: 0x5
> (00.005394) 0x7facd1756000-0x7facd175e000 (32K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x21000 reg fp  shmid: 0x5
> (00.005399) 0x7facd175f000-0x7facd1760000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x29000 reg fp  shmid: 0x5
> (00.005403) 0x7facd1760000-0x7facd1761000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x2a000 reg fp  shmid: 0x5
> (00.005407) 0x7facd1761000-0x7facd1762000 (4K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005412) 0x7ffc028f4000-0x7ffc02915000 (132K) prot 0x3 flags 0x122 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.005416) 0x7ffc029af000-0x7ffc029b2000 (12K) prot 0x1 flags 0x22 fdflags 0 st 0x1201 off 0 reg vvar ap  shmid: 0
> (00.005421) 0x7ffc029b2000-0x7ffc029b4000 (8K) prot 0x5 flags 0x22 fdflags 0 st 0x209 off 0 reg vdso ap  shmid: 0
> (00.005425) 0xffffffffff600000-0xffffffffff601000 (4K) prot 0x5 flags 0x22 fdflags 0 st 0x204 off 0 vsys ap  shmid: 0
> (00.005429) ----------------------------------------
> (00.005433)
> (00.005436) Collecting fds (pid: 2160)
> (00.005439) ----------------------------------------
> (00.005461) Found 3 file descriptors
> (00.005466) ----------------------------------------
> (00.005479) Dump private signals of 2160
> (00.005489) Dump private signals of 2161
> (00.005497) Dump shared signals of 2160
> (00.005514) Parasite syscall_ip at 0x401000
> (00.006133) Set up parasite blob using memfd
> (00.006145) Putting parasite blob into 0x7fda8de3e000->0x7facd03ed000
> (00.006342) Dumping general registers for 2160 in native mode
> (00.006356) Dumping GP/FPU registers for 2160
> (00.006363) ZoZo: 1
> (00.006386) x86: xsave runtime structure
> (00.006402) x86: -----------------------
> (00.006408) x86: cwd:0 swd:0 twd:0 fop:0 mxcsr:0 mxcsr_mask:0
> (00.006413) x86: magic1:0 extended_size:0 xstate_bv:0 xstate_size:0
> (00.006419) x86: xstate_bv: 0
> (00.006423) x86: -----------------------
> (00.006429) Putting tsock into pid 2160
> (00.006662) Wait for parasite being daemonized...
> (00.006674) Wait for ack 2 on daemon socket
> pie: 2160: Running daemon thread leader
> pie: 2160: __sent ack msg: 2 2 0
> pie: 2160: Daemon waits for command
> (00.006809) Fetched ack: 2 2 0
> (00.006821) Parasite 2160 has been switched to daemon mode
> (00.006829) vdso: vDSO hint is reliable - omit checking
> (00.006849) Sent msg to daemon 74 0 0
> (00.006858) Wait for ack 74 on daemon socket
> pie: 2160: __fetched msg: 74 0 0
> pie: 2160: __sent ack msg: 74 74 0
> pie: 2160: Daemon waits for command
> (00.006980) Fetched ack: 74 74 0
> (00.006999) Sent msg to daemon 70 0 0
> (00.007009) Wait for ack 70 on daemon socket
> pie: 2160: __fetched msg: 70 0 0
> pie: 2160: __sent ack msg: 70 70 0
> (00.007090) Fetched ack: 70 70 0
> pie: 2160: Daemon waits for command
> (00.007099) sid=1094 pgid=2160 pid=2160
> (00.007149)
> (00.007155) Dumping opened files (pid: 2160)
> (00.007158) ----------------------------------------
> (00.007169) Sent msg to daemon 71 0 0
> pie: 2160: __fetched msg: 71 0 0
> pie: 2160: __sent ack msg: 71 71 0
> pie: 2160: Daemon waits for command
> (00.007314) Wait for ack 71 on daemon socket
> (00.007328) Fetched ack: 71 71 0
> (00.007361) 2160 fdinfo 0: pos:                0 flags:                2/0
> (00.007388) tty: Dumping tty 12 with id 0x6
> (00.007398) Dumping path for 0 fd via self 12 [/dev/pts/0]
> (00.007412) Sent msg to daemon 73 0 0
> pie: 2160: __fetched msg: 73 0 0
> (00.007417) Wait for ack 73 on daemon socket
> pie: 2160: __sent ack msg: 73 73 0
> pie: 2160: Daemon waits for command
> (00.007446) Fetched ack: 73 73 0
> (00.007554) 2160 fdinfo 1: pos:                0 flags:                2/0
> (00.007588) 2160 fdinfo 2: pos:                0 flags:                2/0
> (00.007607) ----------------------------------------
> (00.007707) Sent msg to daemon 65 0 0
> pie: 2160: __fetched msg: 65 0 0
> (00.007714) Wait for ack 65 on daemon socket
> pie: 2160: __sent ack msg: 65 65 0
> pie: 2160: Daemon waits for command
> (00.007742) Fetched ack: 65 65 0
> (00.007751)
> (00.007756) Dumping pages (type: 52 pid: 2160)
> (00.007762) ----------------------------------------
> (00.007766)    Private vmas 4005/5004 pages
> (00.007785) pagemap-cache: created for pid 2160 (takes 32040 bytes)
> (00.007793) page-pipe: Create page pipe for 5004 segs
> (00.007825) page-pipe: Will grow page pipe (iov off is 0)
> (00.009142) pagemap-cache: 2160: filling VMA 400000-401000 (4K) [l:400000 h:600000]
> (00.009151) pagemap-cache: 	2160:           400000-401000           nr:1     cov:4096
> (00.009156) pagemap-cache: 	2160:           401000-402000           nr:2     cov:8192
> (00.009160) pagemap-cache: 	2160:           402000-403000           nr:3     cov:12288
> (00.009164) pagemap-cache: 	2160:           403000-404000           nr:4     cov:16384
> (00.009168) pagemap-cache: 	2160:           404000-405000           nr:5     cov:20480
> (00.009172) pagemap-cache: 	2160: cache  mode [l:400000 h:600000]
> (00.009185) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009190) page-pipe: Add iov to page pipe (0 iovs, 0/5004 total)
> (00.009194) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009198) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009202) page-pipe: Add iov to page pipe (1 iovs, 1/5004 total)
> (00.009205) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009209) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009212) pagemap-cache: 2160: filling VMA e45000-e66000 (132K) [l:e00000 h:1000000]
> (00.009216) pagemap-cache: 	2160:           e45000-e66000           nr:1     cov:135168
> (00.009220) pagemap-cache: 	2160: simple mode [l:e45000 h:e66000]
> (00.009227) page-pipe: Will grow page pipe (iov off is 2)
> (00.009235) page-pipe: Add iov to page pipe (0 iovs, 2/5004 total)
> (00.009240) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009244) pagemap-cache: 2160: filling VMA 7facd0410000-7facd0411000 (4K) [l:7facd0400000 h:7facd0600000]
> (00.009254) pagemap-cache: 	2160:     7facd0410000-7facd0411000     nr:1     cov:4096
> (00.009258) pagemap-cache: 	2160: simple mode [l:7facd0410000 h:7facd0411000]
> (00.009267) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009271) pagemap-cache: 2160: filling VMA 7facd0411000-7facd13b6000 (16020K) [l:7facd0400000 h:7facd0600000]
> (00.009356) page-pipe: Add iov to page pipe (1 iovs, 3/5004 total)
> (00.009367) page-pipe: Grow pipe 10 -> 20
> (00.009372) page-pipe: Grow pipe 20 -> 40
> (00.009379) page-pipe: Grow pipe 40 -> 80
> (00.009389) page-pipe: Grow pipe 80 -> 100
> (00.009403) page-pipe: Grow pipe 100 -> 200
> (00.009412) page-pipe: Will grow page pipe (iov off is 4)
> (00.009419) page-pipe: Add iov to page pipe (0 iovs, 4/5004 total)
> (00.009425) page-pipe: Grow pipe 10 -> 20
> (00.009430) page-pipe: Grow pipe 20 -> 40
> (00.009436) page-pipe: Grow pipe 40 -> 80
> (00.009443) page-pipe: Grow pipe 80 -> 100
> (00.009456) page-pipe: Grow pipe 100 -> 200
> (00.009465) page-pipe: Will grow page pipe (iov off is 5)
> (00.009472) page-pipe: Add iov to page pipe (0 iovs, 5/5004 total)
> (00.009477) page-pipe: Grow pipe 10 -> 20
> (00.009482) page-pipe: Grow pipe 20 -> 40
> (00.009488) page-pipe: Grow pipe 40 -> 80
> (00.009495) page-pipe: Grow pipe 80 -> 100
> (00.009509) page-pipe: Grow pipe 100 -> 200
> (00.009518) page-pipe: Will grow page pipe (iov off is 6)
> (00.009525) page-pipe: Add iov to page pipe (0 iovs, 6/5004 total)
> (00.009530) page-pipe: Grow pipe 10 -> 20
> (00.009535) page-pipe: Grow pipe 20 -> 40
> (00.009541) page-pipe: Grow pipe 40 -> 80
> (00.009548) page-pipe: Grow pipe 80 -> 100
> (00.009562) page-pipe: Grow pipe 100 -> 200
> (00.009569) Pagemap generated: 1958 pages (0 lazy) 0 holes
> (00.009573) pagemap-cache: 2160: filling VMA 7facd13b6000-7facd13d8000 (136K) [l:7facd1200000 h:7facd1400000]
> (00.009581) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009585) pagemap-cache: 2160: filling VMA 7facd13d8000-7facd1525000 (1332K) [l:7facd1200000 h:7facd1400000]
> (00.009602) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009606) pagemap-cache: 2160: filling VMA 7facd1525000-7facd1571000 (304K) [l:7facd1400000 h:7facd1600000]
> (00.009616) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009620) pagemap-cache: 2160: filling VMA 7facd1571000-7facd1572000 (4K) [l:7facd1400000 h:7facd1600000]
> (00.009629) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009633) pagemap-cache: 2160: filling VMA 7facd1572000-7facd1576000 (16K) [l:7facd1400000 h:7facd1600000]
> (00.009642) page-pipe: Will grow page pipe (iov off is 7)
> (00.009649) page-pipe: Add iov to page pipe (0 iovs, 7/5004 total)
> (00.009653) Pagemap generated: 4 pages (0 lazy) 0 holes
> (00.009657) pagemap-cache: 2160: filling VMA 7facd1576000-7facd1578000 (8K) [l:7facd1400000 h:7facd1600000]
> (00.009666) Pagemap generated: 2 pages (0 lazy) 0 holes
> (00.009670) pagemap-cache: 2160: filling VMA 7facd1578000-7facd157c000 (16K) [l:7facd1400000 h:7facd1600000]
> (00.009678) page-pipe: Will grow page pipe (iov off is 8)
> (00.009686) page-pipe: Add iov to page pipe (0 iovs, 8/5004 total)
> (00.009690) page-pipe: Add iov to page pipe (1 iovs, 9/5004 total)
> (00.009693) Pagemap generated: 3 pages (0 lazy) 0 holes
> (00.009697) pagemap-cache: 2160: filling VMA 7facd157c000-7facd1582000 (24K) [l:7facd1400000 h:7facd1600000]
> (00.009705) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009709) pagemap-cache: 2160: filling VMA 7facd1582000-7facd1592000 (64K) [l:7facd1400000 h:7facd1600000]
> (00.009717) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009721) pagemap-cache: 2160: filling VMA 7facd1592000-7facd1598000 (24K) [l:7facd1400000 h:7facd1600000]
> (00.009729) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009732) pagemap-cache: 2160: filling VMA 7facd1598000-7facd1599000 (4K) [l:7facd1400000 h:7facd1600000]
> (00.009740) page-pipe: Will grow page pipe (iov off is 10)
> (00.009747) page-pipe: Add iov to page pipe (0 iovs, 10/5004 total)
> (00.009751) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009754) pagemap-cache: 2160: filling VMA 7facd1599000-7facd159a000 (4K) [l:7facd1400000 h:7facd1600000]
> (00.009768) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.009772) pagemap-cache: 2160: filling VMA 7facd159a000-7facd159e000 (16K) [l:7facd1400000 h:7facd1600000]
> (00.009780) page-pipe: Will grow page pipe (iov off is 11)
> (00.009784) Pagemap full
> (00.009787) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.009791) page-pipe: Page pipe:
> (00.009794) page-pipe: * 8 pipes 11/5004 iovs:
> (00.009798) page-pipe: 	buf 3 pages, 2 iovs, flags: 0 pipe_off: 0 :
> (00.009802) page-pipe: 		0x401000 1
> (00.009805) page-pipe: 		0x403000 2
> (00.009809) page-pipe: 	buf 512 pages, 2 iovs, flags: 1 pipe_off: 0 :
> (00.009813) page-pipe: 		0xe45000 1
> (00.009816) page-pipe: 		0x7facd0c0f000 511
> (00.009819) page-pipe: 	buf 512 pages, 1 iovs, flags: 1 pipe_off: 0 :
> (00.009823) page-pipe: 		0x7facd0e0e000 512
> (00.009826) page-pipe: 	buf 512 pages, 1 iovs, flags: 1 pipe_off: 0 :
> (00.009830) page-pipe: 		0x7facd100e000 512
> (00.009833) page-pipe: 	buf 423 pages, 1 iovs, flags: 1 pipe_off: 0 :
> (00.009836) page-pipe: 		0x7facd120e000 423
> (00.009840) page-pipe: 	buf 6 pages, 1 iovs, flags: 0 pipe_off: 0 :
> (00.009843) page-pipe: 		0x7facd1572000 6
> (00.009847) page-pipe: 	buf 3 pages, 2 iovs, flags: 1 pipe_off: 0 :
> (00.009850) page-pipe: 		0x7facd1578000 2
> (00.009853) page-pipe: 		0x7facd157b000 1
> (00.009857) page-pipe: 	buf 2 pages, 1 iovs, flags: 0 pipe_off: 0 :
> (00.009860) page-pipe: 		0x7facd1598000 2
> (00.009863) page-pipe: * 0 holes:
> (00.009867) PPB: 3 pages 2 segs 16 pipe 0 off
> (00.009877) Sent msg to daemon 66 0 0
> (00.009885) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> (00.009975) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.009981) PPB: 512 pages 2 segs 512 pipe 2 off
> (00.009990) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.009998) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> (00.010053) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.010058) PPB: 512 pages 1 segs 512 pipe 4 off
> (00.010066) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.010073) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.010281) Fetched ack: 66 66 0
> (00.010289) PPB: 512 pages 1 segs 512 pipe 5 off
> (00.010300) Sent msg to daemon 66 0 0
> (00.010307) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.010474) Fetched ack: 66 66 0
> (00.010485) PPB: 423 pages 1 segs 512 pipe 6 off
> (00.010499) Sent msg to daemon 66 0 0
> (00.010514) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.010720) Fetched ack: 66 66 0
> (00.010731) PPB: 6 pages 1 segs 16 pipe 7 off
> (00.010745) Sent msg to daemon 66 0 0
> (00.010761) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.010891) Fetched ack: 66 66 0
> (00.010902) PPB: 3 pages 2 segs 16 pipe 8 off
> (00.010916) Sent msg to daemon 66 0 0
> (00.010932) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> (00.011031) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.011040) PPB: 2 pages 1 segs 16 pipe 10 off
> (00.011053) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.011064) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> (00.011079) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.011084) page-xfer: Transferring pages:
> (00.011088) page-xfer: 	buf 3/2
> (00.011091) page-xfer: 	p 0x401000 [1]
> (00.011115) page-xfer: 	p 0x403000 [2]
> (00.011137) page-xfer: 	buf 512/2
> (00.011142) page-xfer: 	p 0xe45000 [1]
> (00.011153) page-xfer: 	p 0x7facd0c0f000 [511]
> (00.012991) page-xfer: 	buf 512/1
> (00.013003) page-xfer: 	p 0x7facd0e0e000 [512]
> (00.015031) page-xfer: 	buf 512/1
> (00.015040) page-xfer: 	p 0x7facd100e000 [512]
> (00.017109) page-xfer: 	buf 423/1
> (00.017132) page-xfer: 	p 0x7facd120e000 [423]
> (00.018838) page-xfer: 	buf 6/1
> (00.018846) page-xfer: 	p 0x7facd1572000 [6]
> (00.018880) page-xfer: 	buf 3/2
> (00.018884) page-xfer: 	p 0x7facd1578000 [2]
> (00.018900) page-xfer: 	p 0x7facd157b000 [1]
> (00.018911) page-xfer: 	buf 2/1
> (00.018915) page-xfer: 	p 0x7facd1598000 [2]
> (00.018933) page-pipe: Clean up page pipe
> (00.018939) page-pipe: Will grow page pipe (iov off is 11)
> (00.018944) page-pipe: Will grow page pipe (iov off is 11)
> (00.018947) page-pipe: Add iov to page pipe (0 iovs, 11/5004 total)
> (00.018951) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.018956) pagemap-cache: 2160: filling VMA 7facd159e000-7facd15ab000 (52K) [l:7facd1400000 h:7facd1600000]
> (00.018971) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.018976) pagemap-cache: 2160: filling VMA 7facd15ab000-7facd164b000 (640K) [l:7facd1400000 h:7facd1600000]
> (00.018992) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.018996) pagemap-cache: 2160: filling VMA 7facd164b000-7facd1720000 (852K) [l:7facd1600000 h:7facd1800000]
> (00.019000) pagemap-cache: 	2160:     7facd164b000-7facd1720000     nr:1     cov:872448
> (00.019005) pagemap-cache: 	2160:     7facd1720000-7facd1721000     nr:2     cov:876544
> (00.019009) pagemap-cache: 	2160:     7facd1721000-7facd1722000     nr:3     cov:880640
> (00.019013) pagemap-cache: 	2160:     7facd1722000-7facd1724000     nr:4     cov:888832
> (00.019017) pagemap-cache: 	2160:     7facd1735000-7facd1736000     nr:5     cov:892928
> (00.019021) pagemap-cache: 	2160:     7facd1736000-7facd1756000     nr:6     cov:1024000
> (00.019025) pagemap-cache: 	2160:     7facd1756000-7facd175e000     nr:7     cov:1056768
> (00.019029) pagemap-cache: 	2160:     7facd175f000-7facd1760000     nr:8     cov:1060864
> (00.019033) pagemap-cache: 	2160:     7facd1760000-7facd1761000     nr:9     cov:1064960
> (00.019037) pagemap-cache: 	2160:     7facd1761000-7facd1762000     nr:10    cov:1069056
> (00.019042) pagemap-cache: 	2160: cache  mode [l:7facd164b000 h:7facd1800000]
> (00.019052) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.019056) page-pipe: Will grow page pipe (iov off is 12)
> (00.019060) page-pipe: Add iov to page pipe (0 iovs, 12/5004 total)
> (00.019063) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.019067) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.019071) page-pipe: Will grow page pipe (iov off is 13)
> (00.019074) page-pipe: Add iov to page pipe (0 iovs, 13/5004 total)
> (00.019078) Pagemap generated: 2 pages (0 lazy) 0 holes
> (00.019082) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.019085) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.019089) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.019092) page-pipe: Will grow page pipe (iov off is 14)
> (00.019096) page-pipe: Add iov to page pipe (0 iovs, 14/5004 total)
> (00.019099) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.019103) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.019107) page-pipe: Will grow page pipe (iov off is 15)
> (00.019110) page-pipe: Add iov to page pipe (0 iovs, 15/5004 total)
> (00.019113) Pagemap generated: 1 pages (0 lazy) 0 holes
> (00.019117) pagemap-cache: 2160: filling VMA 7ffc028f4000-7ffc02915000 (132K) [l:7ffc02800000 h:7ffc02a00000]
> (00.019134) page-pipe: Add iov to page pipe (1 iovs, 16/5004 total)
> (00.019138) Pagemap generated: 2 pages (0 lazy) 0 holes
> (00.019142) pagemap-cache: 2160: filling VMA 7ffc029af000-7ffc029b2000 (12K) [l:7ffc02800000 h:7ffc02a00000]
> (00.019148) Pagemap generated: 0 pages (0 lazy) 0 holes
> (00.019151) pagemap-cache: 2160: filling VMA 7ffc029b2000-7ffc029b4000 (8K) [l:7ffc02800000 h:7ffc02a00000]
> (00.019157) page-pipe: Will grow page pipe (iov off is 17)
> (00.019161) page-pipe: Add iov to page pipe (0 iovs, 17/5004 total)
> (00.019165) Pagemap generated: 2 pages (0 lazy) 0 holes
> (00.019169) page-pipe: Page pipe:
> (00.019172) page-pipe: * 8 pipes 18/5004 iovs:
> (00.019176) page-pipe: 	buf 0 pages, 0 iovs, flags: 0 pipe_off: 0 :
> (00.019179) page-pipe: 	buf 1 pages, 1 iovs, flags: 1 pipe_off: 0 :
> (00.019183) page-pipe: 		0x7facd159d000 1
> (00.019187) page-pipe: 	buf 2 pages, 1 iovs, flags: 0 pipe_off: 0 :
> (00.019190) page-pipe: 		0x7facd1720000 2
> (00.019199) page-pipe: 	buf 2 pages, 1 iovs, flags: 1 pipe_off: 0 :
> (00.019203) page-pipe: 		0x7facd1722000 2
> (00.019207) page-pipe: 	buf 2 pages, 1 iovs, flags: 0 pipe_off: 0 :
> (00.019210) page-pipe: 		0x7facd175f000 2
> (00.019214) page-pipe: 	buf 3 pages, 2 iovs, flags: 1 pipe_off: 0 :
> (00.019217) page-pipe: 		0x7facd1761000 1
> (00.019221) page-pipe: 		0x7ffc02913000 2
> (00.019224) page-pipe: 	buf 2 pages, 1 iovs, flags: 0 pipe_off: 0 :
> (00.019228) page-pipe: 		0x7ffc029b2000 2
> (00.019231) page-pipe: * 0 holes:
> (00.019235) PPB: 0 pages 0 segs 16 pipe 11 off
> (00.019257) Sent msg to daemon 66 0 0
> (00.019271) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019437) Fetched ack: 66 66 0
> (00.019451) PPB: 1 pages 1 segs 16 pipe 11 off
> (00.019469) Sent msg to daemon 66 0 0
> (00.019486) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019644) Fetched ack: 66 66 0
> (00.019659) PPB: 2 pages 1 segs 16 pipe 12 off
> (00.019678) Sent msg to daemon 66 0 0
> (00.019694) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019845) Fetched ack: 66 66 0
> (00.019860) PPB: 2 pages 1 segs 512 pipe 13 off
> (00.019878) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.019895) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> (00.019906) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019912) PPB: 2 pages 1 segs 512 pipe 14 off
> (00.019925) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.019933) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> (00.019939) Fetched ack: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019944) PPB: 3 pages 2 segs 512 pipe 15 off
> (00.019951) Sent msg to daemon 66 0 0
> pie: 2160: __fetched msg: 66 0 0
> (00.019958) Wait for ack 66 on daemon socket
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.019974) Fetched ack: 66 66 0
> (00.019979) PPB: 2 pages 1 segs 512 pipe 17 off
> (00.019987) Sent msg to daemon 66 0 0
> (00.019994) Wait for ack 66 on daemon socket
> pie: 2160: __fetched msg: 66 0 0
> pie: 2160: __sent ack msg: 66 66 0
> pie: 2160: Daemon waits for command
> (00.020069) Fetched ack: 66 66 0
> (00.020084) page-xfer: Transferring pages:
> (00.020092) page-xfer: 	buf 0/0
> (00.020099) page-xfer: 	buf 1/1
> (00.020106) page-xfer: 	p 0x7facd159d000 [1]
> (00.020150) page-xfer: 	buf 2/1
> (00.020159) page-xfer: 	p 0x7facd1720000 [2]
> (00.020181) page-xfer: 	buf 2/1
> (00.020187) page-xfer: 	p 0x7facd1722000 [2]
> (00.020205) page-xfer: 	buf 2/1
> (00.020209) page-xfer: 	p 0x7facd175f000 [2]
> (00.020224) page-xfer: 	buf 3/2
> (00.020228) page-xfer: 	p 0x7facd1761000 [1]
> (00.020239) page-xfer: 	p 0x7ffc02913000 [2]
> (00.020254) page-xfer: 	buf 2/1
> (00.020258) page-xfer: 	p 0x7ffc029b2000 [2]
> (00.021499) page-pipe: Killing page pipe
> (00.021556) ----------------------------------------
> (00.021569) Sent msg to daemon 65 0 0
> (00.021574) Wait for ack 65 on daemon socket
> pie: 2160: __fetched msg: 65 0 0
> pie: 2160: __sent ack msg: 65 65 0
> pie: 2160: Daemon waits for command
> (00.021740) Fetched ack: 65 65 0
> (00.021761) Sent msg to daemon 67 0 0
> (00.021771) Wait for ack 67 on daemon socket
> pie: 2160: __fetched msg: 67 0 0
> pie: 2160: __sent ack msg: 67 67 0
> pie: 2160: Daemon waits for command
> (00.021978) Fetched ack: 67 67 0
> (00.022011) Sent msg to daemon 68 0 0
> (00.022022) Wait for ack 68 on daemon socket
> pie: 2160: __fetched msg: 68 0 0
> pie: 2160: __sent ack msg: 68 68 0
> (00.022111) Fetched ack: 68 68 0
> pie: 2160: Daemon waits for command
> (00.022172) Sent msg to daemon 69 0 0
> pie: 2160: __fetched msg: 69 0 0
> (00.022177) Wait for ack 69 on daemon socket
> pie: 2160: __sent ack msg: 69 69 0
> (00.022183) Fetched ack: 69 69 0
> pie: 2160: Daemon waits for command
> (00.022188)
> (00.022192) Dumping core (pid: 2160)
> (00.022195) ----------------------------------------
> (00.022199) Obtaining personality ...
> (00.022236) Sent msg to daemon 64 0 0
> (00.022248) Wait for ack 64 on daemon socket
> pie: 2160: __fetched msg: 64 0 0
> pie: 2160: __sent ack msg: 64 64 0
> pie: 2160: Daemon waits for command
> (00.022447) Fetched ack: 64 64 0
> (00.022464) 2160 has 0 sched policy
> (00.022484) 	dumping 0 nice for 2160
> (00.022494) dumping /proc/2160/loginuid
> (00.022513) dumping /proc/2160/oom_score_adj
> (00.022551) Sent msg to daemon 76 0 0
> (00.022558) Wait for ack 76 on daemon socket
> pie: 2160: __fetched msg: 76 0 0
> pie: 2160: __sent ack msg: 76 76 0
> pie: 2160: Daemon waits for command
> (00.022749) Fetched ack: 76 76 0
> (00.022762) cg: Dumping cgroups for 2160
> (00.022854) cg:  `- New css ID 2
> (00.022864) cg:     `- [blkio] -> [/] [0]
> (00.022870) cg:     `- [cpu,cpuacct] -> [/] [0]
> (00.022875) cg:     `- [cpuset] -> [/] [0]
> (00.022882) cg:     `- [devices] -> [/system.slice/sshd.service] [0]
> (00.022886) cg:     `- [freezer] -> [/] [0]
> (00.022889) cg:     `- [hugetlb] -> [/] [0]
> (00.022893) cg:     `- [memory] -> [/user.slice/user-0.slice/session-3.scope] [0]
> (00.022896) cg:     `- [name=systemd] -> [/user.slice/user-0.slice/session-3.scope] [0]
> (00.022900) cg:     `- [net_cls,net_prio] -> [/] [0]
> (00.022903) cg:     `- [perf_event] -> [/] [0]
> (00.022907) cg:     `- [pids] -> [/user.slice/user-0.slice/session-3.scope] [0]
> (00.030449) cg: adding cgroup /proc/self/fd/12
> (00.030486) cg: Dumping value 1000 from /proc/self/fd/12/blkio.weight
> (00.030508) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.030529) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.030547) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.030564) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.038420) cg: adding cgroup /proc/self/fd/12
> (00.038460) cg: Dumping value 1024 from /proc/self/fd/12/cpu.shares
> (00.038481) cg: Dumping value 100000 from /proc/self/fd/12/cpu.cfs_period_us
> (00.038501) cg: Dumping value -1 from /proc/self/fd/12/cpu.cfs_quota_us
> (00.038513) cg: Couldn't open /proc/self/fd/12/cpu.rt_period_us. This cgroup property may not exist on this kernel
> (00.038523) cg: Couldn't open /proc/self/fd/12/cpu.rt_runtime_us. This cgroup property may not exist on this kernel
> (00.038543) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.038563) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.038581) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.038599) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.038618) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.038638) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.038655) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.038672) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.046414) cg: adding cgroup /proc/self/fd/12
> (00.046455) cg: Dumping value 0-7 from /proc/self/fd/12/cpuset.cpus
> (00.046477) cg: Dumping value 0-1 from /proc/self/fd/12/cpuset.mems
> (00.046496) cg: Dumping value 0 from /proc/self/fd/12/cpuset.memory_migrate
> (00.046516) cg: Dumping value 1 from /proc/self/fd/12/cpuset.cpu_exclusive
> (00.046536) cg: Dumping value 1 from /proc/self/fd/12/cpuset.mem_exclusive
> (00.046556) cg: Dumping value 0 from /proc/self/fd/12/cpuset.mem_hardwall
> (00.046576) cg: Dumping value 0 from /proc/self/fd/12/cpuset.memory_spread_page
> (00.046595) cg: Dumping value 0 from /proc/self/fd/12/cpuset.memory_spread_slab
> (00.046614) cg: Dumping value 1 from /proc/self/fd/12/cpuset.sched_load_balance
> (00.046633) cg: Dumping value -1 from /proc/self/fd/12/cpuset.sched_relax_domain_level
> (00.046653) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.046672) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.046690) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.046707) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.054379) cg: adding cgroup /proc/self/fd/12/system.slice/sshd.service
> (00.054417) cg: Dumping value a *:* rwm from /proc/self/fd/12/system.slice/sshd.service/devices.list
> (00.054440) cg: Dumping value 0 from /proc/self/fd/12/system.slice/sshd.service/cgroup.clone_children
> (00.054469) cg: Dumping value 0 from /proc/self/fd/12/system.slice/sshd.service/notify_on_release
> (00.054488) cg: Dumping value  from /proc/self/fd/12/system.slice/sshd.service/cgroup.procs
> (00.054507) cg: Dumping value  from /proc/self/fd/12/system.slice/sshd.service/tasks
> (00.062371) cg: adding cgroup /proc/self/fd/12
> (00.062409) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.062430) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.062447) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.062465) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.070413) cg: adding cgroup /proc/self/fd/12
> (00.070452) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.070472) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.070490) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.070508) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.078411) cg: adding cgroup /proc/self/fd/12/user.slice/user-0.slice/session-3.scope
> (00.078452) cg: Dumping value 9223372036854771712 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.limit_in_bytes
> (00.078474) cg: Dumping value 9223372036854771712 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.memsw.limit_in_bytes
> (00.078497) cg: Dumping value 60 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.swappiness
> (00.078518) cg: Dumping value 9223372036854771712 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.soft_limit_in_bytes
> (00.078540) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.move_charge_at_immigrate
> (00.078569) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.oom_control
> (00.078592) cg: Dumping value 1 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.use_hierarchy
> (00.078614) cg: Dumping value 9223372036854771712 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.kmem.limit_in_bytes
> (00.078636) cg: Dumping value 9223372036854771712 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/memory.kmem.tcp.limit_in_bytes
> (00.078657) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.clone_children
> (00.078677) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/notify_on_release
> (00.078697) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.procs
> (00.078716) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/tasks
> (00.086445) cg: adding cgroup /proc/self/fd/12/user.slice/user-0.slice/session-3.scope
> (00.086482) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.clone_children
> (00.086505) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/notify_on_release
> (00.086524) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.procs
> (00.086543) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/tasks
> (00.094417) cg: adding cgroup /proc/self/fd/12
> (00.094458) cg: Dumping value 0 from /proc/self/fd/12/net_cls.classid
> (00.094480) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.094499) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.094517) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.094534) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.094558) cg: Dumping value lo 0
> enp3s0 0
> enp2s0 0
> enp8s0f0 0
> enp8s0f1 0
> xenbr0 0 from /proc/self/fd/12/net_prio.ifpriomap
> (00.094577) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.094595) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.094612) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.094630) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.102421) cg: adding cgroup /proc/self/fd/12
> (00.102461) cg: Dumping value 0 from /proc/self/fd/12/cgroup.clone_children
> (00.102481) cg: Dumping value 0 from /proc/self/fd/12/notify_on_release
> (00.102508) cg: Dumping value  from /proc/self/fd/12/cgroup.procs
> (00.102525) cg: Dumping value  from /proc/self/fd/12/tasks
> (00.110413) cg: adding cgroup /proc/self/fd/12/user.slice/user-0.slice/session-3.scope
> (00.110451) cg: Dumping value max from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/pids.max
> (00.110472) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.clone_children
> (00.110493) cg: Dumping value 0 from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/notify_on_release
> (00.110513) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/cgroup.procs
> (00.110531) cg: Dumping value  from /proc/self/fd/12/user.slice/user-0.slice/session-3.scope/tasks
> (00.110558) cg: Set 2 is root one
> (00.110621) ----------------------------------------
> (00.110635) Waiting for 2160 to trap
> (00.110767) Daemon 2160 exited trapping
> (00.110788) Sent msg to daemon 3 0 0
> (00.110806) Force no-breakpoints restore
> (00.110980) 2160 was trapped
> (00.110994) 2160 (native) is going to execute the syscall 45, required is 15
> (00.111137) 2160 was trapped
> (00.111148) `- Expecting exit
> (00.111305) 2160 was trapped
> (00.111318) 2160 (native) is going to execute the syscall 186, required is 15
> (00.111372) 2160 was trapped
> (00.111377) `- Expecting exit
> (00.111410) 2160 was trapped
> (00.111423) 2160 (native) is going to execute the syscall 1, required is 15
> pie: 2160: __fetched msg: 3 0 0
> (00.111555) 2160 was trapped
> (00.111566) `- Expecting exit
> (00.111700) 2160 was trapped
> (00.111713) 2160 (native) is going to execute the syscall 186, required is 15
> (00.111844) 2160 was trapped
> (00.111854) `- Expecting exit
> (00.111973) 2160 was trapped
> (00.111986) 2160 (native) is going to execute the syscall 186, required is 15
> (00.112008) 2160 was trapped
> (00.112018) `- Expecting exit
> (00.112038) 2160 was trapped
> (00.112047) 2160 (native) is going to execute the syscall 1, required is 15
> pie: 2160: 2160: new_sp=0x7facd0406008 ip 0x7facd1585acb
> (00.112070) 2160 was trapped
> (00.112074) `- Expecting exit
> (00.112085) 2160 was trapped
> (00.112091) 2160 (native) is going to execute the syscall 3, required is 15
> (00.112109) 2160 was trapped
> (00.112113) `- Expecting exit
> (00.112133) 2160 was trapped
> (00.112174) 2160 (native) is going to execute the syscall 3, required is 15
> (00.112188) 2160 was trapped
> (00.112193) `- Expecting exit
> (00.112205) 2160 was trapped
> (00.112211) 2160 (native) is going to execute the syscall 15, required is 15
> (00.112223) 2160 was stopped
> (00.112230)
> (00.112233) Dumping core for thread (pid: 2161)
> (00.112237) ----------------------------------------
> (00.112241) Dumping general registers for 2161 in native mode
> (00.112245) Dumping GP/FPU registers for 2161
> (00.112248) ZoZo: 1
> (00.112442) 2161 has 0 sched policy
> (00.112452) 	dumping 0 nice for 2161
> (00.112538) ----------------------------------------
> (00.112625)
> (00.112634) Dumping mm (pid: 2160)
> (00.112639) ----------------------------------------
> (00.112643) 0x400000-0x401000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x1
> (00.112649) 0x401000-0x402000 (4K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x1000 reg fp  shmid: 0x1
> (00.112654) 0x402000-0x403000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x2000 reg fp  shmid: 0x1
> (00.112658) 0x403000-0x404000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x2000 reg fp  shmid: 0x1
> (00.112662) 0x404000-0x405000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x3000 reg fp  shmid: 0x1
> (00.112689) 0xe45000-0xe66000 (132K) prot 0x3 flags 0x22 fdflags 0 st 0x221 off 0 reg heap ap  shmid: 0
> (00.112695) 0x7facd0410000-0x7facd0411000 (4K) prot 0 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112699) 0x7facd0411000-0x7facd13b6000 (16020K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112704) 0x7facd13b6000-0x7facd13d8000 (136K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x2
> (00.112708) 0x7facd13d8000-0x7facd1525000 (1332K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x22000 reg fp  shmid: 0x2
> (00.112713) 0x7facd1525000-0x7facd1571000 (304K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x16f000 reg fp  shmid: 0x2
> (00.112724) 0x7facd1571000-0x7facd1572000 (4K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x1bb000 reg fp  shmid: 0x2
> (00.112729) 0x7facd1572000-0x7facd1576000 (16K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x1bb000 reg fp  shmid: 0x2
> (00.112733) 0x7facd1576000-0x7facd1578000 (8K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1bf000 reg fp  shmid: 0x2
> (00.112737) 0x7facd1578000-0x7facd157c000 (16K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112742) 0x7facd157c000-0x7facd1582000 (24K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x3
> (00.112746) 0x7facd1582000-0x7facd1592000 (64K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x6000 reg fp  shmid: 0x3
> (00.112751) 0x7facd1592000-0x7facd1598000 (24K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x16000 reg fp  shmid: 0x3
> (00.112755) 0x7facd1598000-0x7facd1599000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x1b000 reg fp  shmid: 0x3
> (00.112759) 0x7facd1599000-0x7facd159a000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1c000 reg fp  shmid: 0x3
> (00.112764) 0x7facd159a000-0x7facd159e000 (16K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112768) 0x7facd159e000-0x7facd15ab000 (52K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x4
> (00.112772) 0x7facd15ab000-0x7facd164b000 (640K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0xd000 reg fp  shmid: 0x4
> (00.112777) 0x7facd164b000-0x7facd1720000 (852K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0xad000 reg fp  shmid: 0x4
> (00.112781) 0x7facd1720000-0x7facd1721000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x181000 reg fp  shmid: 0x4
> (00.112785) 0x7facd1721000-0x7facd1722000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x182000 reg fp  shmid: 0x4
> (00.112790) 0x7facd1722000-0x7facd1724000 (8K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112794) 0x7facd1735000-0x7facd1736000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0 reg fp  shmid: 0x5
> (00.112798) 0x7facd1736000-0x7facd1756000 (128K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0x1000 reg fp  shmid: 0x5
> (00.112802) 0x7facd1756000-0x7facd175e000 (32K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x21000 reg fp  shmid: 0x5
> (00.112807) 0x7facd175f000-0x7facd1760000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x29000 reg fp  shmid: 0x5
> (00.112811) 0x7facd1760000-0x7facd1761000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x2a000 reg fp  shmid: 0x5
> (00.112815) 0x7facd1761000-0x7facd1762000 (4K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112820) 0x7ffc028f4000-0x7ffc02915000 (132K) prot 0x3 flags 0x122 fdflags 0 st 0x201 off 0 reg ap  shmid: 0
> (00.112824) 0x7ffc029af000-0x7ffc029b2000 (12K) prot 0x1 flags 0x22 fdflags 0 st 0x1201 off 0 reg vvar ap  shmid: 0
> (00.112828) 0x7ffc029b2000-0x7ffc029b4000 (8K) prot 0x5 flags 0x22 fdflags 0 st 0x209 off 0 reg vdso ap  shmid: 0
> (00.112833) 0xffffffffff600000-0xffffffffff601000 (4K) prot 0x5 flags 0x22 fdflags 0 st 0x204 off 0 vsys ap  shmid: 0
> (00.112837) Obtaining task auvx ...
> (00.112962) Dumping path for -3 fd via self 15 [/root/container/rootfs]
> (00.113002) Dumping path for -3 fd via self 15 [/]
> (00.113012) Dumping task cwd id 0x8 root id 0x9
> (00.113186) Dumping file-locks
> (00.113194)
> (00.113198) Dumping pstree (pid: 2160)
> (00.113201) ----------------------------------------
> (00.113206) Process: 2160(2160)
> (00.113246) ----------------------------------------
> (00.113284) cg: Dumping 1 sets
> (00.113291) cg:    `- Dumping blkio of /
> (00.113294) cg:    `- Dumping cpu,cpuacct of /
> (00.113298) cg:    `- Dumping cpuset of /
> (00.113301) cg:    `- Dumping devices of /system.slice/sshd.service
> (00.113305) cg:    `- Dumping freezer of /
> (00.113308) cg:    `- Dumping hugetlb of /
> (00.113311) cg:    `- Dumping memory of /user.slice/user-0.slice/session-3.scope
> (00.113315) cg:    `- Dumping name=systemd of /user.slice/user-0.slice/session-3.scope
> (00.113319) cg:    `- Dumping net_cls,net_prio of /
> (00.113322) cg:    `- Dumping perf_event of /
> (00.113325) cg:    `- Dumping pids of /user.slice/user-0.slice/session-3.scope
> (00.113359) cg: Writing CG image
> (00.113441) unix: Dumping external sockets
> (00.113453) tty: Unpaired slave 0
> (00.113459) Writing image inventory (version 1)
> (00.113668) Running post-dump scripts
> (00.113675) Unlock network
> (00.113682) Unfreezing tasks into 1
> (00.113686) 	Unseizing 2160 into 1
> (00.113700) Writing stats
> (00.113770) Dumping finished successfully

> _______________________________________________
> CRIU mailing list
> CRIU at openvz.org
> https://lists.openvz.org/mailman/listinfo/criu



More information about the CRIU mailing list