[Users] unable to stop/suspend/kill CT

Scott Dowdle dowdle at montanalinux.org
Thu Feb 4 10:06:17 PST 2016


Greetings,

----- Original Message -----
> Hi there,
> 
> We are having issues with one container that cannot be
> stopped/suspended
> or killed, all commands remain in Sleep or Running Sleep.
> Any ideea how to stop this container withour rebooting the main
> machine?
> We did try to kill all proceeses, they do not die.
> 
>        CTID      NPROC STATUS    IP_ADDR         HOSTNAME
>         111        100 running   a.b.c.d server.name
> 
> 
> [3839648.976835] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839648.976842] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839649.977756] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839649.977764] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839650.978718] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839650.978726] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839665.639557] CPT ERR: ffff880839216000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839665.639564] CPT ERR: ffff880839216000,111 :suspend is impossible
> now.
> [3839666.640019] CPT ERR: ffff880839216000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> 
> root       19890  0.0  0.0  25460   376 ?        Rs   03:34   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root       39626  0.0  0.0  25460   376 ?        Rs   03:44   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root       65503  0.0  0.0  27560   412 ?        Rs   11:59   0:00
> vzctl
> enter 111
> root       65508  0.0  0.0  27560   416 ?        Rs   11:59   0:00
> vzctl
> enter 111
> root       65522  0.0  0.0  27560   416 ?        Rs   11:59   0:00
> vzctl
> enter 111
> root       73329  0.0  0.0  25460   372 ?        Rs   12:00   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root       73371  0.0  0.0  25460   380 ?        Rs   12:00   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root       74865  0.0  0.0  25464   408 ?        Rs   12:00   0:00
> vzctl
> stop 111
> root       75864  0.0  0.0  25464   412 ?        Rs   12:04   0:00
> vzctl
> stop 111
> root       85384  0.0  0.0  25464   404 ?        Rs   12:08   0:00
> vzctl
> stop 111
> root       96674  0.0  0.0  25464   412 ?        Rs   12:12   0:00
> vzctl
> stop 111
> root       96787  0.0  0.0  25464   408 ?        Rs   12:13   0:00
> vzctl
> stop 111 --fast
> root      107300  0.0  0.0  27560   412 ?        Rs   12:18   0:00
> vzctl
> enter 111
> root      107398  0.0  0.0  25460   396 ?        Rs   12:19   0:00
> vzctl
> exec 111 ps
> root      116638  0.0  0.0 108168  1368 ?        S    12:21   0:00 sh
> -c
> /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root      116639  0.0  0.0  25460  1024 ?        S    12:21   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      116642  0.0  0.0  25460   364 ?        S    12:21   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      116643  0.0  0.0  25460   384 ?        Rs   12:21   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      116650  0.0  0.0  25460   384 ?        Rs   12:21   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117653  0.0  0.0  25460   380 ?        Rs   12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117746  0.0  0.0 108168  1368 ?        S    12:22   0:00 sh
> -c
> /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root      117747  0.0  0.0 108168  1368 ?        S    12:22   0:00 sh
> -c
> /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root      117748  0.0  0.0  25460  1016 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117749  0.0  0.0  25460  1020 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117754  0.0  0.0  25460   360 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117755  0.0  0.0  25460   356 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117756  0.0  0.0  25460   380 ?        Rs   12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      117757  0.0  0.0  25460   376 ?        Rs   12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      118191  0.0  0.0 108168  1372 ?        S    12:22   0:00 sh
> -c
> /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemTotal' | awk '{print $2}'
> root      118192  0.0  0.0  25460  1020 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      118195  0.0  0.0  25460   360 ?        S    12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      118196  0.0  0.0  25460   380 ?        Rs   12:22   0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root      126585  0.0  0.0  25464   408 ?        Rs   12:25   0:00
> vzctl
> stop 111
> root      129412  0.0  0.0  25464   352 ?        Rs   12:26   0:00
> vzctl
> stop 111
> root      138146  0.0  0.0  25464   404 ?        Rs   12:28   0:00
> vzctl
> stop 111
> root      147844  0.0  0.0  25464   408 ?        Rs   12:33   0:00
> vzctl
> stop 111
> root      157178  0.0  0.0  25464   412 ?        Rs   12:36   0:00
> vzctl
> stop 111
> root      158300  0.0  0.0  25464   400 ?        Rs   12:39   0:00
> vzctl
> stop 111
> root      179962  0.0  0.0  25464   408 ?        Rs   12:49   0:00
> vzctl
> stop 111
> root      180039  0.0  0.0  25464   408 ?        Rs   12:49   0:00
> vzctl
> stop 111
> root      220918  0.0  0.0  25464   412 ?        Rs   13:04   0:00
> vzctl
> stop 111
> root      240631  0.0  0.0  25464   408 ?        Rs   13:14   0:00
> vzctl
> stop 111
> root      247169  0.0  0.0  25464   412 ?        Rs   13:15   0:00
> vzctl
> stop 111
> root      250371  0.0  0.0  25464   400 ?        Rs   13:19   0:00
> vzctl
> stop 111 --fast

If you can get a pid list of what remains within the container, you can try to kill -9 it from the host node.  You can use vzpid to map container pid references to host node pid references.

If that doesn't work for you... it would be interesting to know your vzkernel and vzctl versions.  If not current, most likely you have run into an issue that has long since been fixed.

If nothing you try will resolve it, ideally you have another host you can migrate any other containers too (so they will have minimal downtime) and then you can reboot the host node... and if it isn't running the latest greatest vzkernel/vzctl, get it upgraded while you are at it. :)

Good luck!

TYL,
-- 
Scott Dowdle
704 Church Street
Belgrade, MT 59714
(406)388-0827 [home]
(406)994-3931 [work]


More information about the Users mailing list