[Users] unable to stop/suspend/kill CT
Kir Kolyshkin
kir at virtuozzo.com
Thu Feb 4 11:02:26 PST 2016
Hi Bogdan,
This looks very much like a cpu scheduler lockup, as many of the processes
belonging to the container are in R state but not running.
Can you try resetting the cpulimit for the container in question,
something like
vzctl set $CTID --cpulimit 0
and see if anything changes?
Also, take a look at cpu.stat for some of the processes that is in such
state?
Say, this one:
root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00 vzctl
exec 111 ps
cat /proc/vz/fairsched/107398/cpu.stat
If throttled_time is big, it means my hypothesis makes sense.
I am also ccing Vladimir, who knows a thing or two about our fair cpu
scheduler.
Kir.
On 02/04/2016 05:48 AM, Bogdan-Stefan Rotariu wrote:
> Hi there,
>
> We are having issues with one container that cannot be
> stopped/suspended or killed, all commands remain in Sleep or Running
> Sleep.
> Any ideea how to stop this container withour rebooting the main machine?
> We did try to kill all proceeses, they do not die.
>
> CTID NPROC STATUS IP_ADDR HOSTNAME
> 111 100 running a.b.c.d server.name
>
>
> [3839648.976835] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839648.976842] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839649.977756] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839649.977764] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839650.978718] CPT ERR: ffff8803dd109000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839650.978726] CPT ERR: ffff8803dd109000,111 :suspend is impossible
> now.
> [3839665.639557] CPT ERR: ffff880839216000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
> [3839665.639564] CPT ERR: ffff880839216000,111 :suspend is impossible
> now.
> [3839666.640019] CPT ERR: ffff880839216000,111 :foreign process
> 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>
> root 19890 0.0 0.0 25460 376 ? Rs 03:34 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 39626 0.0 0.0 25460 376 ? Rs 03:44 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 65503 0.0 0.0 27560 412 ? Rs 11:59 0:00
> vzctl enter 111
> root 65508 0.0 0.0 27560 416 ? Rs 11:59 0:00
> vzctl enter 111
> root 65522 0.0 0.0 27560 416 ? Rs 11:59 0:00
> vzctl enter 111
> root 73329 0.0 0.0 25460 372 ? Rs 12:00 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 73371 0.0 0.0 25460 380 ? Rs 12:00 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 74865 0.0 0.0 25464 408 ? Rs 12:00 0:00
> vzctl stop 111
> root 75864 0.0 0.0 25464 412 ? Rs 12:04 0:00
> vzctl stop 111
> root 85384 0.0 0.0 25464 404 ? Rs 12:08 0:00
> vzctl stop 111
> root 96674 0.0 0.0 25464 412 ? Rs 12:12 0:00
> vzctl stop 111
> root 96787 0.0 0.0 25464 408 ? Rs 12:13 0:00
> vzctl stop 111 --fast
> root 107300 0.0 0.0 27560 412 ? Rs 12:18 0:00
> vzctl enter 111
> root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00
> vzctl exec 111 ps
> root 116638 0.0 0.0 108168 1368 ? S 12:21 0:00 sh
> -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root 116639 0.0 0.0 25460 1024 ? S 12:21 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 116642 0.0 0.0 25460 364 ? S 12:21 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 116643 0.0 0.0 25460 384 ? Rs 12:21 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 116650 0.0 0.0 25460 384 ? Rs 12:21 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117653 0.0 0.0 25460 380 ? Rs 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117746 0.0 0.0 108168 1368 ? S 12:22 0:00 sh
> -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root 117747 0.0 0.0 108168 1368 ? S 12:22 0:00 sh
> -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemFree' | awk '{print $2}'
> root 117748 0.0 0.0 25460 1016 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117749 0.0 0.0 25460 1020 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117754 0.0 0.0 25460 360 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117755 0.0 0.0 25460 356 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117756 0.0 0.0 25460 380 ? Rs 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 117757 0.0 0.0 25460 376 ? Rs 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 118191 0.0 0.0 108168 1372 ? S 12:22 0:00 sh
> -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
> 'MemTotal' | awk '{print $2}'
> root 118192 0.0 0.0 25460 1020 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 118195 0.0 0.0 25460 360 ? S 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 118196 0.0 0.0 25460 380 ? Rs 12:22 0:00
> /usr/sbin/vzctl exec 111 cat /proc/meminfo
> root 126585 0.0 0.0 25464 408 ? Rs 12:25 0:00
> vzctl stop 111
> root 129412 0.0 0.0 25464 352 ? Rs 12:26 0:00
> vzctl stop 111
> root 138146 0.0 0.0 25464 404 ? Rs 12:28 0:00
> vzctl stop 111
> root 147844 0.0 0.0 25464 408 ? Rs 12:33 0:00
> vzctl stop 111
> root 157178 0.0 0.0 25464 412 ? Rs 12:36 0:00
> vzctl stop 111
> root 158300 0.0 0.0 25464 400 ? Rs 12:39 0:00
> vzctl stop 111
> root 179962 0.0 0.0 25464 408 ? Rs 12:49 0:00
> vzctl stop 111
> root 180039 0.0 0.0 25464 408 ? Rs 12:49 0:00
> vzctl stop 111
> root 220918 0.0 0.0 25464 412 ? Rs 13:04 0:00
> vzctl stop 111
> root 240631 0.0 0.0 25464 408 ? Rs 13:14 0:00
> vzctl stop 111
> root 247169 0.0 0.0 25464 412 ? Rs 13:15 0:00
> vzctl stop 111
> root 250371 0.0 0.0 25464 400 ? Rs 13:19 0:00
> vzctl stop 111 --fast
>
>
> _______________________________________________
> Users mailing list
> Users at openvz.org
> https://lists.openvz.org/mailman/listinfo/users
More information about the Users
mailing list