[Users] unable to stop/suspend/kill CT
Bogdan-Stefan Rotariu
bogdan at rotariu.ro
Thu Feb 4 13:21:07 PST 2016
> On 04 Feb 2016, at 21:02, Kir Kolyshkin <kir at virtuozzo.com> wrote:
>
> Hi Bogdan,
Hi!
>
> This looks very much like a cpu scheduler lockup, as many of the processes
> belonging to the container are in R state but not running.
>
> Can you try resetting the cpulimit for the container in question, something like
>
> vzctl set $CTID --cpulimit 0
Hah, this was the fix, i did try all the possibilities I know.
The CT did shut down correctly after this!
>
> and see if anything changes?
>
> Also, take a look at cpu.stat for some of the processes that is in such state?
> Say, this one:
> root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00 vzctl exec 111 ps
>
> cat /proc/vz/fairsched/107398/cpu.stat
>
> If throttled_time is big, it means my hypothesis makes sense.
>
> I am also ccing Vladimir, who knows a thing or two about our fair cpu scheduler.
Sorry, forgot to retrieve the info before setting cpulimit
Thank you Sir, you saved me from a reboot!
>
> Kir.
>
> On 02/04/2016 05:48 AM, Bogdan-Stefan Rotariu wrote:
>> Hi there,
>>
>> We are having issues with one container that cannot be stopped/suspended or killed, all commands remain in Sleep or Running Sleep.
>> Any ideea how to stop this container withour rebooting the main machine?
>> We did try to kill all proceeses, they do not die.
>>
>> CTID NPROC STATUS IP_ADDR HOSTNAME
>> 111 100 running a.b.c.d server.name
>>
>>
>> [3839648.976835] CPT ERR: ffff8803dd109000,111 :foreign process 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>> [3839648.976842] CPT ERR: ffff8803dd109000,111 :suspend is impossible now.
>> [3839649.977756] CPT ERR: ffff8803dd109000,111 :foreign process 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>> [3839649.977764] CPT ERR: ffff8803dd109000,111 :suspend is impossible now.
>> [3839650.978718] CPT ERR: ffff8803dd109000,111 :foreign process 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>> [3839650.978726] CPT ERR: ffff8803dd109000,111 :suspend is impossible now.
>> [3839665.639557] CPT ERR: ffff880839216000,111 :foreign process 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>> [3839665.639564] CPT ERR: ffff880839216000,111 :suspend is impossible now.
>> [3839666.640019] CPT ERR: ffff880839216000,111 :foreign process 14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
>>
>> root 19890 0.0 0.0 25460 376 ? Rs 03:34 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 39626 0.0 0.0 25460 376 ? Rs 03:44 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 65503 0.0 0.0 27560 412 ? Rs 11:59 0:00 vzctl enter 111
>> root 65508 0.0 0.0 27560 416 ? Rs 11:59 0:00 vzctl enter 111
>> root 65522 0.0 0.0 27560 416 ? Rs 11:59 0:00 vzctl enter 111
>> root 73329 0.0 0.0 25460 372 ? Rs 12:00 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 73371 0.0 0.0 25460 380 ? Rs 12:00 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 74865 0.0 0.0 25464 408 ? Rs 12:00 0:00 vzctl stop 111
>> root 75864 0.0 0.0 25464 412 ? Rs 12:04 0:00 vzctl stop 111
>> root 85384 0.0 0.0 25464 404 ? Rs 12:08 0:00 vzctl stop 111
>> root 96674 0.0 0.0 25464 412 ? Rs 12:12 0:00 vzctl stop 111
>> root 96787 0.0 0.0 25464 408 ? Rs 12:13 0:00 vzctl stop 111 --fast
>> root 107300 0.0 0.0 27560 412 ? Rs 12:18 0:00 vzctl enter 111
>> root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00 vzctl exec 111 ps
>> root 116638 0.0 0.0 108168 1368 ? S 12:21 0:00 sh -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1 'MemFree' | awk '{print $2}'
>> root 116639 0.0 0.0 25460 1024 ? S 12:21 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 116642 0.0 0.0 25460 364 ? S 12:21 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 116643 0.0 0.0 25460 384 ? Rs 12:21 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 116650 0.0 0.0 25460 384 ? Rs 12:21 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117653 0.0 0.0 25460 380 ? Rs 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117746 0.0 0.0 108168 1368 ? S 12:22 0:00 sh -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1 'MemFree' | awk '{print $2}'
>> root 117747 0.0 0.0 108168 1368 ? S 12:22 0:00 sh -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1 'MemFree' | awk '{print $2}'
>> root 117748 0.0 0.0 25460 1016 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117749 0.0 0.0 25460 1020 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117754 0.0 0.0 25460 360 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117755 0.0 0.0 25460 356 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117756 0.0 0.0 25460 380 ? Rs 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 117757 0.0 0.0 25460 376 ? Rs 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 118191 0.0 0.0 108168 1372 ? S 12:22 0:00 sh -c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1 'MemTotal' | awk '{print $2}'
>> root 118192 0.0 0.0 25460 1020 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 118195 0.0 0.0 25460 360 ? S 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 118196 0.0 0.0 25460 380 ? Rs 12:22 0:00 /usr/sbin/vzctl exec 111 cat /proc/meminfo
>> root 126585 0.0 0.0 25464 408 ? Rs 12:25 0:00 vzctl stop 111
>> root 129412 0.0 0.0 25464 352 ? Rs 12:26 0:00 vzctl stop 111
>> root 138146 0.0 0.0 25464 404 ? Rs 12:28 0:00 vzctl stop 111
>> root 147844 0.0 0.0 25464 408 ? Rs 12:33 0:00 vzctl stop 111
>> root 157178 0.0 0.0 25464 412 ? Rs 12:36 0:00 vzctl stop 111
>> root 158300 0.0 0.0 25464 400 ? Rs 12:39 0:00 vzctl stop 111
>> root 179962 0.0 0.0 25464 408 ? Rs 12:49 0:00 vzctl stop 111
>> root 180039 0.0 0.0 25464 408 ? Rs 12:49 0:00 vzctl stop 111
>> root 220918 0.0 0.0 25464 412 ? Rs 13:04 0:00 vzctl stop 111
>> root 240631 0.0 0.0 25464 408 ? Rs 13:14 0:00 vzctl stop 111
>> root 247169 0.0 0.0 25464 412 ? Rs 13:15 0:00 vzctl stop 111
>> root 250371 0.0 0.0 25464 400 ? Rs 13:19 0:00 vzctl stop 111 --fast
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at openvz.org
>> https://lists.openvz.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Users at openvz.org
> https://lists.openvz.org/mailman/listinfo/users
More information about the Users
mailing list