[Users] openvz 7> centos 8 container

mailinglist at tikklik.nl mailinglist at tikklik.nl
Tue Jun 9 15:02:42 MSK 2020


Thanxs for helping me out...

for i in /proc/[0-9]*/ns/mnt; do readlink $i; done | uniq
mnt:[4026531840]

cat /proc/mounts | wc -l
593

Oke soy the problem here was that centos8 is using namesapces and centos 6
not.

I will contact the vendor to see if i can reduse the total of mounts
This mounts are all website log directory, but i dont offer SSH access so
dont need them.
An dit is only slowing down the restarts if needed.

Thanxs

Steffan

-----Oorspronkelijk bericht-----
Van: users-bounces at openvz.org <users-bounces at openvz.org> Namens Konstantin
Khorenko
Verzonden: dinsdag 9 juni 2020 12:32
Aan: OpenVZ users <users at openvz.org>
Onderwerp: Re: [Users] openvz 7> centos 8 container

On 06/09/2020 12:19 PM, mailinglist at tikklik.nl wrote:
> Hello Konstantin,
>
>
> 1: this is a centos 6 container
>     lsns is not there

i don't expect namespaces in centos 6 Containers, i think there is only one,
but you can verify it:
# for i in /proc/[0-9]*/ns/mnt; do readlink $i; done | uniq

Just check the number of lines in output.

And how many mounts in CentOS 6 Container?
(Assuming only 1 mount namespace, just cat /proc/mounts | wc -l)


> 2:
> PID: 1, # of mounts: 600,       cmdline: init-z
> PID: 707,       # of mounts: 1199,      cmdline:
> /usr/lib/systemd/systemd-udevd
> PID: 724,       # of mounts: 1205,      cmdline:
> /usr/sbin/NetworkManager--no-daemon
> PID: 11063,     # of mounts: 1201,      cmdline:
/usr/sbin/httpd-DFOREGROUND
> PID: 10410,     # of mounts: 1203,      cmdline:
> /usr/libexec/postfix/master-w
> PID: 1118,      # of mounts: 1201,      cmdline:
> /usr/libexec/mysqld--basedir=/usr
> PID: 1029,      # of mounts: 1201,      cmdline: php-fpm: master process
> (/etc/opt/remi/php73/php-fpm.conf)
> PID: 1037,      # of mounts: 1201,      cmdline: php-fpm: master process
> (/etc/opt/remi/php71/php-fpm.conf)
> PID: 1039,      # of mounts: 1201,      cmdline: php-fpm: master process
> (/etc/opt/remi/php56/php-fpm.conf)
> PID: 1041,      # of mounts: 1202,      cmdline: php-fpm: master process
> (/etc/php-fpm.conf)


> Total number of mounts: 28169

He-he, there is a mistake in the script - "Total number of mounts" prints
sum of pids. :) But ok, the real sum of mounts is 10000+.

--
Konstantin

> -----Oorspronkelijk bericht-----
> Van: users-bounces at openvz.org <users-bounces at openvz.org> Namens 
> Konstantin Khorenko
> Verzonden: dinsdag 9 juni 2020 10:53
> Aan: OpenVZ users <users at openvz.org>
> Onderwerp: Re: [Users] openvz 7> centos 8 container
>
> On 06/09/2020 09:42 AM, mailinglist at tikklik.nl wrote:
>> Is this on a openvz6 a different setting.
>
> That's strange, the mount limit presents in vz6 as well.
>
> At some point we faced the situation when some Container stop took 
> enormous amount of time, we found out that there was a software inside
which "leaked"
> mounts, but this does not matter, it means any "bad guy" can create a 
> lot of mounts and start/stop Containers affecting other Containers on 
> the same node (global locks taken - namespace_sem, vfsmount_lock).
>
> Thus we've implemented the precaution limit for mounts.
>
> Can you check the total number of mounts on
> 1) vz6 ("old" server running in centos7 Container?) and
> 2) vz7 ("new" server running in centos8 Container?)
>
> # export total=0; for i in `lsns | grep mnt | awk -e '{print $4;}'`; 
> do echo -en "PID: $i,\t# of mounts: "; echo -n `cat /proc/$i/mounts | 
> wc -l`; echo -en ",\tcmdline: "; cat /proc/$i/cmdline; echo ""; 
> total=$((total + $i)); done; echo "Total number of mounts: $total"
>
> Thank you.
>
> --
> Konstantin
>
>> The old server is now running on a centos 7 vps
>>
>>
>> -----Oorspronkelijk bericht-----
>> Van: users-bounces at openvz.org <users-bounces at openvz.org> Namens 
>> Konstantin Khorenko
>> Verzonden: maandag 8 juni 2020 23:07
>> Aan: OpenVZ users <users at openvz.org>
>> Onderwerp: Re: [Users] openvz 7> centos 8 container
>>
>> On 06/08/2020 09:15 PM, mailinglist at tikklik.nl wrote:
>>> If 4096 is the default
>>> Then i dont get it why this error is there
>>>
>>> Its 'only' 597
>>>
>>> mount | wc -l
>>> 597
>>
>> Most probably you have mount namespaces with more mounts inside.
>>
>>>
>>>
>>> Best regards,
>>>
>>> Steffan
>>> -----Oorspronkelijk bericht-----
>>> Van: users-bounces at openvz.org <users-bounces at openvz.org> Namens 
>>> Konstantin Khorenko
>>> Verzonden: maandag 8 juni 2020 17:45
>>> Aan: OpenVZ users <users at openvz.org>
>>> Onderwerp: Re: [Users] openvz 7> centos 8 container
>>>
>>> On 06/08/2020 03:31 PM, mailinglist at tikklik.nl wrote:
>>>> I now see on my node:
>>>>
>>>> kernel: CT#402 reached the limit on mounts.
>>>
>>> You can increase the limit of mounts inside a Container via sysctl 
>>> "fs.ve-mount-nr" (4096 by default).
>>>
>>> Warning: stopping of a Container with many mounts inside can take 
>>> quite a long time.
>>> Say, if you have 200000 of mounts in a Container, Container stop may 
>>> take
>>> ~10 minutes.
>>>
>>> --
>>> Best regards,
>>>
>>> Konstantin Khorenko,
>>> Virtuozzo Linux Kernel Team
>>>
>>>> So think that is the problem.
>>>>
>>>> I see a old toppic onlin
>>> https://forum.openvz.org/index.php?t=rview&th=12902&goto=52002
>>>>
>>>> Any idee if that is the solution that is needed today?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *Van:* users-bounces at openvz.org <users-bounces at openvz.org> *Namens
>>> *mailinglist at tikklik.nl
>>>> *Verzonden:* maandag 8 juni 2020 14:17
>>>> *Aan:* 'OpenVZ users' <users at openvz.org>
>>>> *Onderwerp:* [Users] openvz 7> centos 8 container
>>>>
>>>>
>>>>
>>>>     Hello,
>>>>
>>>>
>>>>
>>>>     If installed a centos 8 op-envz container
>>>>
>>>>     It was working, but after migration my data from an older 
>>>> container im
>>> keep getting errors like this:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>     php71-php-fpm.service: Failed to set up mount namespacing:
>>>> Cannot
>>> allocate memory
>>>>
>>>>     php71-php-fpm.service: Failed at step NAMESPACE spawning
>>> /opt/remi/php71/root/usr/sbin/php-fpm: Cannot allocate memory
>>>>
>>>>
>>>>
>>>>     php73-php-fpm.service: Failed to set up mount namespacing:
>>>> Cannot
>>> allocate memory
>>>>
>>>>     php73-php-fpm.service: Failed at step NAMESPACE spawning
>>> /opt/remi/php73/root/usr/sbin/php-fpm: Cannot allocate memory
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>     httpd.service: Failed to set up mount namespacing: Cannot 
>>>> allocate
>>> memory
>>>>
>>>>     httpd.service: Failed at step NAMESPACE spawning /usr/sbin/httpd:
>>> Cannot allocate memory
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>     cat /proc/user_beancounters
>>>>
>>>>     Version: 2.5
>>>>
>>>>     resource                     held              maxheld
>>> barrier                limit              failcnt
>>>>
>>>>     kmemsize                 92078080            121937920
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     lockedpages                     0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     privvmpages                 52155                75857
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     shmpages                      659                 2636
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dummy                           0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numproc                        39                   39
>>> 4194304              4194304                    0
>>>>
>>>>     physpages                   97697               111964
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     vmguarpages                     0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     oomguarpages                97697               111964
>>> 0                    0                    0
>>>>
>>>>     numtcpsock                      0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numflock                        2                    5
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numpty                          0                    1
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numsiginfo                      0                   57
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     tcpsndbuf                       0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     tcprcvbuf                       0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     othersockbuf                    0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dgramrcvbuf                     0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numothersock                    0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dcachesize               51408896             72798208
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     numfile                       711                  995
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dummy                           0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dummy                           0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>     dummy                           0                    0
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>         numiptent                       8                   16
>>> 9223372036854775807  9223372036854775807                    0
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>     uname -r      3.10.0-1062.12.1.vz7.131.10
>>>>
>>>>
>>>>
>>>>     any idees what went wrong and how to repair?
>>>>
>>>>
>>>>
>>>>     Thanxs
>>>>
>>>>
>>>>     Steffan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at openvz.org
>>>> https://lists.openvz.org/mailman/listinfo/users
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at openvz.org
>>> https://lists.openvz.org/mailman/listinfo/users
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at openvz.org
>>> https://lists.openvz.org/mailman/listinfo/users
>>> .
>>>
>> _______________________________________________
>> Users mailing list
>> Users at openvz.org
>> https://lists.openvz.org/mailman/listinfo/users
>>
>> _______________________________________________
>> Users mailing list
>> Users at openvz.org
>> https://lists.openvz.org/mailman/listinfo/users
>> .
>>
> _______________________________________________
> Users mailing list
> Users at openvz.org
> https://lists.openvz.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Users at openvz.org
> https://lists.openvz.org/mailman/listinfo/users
> .
>
_______________________________________________
Users mailing list
Users at openvz.org
https://lists.openvz.org/mailman/listinfo/users



More information about the Users mailing list