[Users] OOM and /etc/vz/oom-groups.conf

Wed Oct 2 10:10:11 PDT 2013

On Tue, Oct 01, 2013 at 04:37:52PM -0700, Kir Kolyshkin wrote:
> I'd suggest you to reproduce the bug on 081.3 kernel booted for real,
> and if you succeed, file a kernel bug to bugzilla.openvz.org.

Will put in the request to try and get one upgraded (or next node build on the new kernel)

> Also, it would be helpful to see output of pstree or ps axf inside a
> container.
> Note that you have specified oom adjustment for mysqld processes
> which parent is init. I guess that OOM killer kills a mysqld process
> which parent is not init, so oom adj is not applied.

ps axf:

  PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:00 init
    2 ?        S      0:00 [kthreadd/3679]
    3 ?        S      0:00  \_ [khelper/3679]
  127 ?        S<s    0:00 /sbin/udevd -d
  481 ?        Ss     0:00 /sbin/portreserve
  523 ?        Ss     0:00 nsd
  525 ?        S      0:00  \_ nsd
  526 ?        S      0:07  \_ nsd
 1923 ?        Ss     0:00 /usr/sbin/sshd
 1930 ?        Ss     0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
 2647 ?        Ss     0:01 /usr/sbin/exim -bd -q60m
 2668 ?        Ss     0:04 /usr/sbin/dovecot
 2670 ?        S      0:01  \_ dovecot-auth
 2697 ?        S      0:02  \_ pop3-login
 2698 ?        S      0:00  \_ pop3-login
 2699 ?        S      0:01  \_ imap-login
 2700 ?        S      0:02  \_ imap-login
 2675 ?        Ss     0:23 /usr/local/bin/spamd -d --allowed-ips=127.0.0.1 --pidfile=/var/run/spamd.pid --max-children=3 --max-spare=1
 8921 ?        S      0:12  \_ spamd child
 2701 ?        Ss     0:08 /usr/local/apache/bin/httpd -k start -DSSL
27690 ?        S      0:00  \_ /usr/local/apache/bin/httpd -k start -DSSL
27691 ?        S      0:00  \_ /usr/bin/perl /usr/local/cpanel/bin/leechprotect
27697 ?        S      0:00  \_ /usr/local/apache/bin/httpd -k start -DSSL
27709 ?        Sl     0:15  \_ /usr/local/apache/bin/httpd -k start -DSSL
31777 ?        S      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
31881 ?        S      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
31917 ?        R      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
27774 ?        Sl     0:15  \_ /usr/local/apache/bin/httpd -k start -DSSL
27801 ?        Sl     0:16  \_ /usr/local/apache/bin/httpd -k start -DSSL
31308 ?        Z      0:01  |   \_ [php] <defunct>
31890 ?        S      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
31911 ?        R      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
27831 ?        Sl     0:16  \_ /usr/local/apache/bin/httpd -k start -DSSL
31913 ?        S      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
31915 ?        R      0:00  |   \_ /usr/bin/php /home/noticias/public_html/wp-admin/admin-ajax.php
27858 ?        Sl     0:16  \_ /usr/local/apache/bin/httpd -k start -DSSL
31898 ?        S      0:00  |   \_ /usr/bin/php /home/noticias/public_html/index.php
28021 ?        Sl     0:16  \_ /usr/local/apache/bin/httpd -k start -DSSL
31796 ?        S      0:00      \_ /usr/bin/php /home/noticias/public_html/index.php
31886 ?        S      0:00      \_ /usr/bin/php /home/noticias/public_html/index.php
 2707 ?        Ss     0:00 pure-ftpd (SERVER)                                                                                                        
 2709 ?        S      0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin/pureauth
 2716 ?        Ss     0:00 crond
 2726 ?        Ss     0:00 /usr/sbin/atd
 2896 ?        S      0:00 cPhulkd - processor
 2918 ?        S      0:07 cpsrvd (SSL) - waiting for connections
 2929 ?        S      0:00 cpdavd - accepting connections on 2077 and 2078
 2944 ?        S      0:01 queueprocd - wait to process a task
 2985 ?        S      0:01 tailwatchd
 3018 ?        Ss     0:01 /usr/sbin/cf-execd
18794 ?        Ssl    0:00 /usr/sbin/nslcd
15195 ?        S      0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/server.noticias365.net.pid
31661 ?        Sl     1:43  \_ /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/log/mysql.error.log --open-files-limit=8192 --pid-file=/var/lib/mysql/server.noticias365.net.pid --socket=/tmp/mysql.sock --port=3306
32002 ?        SN     0:00 cpanellogd - sleeping for logs
29446 ?        S<L    0:08 /usr/bin/atop -a -w /var/log/atop/atop_20131002 60
31825 ?        Ss     0:00 vzctl: pts/0    
31826 pts/0    Ss     0:00  \_ -bash
31919 pts/0    R+     0:00      \_ ps axf

> Generally speaking, I think you are doing something wrong here.
> 
> First, mysqld is a service which controls its own memory consumption
> pretty well, you just need to set a few parameters in its /etc/my.cnf,
> and that would be the preferred way of managing it.

These do have tweaked my.cnf files for cPanel. 

> Second, OOM should not happen in general. In case you are using
> vswap, the sum of ram and swap configured for all containers
> should not exceed real ram and swap on the system. Worst case
> scenario is swap will be used heavily, resulting in a slowdown
> (which is still way better than killing processes). In case you are
> not using VSwap -- well, you should, see http://openvz.org/VSwap.
>

These are vswap enabled VM's and the nodes do have the /proc/vz/vswap file. If I need to tweak the conf files more thats not a problem.