[Users] Kswapd usage

[Paulo Coghi - Coghi IT]

*First - Possible workarounds*
If you feel secure to use the remaining RAM, you need to reduce swappiness
value (and clean swap) or to disable swap. There are two drawbacks in this
procedure, however. One is that you will need to better monitor the memory
usage to avoid containers and vms stop working. And second, you will still
get some performance degradation by the lack of RAM space to provide
filesystem caching. This performance degradation, though,
theoretically could be less worse than the generated by the swapping
movement.

*Why you are seeing high load average and performance degradation*
The increase in load average is not only incurred by the CPU usage of
kswapd process, but by the I/O and "I/O wait" it generates on your storage.
Even if it is a SSD (NVMe or not), its speed is not unlimited and will be
saturated until the swap movement finishes, as the RAM speed is noticeably
faster. This saturation on your storage I/O capacity impacts everything on
your server, including its containers and vms.

*Final considerations - My 2 cents*
After the observations above, I do note that 30GB of free RAM on a 128GB
server indicates that it has only 25% of free memory. Server admin
decisions depend on the type of use and server scenario. *But*, it could be
beneficial to your server (and to the workloads it is responsible to
provide, like OpenVZ containers) if you maintain at least 25% of free RAM.

Operating with less than this threshold could represent a risk to your
operations if, for whatever reason, your server and/or its containers
suffer a spike on processing/access/usage/etc, since it will not have a
secure reserve of resources to respond properly.

Again, as the server administrator, you can make a better decision on this
threshold since you know what your server scenario and use is.



Cordially,

Paulo Coghi

On Sun, Jun 6, 2021 at 11:37 PM George <g at sturley.me> wrote:

> Hello,
>
> For a very long time now (we're talking the past few years) I've faced an
> issue with a number of my nodes where by two kswapd processes fire up
> frequently (using ~200% CPU between them).
> It's at those times that the overall load average shoots up (from a normal
> range of 30-50 to 100+), resulting in temporary performance degradation.
>
> I'm currently using the latest stable kernel (3.10.0-1160.21.1.vz7.174.13)
> with all available updates installed, but have seen it on every other
> kernel in the past as well.
>
> There has been some suggestion from various people that upgrading the
> amount of physical RAM would be beneficial, which doesn't make a great deal
> of sense to me as there's always at least 30 GB RAM available on the nodes,
> and 40 GB+ in most cases.
>
> The node specifications are as follows:
>
> 2x Intel Xeon E5-2670v2
> 128 GB DDR3 ECC RAM (upgradable to 256 GB if needed)
> 4x 2 TB Samsung 870 SSD
> Hardware RAID 10 w/ BBU (WriteBack, ReadAheadNone, Direct, No Write Cache
> if Bad BBU)
>
> Output from top -c: https://i.imgur.com/C4Yo0V1.png
> Output from free -m: https://i.imgur.com/utkZNFf.png
>
> My question is: has anyone seen anything like this? Is there anything that
> I should consider doing to help remedy the problem? Upgrading the RAM is an
> option, but I am not entirely convinced it will be beneficial.
>
> Many thanks.
>
> _______________________________________________
> Users mailing list
> Users at openvz.org
> https://lists.openvz.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/users/attachments/20210610/80d848b1/attachment.html>