[Devel] [PATCH rh7 4/4] /proc/<pid>/vz_latency: Show maximal allocation latency in the last second.

[Andrey Ryabinin]

On 08/20/2018 10:57 AM, Pavel Borzenkov wrote:
> 
> 
>> On 17 Aug 2018, at 19:40, Andrey Ryabinin <aryabinin at virtuozzo.com> wrote:
>>
>> Add to '/proc/<pid>/vz_latency' column with maximal latency task have seen
>> in the last second.
>>
>> E.g.:
>>
>> cat /proc/1/vz_latency
>> Type                    Total_lat                Calls           Max (1sec)
>> allocatomic:                    0                  294                    0
>> alloc:                    3000000                43394                    0
>> allocmp:                        0                 1018                    0
> 
> 1 second is too small.
> Prometheus doesn't scrape our services that frequently as we configure it to do this once a minute.
> In this case we are gonna miss 59/60 of the data.
> 
> Please use 2-minute maximums.
> 

Sure, no problem.

Also note that technically this is not maximum in last period of time.
It's maximum since the last reset of max latency with reset happening once
per period of time (2 minutes in this patch).
The true max latency in last X-sec/min is technically impossible to implement
as this would require to remember all allocations times and and latencies
in last X-sec/min. 
We can record max latencies every 10 sec for the past 2min and based on that
calculate the max latency in last 2 minutes (+/-10sec).
But the same logic can be implemented in user-space with less performance cost.
In the kernel we would need to put that complex logic into every allocation,
while in userspace this can be done only for tasks we want to monitor.