[Users] Lots of interrupts since latest el6 kernel

[Vasily Averin]

On 5/30/19 10:39 PM, Karl Johnson wrote:
> Hello,
> 
> It's always related to swapper and ksoftirqd:
"swapper" is idle thread, it is called if CPU does not have any active tasks
it would be interesting to look at state of "ksoftirqd" processes several times, to see any changes.

In provided example I see that this process was captured during processing of top-level function handles soft interrupts:
do_softirq()-> call_softirq(). Usually these function handles network packets and I expected your example will contain more deep calltraces.
Probably this happen next time.

Anyway, these calltraces shows that CPUs are NOT 100% busy by processing of timer interrupts,
so in general the situation looks like expected: in current theory ksoftirq processes handles network traffic.

Thank you,
	Vasily Averin
 
> Some examples here: https://pastebin.com/wn0nCwce
> 
> Karl
> 
> On Thu, May 30, 2019 at 3:11 PM Vasily Averin <vvs at virtuozzo.com <mailto:vvs at virtuozzo.com>> wrote:
> 
>     Dear Karl,
>     thank you for reporting the problem.
> 
>     no, it is not known issue.
>     moreover, I doubt it is related to real hardware interrupts,
>     soft-interrupts handles delayed procedures like processing of network packets.
> 
>     For troubleshooting is to look at stack of affected running processes via /proc/<pid>/stack
>     alternatively you can use magic sysrq key
>     # echo l > /proc/sysrq-trigger
>     it should dump current state of all running processors.
>     you can do it few times to monitor state of affected processes.
> 
>     Thank you,
>             Vasily Averin
> 
> 
>     On 5/30/19 7:54 PM, Karl Johnson wrote:
>     > Hello,
>     >
>     > I've upgraded from 2.6.32-042stab133.2 to 2.6.32-042stab138.1 and since boot, 2 cores are using 100% cpu on ksoftirqd:
>     >
>     > root          21 99.9  0.0      0     0 ?        R    May29 1178:07  \_ [ksoftirqd/4]
>     > root          25 99.9  0.0      0     0 ?        R    May29 1177:51  \_ [ksoftirqd/5]
>     >
>     > From /proc/interrupts I can see that it's caused by IR-IO-APIC-edge      timer:
>     >
>     >            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7      
>     >   0:     136922     103603      26928      27528  112318229   71888343      73755     285735  IR-IO-APIC-edge      timer
>     >
>     > kernel /vmlinuz-2.6.32-042stab138.1 ro root=UUID=7367aa0f-8216-44ca-9cc4-affed22bbd9c rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM nohz=off nopti
>     >
>     > Any way to troubleshoot this? Is it a known issue?
>     >
>     > Karl
>     >
>     >
>     > _______________________________________________
>     > Users mailing list
>     > Users at openvz.org <mailto:Users at openvz.org>
>     > https://lists.openvz.org/mailman/listinfo/users
>     >
>