[Users] crash

[Rene C.]

We had a server crash today, and after the server came back up I found
this in /var/log/messages of the hardware node immediately before the
crash.  I know it's not the very latest kernel but it's not that old
either.   Does this give any idea what went wrong ?  There are some
references to bad swap, something we need to fix?

Jun 18 04:02:30 server16 kernel: [2066983.317141] swap_free: Bad swap
offset entry 10000000
Jun 18 04:02:30 server16 kernel: [2066983.317183] BUG: Bad page map in
process php  pte:2000000000 pmd:3e90c0067
Jun 18 04:02:30 server16 kernel: [2066983.317226]
addr:00007fe8d9f48000 vm_flags:08000070 anon_vma:(null)
mapping:ffff8805e037ec00 index:10b
Jun 18 04:02:30 server16 kernel: [2066983.317303] vma->vm_ops->fault:
filemap_fault+0x0/0x5d0
Jun 18 04:02:30 server16 kernel: [2066983.317356]
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0xd0 [ext4]
Jun 18 04:02:30 server16 kernel: [2066983.317398] Pid: 527330, comm:
php veid: 1602 Not tainted 2.6.32-042stab076.8 #1
Jun 18 04:02:30 server16 kernel: [2066983.317469] Call Trace:
Jun 18 04:02:30 server16 kernel: [2066983.317508]
[<ffffffff811560a8>] ? print_bad_pte+0x1d8/0x290
Jun 18 04:02:30 server16 kernel: [2066983.317549]
[<ffffffff811579a6>] ? unmap_vmas+0x5f6/0xce0
Jun 18 04:02:30 server16 kernel: [2066983.317591]
[<ffffffff8113ff5c>] ? release_pages+0x3dc/0x490
Jun 18 04:02:30 server16 kernel: [2066983.317632]
[<ffffffff81161501>] ? unmap_region+0x91/0x130
Jun 18 04:02:30 server16 kernel: [2066983.317672]
[<ffffffff8116316c>] ? do_munmap+0x26c/0x360
Jun 18 04:02:30 server16 kernel: [2066983.317712]
[<ffffffff811632b6>] ? sys_munmap+0x56/0x80
Jun 18 04:02:30 server16 kernel: [2066983.317754]
[<ffffffff8100b102>] ? system_call_fastpath+0x16/0x1b
Jun 18 04:02:30 server16 kernel: [2066983.317802] Disabling lock
debugging due to kernel taint
Jun 20 12:59:32 server16 kernel: [2271550.512507] swap_free: Bad swap
offset entry 10000000
Jun 20 12:59:32 server16 kernel: [2271550.512549] BUG: Bad page map in
process httpd  pte:2000000000 pmd:3e90c0067
Jun 20 12:59:32 server16 kernel: [2271550.512621]
addr:00007f75ddf48000 vm_flags:08000070 anon_vma:(null)
mapping:ffff8805ed9de260 index:1cc
Jun 20 12:59:32 server16 kernel: [2271550.512698] vma->vm_ops->fault:
filemap_fault+0x0/0x5d0
Jun 20 12:59:32 server16 kernel: [2271550.512750]
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0xd0 [ext4]
Jun 20 12:59:32 server16 kernel: [2271550.512792] Pid: 867375, comm:
httpd veid: 1602 Tainted: G    B      ---------------
2.6.32-042stab076.8 #1
Jun 20 12:59:32 server16 kernel: [2271550.512868] Call Trace:
Jun 20 12:59:32 server16 kernel: [2271550.512906]
[<ffffffff811560a8>] ? print_bad_pte+0x1d8/0x290
Jun 20 12:59:32 server16 kernel: [2271550.512947]
[<ffffffff811579a6>] ? unmap_vmas+0x5f6/0xce0
Jun 20 12:59:32 server16 kernel: [2271550.512989]
[<ffffffff8117c44a>] ? alloc_pages_current+0xaa/0x110
Jun 20 12:59:32 server16 kernel: [2271550.513030]
[<ffffffff811617f7>] ? exit_mmap+0x87/0x1a0
Jun 20 12:59:32 server16 kernel: [2271550.513072]
[<ffffffff8106a5dc>] ? mmput+0x5c/0x1f0
Jun 20 12:59:32 server16 kernel: [2271550.513112]
[<ffffffff81070dc9>] ? exit_mm+0x109/0x150
Jun 20 12:59:32 server16 kernel: [2271550.513152]
[<ffffffff81072bb7>] ? do_exit+0x187/0x920
Jun 20 12:59:32 server16 kernel: [2271550.513191]
[<ffffffff810733a8>] ? do_group_exit+0x58/0xd0
Jun 20 12:59:32 server16 kernel: [2271550.513231]
[<ffffffff81073437>] ? sys_exit_group+0x17/0x20
Jun 20 12:59:32 server16 kernel: [2271550.513281]
[<ffffffff8100b102>] ? system_call_fastpath+0x16/0x1b
Jun 20 13:14:51 server16 kernel: [2272467.322991] swap_free: Bad swap
offset entry 10000000
Jun 20 13:14:51 server16 kernel: [2272467.323033] BUG: Bad page map in
process httpd  pte:2000000000 pmd:3e90c0067
Jun 20 13:14:51 server16 kernel: [2272467.323114]
addr:00007f75ddd48000 vm_flags:08000070 anon_vma:(null)
mapping:ffff8805ed9deb40 index:2cc
Jun 20 13:14:51 server16 kernel: [2272467.323196] vma->vm_ops->fault:
filemap_fault+0x0/0x5d0
Jun 20 13:14:51 server16 kernel: [2272467.323249]
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0xd0 [ext4]
Jun 20 13:14:51 server16 kernel: [2272467.323291] Pid: 874363, comm:
httpd veid: 1602 Tainted: G    B      ---------------
2.6.32-042stab076.8 #1
Jun 20 13:14:51 server16 kernel: [2272467.323367] Call Trace:
Jun 20 13:14:51 server16 kernel: [2272467.323406]
[<ffffffff811560a8>] ? print_bad_pte+0x1d8/0x290
Jun 20 13:14:51 server16 kernel: [2272467.323446]
[<ffffffff811579a6>] ? unmap_vmas+0x5f6/0xce0
Jun 20 13:14:51 server16 kernel: [2272467.323488]
[<ffffffff8117c44a>] ? alloc_pages_current+0xaa/0x110
Jun 20 13:14:51 server16 kernel: [2272467.323530]
[<ffffffff811617f7>] ? exit_mmap+0x87/0x1a0
Jun 20 13:14:51 server16 kernel: [2272467.323572]
[<ffffffff8106a5dc>] ? mmput+0x5c/0x1f0
Jun 20 13:14:51 server16 kernel: [2272467.323612]
[<ffffffff81070dc9>] ? exit_mm+0x109/0x150
Jun 20 13:14:51 server16 kernel: [2272467.323652]
[<ffffffff81072bb7>] ? do_exit+0x187/0x920
Jun 20 13:14:51 server16 kernel: [2272467.323691]
[<ffffffff811631be>] ? do_munmap+0x2be/0x360
Jun 20 13:14:51 server16 kernel: [2272467.323731]
[<ffffffff810733a8>] ? do_group_exit+0x58/0xd0
Jun 20 13:14:51 server16 kernel: [2272467.323771]
[<ffffffff81073437>] ? sys_exit_group+0x17/0x20
Jun 20 13:14:51 server16 kernel: [2272467.323814]
[<ffffffff8100b102>] ? system_call_fastpath+0x16/0x1b
Jun 20 13:15:06 server16 kernel: [2272482.628424] swap_free: Bad swap
offset entry 10000000
Jun 20 13:15:06 server16 kernel: [2272482.628466] BUG: Bad page map in
process php  pte:2000000000 pmd:3e90c0067
Jun 20 13:15:06 server16 kernel: [2272482.628509]
addr:00007fe8cf548000 vm_flags:08000070 anon_vma:(null)
mapping:ffff8805e00fac80 index:266
Jun 20 13:15:06 server16 kernel: [2272482.628586] vma->vm_ops->fault:
filemap_fault+0x0/0x5d0
Jun 20 13:15:06 server16 kernel: [2272482.628638]
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0xd0 [ext4]
Jun 20 13:15:06 server16 kernel: [2272482.628680] Pid: 875743, comm:
php veid: 1602 Tainted: G    B      ---------------
2.6.32-042stab076.8 #1
Jun 20 13:15:06 server16 kernel: [2272482.628756] Call Trace:
Jun 20 13:15:06 server16 kernel: [2272482.628795]
[<ffffffff811560a8>] ? print_bad_pte+0x1d8/0x290
Jun 20 13:15:06 server16 kernel: [2272482.628835]
[<ffffffff811579a6>] ? unmap_vmas+0x5f6/0xce0
Jun 20 13:15:06 server16 kernel: [2272482.628877]
[<ffffffff8113ff5c>] ? release_pages+0x3dc/0x490
Jun 20 13:15:06 server16 kernel: [2272482.628918]
[<ffffffff81161501>] ? unmap_region+0x91/0x130
Jun 20 13:15:06 server16 kernel: [2272482.628958]
[<ffffffff8116316c>] ? do_munmap+0x26c/0x360
Jun 20 13:15:06 server16 kernel: [2272482.628998]
[<ffffffff811632b6>] ? sys_munmap+0x56/0x80
Jun 20 13:15:06 server16 kernel: [2272482.629048]
[<ffffffff8100b102>] ? system_call_fastpath+0x16/0x1b