uname -a
Linux xxxxxxxx.com 2.6.18-164.11.1.el5 #1 SMP Wed Jan 20 07:32:21 EST 2010 x86_64 x86_64 x86_64 GNU/Linux
Code: Select all
Jan 11 12:30:25 cloud1 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [swapper:0]
Jan 11 12:30:25 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11135 duration=136(sec)
Jan 11 12:30:34 cloud1 kernel: CPU 0:
Jan 11 12:30:35 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11189 duration=44(sec)
Jan 11 12:30:35 cloud1 kernel: Modules linked in: dm_crypt ecb cbc aes_x86_64 testmgr_cipher testmgr aead crypto_blkcipher crypto_algapi aes_generic cifs fuse tun ipt_LOG ipt_ecn ipt_ECN xt_string xt_connlimit xt_mark xt_tcpmss xt_DSCP xt_MARK xt_multiport xt_pkttype xt_physdev bridge xt_mac xt_limit xt_length xt_conntrack ipt_REDIRECT ipt_recent ipt_owner ipv6 xfrm_nalgo crypto_api iptable_nat ip_nat iptable_mangle ipt_REJECT xt_tcpudp xt_state ip_conntrack nfnetlink iptable_filter ip_tables x_tables dm_mirror dm_multipath scsi_dh video hwmon backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac lp floppy ide_cd joydev i2c_piix4 8139too i2c_core parport_pc parport 8139cp cdrom serio_raw xen_platform_pci mii pcspkr xfs raid456 xor multipath ahci linear sata_nv dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache mptspi mptscsih scsi_transport_spi mptbase sata_sis pata_sis raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd ata_piix libata arcmsr 3w_9xxx 3w_xxxx sd_mod scsi_mod
Jan 11 12:30:35 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11195 duration=36(sec)
Jan 11 12:30:35 cloud1 kernel: Pid: 0, comm: swapper Not tainted 2.6.18-164.11.1.el5 #1
Jan 11 12:30:35 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11206 duration=34(sec)
Jan 11 12:30:36 cloud1 kernel: RIP: 0010:[<ffffffff80064b50>] [<ffffffff80064b50>] _spin_unlock_irqrestore+0x8/0x9
Jan 11 12:30:36 cloud1 xinetd[13711]: START: smtp pid=11256 from=178.187.72.234
Jan 11 12:30:36 cloud1 kernel: RSP: 0018:ffffffff8043dd60 EFLAGS: 00000246
Jan 11 12:30:38 cloud1 kernel: RAX: 0000000000000001 RBX: ffffffff804e3898 RCX: 0000000000000000
Jan 11 12:30:38 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11213 duration=36(sec)
Jan 11 12:30:38 cloud1 kernel: RDX: 00000000000001f0 RSI: 0000000000000246 RDI: ffffffff803e8600
Jan 11 12:30:40 cloud1 kernel: RBP: ffffffff8043dce0 R08: ffff810001000058 R09: 0000000000000e00
Jan 11 12:30:41 cloud1 kernel: R10: ffff81007f39a000 R11: ffffffff8005a3e7 R12: ffffffff8005dc8e
Jan 11 12:30:41 cloud1 kernel: R13: 0000000000000000 R14: ffffffff80077d08 R15: ffffffff8043dce0
Jan 11 12:30:41 cloud1 kernel: FS: 00002b18bcd97f70(0000) GS:ffffffff803c1000(0000) knlGS:00000000f7cf2ad0
Jan 11 12:30:41 cloud1 kernel: CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jan 11 12:30:41 cloud1 kernel: CR2: 0000000001d27a98 CR3: 000000005fc15000 CR4: 00000000000006e0
Jan 11 12:30:42 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11197 duration=43(sec)
Jan 11 12:30:42 cloud1 kernel:
Jan 11 12:30:42 cloud1 xinetd[13711]: START: smtp pid=11262 from=66.210.221.100
Jan 11 12:30:42 cloud1 kernel: Call Trace:
Jan 11 12:30:42 cloud1 kernel: <IRQ> [<ffffffff8000d7e7>] ide_intr+0x1d1/0x1df
Jan 11 12:30:42 cloud1 kernel: [<ffffffff80010b81>] handle_IRQ_event+0x51/0xa6
Jan 11 12:30:42 cloud1 kernel: [<ffffffff800ba2c5>] __do_IRQ+0xa4/0x103
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8006c9bf>] do_IRQ+0xe7/0xf5
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8005d615>] ret_from_intr+0x0/0xa
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8024418d>] rt_check_expire+0xdc/0x21d
Jan 11 12:30:42 cloud1 kernel: [<ffffffff802440b1>] rt_check_expire+0x0/0x21d
Jan 11 12:30:42 cloud1 kernel: [<ffffffff800972b4>] run_timer_softirq+0x133/0x1af
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8001231e>] __do_softirq+0x89/0x133
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8005e2fc>] call_softirq+0x1c/0x28
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8006cb3c>] do_softirq+0x2c/0x85
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8006c9c4>] do_IRQ+0xec/0xf5
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8006b2f4>] default_idle+0x0/0x50
Jan 11 12:30:42 cloud1 kernel: [<ffffffff8005d615>] ret_from_intr+0x0/0xa
Jan 11 12:30:43 cloud1 kernel: <EOI> [<ffffffff8006b31d>] default_idle+0x29/0x50
Jan 11 12:30:43 cloud1 kernel: [<ffffffff8004947b>] cpu_idle+0x95/0xb8
Jan 11 12:30:43 cloud1 kernel: [<ffffffff803fd7fd>] start_kernel+0x220/0x225
Jan 11 12:30:43 cloud1 kernel: [<ffffffff803fd22f>] _sinittext+0x22f/0x236
Jan 11 12:30:43 cloud1 kernel:
Jan 11 12:31:05 cloud1 xinetd[13711]: START: smtp pid=11266 from=64.78.150.46
Jan 11 12:31:22 cloud1 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [pdflush:23797]
Jan 11 12:31:22 cloud1 kernel: CPU 0:
Jan 11 12:31:22 cloud1 kernel: Modules linked in: dm_crypt ecb cbc aes_x86_64 testmgr_cipher testmgr aead crypto_blkcipher crypto_algapi aes_generic cifs fuse tun ipt_LOG ipt_ecn ipt_ECN xt_string xt_connlimit xt_mark xt_tcpmss xt_DSCP xt_MARK xt_multiport xt_pkttype xt_physdev bridge xt_mac xt_limit xt_length xt_conntrack ipt_REDIRECT ipt_recent ipt_owner ipv6 xfrm_nalgo crypto_api iptable_nat ip_nat iptable_mangle ipt_REJECT xt_tcpudp xt_state ip_conntrack nfnetlink iptable_filter ip_tables x_tables dm_mirror dm_multipath scsi_dh video hwmon backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac lp floppy ide_cd joydev i2c_piix4 8139too i2c_core parport_pc parport 8139cp cdrom serio_raw xen_platform_pci mii pcspkr xfs raid456 xor multipath ahci linear sata_nv dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache mptspi mptscsih scsi_transport_spi mptbase sata_sis pata_sis raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd ata_piix libata arcmsr 3w_9xxx 3w_xxxx sd_mod scsi_mod
Jan 11 12:31:22 cloud1 kernel: Pid: 23797, comm: pdflush Not tainted 2.6.18-164.11.1.el5 #1
Jan 11 12:31:22 cloud1 kernel: RIP: 0010:[<ffffffff80010b72>] [<ffffffff80010b72>] handle_IRQ_event+0x42/0xa6
Jan 11 12:31:22 cloud1 kernel: RSP: 0000:ffffffff8043de48 EFLAGS: 00000246
Jan 11 12:31:22 cloud1 kernel: RAX: 0000000002010000 RBX: ffff81007e41e740 RCX: 000000000000000e
Jan 11 12:31:22 cloud1 kernel: RDX: ffff81007e41e740 RSI: ffffffff8043deb8 RDI: 000000000000000e
Jan 11 12:31:22 cloud1 kernel: RBP: ffffffff8043ddc0 R08: ffffffff8005e2fc R09: ffff810081f42000
Jan 11 12:31:22 cloud1 kernel: R10: ffffffff8043df98 R11: ffffffff8005a3e7 R12: ffffffff8005dc8e
Jan 11 12:31:22 cloud1 kernel: R13: ffffffff803da180 R14: ffffffff80077d08 R15: ffffffff8043ddc0
Jan 11 12:31:22 cloud1 kernel: FS: 00002ac8589fe240(0000) GS:ffffffff803c1000(0000) knlGS:00000000f7cf2ad0
Jan 11 12:31:26 cloud1 kernel: CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jan 11 12:31:26 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11256 duration=50(sec)
Jan 11 12:31:26 cloud1 kernel: CR2: 00002ae22a3b9ee0 CR3: 000000006c679000 CR4: 00000000000006e0
Jan 11 12:31:26 cloud1 xinetd[13711]: EXIT: smtp status=0 pid=11262 duration=44(sec)
Jan 11 12:31:26 cloud1 kernel:
Jan 11 12:31:26 cloud1 xinetd[13711]: START: smtp pid=11271 from=65.98.103.130
Jan 11 12:31:26 cloud1 kernel: Call Trace:
Jan 11 12:31:26 cloud1 xinetd[13711]: START: smtp pid=11272 from=189.13.85.142
Jan 11 12:31:26 cloud1 kernel: <IRQ> [<ffffffff800ba2c5>] __do_IRQ+0xa4/0x103
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8006c9bf>] do_IRQ+0xe7/0xf5
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8005d615>] ret_from_intr+0x0/0xa
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8005a3e7>] ide_outsw+0x0/0x9
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8005e2fc>] call_softirq+0x1c/0x28
Jan 11 12:31:27 cloud1 kernel: [<ffffffff800122e6>] __do_softirq+0x51/0x133
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8005e2fc>] call_softirq+0x1c/0x28
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8006cb3c>] do_softirq+0x2c/0x85
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8005dc8e>] apic_timer_interrupt+0x66/0x6c
Jan 11 12:31:27 cloud1 kernel: <EOI> [<ffffffff8005a3e7>] ide_outsw+0x0/0x9
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8000c0b2>] __make_request+0x3f0/0x401
Jan 11 12:31:27 cloud1 kernel: [<ffffffff8001c028>] generic_make_request+0x211/0x228
Jan 11 12:31:28 cloud1 kernel: [<ffffffff882b5cb7>] :xfs:xfs_trans_read_buf+0x47/0x2af
Jan 11 12:31:28 cloud1 kernel: [<ffffffff80033437>] submit_bio+0xe4/0xeb
Jan 11 12:31:28 cloud1 kernel: [<ffffffff882bf357>] :xfs:_xfs_buf_ioapply+0x1ee/0x219
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882bf3bb>] :xfs:xfs_buf_iorequest+0x39/0x64
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882c334e>] :xfs:xfs_bdstrat_cb+0x36/0x3a
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882bf450>] :xfs:xfs_buf_iostart+0x6a/0x8e
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882a5aeb>] :xfs:xfs_iflush+0x22b/0x2a4
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882bb72f>] :xfs:xfs_inode_flush+0xbf/0xe1
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882c5c06>] :xfs:xfs_fs_write_inode+0x36/0x4d
Jan 11 12:31:29 cloud1 kernel: [<ffffffff8002fcfd>] __writeback_single_inode+0x1e9/0x328
Jan 11 12:31:29 cloud1 kernel: [<ffffffff882b5279>] :xfs:xfs_trans_first_ail+0x12/0x1d
Jan 11 12:31:29 cloud1 kernel: [<ffffffff80020d0d>] sync_sb_inodes+0x1b5/0x26f
Jan 11 12:31:29 cloud1 kernel: [<ffffffff8009fe9f>] keventd_create_kthread+0x0/0xc4
Jan 11 12:31:29 cloud1 kernel: [<ffffffff8005121a>] writeback_inodes+0x82/0xd8
Jan 11 12:31:29 cloud1 kernel: [<ffffffff800c8b9c>] wb_kupdate+0xd4/0x14e
Jan 11 12:31:29 cloud1 kernel: [<ffffffff8005685a>] pdflush+0x0/0x1fb
Jan 11 12:31:30 cloud1 kernel: [<ffffffff800569ab>] pdflush+0x151/0x1fb
Jan 11 12:31:30 cloud1 kernel: [<ffffffff800c8ac8>] wb_kupdate+0x0/0x14e
Jan 11 12:31:30 cloud1 kernel: [<ffffffff80032950>] kthread+0xfe/0x132
Jan 11 12:31:30 cloud1 kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
Jan 11 12:31:30 cloud1 kernel: [<ffffffff8009fe9f>] keventd_create_kthread+0x0/0xc4
Jan 11 12:31:30 cloud1 kernel: [<ffffffff80032852>] kthread+0x0/0x132
Jan 11 12:31:30 cloud1 kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
Jan 11 12:31:30 cloud1 kernel: