排查結果:
查詢日志獲取到的信息
虛擬機內核版本:
Jun 20 03:34:51 : Linux 2.6.32-642.1.1.el6. () (gcc 4.4.7 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Tue May 31 21:57:07 UTC 2016
查看到虛擬機重啟時間約為:2019/6/20 03:34 CST
Jun 20 03:31:04 : : -
Jun20 03:31:04 init: tty (/dev/tty1) main (2703) by TERM
Jun20 03:31:04 init: tty (/dev/tty2) main (2705) by TERM
Jun20 03:31:04 init: tty (/dev/tty3) main (2707) by TERM
Jun20 03:31:04 init: tty (/dev/tty4) main (2709) by TERM
Jun20 03:31:04 init: tty (/dev/tty5) main (2711) by TERM
Jun20 03:31:04 init: tty (/dev/tty6) main (2713) by TERM
Jun20 03:31:04 init: (ttyS0) main (2723) by TERM
Jun20 03:31:05 abrtd: Got 15,
Jun20 03:31:10 [1710]: on of
Jun20 03:31: acpid:
Jun20 03:31:13 : on . with " -w"Jun20 03:31: init: from bus
Jun20 03:31:14 [1424]: The audit is .
Jun20 03:31:14 : type=1305 audit(.940:): =0 old=1424 auid= ses= subj=:::s0 res= 03:31:15 : type=1305 audit(.039:): =0 old=1 auid= ses= subj=:::s0 res= 03:31: : (proc) .
Jun20 03:31:15 : [ ="" ="5.8.10" x-pid="1458" x-info=""] on 15.
Jun20 03:34:51 : 5.8.10, log = /proc/kmsg .
Jun20 03:34:51 : [ ="" ="5.8.10" x-pid="1457" x-info=""] start
Jun20 03:34: :
Jun20 03:34: : cpu
Jun20 03:34:51 : Linux 2.6.32-642.1.1.el6. () (gcc 4.4.7 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Tue May 31 21:57:07 UTC 03:34:51 : line: ro root=UUID=-fef6-4075-941e- =pc =us LANG=en_US.UTF-8 =-sun16 =ttyS0, =ttyS0, =
Jun20 03:34: : cpus:
Jun20 03:34: : Intel
Jun20 03:34: : AMD
Jun20 03:34:51 :
查詢到虛擬機重啟之前出現了 Panic的錯誤,提示:INFO: task jbd2/sda1-8:540 for more than 120 .備注:jdb2進程是一個文件系統的寫的進程
Jun 19 00:27:01 [1424]: Audit log files
Jun20 03:28:27 : INFO: task jbd2/sda1-8:540 for more than .
Jun20 03:28:27 : Not 2.6.32-642.1.1.el6. # 03:28:27 : "echo 0 > /proc/sys//cs" this .
Jun20 03:28:27 : jbd2/sda1-8 D 00000 0 540 2 03:28:27 :
Jun20 03:28: :
Jun20 03:28:27 :
Jun20 03:28: : Call Trace:
Jun20 03:28:27 : [] ? +0x38/0x80[]
Jun20 03:28:27 : [] ? +0x0/ 03:28:27 : [] +0x73/ 03:28:27 : [] +0x3d/ 03:28:27 : [] +0x5f/ 03:28:27 : [] +0x73/ 03:28:27 : [] ? +0x0/ 03:28:27 : [] ? +0x25/ 03:28:27 : [] +0xfb/ 03:28:27 : [] +0x2f/ 03:28:27 : [] +0x7e9/[jbd2]
Jun20 03:28:27 : [] ? c+0x7b/ 03:28:27 : [] +0xb8/0x220[jbd2]
Jun20 03:28:27 : [] ? tion+0x0/ 03:28:27 : [] ? +0x0/0x220[jbd2]
Jun20 03:28:27 : [] +0x9e/ 03:28:27 : [] +0xa/ 03:28:27 : [] ? +0x0/ 03:28:27 : [] ? +0x0/0x20
查詢了日志,僅僅發現有關機信號的記錄,沒有更加詳細的信息了 備注: 15 A 終止信號
Jun 20 03:31:05 sshd[1811]: 15; .
Jun20 03:34:59 sshd[1808]: on 0.0.0.0 port 22.
Jun20 03:34:59 sshd[1808]: on :: port 22.
查詢了Linux問題虛擬機所在的物理節點,發現早在6/20 03:10 CST左右linux重啟網絡服務報錯,物理節點已經出現了系統故障,虛擬機被自動遷移至其他可用節點,此過程造成了虛擬機的自動重啟
總結:在某些情況下,服務器系統日志出現了call trace的報錯,也不一定是操作系統層面引起的問題linux重啟網絡服務報錯,還可能是物理節點(針對虛擬機)或硬件故障。