Details
-
Bug
-
Resolution: Done
-
Critical
-
None
-
7.1
-
None
Description
Migrating an offline VM from one storage SR to another causes huge amounts of errors to be logged on a (seemingly random) host in the pool, and segfaults in vhd-util on the pool master.
Seemingly random host in the pool:
[2515464.422851] systemd-journald[520]: /dev/kmsg buffer overrun, some messages lost. [2515464.422873] sd 22:0:0:1: rejecting I/O to offline device [2515464.422896] sd 22:0:0:1: rejecting I/O to offline device [2515464.422997] sd 22:0:0:1: rejecting I/O to offline device [2515464.423021] sd 22:0:0:1: rejecting I/O to offline device [2515464.423111] sd 22:0:0:1: rejecting I/O to offline device [2515464.423133] sd 22:0:0:1: rejecting I/O to offline device [2515464.423216] sd 22:0:0:1: rejecting I/O to offline device [2515464.423238] sd 22:0:0:1: rejecting I/O to offline device [2515464.423331] sd 22:0:0:1: rejecting I/O to offline device [2515464.423355] sd 22:0:0:1: rejecting I/O to offline device [2515464.423395] systemd-journald[520]: /dev/kmsg buffer overrun, some messages lost. [2515464.423449] sd 22:0:0:1: rejecting I/O to offline device [2515464.423481] sd 22:0:0:1: rejecting I/O to offline device [2515464.423571] sd 22:0:0:1: rejecting I/O to offline device [2515464.423588] sd 22:0:0:1: rejecting I/O to offline device [2515464.423665] sd 22:0:0:1: rejecting I/O to offline device [2515464.423685] sd 22:0:0:1: rejecting I/O to offline device
The disk it's complaining about (tdk) is the 'new' disk / block device XenServer created for the aforementioned storage migration (LVMoiSCSI).**
After this occurred for 10 minutes or so the whole server locked up and took down all VMs running on the host.
dmesg on the pool master:
[271308.109158] vhd-util[6020]: segfault at 1 ip 00007f8f557ebcdc sp 00007ffe3c055988 error 4 in libc-2.17.so[7f8f556e1000+1b6000] [271337.929075] vhd-util[7295]: segfault at 1 ip 00007f29fc245cdc sp 00007ffda9c28688 error 4 in libc-2.17.so[7f29fc13b000+1b6000] [271368.211487] vhd-util[9022]: segfault at 1 ip 00007fb87e0d7cdc sp 00007ffcc1491458 error 4 in libc-2.17.so[7fb87dfcd000+1b6000] [271398.126917] vhd-util[9762]: segfault at 1 ip 00007f00c7249cdc sp 00007ffd8ec6b808 error 4 in libc-2.17.so[7f00c713f000+1b6000] [271428.134658] vhd-util[11026]: segfault at 1 ip 00007f570c0c3cdc sp 00007ffc097aee78 error 4 in libc-2.17.so[7f570bfb9000+1b6000] [271458.230058] vhd-util[12366]: segfault at 1 ip 00007ff686205cdc sp 00007fffec2e00d8 error 4 in libc-2.17.so[7ff6860fb000+1b6000] [271488.284843] vhd-util[14107]: segfault at 1 ip 00007f3ea6d62cdc sp 00007fff6ea53b28 error 4 in libc-2.17.so[7f3ea6c58000+1b6000] [271518.157534] vhd-util[14949]: segfault at 1 ip 00007fc96a359cdc sp 00007ffe0b7ac178 error 4 in libc-2.17.so[7fc96a24f000+1b6000] [271548.376274] vhd-util[16204]: segfault at 1 ip 00007f1f9287fcdc sp 00007ffc3d1c0c18 error 4 in libc-2.17.so[7f1f92775000+1b6000] [271578.323336] vhd-util[17410]: segfault at 1 ip 00007f3c37907cdc sp 00007ffda2a62508 error 4 in libc-2.17.so[7f3c377fd000+1b6000] [271608.366773] vhd-util[18641]: segfault at 1 ip 00007f5146f2ecdc sp 00007ffea6df0638 error 4 in libc-2.17.so[7f5146e24000+1b6000] [271638.350305] vhd-util[19842]: segfault at 1 ip 00007f74856e0cdc sp 00007ffda95180d8 error 4 in libc-2.17.so[7f74855d6000+1b6000] [271668.449327] vhd-util[21070]: segfault at 1 ip 00007f0b26f3bcdc sp 00007ffe694a37d8 error 4 in libc-2.17.so[7f0b26e31000+1b6000] [271698.416783] vhd-util[22281]: segfault at 1 ip 00007f9be6197cdc sp 00007fff20ad15b8 error 4 in libc-2.17.so[7f9be608d000+1b6000] [271728.503662] vhd-util[23214]: segfault at 1 ip 00007fc9fe40dcdc sp 00007ffccb522618 error 4 in libc-2.17.so[7fc9fe303000+1b6000] [271758.471919] vhd-util[24390]: segfault at 1 ip 00007fef56700cdc sp 00007ffd6ddd1f48 error 4 in libc-2.17.so[7fef565f6000+1b6000] [271788.550563] vhd-util[25637]: segfault at 1 ip 00007f1314a73cdc sp 00007ffd617a30a8 error 4 in libc-2.17.so[7f1314969000+1b6000] [271818.566013] vhd-util[26860]: segfault at 1 ip 00007f249ee86cdc sp 00007ffe661fda68 error 4 in libc-2.17.so[7f249ed7c000+1b6000] [271848.664730] vhd-util[28479]: segfault at 1 ip 00007fe568ae0cdc sp 00007ffd2eefa698 error 4 in libc-2.17.so[7fe5689d6000+1b6000] [271878.529300] vhd-util[30138]: segfault at 1 ip 00007f83e212bcdc sp 00007fffeeece7d8 error 4 in libc-2.17.so[7f83e2021000+1b6000] [271908.600749] vhd-util[31381]: segfault at 1 ip 00007f8f0c406cdc sp 00007fffe2832378 error 4 in libc-2.17.so[7f8f0c2fc000+1b6000] [271938.604470] vhd-util[32570]: segfault at 1 ip 00007f71be8f9cdc sp 00007ffeb169f8e8 error 4 in libc-2.17.so[7f71be7ef000+1b6000] [271968.686467] vhd-util[913]: segfault at 1 ip 00007f10523d4cdc sp 00007fff58a22828 error 4 in libc-2.17.so[7f10522ca000+1b6000] [271998.690890] vhd-util[2633]: segfault at 1 ip 00007f1ef615ccdc sp 00007ffe582eb338 error 4 in libc-2.17.so[7f1ef6052000+1b6000] [272028.745409] vhd-util[3959]: segfault at 1 ip 00007fbd54f40cdc sp 00007ffc158d5f18 error 4 in libc-2.17.so[7fbd54e36000+1b6000] [272058.837410] vhd-util[5201]: segfault at 1 ip 00007ff98bb9bcdc sp 00007ffd5bf5c5e8 error 4 in libc-2.17.so[7ff98ba91000+1b6000] [272088.819969] vhd-util[7048]: segfault at 1 ip 00007f5fe31edcdc sp 00007fffdd3dd8d8 error 4 in libc-2.17.so[7f5fe30e3000+1b6000] [272118.763481] vhd-util[8007]: segfault at 1 ip 00007f8a60e78cdc sp 00007fffe5d05d78 error 4 in libc-2.17.so[7f8a60d6e000+1b6000] [272148.839825] vhd-util[9262]: segfault at 1 ip 00007f035eb5dcdc sp 00007ffe88f938c8 error 4 in libc-2.17.so[7f035ea53000+1b6000] [272178.859123] vhd-util[10478]: segfault at 1 ip 00007f4c0ca3ccdc sp 00007ffe05175ad8 error 4 in libc-2.17.so[7f4c0c932000+1b6000] [272208.901499] vhd-util[12835]: segfault at 1 ip 00007f5a4a3b0cdc sp 00007ffe71640568 error 4 in libc-2.17.so[7f5a4a2a6000+1b6000] [272238.913549] vhd-util[13526]: segfault at 1 ip 00007ff746352cdc sp 00007ffd3a08ea58 error 4 in libc-2.17.so[7ff746248000+1b6000] [272268.936067] vhd-util[14780]: segfault at 1 ip 00007fc472a72cdc sp 00007ffcf449b1d8 error 4 in libc-2.17.so[7fc472968000+1b6000] [272298.859596] vhd-util[16225]: segfault at 1 ip 00007f6b52964cdc sp 00007ffdb8d74f48 error 4 in libc-2.17.so[7f6b5285a000+1b6000] [272329.019354] vhd-util[18037]: segfault at 1 ip 00007f67a465fcdc sp 00007ffc0b36ac28 error 4 in libc-2.17.so[7f67a4555000+1b6000] [272359.098866] vhd-util[19270]: segfault at 1 ip 00007fbab98e6cdc sp 00007fff335b2918 error 4 in libc-2.17.so[7fbab97dc000+1b6000] [272389.162301] vhd-util[20499]: segfault at 1 ip 00007fd373391cdc sp 00007ffc122f8188 error 4 in libc-2.17.so[7fd373287000+1b6000] [272418.965419] vhd-util[21275]: segfault at 1 ip 00007fb4229cecdc sp 00007fff73258308 error 4 in libc-2.17.so[7fb4228c4000+1b6000] [272448.988945] vhd-util[22671]: segfault at 1 ip 00007f4e15c7fcdc sp 00007fffef407a18 error 4 in libc-2.17.so[7f4e15b75000+1b6000] [272479.176011] vhd-util[24610]: segfault at 1 ip 00007fec63ae8cdc sp 00007ffec57b03f8 error 4 in libc-2.17.so[7fec639de000+1b6000] [272509.219720] vhd-util[25829]: segfault at 1 ip 00007fda1c709cdc sp 00007ffd9a5d3228 error 4 in libc-2.17.so[7fda1c5ff000+1b6000] [272539.143036] vhd-util[27053]: segfault at 1 ip 00007fe225892cdc sp 00007ffdece856e8 error 4 in libc-2.17.so[7fe225788000+1b6000] [272569.253663] vhd-util[28245]: segfault at 1 ip 00007f7b0c82dcdc sp 00007ffe29bb6398 error 4 in libc-2.17.so[7f7b0c723000+1b6000] [272599.237084] vhd-util[29495]: segfault at 1 ip 00007f6fbc7c0cdc sp 00007ffcd565c4a8 error 4 in libc-2.17.so[7f6fbc6b6000+1b6000]
