Details
-
Bug
-
Resolution: Done
-
Blocker
-
None
-
6.5 SP1
-
Dell PowerEdge R730
Citrix XenServer 6.5 SP1
Description
At 01:06 this morning our server stopped responding. All VMs appeared to be unresponsive (i.e. could not access via RDP/Console, HTTP, HTTPS etc), although they could be pinged.
SSH to the host was responding. See attached file containing output from SSH.
This issue has occurred once before on 16th October 2015 but never prior. Hot fixes are installed as and when they become available.
To resolve, had to do a full power-cycle using iDRAC as running "reboot" via SSH did not work and rebooting via XenCenter was unsuccessful.
Snippet from /var/log/messages output:
Nov 23 01:06:31 L1PS02XEN xenstored: D76 write data/meminfo_free 2688812 Nov 23 01:06:31 L1PS02XEN xenstored: A834631 w event /local/domain/76/data/meminfo_free /local/domain/76/data/meminfo_free Nov 23 01:06:35 L1PS02XEN xapi: [ info|L1PS02XEN|1905130 UNIX /var/xapi/xapi|session.slave_login D:e9a5faf8a762|xapi] Session.create trackid=aa7197e4ebe1a67ca9f5315e515b3329 pool=true uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49 Nov 23 01:06:35 L1PS02XEN xapi: [ info|L1PS02XEN|1905132 UNIX /var/xapi/xapi|dispatch:SR.scan D:5a6972b539ec|taskhelper] task SR.scan R:b8f776fa35e0 (uuid:00997835-7255-50df-7b16-3704521537a2) created (trackid=aa7197e4ebe1a67ca9f5315e515b3329) by task D:9fc100f1a087 Nov 23 01:06:35 L1PS02XEN xapi: [ info|L1PS02XEN|1905132 UNIX /var/xapi/xapi|SR.scan R:b8f776fa35e0|storage_impl] SR.scan dbg:OpaqueRef:b8f776fa-35e0-b2ad-b23e-044b1aa50d80 sr:702a92f0-70cf-d895-d579-e20e94bcec30 Nov 23 01:06:35 L1PS02XEN xapi: [ info|L1PS02XEN|1905132 UNIX /var/xapi/xapi|sm_exec D:c08f5bef122c|xapi] Session.create trackid=a0bc9d0cc125fe1aef82346589a0f278 pool=false uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49 Nov 23 01:06:36 L1PS02XEN xapi: [ info|L1PS02XEN|1905132 UNIX /var/xapi/xapi|sm_exec D:c08f5bef122c|xapi] Session.destroy trackid=a0bc9d0cc125fe1aef82346589a0f278 Nov 23 01:06:36 L1PS02XEN xapi: [ info|L1PS02XEN|1905143 UNIX /var/xapi/xapi|session.logout D:910d84bc65e9|xapi] Session.destroy trackid=aa7197e4ebe1a67ca9f5315e515b3329 Nov 23 01:06:36 L1PS02XEN tapdisk[18065]: ERROR: errno -12 at tapdisk_control_accept: failed to allocate new control connection Nov 23 01:06:36 L1PS02XEN tapdisk[18065]: tap-err:scheduler_wait_for_events: select failed: Bad file descriptor Nov 23 01:06:36 L1PS02XEN last message repeated 20 times Nov 23 01:06:36 L1PS02XEN tap-ctl[23453]: tap-err:tap_ctl_read_raw: failure reading data 0/544 Nov 23 01:06:36 L1PS02XEN tap-ctl[23453]: tap-err:tap_ctl_send_and_receive: failed to receive 'pid' message Nov 23 01:06:36 L1PS02XEN tapdisk[18065]: tap-err:scheduler_wait_for_events: select failed: Bad file descriptor Nov 23 01:06:37 L1PS02XEN last message repeated 43592 times
