/askapbuffer unavailable overnight
Resolved
One of the server pair for the /askapbuffer metadata service rebooted overnight. HA handover to its partner doesn't seem to have worked correctly, causing the filesystem to be unresponsive.

Staff have restarted services and are investigating the root cause

askap-fs1-mds01-fence (stonith:fence_ipmilan): Started askap-fs1-mds02.pawsey.org.au
askap-fs1-mds02-fence (stonith:fence_ipmilan): Stopped
askapfs1-MGS (ocf::lustre:Lustre): Stopped
askapfs1-MDT0000 (ocf::lustre:Lustre): Stopped
askapfs1-MDT0001 (ocf::lustre:Lustre): Stopped

Failed Actions:
* askapfs1-MDT0000_start_0 on askap-fs1-mds02.pawsey.org.au 'unknown error' (1): call=26, status=complete, exitreason='',
last-rc-change='Wed May 27 23:24:44 2020', queued=0ms, exec=21222ms
* askapfs1-MGS_start_0 on askap-fs1-mds02.pawsey.org.au 'unknown error' (1): call=24, status=complete, exitreason='',
last-rc-change='Wed May 27 23:24:44 2020', queued=0ms, exec=21363ms
* askapfs1-MDT0001_start_0 on askap-fs1-mds02.pawsey.org.au 'unknown error' (1): call=25, status=complete, exitreason='',
last-rc-change='Wed May 27 23:24:44 2020', queued=0ms, exec=21197ms
Posted May 27, 2020 - 22:00 AWST