Garrawarla draining nodes due to OST usage

Incident Report for Pawsey Supercomputing Research Centre

Resolved

We have manually restriped some large files to reduce pressure on the OSTs. There are still 4 OSTs (4,9,39, and 42) that have far less free space than the others and should be migrated by users

Users are reminded about the importance of striping large files over several OSTs for both improved performance and reducing the risk of nodes being set to automatically drain.

Posted May 30, 2022 - 10:59 AWST

Update

Our administrators are attempting to free space on the critically full OSTs by migrating some large files to less full OSTs.

Posted May 26, 2022 - 15:53 AWST

Investigating

Garrawarla has begun draining nodes in response to the /astro filesystem nearing critical OST usage.

Posted May 26, 2022 - 15:12 AWST

This incident affected: Garrawarla (Garrawarla workq partition).